A Disaster Non-Recovery Story
Based on true events, as relayed by the Senior Demon In Charge of Infernal Machines
Like many abyssal demesnes, mine was required by various diabolical agreements to have some sort of “auditing” that showed that, in the event that our company could not function, somehow its functions would still be accomplished. This always struck me as a bit of an impossibility, but what do we expect from boilerplate contracts drafted by drop-outs of the seventh circle of Hell and lazily used by people who barely qualify for the second circle?
It started off as a typical Monday: wake up, morning routine, kiss the mistress goodbye and wish the little imps to go to hell, then off to the “commute,” which, by the way, has nothing to do with “commutation,” which is why all those on the road who think they are getting to their abyssal domains before me are only wasting their money passing me. I got into the office and sat down, examined my empty schedule and began thinking of new and clever tortures for my company. First up: how to make Quickbooks run even slower.
To be honest, it was looking like an “up” sort of day.
But then, around 10:30–11:00 — the confusion about when is caused by the fact that no one called it in to the power company when it happened, but waited 20 minutes or so before finally resigning themselves to the fact that the power was not, magically, going to come back on — the power was cut to our facility. It was an impressive display of darkness, punctuated by the then-blinding lights of computer screens that hadn’t gotten the memo.
I shut down my computer, stepped into the server room, awash in the sonic distress of several protesting UPSes, and shut down the servers. At this point, everything I had designed for a moment like this had worked perfectly, which, frankly, as Senior Demon In Charge of Infernal Machines, was really to be expected, but c’mon, let me revel in my work, okay?
As I’m in the server room, the Facilities manager, an old demon, survivor of infernal wars I can’t even imagine, stepped in, climbed atop a chair, and opened a box on the wall, disconnecting the battery to our door maglocks. “Don’t want to repeat that fiasco from a few years ago when we were locked in,” he chuckles.
“Aye, but I thought you were going to connect a switch to make that easier?”
With a twinkle in his eye, he says, “That wouldn’t be nearly as diabolical, would it?”
Have to give it to the old man: he does think of everything.
As I wander through the now pleasantly quiet and refreshingly dark building, I note the chorus of UPSes sounding their pathetic alarms and wonder, “Would this be better in some different sound wave, like “Infernal Chorus?”” but that’s a moneymaking idea that I’ll leave to someone else. What impresses me is that some demons are reporting that their UPSes didn’t work at all: their work was abruptly destroyed. This brings a frown to my face. The point of these things is to give the illusion of control. I shall have to have words with someone at the poorly-named “CyberPower” abyss. But what really arches my brow is when I hear from some of these demons — people who have known me for years as an approachable demon who is easy to talk to once you fill out the appointment form in triplicate and wait for me to open my locked panic room door — that they were aware that their UPS batteries had failed some months before and never bothered to so much as yell or send an email about it.
It’s enough to make an old demon cry. How much of my carefully-constructed Disaster Recovery plan is based on such lies?
I slide into a discussion that the Senior Management Demons are having with the facilities manager. Apparently, the imps responsible for providing electricity are signaling that power will be cut off for at least three hours. With a full day crew sitting around wishing they were anywhere but here, Senior Management makes the decision to reach out to their lowest-priced generator vendor. Better to pay a few dollars now and get things back online early rather than lose half a day and pay for the labor. “Ruthless efficiency,” I think is the term.
The facilities manager conducts the arcane ritual to summon the generator, but the spell fizzles. He tries again, with no success. Finally, he mounts his ram and speeds over to that abyss, only to discover that, sometime during the pandemic, when many people were working from home, the market seems to have dried up in the portable-on-demand generator business, and an important part of our Disaster Recovery Plan is, in fact, missing, rather permanently.
He so reports and I catch the faintest twinkle in Senior Management’s eye when they announce that we shall have to schedule a Meeting — code for “opportunity to berate and belittle underlings” — to discuss alternatives, since clearly we can’t be expected to not do our jobs simply because we lack the power to do our jobs.
We can’t be expected to not do our jobs simply because we lack the power to do our jobs!
And with tax season right around the corner, what we really need to punish people right now is the high cost of capital purchases. Excellent.
One of the Senior Management demons is able to find some “busywork” for the office demons to do: stuffing envelopes that, if our power was on, would take one demon mere minutes, but even with twenty working on it for two hours, will not be finished in that time. I laugh inside at the inanity of it all as I slip outside with the lamest of excuses, “I want to see how the imps are getting on with restoring our power.”
Surprisingly, the imps come through. Almost exactly three hours later, the lights flicker back on. Of course, I’m still down the street watching the imps, so Senior Management summons me to return and bring everything back on. I arrive 15 minutes after the power is on.
The Junior IT Demon is there when I get back, floundering as other demons are complaining that they can’t do their work, despite their computers being on. “I don’t know how to turn the servers on!” He tells me.
I smile: “The instructions are in the “\IT Stuff” folder. Did you look there?”
He frowns, ashamed. “No.” I wait at least three seconds, staring at him with the look of, “It was a joke and you didn’t get it,” before he does, in fact, get it. “Oh! That folder isn’t accessible with the servers down!” We laugh and I tell him that I wrote out the instructions while watching the imps work and we’ll just copy it off so he can store the instructions on his person. I don’t tell him that without the passwords — also stored on the server — he won’t be able to do these steps, but then, we need to leave SOME things to be discovered, don’t we?
Everything comes up as expected and within 15 minutes, the demons in the abyss are able to get back to their wage-slavery and/or time-killing. I spend another hour showing the Junior Demon that one must not rely on “the computer came back up” to ensure that the apps and services on that computer are running, and then, blessedly, the moment is over.
As I drive home I muse on the fact that, after twenty years, my employer is finally going to buy a generator to power at least some of the building and the fresh hell that will be for the facilities manager to rewire. And the cost! Oh, the cost! I rub my claws together just thinking of how Senior Management is always saying my department is a cost-sink. Their jaws will fall off when they see the estimates for what they’re planning.
Disaster Recovery is, in its own way, its own disaster. Truly wise Demons know that you need to have a recovery plan for the disaster recovery plan.