“Why has my screen gone black?”
Recognize this question? Sadly, it’s one IT admins receive far too often. When it’s a workstation maybe it’s not so much of a big deal… but when it’s a server, chances are your day is about to get really interesting. And by interesting I mean long.
Unfortunately, unless flames or audible grinding or clicking noises can be heard from the drive array, it can be really difficult to understand how a server goes from working perfectly one minute to being a pile of junk the next.
There are, of course, times when it’s a miraculous, or easy fix. Simply using the ubiquitous "power it off/power it on" routine can work wonders. Or it could simply be due to a USB stick or CD-ROM being left in a drive, and “tricking” the server into trying to boot with no OS. Simply removing the media puts you back in business in no time. Other times it’s the array controller, or BIOS that, post OS patches and updates, needs a firmware update before you’re back in business.
As an example, after one shutdown followed by a move of the server across the room, the system hung on boot just after the RAID card BIOS loaded. Yes, we moved it carefully in case you were wondering; very carefully. Both drives in the OS RAID 1 partition had flipped to “Foreign”. No big deal… only the customer’s Exchange and Accounting Databases were located on the server as well as 10 years worth of files.
Fortunately, in this case a phone call to Dell, tech support let us into a secret: pressing the shift key twice yielded a ‘secret screen’. This allowed us to import the “Foreign” drive configuration and “hey presto!”, we were back in business.
Other times it’s a problem somewhere between the hardware and the layers of software found in a modern server. When it’s looking like something more complex, the pressure really starts to mount.
So how do you reduce that pressure?
Ultimately, anyone who has been in IT for a few years has faced the daunting task of a server down scenario and realistically if you’re relatively new to IT you can look forward to your baptism by fire.
How hot the fire burns depends on the quality of your backup and how much you have thought about what to do when you see the Black Screen of Death.
Ian Thornton-Trump, CSA+, CD, CEH, CNDA, CPM, BA is CTO at Octopi Managed Services Inc. Ian is an ITIL certified Information Technology (IT) consultant with more than 20 years of experience in IT security and information technology. He enjoys and maintains a strong commitment to the security community. From 1989 to 1992, Ian served with the Canadian Forces (CF), Military Intelligence Branch; in 2002, he joined the CF Military Police Reserves and retired as a Public Affairs Officer in 2013.
You can follow Ian on Twitter® at @phat_hobbit.