No doubt you’ve probably heard about Delta’s no good, horrible, very bad week. It began early Monday morning when a power failure at its Atlanta data center led to an outage of epic proportion, causing a domino effect in which all its systems shut down.
The entire airline was affected, causing over 700 flights to be canceled worldwide and delaying countless others on Monday alone.
While it’s beginning to get on its feet today, this should serve as an object lesson to those who think that a private data center is a better option than the cloud. What this episode shows is that even when you think you’re in total control, this is in fact a fallacy — and your entire business could be pivoting on a single point of failure.
Surely, we’ve seen power outages at cloud data centers too, usually due to severe electrical storms, but there were other data centers they could fail over to.
In fact, Delta did have backup servers, as you would expect for an operation as big as a major airline, but some of the systems failed to switch over to the backups, according to reports.
If Delta had some of its operations running on cloud servers, chances are it would have had operations spread across multiple vendors, following the pattern of most big companies. An outage in one place might have taken down one or two systems, but it wouldn’t have taken down every system.
Time to pay the piper
The result is going to be pretty devastating financially — costing the airline millions of dollars, according the Wall Street Journal. Delta reported the outage could wreak havoc with the schedule for the entire week, even when they get back up to full strength. They’ve said they will issue full refunds to passengers whose flights have been canceled or significantly delayed, and they have waived change fees for the week. That adds up quickly.
We’ve also seen the problem with single points of failure in security breaches. When all your content is sitting on one network, if someone gets in — as happened with Sony — everything from your email to your intellectual property is open to attack.
As Aaron Levie, CEO of Box pointed out last year in an interview if Sony had spread out its content over various cloud services, it’s possible hackers might have gotten access to some IP, but the chances of getting access to all of it would have been highly unlikely.
The WSJ reported that Delta is running 1990s technology in a single data center. When something goes wrong, it can have a huge impact, as we saw this week.
That means even if you think you’re in control, you’re one disaster away from your whole business going down. Delta certainly learned that harsh lesson this week. Who’s next?
