Verizon's data center downtime: A lesson in power management

Verizon's data center downtime: A lesson in power management

Flight delays are only one possible outcome of data center downtime.

Data center power management is a frequently discussed topic in the IT universe, and for good reason. A lapse in power management can result in the needless consumption of energy, shorter life spans for critical electronics, or worse, downtime that can cost millions of dollars or more. 

Surely, anyone who keeps up to date with the happenings in the world of data centers is aware of just how crippling the d-word can be. But what exactly does downtime look like down the line? A recent power outage in a Verizon-owned data center shed some light on the calamity that can ensue from just a single outage and provided further clarification of what's at stake when power management is anything less than meticulous. 

Downtime leads to delayed flights

On Jan. 14, thousands of JetBlue passengers were notified of flight delays, shortly after the airline's website crashed. According to Data Center Knowledge, the site stayed down for several hours and disruptions in flight schedules resulted in long airport lines and general frustration among travelers. Even after the site became operational, complications continued to wreak havoc at check-in points and other airport systems, and as a result, passengers flooded social media with their grievances. In total, approximately 200 flights were delayed. 

JetBlue's response for the situation pointed to one of the largest Internet service providers in the U.S.: Verizon Communications. 

"The power was disrupted during a maintenance operation at the Verizon data center," JetBlue wrote in a blog soon after the issue first occurred. "Verizon can provide more details into the cause."

According to TechTarget, IDC research manager Kelly Quinn noted that TechTarget, IDC research manager Kelly Quinn noted that a three-hour long outage in a data center is very unusual, and raises some questions about Verizon's disaster recover plan, which in theory, should have backed up the facility. More specific details regarding the outage have yet to be provided by either JetBlue or Verizon. 

Verizon's data center outage left a lot of passengers grounded, and feeling frustrated.Verizon's data center outage left a lot of passengers grounded, and feeling frustrated.

How could this have been avoided?

Threats to data center functionality abound, and as such, it's a matter of "when" and not "if" an outage will risk causing downtime that could impact a customer's bottom line. In this case, Verizon was the colocation provider, and JetBlue was the customer. So the question really is, how can data center managers on both sides help ensure that this doesn't happen to them?

"Every data center must have an uninterruptible power supply."

For one thing, in the event that there is a power disruption, every data center must have at least one uninterruptible power supply that would allow it to stay up and running for at least a few hours once the outage occurs. Transfer switches should kick into action immediately, and automatically route electricity from the UPS into certain pieces of mission-critical equipment. If everything functions accordingly, outages should be ephemeral at worst, and not nearly substantial enough to impact thousands of customers and ground 200 flights.

Power management is not an easy job, and the possible number of sources of an outage is significant. Nevertheless, power monitoring systems that link with data center infrastructure management systems give management a way to know if they are achieving operational efficiency - loads must stay balanced to ensure that resource consumption is sensible, and that sensitive electronics run at optimal levels. This can help reduce the risk of outages that lead to unplanned downtime. 

However, as the Verizon data center outage shows, it's not enough to just have preventative tools in place. The UPS is vital to ensuring that in the unfortunate, but all-too-common event of an outage, equipment can still be sustained. Geist transfer switches, for example, are designed to respond to an outage of the primary source of power within milliseconds. Once the power is restored to the original source, equipment automatically switches back over.

Don't leave customers in the dark. Leverage a UPS and reliable transfer switches to achieve a failsafe disaster recovery plan for your data center or server room. 

Share