Aug 18, 2003 : Massive Power Outage Tests Web Hosts

📅 August 18, 2003 - Thursday's power outage was a significant test for Web hosting companies located in eastern North America, as most firms scrambled to address a massive failure that forced some regions to spend the entire weekend in the dark.

And while the electrical blackout that affected the Eastern Seaboard severely tested the robustness of the Internet architecture, the network did maintain its integrity.
According to Keynote Systems (keynote.com), an Internet performance management and testing firm, the Internet's major backbones and Web sites did not show any adverse effects from the massive power outage in the Northeast. The company watched all of the major U.S. Internet backbones from the 25 largest U.S. metropolitan areas and did not detect any major difficulties during the outage.
A number of news sites, such as CNN (cnn.com) and USA Today (usatoday.com) did experience minor performance and availability problems during the initial phases of the blackout, but Keynote maintains that such performance issues often result from heavy workloads during major news events. The two news sites did rapidly recover and returned to excellent performance on Thursday.
Network operators did report a drop between 2,600 and 3,510 prefixes within Border Gateway Protocol (BGP) tables. BGP is the protocol for exchanging routing information between gateway hosts. The drop constituted an overall decline in optimum traffic levels by one or two per cent. Network traffic is expected to return to normal as users regain Internet access.
Keynote, however, is warning that Internet access cannot be guaranteed until the electrical grid is stabilized. "As is true of the telephone system, the Internet and major Web sites have been engineered with redundancy and backup power systems to withstand power outages. However, this may change if emergency power generation facilities at some Internet switching centers of telephone centers begin to fail because of the extended outage," said Eris Siegel, a principal Internet consultant at Keynote Systems, in a prepared statement.
The outage reportedly did effect operations at Toronto's primary carrier hotel at 151 Front Street (151frontstreet.com). According to reports from the North American Network Operators Group (NANOG) mailing list, the facility apparently experienced air conditioning problems from Thursday night into Friday morning, which caused some major providers in the building to go offline. As a result, the outage apparently affected the Toronto Internet Exchange (torix.net), a major Internet junction within the city.
Other peering networks in Toronto, however, were able to provide continuous service. Peer 1 Network (peer1.net -> peer1.com) succeeded in maintaining its delivery of high bandwidth and server colocation services to its customers in both its Toronto and New York facilities.
"We are pleased to report that all of our facilities were up and running and continuous," said Geoff Hampson, President & CEO of Peer 1 Network. "Both facilities were effected by the outage - but they operated as designed - and switched over to diesel generation. Our facilities and most networks were up, though some network suppliers did experience some problems. The situation kept our engineers busy, but it is great to know that all the planning and procedures we instituted for such contingencies worked."
The company was even hailed by the Toronto Star, one of Canada's largest daily newspapers, as being "instrumental" in helping the paper publish a special edition Friday after sharing power from its generators. Peer 1 allowed the newspaper to set up a mini-command center within their facility, which permitted approximately 80 journalists to file their stories and transmit them via the Internet to the Star's printing facility.
Toronto-based Web hosting firms Q9 Networks (q9.com -> equinix.com), which maintains two diesel generators on the roof of an office tower in Toronto's financial district, and Hostingplex.com (hostingplex.com 👉 Total Reviews: 1
🙌 Average Rating: 2 / 10
👍 Good Reviews: 0
👎 Bad Reviews: 1
👈 Official Responses: 0 -> hostmds.com) also reported seamless transitions to backup power.
In the United States, many communications firms, including Equinix (equinix.com), Level 3 Communications (level3.com) and NaviSite (navisite.com 👉 Total Reviews: 1
🙌 Average Rating: 1 / 10
👍 Good Reviews: 0
👎 Bad Reviews: 1
👈 Official Responses: 0), reported an immediate and seamless transition to backup power systems supplied by generators that were able to last for several days if required.
The blackout left 50 million people through northeastern Canada and the United States without power.