Oracle, IBM Caused American Eagle's Eight-Day Blackout

Evan Schuman reports in StoreFrontBackTalk that: "It seems a failure in an Oracle backup utility coupled with the failure of IBM hosting managers to detect it and to verify that a disaster recovery site was operational were the key factors in turning a standard site outage at American Eagle Outfitters into an eight-day-long disaster, according to an IT source involved in the probe.

"The initial problem was pretty much along the lines of what StorefrontBacktalk reported on July 29, which was a series of server failures. But the problems with two of the biggest names in retail tech–IBM and Oracle–are what made this situation balloon into a nightmare."

The lesson here is a warning to all CIOs and systems managers that outsourcing your back-up, disaster recovery and Web Server platform to "the cloud" does not resolve you of responsibility for assuring that these are all being managed properly (even if this includes on site inspections) -- no matter how big and "reliable" the third-party providers are, and to review your Service Level Agreements to make sure that appropriate financial penalties are in place in the event that these services are not provided as specified.