August 17th, 2009 Case Studies in Project Failure

Digital trunked radio system failures

August 17th, 2009  -  Only a small percentage of problems with digital trunked radio systems are reported by the news media, however there are a sufficient number of stories to establish reasonable doubt as to the suitability of these systems. In fact, there is a clear pattern that would cause a reasonable and prudent person to entertain a strong and honest suspicion as to the suitability of this technology for mission-critical public safety radio communication. 

LSE Breaks Down Biggest Failure in 8 Years

Sept. 8 (Bloomberg) 2008 - London Stock Exchange Group Plc, operator of the world's fourth-largest market, broke down on the day European equities posted their biggest gain in five months, hurting clients who trade an average $17.5 billion a day.  Trading resumed at 4 p.m. after an earlier computer failure left clients unable to buy or sell shares for about seven hours, according to the LSE's Web site. About 352 million shares, worth $2.5 billion, changed hands in the first hour of trading before the halt, more than twice the amount in the same period a week ago, sending the FTSE-100 index up as much as 3.8 percent. The breakdown left traders in Europe's financial capital in limbo as equities around the world rallied on the U.S. government's takeover of mortgage lenders Fannie Mae and Freddie Mac. The LSE, Europe's oldest independent exchange, said attempts to fix its biggest computer failure in more than eight years took ``longer than expected.''

Overstock.com's 4 Year ERP Nightmare

October 8, 2008 - Looking back through its short history, it appears that the ERP implementation has been nothing but trouble. The strife started in 2005. That's when Byrne's "my bad" mea culpa over the $14.2 million loss was further exacerbated by a customer service meltdown related to the hurried Oracle ERP rollout.

"I bit off more technology projects than my colleagues could chew," Byrne told investors at the time. "The last bite, an ERP implementation, was one bite too many, and we choked on it."

Overstock's transition from a homegrown ERP package to the new Oracle system, which was rushed to completion before the 2005 holiday shopping season, left it unable to tell customers the status of their orders or to provide any shipping information, reported The Register's Ashlee Vance in late October 2005. The result: Overstock.com's customer tracking systems were down for nearly a week. "In total," Vance added, "it's a shocking display for a supposed Internet-savvy retailer meant to compete with the likes of Amazon.com and eBay."

Canadian Problems with large IT projects

Only two of the seven projects looked at: the 2006 Census Online and My Account, My Business Account projects met all audit criteria for well-managed projects. Five of the projects were allowed to proceed with a business case that was incomplete or out-of-date or contained information that could not be supported. The majority of projects examined were undertaken even though departments lacked the appropriate skills and experience to manage the projects or the capacity to use the system to improve the way they deliver their programs.

Sainsbury's $526m Project Failure

In October 2005 giant British food retailer J Sainsbury had to write off $526m it had invested in an automated supply-chain management system. It seems that merchandise was stuck in the company's depots and warehouses and was not getting through to many of its stores. As a result, Sainsbury was forced to hire about 3000 additional clerks to stock its shelves manually. "If an ERP project costs more than $10m, your chances of coming in on time and on budget are statistically zero," Jim Johnson, chairman of Standish Group International, surveyed more than 8,000 software-application projects over the past few years. "You also have a 50/50 chance of its being canceled before it's completed after you've spent 200% of your budget."

Verdict on a $64m Project Failure

In April 2005 inter-departmental warfare played a significant role in the failure of a $64m federal IT project, according to the Auditor General, who had the last word on the long-running debacle yesterday. Among a series of criticisms the audit report notes ongoing tensions between the two agencies involved, Centrelink and the Department of Family and Community Services (FaCS). “There were tensions between FaCS and Centrelink at all levels during the project,” says the report. “It would have been surprising if there were not, as the two agencies had somewhat differing needs from the project. “Generally, work on the project progressed despite the tensions. The tensions were greater at more senior levels, where funding could not be agreed, and effective high-level governance of the project was not evident.” A memorandum of understanding covering funding and savings from the project was never concluded between the two agencies as intended, leaving responsibility for costs in dispute.

Software failure in hybrid vehicles

In May 2005 automaker Toyota recalled 160,000 of its Prius hybrid vehicles following reports of vehicle warning lights illuminating for no reason, and cars' gasoline engines stalling unexpectedly. But unlike the large-scale auto recalls of years past, the root of the Prius issue wasn't a hardware problem -- it was a programming error in the smart car's embedded code. The Prius had a software bug.

MCI 's Operational Infrastructure Failure

MCI had recently upgraded to a more scalable infrastructure, a move that reportedly caused the initial congestion and led to under-performance and complete network instability for over a week. As efforts to fix the problem repeatedly failed, MCI was forced to shut down the whole system for 24 hours. The Chicago Board of Trade was one of MCI's 3,000 customers rendered helpless by the outage. The failure disabled the electronic system that governs the board's exchange leading to an estimated loss of some 180,000 trades. At anywhere between $10,000 and $100,000 per trade, the loss of business was significant and tough to calculate.

UK Air Traffic Control Upgrade Project 

Flights across the UK were grounded on Thursday after an air traffic control computer failure at West Drayton control centre. Nats' Flight Data Processing System failed at around 0600 BST for an hour, after overnight testing of an upgrade. Thousands of passengers have been experiencing delays as airlines work to clear the backlog of flights. Planes had to be grounded at airports including Gatwick, Heathrow, Manchester and Inverness. By mid-afternoon, delays at Heathrow and Gatwick were still 90 minutes, while at Stansted and Scottish airports the delay was about 30 minutes.

Comair Back in Air After Outage

Comair airlines was scrambling to contain the fallout from a cancellation of over 1,000 flights on Christmas Day after its computer systems for reservations crashed, saying it expects to be running at 60% by Monday evening. Officials of the Cincinatti-based subsidiary of Delta Air Lines (Quote) blamed severe weather in the middle state regions as the cause of a surge in crew flight re-assignments that knocked out its computer reservations system. The disruption snarled airline traffic and stranded passengers up and down the eastern- and middle-region states through the weekend.

UK Government Operational Failure

A computer failure at the Department for Work and Pensions (DWP) was "blown out of proportion", says the government. The DWP admitted 80,000 staff were not able to process new pensions and benefits claims for several days, but regular payments were unaffected. DWP computer specialists have launched an investigation into the massive IT failure which took out a third of their computer network. BBC political correspondent James Hardy said on Friday: "After nearly five days of chaos and round the clock repairs, the system was finally given the all clear this afternoon."