December 2010: (via Y)
"The largest Israeli mobile telephony operator (Cellcom) ceased most operations for half a day for most of its 3.3 million customers. There is no full official explanation yet, but reports (Haaretz) and elsewhere said that they were updating the customer database with a new script that provides new services. It is a process that they do every few weeks. They checked the script at the lab and it was fine. So, during the night they uploaded it to the production system. They have a central Database and two backup systems. The spec is to update the production system, if all is fine, to update each of the backups. What they actually did was to update the three systems in parallel, in order to save work and time. The problem was that the script was faulty and all three systems (production and the backups) failed gradually and it was very difficult to restore operations, as no backup was alive ... Sounds like a Chernobyl style exercise, the good thing that this time it was just a telephony system. There are already some legal class actions against Cellcom – two of them claim 1.2B Shekels (around 250M Euro) damages."
Article on Reuters (reuters.com) TEL AVIV, Wed Dec 1, 2010.
Article on PR Newswire www.prnewswire.com NETANYA, Thu Dec 2, 2010.
