- In short: Optus says close to 2,700 customers tried and failed to call emergency services from their mobile phones during the November 2023 network outage.
- This number is more than 10 times higher than what the telco previously told the Senate.
- What’s next? Optus says it is writing to each customer individually to apologise, and the federal government is conducting a post-incident review.
Man I’d love to have beers down the pub with the engineer(s) pushing that change that night. I’m sure the story of how that incident unfolded would be insane.
I would imagine they feel pretty shit, especially if people literally died due to the delays caused by not being able to access 000 😬
Imagine if missing a dot in an IP or some other tiny typo caused a cascade effect that resulted in people dying. You’d carry that shit forever, even if technically it wasn’t “your” fault directly.
Any comment I made on the cause of the incident would be pure speculation, but I’d be astonished if it were something so trivial. I suspect it more likely to be through some unknown hardware bug that didn’t happen in the Dev/Test environments for some reason. From what they have told the Senate Committee, this was a routine update. Those don’t usually carry significant risk of failure.
So all we do know for real is that it was very unexpected and complicated to fix. Something minor like you are describing would not generally take 10 hours to roll back.