Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

A one-day outage in December can be crippling for retail.

I don't doubt that many functions are difficult to failover, but a bare-bones minimum seems straightforward. For example, evidence of delivery is append-only and only needs to be globally consistent later, after a dispute.



I totally understand why Amazon halted everything. Sure, one could deliver shipments off-line and sort them out afterwards manually. And at a lower scale Amazon might have tried it (at least it would have been on the table when I worked there a couple of years ago).

But what then? You had a complete loss of traceability of shipments and operations, once you regain it, a junk of shipments isn't there anymore where there are supposed to be. No you have not one potential root cause for this, the outage, that could be resolved by retriggering those shipments (not loss, as you didn't deliver anything to customers) but two: the outage and some off-line shipments. In case it was just one FC, sure that would be doable. If the whole network in a complete region goes down, no way to handle that. It is much easier and safer to just stop operations until the outage is resolved, re-route orders to other regions in the meantime, and then work through the backlog. Amazon's ops are good at that, specifically because they have almost complete transparency on their material flows. Going off-line would have jeopardized that transparency, making a quick recovery after the outage all the harder.


Any idea if Amazon has insurance to cover this type of event?


Can't speak for Amazon. Generally so I don't see how one could insure against it. I know that e.g. Allianz offers policies against IT outages. In that case so, what is the actual damage? Probably the delivery drivers paid without delivering and salaries, plus potential overtime to solve the backlog. Depending on the conditions a company the size of Amazon would get, maybe it's not worth it.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: