You must have worked on some incredibly bad software to have non-determinism dominate your life. If a request fails then retry it just like a user would. If it keeps failing there's a problem. If it works then move on. I doubt your bank just throws in the towel and says "whelp software systems are inherently non-deterministic so we'll just forget some transactions here, allow the wrong amount over there, forget tests they're hard we can handle a little chance in our payment flows". The closest thing I've heard to that is amazon very occasionally shipping multiples of the same item because it was allegedly more expensive to implement immediate consistency than to ship a few duplicate items.