After some time you have to consider the test is failed and investigate, even if it would have succeeded had the timeout been 1 second larger. I cannot believe they do not have quality of service requirements. Testing those requirements is of course not easy. It may take to much time to run on every release or may be considered out of the scope of E2E tests and compliance is checked with telemetry results.
However pick any response time mandated by the QOS requirements, multiply by an appropriate x and use this as the pass/fail timeout for your test. Take a value large enough that can easily be considered a bug (because e.g. the customer would think the operation failed and would hit refresh or back). You then have an issue that is definitely worth investigating. You may actually have reproduced a rare issue that is part of the long tail of your telemetry.
Right have a timeout measured in minutes. The timeouts have zero effect on a clean run, so large timeouts have no effect on time to deploy if you require a clean run of tests for deploying.
However pick any response time mandated by the QOS requirements, multiply by an appropriate x and use this as the pass/fail timeout for your test. Take a value large enough that can easily be considered a bug (because e.g. the customer would think the operation failed and would hit refresh or back). You then have an issue that is definitely worth investigating. You may actually have reproduced a rare issue that is part of the long tail of your telemetry.