I think the premise (neural nets get stuck in local optima) is not trivial, and there has been a lot of research about non-convex optimisation showing that this is not much of an issue. I am not a researcher, but this answer [0] points to this research.

I would also say that there are multiple ways to escape local optima (setting a larger learning rate, multiple random initialisations, ensembling).

https://www.quora.com/How-come-neural-networks-dont-get-stuc...