Well, generally you want something with better performance characteristics, which increasingly is not the case (see e.g. https://www.nature.com/articles/s41550-020-1208-y%C2%A0). Though, even if your stuff is implemented in Fortran or C or C++ or on a GPU, Python is still a natural scripting language that is often the primary interface to some other piece of code, thanks to swig or boost::python or cppyy. But Python's fast rate of change and lack of backwards compatibility makes it problematic for this use. Our HPC cluster recently forced everyone to upgrade to Python 3 and that caused all sorts of problems for everyone. I understand there isn't really a choice with the lack of Python 2.7 support, but this has not been a great experience for anybody.
Yeah totally. Our use cases seem very similar. We still haven't transitioned to python3 at our HPC.
Well.. we do have it available but only around ~10% of the workloads make use of python3 and their libraries. We have created the two different paths in our NFS and if you're in python3 then you can't access python2 and vice-versa.
We will be migrating everything to python3 eventually but as you noted it is a big effort with a lot of possibility for chaos.
