Mxnet does. The commenter just left that part out. "The library is portable and lightweight, and is ready scales to multiple GPUs, and multiple machines." https://mxnet.readthedocs.org/en/latest/distributed_training...