I, too, really love Elm, and have worked on some tooling for getting it running on the server. (Self promotion ahead). I wrote an NPM module for compiling and loading Elm[1], and also put together a more comprehensive project for scaffolding an Express application to communicate between Elm in the browser and Elm on the server, via ports (this is the first time I announce it anywhere, since I want to have some full examples before anything "official", but it is usable today).
It performs perfectly fine on my machine, but I have not done any comprehensive testing. I was actually quite surprised when I first got everything working and was streaming input to the server, state to the client, and rendering the latter without any obvious latency. There are even some trivial optimizations, such as dropping repeats from the state signal on the server, which should speed things up significantly. I have no idea if it will scale, but I do plan on adding some infrastructure for writing bots in Elm and having them run in a cluster, so it should be easy enough to test at some point.
1. https://www.npmjs.com/package/elm-loader
2. https://www.npmjs.com/package/elm-expressway