The paper begins with:

  Language modeling has long been an important research area since Shannon (1951) estimated the information in language with next word prediction.

Man, I wonder what Claude Shannon would think of all this if he were alive today...