A transformer will attend to previous tokens, and so it is free to ignore prior errors. I don't get LeCun's comment on error propagation, and look forward to a more thorough exposition of the problem.