More previous tokens in this case would mean more previous frames. But there's really no reason to just stick to rendered pixels as input (except for novelty's sake) because we could train directly on snapshots of full game state.