Seriously. The transformer coupled with tons of compute is why we got here. When that paper came out and people (AI researchers) saw the results many were confused or unconvinced. No one has any clue such an architecture would yield the results it has. AI systems has always been far more art than science and we still don’t even really know why it works. I feel like that idea being stumbled upon was sort of more luck than anything…