That's true. I don't see any reason why they couldn't cache non-amp content with a combination of checking for speed benchmarks and schema markup. When you think about it that way, it seems like they are more concerned with controlling the user experience than they are with speed improvements.