Update: So if you run the server locally and select maps, you're presented with this screen, saying you need a key from openmaptiles.com: https://i.imgur.com/NTb5zIL.jpg.
Docs for building the tiles yourself are here [0,1]. It's worked really well for me so far. If you want a visual example of self hosted tiles, you can check out my (in progress) website [2]. The tiles are all hosted statically in S3 with cloudflare as a proxy, so it was pretty cheap.
Thanks, I will look into that. It's not altogether clear from their site — their Docker image basically implies that you're supposed to use their pre-existing datasets, which aren't free.
If you select "commercial product or company web", you're asked to pay for it: https://i.imgur.com/TrAmju4.jpg
If you select any of the non-commercial ones, the newest map data you get is from 2017 (!).
I'm not against paying, but it's not free!