To add to this already good explanation, it's my understanding that base64 and the MIME packaging scheme were invented because many early Internet protocols made a distinction between text mode and binary mode due to the majority of the protocol framing being text. You can thank them for that textual decision(?) when you're debugging SMTP with a copy of netcat, but it means that a PDF attachment needs to be encoded to travel over email.

To bring this conversation back to PKI, base64 encoding means you can embed the GPG signature of your email body in the text itself, without having to deal with attachments like S/MIME does