Using a YubiKey does require that you provide your PIN every time you want to do a signature, which limits how much you can automate things. A YubiHSM would remove that requirement, and might be able to work with a self-hosted GitHub Actions Runner, but it's more expensive, and you'd want confirmation from your CA (Sectigo, for example) that a YubiHSM is OK.
> Using a YubiKey does require that you provide your PIN every time you want to do a signature, which limits how much you can automate things.
AutoHotkey is your friend!
When I set up our code signing machine at AltspaceVR ten years ago, I wrote a simple little AutoHotkey script that watched for the signtool PIN popup and typed in the PIN. It was maybe 15-20 lines of code.
https://support.sectigo.com/IS_KnowledgeDetailPage?Id=kA03l0... is an example explaining how to get both a CSR and an attestation certificate from a YubiKey 5 FIPS, on Windows.
https://support.yubico.com/hc/en-us/articles/360016614840-Co... explains how to use Windows' signtool with a Yubikey.
Using a YubiKey does require that you provide your PIN every time you want to do a signature, which limits how much you can automate things. A YubiHSM would remove that requirement, and might be able to work with a self-hosted GitHub Actions Runner, but it's more expensive, and you'd want confirmation from your CA (Sectigo, for example) that a YubiHSM is OK.