I sort of have a compromise solution for this. I use my banks autopay whenever possible which is a push instead of a pull. That way I can just shut it off instead of finding some weird website I haven't used in a million years or call and wait on hold for 20 minutes. I started using it mostly so I didn't have to buy checks but I saw it had some advantages beyond that.

It doesn't work well with variable bills though because I can't schedule an amount I don't know yet to be paid. I'm stuck using a pull for my power bill for instance.