You always need to outsource things. Unless you live on a desert island alone some things are out sourced. Most of us out source barter to a government run cash system, which we then further outsource to various banks.
While out sourcing is a risk, it is also an opportunity to offload the hard work. You need to figure out what is worth doing yourself; what you outsource completely and ignore; and what you out source and audit.
Accounting - including billing, is normally something you should out source and audit. Let someone else worry about all the hard and weird rules. However you need to audit it because if you do not someone corrupt will steal your money and you are still required to pay whoever you owe money after that happens - this is one way your company can go bankrupt.
That's why you need an abstraction layer for each 3rd party provider. I've been doing online businesses 20+ years and every time we didn't create an abstraction layer our switching cost rose.
Many "startups" who are hungry enough to provide good value for money switch to "enterprise" pricing when investor money run out and your operations costs may rise a lot or you get "discontinued". Look what Atlassian did with self-hosted versions.
Also for tracking/logging layers it may be beneficial to mirror the data you're sending to 3rd party providers to your own db/logs, because you won't get data locked when switching.
So my point is that you need to outsource smartly and keep in mind that the 3rd party dependencies increase operations risk.
BTW that's why any user request shouldn't depend on 3rd party resources in a blocking matter (e.g. synchronous JS from vendors' CDN). You might pay more for your hosting but if you factor the service disruption risks you may be better off for mission critical parts.
While out sourcing is a risk, it is also an opportunity to offload the hard work. You need to figure out what is worth doing yourself; what you outsource completely and ignore; and what you out source and audit.
Accounting - including billing, is normally something you should out source and audit. Let someone else worry about all the hard and weird rules. However you need to audit it because if you do not someone corrupt will steal your money and you are still required to pay whoever you owe money after that happens - this is one way your company can go bankrupt.