If I pay with cash at a store then I'm literally walking out with the goods. For services, I've already received the services before paying.

You're correct that there is no physical protection for the cash, but that is why most people do not carry lots of cash. Most store it in an insured bank account.