This method is actually used by many countries, most of them in Africa, where thanks to MPESA and such the need of protection against SIM swapping is even higher, since your SIM is literally your bank account.

https://www.wired.com/story/sim-swap-fix-carriers-banks/

"The SIM Swap Fix That the US Isn't Using While foreign phone carriers are sharing data to stop SIM swap fraud, US carriers are dragging feet."