If you have compiled a pretty fat kernel with all useful, non-exotic, modules built in you will never have to recompile the kernel (unless a new release drops) when switching out hardware. You (most likely) only need the firmware, which apparently is quite a hassle in Debian based environments.
For NixOS and Gentoo, for example, all the firmware you will probably ever need is included in one big package: linux-firmware. Install that, and you are pretty much done and ready to use almost everything connected on your computer without any problems.
Almost, because there's one big exception: Nvidia video cards. Which will need their own proprietary drivers to run properly. AMD works out of the box...
For NixOS and Gentoo, for example, all the firmware you will probably ever need is included in one big package: linux-firmware. Install that, and you are pretty much done and ready to use almost everything connected on your computer without any problems.
Almost, because there's one big exception: Nvidia video cards. Which will need their own proprietary drivers to run properly. AMD works out of the box...