There are many ways. A simple way is to simulate a USB hub with an input device and a usb drive. You use the input device to execute whatever is on the drive. Another way is to identify as a device whose driver has some vulnerability. Windows auto-installs that driver, then you exploit it.
Sure, if you're the one who created the USB drive then you could make it not actually a USB drive. But this sounds like an infected machine infecting previously safe USB drives and turn them into malicious ones. And I'm not sure I get how a USB drive can be turned malicious. I vaguely remember there was a bit you could flip in older USB drives to make them appear as disk drives and enable autorun, but I doubt that's how this is done.
I think its the firmware. Outside of the main drive, there are smaller chips that work with the OS to r/w the main drive. Each chip has firmware whose memory is usually r/w as well.
Once you can manipulate the code on the firmware, its probably pretty easy to find a kernel level exploit.
> simulate a USB hub with an input device and a usb drive
Yea but that has to be a custom or specific kind of programmable USB device. Or one that somehow unintentionally allows you to reflash its firmware to something else.
And also if anyone ever plugs your malicious USB device into a Mac, they will get a pop-up from macOS that asks you to identify the keyboard. Although maybe if it fakes a specific USB keyboard that macOS knows out of the box, you could avoid that?