I think this is just a bug though. But yes, I have been thinking of creating a new version of x86 that drops things like segmentation, real mode, virtual 8086 mode, and 16-bit protected mode.
Coincidentally I've also thought about creating a "better x86", but in the opposite direction: on the basis that segmentation is extremely powerful and useful, I think it would've been better for all the various virtualisation extensions to be based on the same model that V86 and protected mode started: segment descriptors of different types. That would allow V86, 16, 32, and 64-bit (using "double-wide" descriptors, to allow a 64-bit base and limit to fit) protected mode "tasks" or VMs to coexist completely, and the OS to switch between them easily. There are enough reserved spaces in the existing specification to allow these to fit nicely (in fact, what I came up with fits so neatly that I wonder if Intel had originally envisioned them for this purpose, if AMD hadn't beaten them with AMD64.)
Instead, now we have partially-backwards-compatible (and even slightly different) AMD64/Intel64 and at least two non-interoperable virtualisation extensions (AMD-V, VT-x).
If you're going to create a wildly incompatible "better x86", just start with a RISC design and be done with it. Compatibility with modern x86 OS's is the reason for x86's existence (note that v8086 mode is not required for this). Drop that and you might as well just fix all the other problems with the ISA while you're at it.
I'd be interested to know how much ARM have an advantage here in terms of their ISA? Presumably if more resources were dedicated to it, ARM cpus could be quite a lot faster than x86?
I doubt it. For one, 32-bit ARM is pretty complex too (though 64-bit ARM is a very nice simplification). More importantly, though, you can overcome the x86 complexity by simply throwing enormous manpower at the problem, and that is what Intel has done over the years.
(That doesn't mean the complexity is justified, just that it can be overcome. I think that people often assume that x86's dominance means that there's something inherently amazing about x86 from a microarchitectural point of view. That isn't true; rather it's simply that x86/Windows has historically earned Intel so much revenue that they've been able to fund the manpower needed to keep it alive and on top.)
It's complicated. There's a fixed amount of design overhead in supporting all the various x86 modes but compared to designing a top-end processor those aren't too big. And being able to decode 4 x86 instruction at once is hard compared to doing that with ARM but compared to the size of a 192 entry reorder buffer for deep out of order execution the cost isn't huge. Both x86 and ARM want to convert their ISA operations to a different format for use internally. ARM's is closer to their ISA instructions but it's not clear that that makes a big difference. The biggest issue in practice at the high end might memory ordering constraints with x86 being very tight and ARM being very loose. But I can't actually say for sure.
But this is probably why Intel's efforts to extend x86 down to lower power processors haven't been huge successes.
Keep in mind that if you drop the old stuff, you'll have to provide a bootloader and even then it'll be weird; modern systems still boot in 16 bits, bootstrap in 32 bit mode, then configure 64 bit. Unless UEFI fixed that madness?
The processor itself still boots up in 16-bit mode. UEFI does switch into 64-bit mode before booting an OS loader, but that doesn't mean the processor itself starts in 64-bit mode.
BIOS itself can be implemented as 32-bit code, which contains software emulator of 16-bit code. This emulator runs until CPU is switched to 32-bit protected mode by OS bootloader.
