Ahem, you need to full supervisor support as well with virtual memory (page table walkers, TLBs etc). And atomics. And floating point (etc).
This is all non-trivial and would make the design ~ twice as big and likely impact the cycle times in a rather sad way. But possible of course.
Anecdata: Full RocketChip (RV64GC) built for ECP5 85F comes in at 54k LUTs (out of 84k) and clocked at 14.8 MHz. However, the cycle time is related to the FPU which assumes retiming which yosys can't do. Without the FPU it's a more reasonable 50-60 MHz.