I rather dislike that weird side ribbon thing. Some time ago I thought I'd try and build a version without it, but that wasn't as easy as I would have liked. Maybe I'll give it another go.
The side ribbon of VSCode is called the Activity Bar. You can put it on top of the sidebar (making it more like a tab bar) or disable it entirely (which makes it a lot harder to open the sidebar or change the page it's on).
It's as simple as right click it -> Activity Bar Position -> ...