Not only should you be overriding stuff, but nowadays they include the LESS files (with Bower at least) so in essence you can pick and/or rewrite the parts you want.

The complaint that Bootstrap is so huge is no longer valid, people are just super lazy.

That's my take as well... imho, you should be using bootstrap from the source... I usually start with the source, copy the mixins and variables files, and then create a new main less file that references all the parts I need with my copy of mixins/vars... works out really well, and less need to worry about new versions.