Often, one of those cost controls will be penny-pinching on salary. Don't get me wrong, I'd love a place that pays a generous salary and then buys knockoff/used aerons instead of brand new / brand name.
But often the company is either stingy or generous, and they view the chairs and the engineer comp as both things to invest in, or neither.
You know your salary before accepting the offer, so if they're that kind of cost control you should have advance warning when they lowball you.
I think the big reason why this is so common is because the type of talent a lot of these startups want to attract is explicitly immature talent. People who are drawn to fluffy perks without realizing that they (a) don't actually benefit from them in any real way and (b) are actually a warning sign of what's to come financially.
But up to you what kind of talent you want to attract.