Hedonic adjustments are important but, as you point out, have issues.
But, affordability isn't represented well by inflation, either. Median family income in 1950 was equivalent to $45k today; the actual number is $100k.
A big part of the issue is that workers stopped benefitting from productivity growth in the mid-1990s. That's what's really made the housing crisis particularly sharp.
It has, because people are greedy and self centered and capitalism exploits this better. Nobody's gonna work harder and smarter if they know they'll get the same outcome as those who don't, as per socialism.
That's why Europe is falling behind the US at tech innovations. People aren't gonna put in the risk, the grind and the hours if they'll get the same living standards as a unionized tram driver.
Ultimately it doesnt change the fact of affordability unless one can also find homes w/o indoor plumbing to make up for inflation.