I agree that it isn't about rehabilitation. And to a certain degree it definitely seems our system is geared towards retribution. But study after study[1] has shown that an individual's perception of the likely punishment (e.g., prison time) doesn't deter crime. Rather, an individual's perception of the likelihood they'll get _caught_ seems to be the main deterrent.
My understanding was that Bill Clinton and congressional democrats in the 90s felt they had to look tough on crime to appease the public and avoid losing elections to Republicans who favored similar policies.