If your hashes are perfectly distributed, as in each bit having a 50% probability of being 1, independent of all other bits, then the low order bits will be just as random as the high order bits, no "making the impact less bad" required.