Places where I've seen it, people who have the right of way take that to mean they get to play "traffic cop". E.g., at a 4-way stop the "extra polite" person who gets there first, instead of going through the intersection first, waits and waves the person who does not have the right of way to go through first, which makes that person (and everyone else) confused, and then causes a lot of slowdown, frustration, etc. Of course only happens in certain locales, where it's part of the local driving culture.
Or they slam on the brakes at the end of what should be a merging ramp or lane because the fore and aft distances to the other traffic that were good enough for the prior N people would be too close for them.
if you have the right of way, and you do not take the right of way, in most cases you are effectively disobeying traffic rules and disrupting the flow of traffic.
In rural areas where this is common there isn't traffic. Thus the politeness doesn't cost you anything. Also in those rural areas it is somewhat common for someone with the right away to have a reason to drive much slower than the speed limit and so giving up your right away makes things better for everyone - the other person doesn't have to try to pass you latter.
When there is traffic you giving up the right away is impolite to the person behind you and shouldn't be done. unless traffic is so heavy that you won't be able to move, and so you should give you your right away to someone who can go.