It is not their residence. They broke in and just moved in.
Kicking out people who are trespassing is how you have a safe and lawful society. Allowing private citizens to do whatever they feel like to innocent people is mafia like behavior.
It is their residence. That's the semantic difference between residence and ownership. You may reside somewhere without owning it, and vice-versa.
Now, breaking into people's residence is a different matter, and is already highly criminalized. Laws about squatting and tenants rights don't exist in the void without a reason: they are supposed to be a balance between ownership rights and housing rights. Allowing the real estate mafia to make its own law is not exactly a balance...
Kicking out people who are trespassing is how you have a safe and lawful society. Allowing private citizens to do whatever they feel like to innocent people is mafia like behavior.