And a similar concept to this is that given a set of accepted axioms, two parties should always come to the same conclusion if both parties are behaving rationally. All debates are at best disagreements of accepted premises and definitions.
It's a important concept to remember for anyone who's argumentative/legalistic since understanding the oppossing sides definitions and desired conclusion is often the cause for disagreement. If there are no first principles, then there are no definitive last principles.
Also, in some ideal sense, since we attain consciousness as pure minds untainted by experience, tabula rasa, all that follows is the result of sense information on the network. The concept of individuality is weak. Two pure minds, fully rational and capable of honest communication, can be different by having different sense information, but if they are able to communicate with each other they should, in time, be identical. i.e. once I have given you everything I know completely and you have given me everything you know completely, we are not distinct, you and I are the same. A full mind meld.
We can't do this for all sorts of reasons, but two AGIs truly meeting may, which means that there is a sort of greyness to this true meeting of AGIs: by meeting they achieve unity. All AGIs behaving in this manner then become the same purely by meeting. Interesting.
This makes total sense, because of the assumption of common priors.
Drop that assumption, and fully rational people can indeed agree to disagree. This is a large part of why rational people can look at the same evidence, and come to different conclusions.
> This makes total sense, because of the assumption of common priors.
That's the Wiki article, yes. But scarejunba is going past that, to something more fundamental.
If you have "pure minds untainted by experience, tabula rasa", then you remove the issue of priors. No longer is there any matrix of prejudices that have to agree. You start with observations about the world and logic, and nothing else, and everyone ends up in agreement.
(To an extent you might bootstrap with priors, but over time you'd use observations and logic to replace them, especially when they disagree with someone else you're debating.)
This seems unworkable because it ignores the role instincts have in living things, which are prior beliefs about the world put immediately into action.
For example, an infant has an instinct to breathe right after birth. Is that a belief or an action? It’s both—a strategy to survive in the world encoded in genes attached to an implicit belief. An infant that tries to breathe two minutes later would suffer hypoxia and brain damage, while one that tries sooner will choke.
There is no pure tabula rasa untainted by priors, as soon as you acknowledge minds are embedded in bodies shaped by millions of years of evolution and steeped in survival tactics, and what are tactics except a set of useful priors about good actions?
Also, I believe this requires full mutual trust between the two parties, and perfect communication.
Any breaks in trust, or breakdowns in communication can disrupt the process of mind melding. And it seems to me that trust and failure of communication are features of boundaries between individuals.
