>Correct me if I'm wrong, but I think this is how smalltalk/squeak works?
In Smalltalk those would be classes' methods rather than freestanding functions, but yes: all objects within the image directly send messages to each other, i.e. invoke each other's methods. Lisp machine operating systems are somewhat closer, since initially they had no object system and used freestanding functions calling each other, though later they became generic functions specialized to their arguments' classes.
Ironically, there are so many pointers at this point that OSes often now do use a single address space for every program (with escape hatches in case you need aliasing).
Virtual memory and vtables are now more about access control than about managing the scarcity of pointers.
Can you provide any citations for this (extraordinary, if true) claim? My knowledge of operating systems (which is based on experience in designing toy ones, but also studying other operating systems) suggests this is not the case on at least windows, macOS and linux. Feel free to correct me if I am wrong however!
In Smalltalk those would be classes' methods rather than freestanding functions, but yes: all objects within the image directly send messages to each other, i.e. invoke each other's methods. Lisp machine operating systems are somewhat closer, since initially they had no object system and used freestanding functions calling each other, though later they became generic functions specialized to their arguments' classes.