Another issue with remote desktop (using VNC) on Linux is keyboard layouts (for inputting non-latin characters). With X server, the server passes a physical key number (keycode) and modifiers state (whether keys like Shift or Caps Lock are active) to the client and the client converts it to the corresponging character (keysym) using translation tables loaded earlier from the X server.
When you are accessing another computer remotely using protocols like VNC the local computer's keyboard layout can be different and it can produce characters not present in remote computer's translation tables. The VNC protocol itself transmits keysyms, not keycodes. So there can be a situation when there is no matching physical key number (keycode) for given character. So VNC servers like x11vnc had to modify translation tables on the fly and insert non-existing key into the tables and pass that key's number to X clients.
I wonder if this problem is solved with Wayland. As I understand, Wayland uses similar scheme and the compositor sends only a physical key number (keycode) along with the modifiers to the client and the latter does the translation. By the way, would not it be easier if the compositor did the translation itself?
The hack I described can also cause issues with hot keys. If I run application locally and it has a hotkey like Ctrl + S, and my keyboard has two layouts (English and Russian) then it will probably work with both of them because the application can bind the hotkey to physical key number. But when we send key presses remotely over VNC we send only a keysym (a letter code), not a keycode. If the remote server doesn't have Russian layout it is unable to guess what physical key was pressed and cannot recognize that I pressed Ctrl + S in Russian layout.
When you are accessing another computer remotely using protocols like VNC the local computer's keyboard layout can be different and it can produce characters not present in remote computer's translation tables. The VNC protocol itself transmits keysyms, not keycodes. So there can be a situation when there is no matching physical key number (keycode) for given character. So VNC servers like x11vnc had to modify translation tables on the fly and insert non-existing key into the tables and pass that key's number to X clients.
I wonder if this problem is solved with Wayland. As I understand, Wayland uses similar scheme and the compositor sends only a physical key number (keycode) along with the modifiers to the client and the latter does the translation. By the way, would not it be easier if the compositor did the translation itself?
The hack I described can also cause issues with hot keys. If I run application locally and it has a hotkey like Ctrl + S, and my keyboard has two layouts (English and Russian) then it will probably work with both of them because the application can bind the hotkey to physical key number. But when we send key presses remotely over VNC we send only a keysym (a letter code), not a keycode. If the remote server doesn't have Russian layout it is unable to guess what physical key was pressed and cannot recognize that I pressed Ctrl + S in Russian layout.