Authentication

See #246

commented:

I know this bug is closed, but I wanted to share my own meagre findings for posterity.

Mac's built-in VNC viewer (Cmd+K in Finder) supports the following auth types, listed in its own preference order: 33, 36, 30. As previously noted, Remmina has partial support for the '30' auth type.

On the '30' auth type: Mac's VNC client sends a ClientInit message with its shared flag set to 0x1C. In response, the server modifies its ServerInit message to include non-textual information at the start of the 'name' string. There follows an exchange of unencrypted packets which I suppose is related to a more secure crypto setup (not using ECB/MD5?). From that point forward, the client appears to send only encrypted data, with an unencrypted 2-byte header. The server continues to send unencrypted framebuffer updates. I suspect it sends clipboard updates as encrypted messages, but I haven't confirmed this. The encrypted messages begin '0x00 0xYY', where YY is some nonzero number; from this I suppose they can be distinguished from RFB-compliant SetPixelFormat and FramebufferUpdate packets, which always have their second byte set to 0x00.

With some experimentation (and perhaps a bit of binary analysis) I think a sufficiently dedicated Remmina contributor could work out the details of this protocol. The first trick is to write a little VNC proxy that forces a Mac client to use the '30' auth type when connecting to a Mac server.

On the '33' auth type (currently preferred by Mac's VNC viewer):

Seems to have been implemented in Apple Remote Desktop 3.9 in response to CVE-2017-488. Apple mention that it uses the Secure Remote Password algorithm and 2048 bit RSA keys Apple uses SRP in at least a couple of other places: iCloud storage and HomeKit Accessory Protocol. The implementations may be similar. Here are my notes on the auth type 33 packet formats:

C -> S:
    4 bytes: packet length
    2 bytes: version?
    4 bytes: algorithm? ('RSA1' ascii)
    4 bytes: ????

S -> C:
    4   bytes: packet length
    2   bytes: ???
    ... bytes: ASN.1 data (server public key?)
    1   bytes: ???

C -> S:
    4   bytes: packet length
    2   bytes: ???
    4   bytes: algorithm? ('RSA1' ascii)
    2   bytes: ???
    2   bytes: key length? (256)
    256 bytes: ???
    128 bytes: ??? (all zero)
    256 bytes: ???

I doubt it's possible to get much further with implementing auth type '33' without some skilful reverse-engineering of Apple binaries, which is beyond my present abilities.

I didn't look much at auth type '36' except to note that in debugging messages from Mac's 'Console' app, it appears to use SRP, like '33'.

Hello to any future programmers finding this bug through a search engine! Apple's developer documentations fucking sucks eh?