Complete rework of terminal userpass input system.

The system for handling seat_get_userpass_input has always been
structured differently between GUI PuTTY and CLI tools like Plink.

In the CLI tools, password input is read directly from the OS
terminal/console device by console_get_userpass_input; this means that
you need to ensure the same terminal input data _hasn't_ already been
consumed by the main event loop and sent on to the backend. This is
achieved by the backend_sendok() method, which tells the event loop
when the backend has finished issuing password prompts, and hence,
when it's safe to start passing standard input to backend_send().

But in the GUI tools, input generated by the terminal window has
always been sent straight to backend_send(), regardless of whether
backend_sendok() says it wants it. So the terminal-based
implementation of username and password prompts has to work by
consuming input data that had _already_ been passed to the backend -
hence, any backend that needs to do that must keep its input on a
bufchain, and pass that bufchain to seat_get_userpass_input.

It's awkward that these two totally different systems coexist in the
first place. And now that SSH proxying needs to present interactive
prompts of its own, it's clear which one should win: the CLI style is
the Right Thing. So this change reworks the GUI side of the mechanism
to be more similar: terminal data now goes into a queue in the Ldisc,
and is not sent on to the backend until the backend says it's ready
for it via backend_sendok(). So terminal-based userpass prompts can
now consume data directly from that queue during the connection setup
stage.

As a result, the 'bufchain *' parameter has vanished from all the
userpass_input functions (both the official implementations of the
Seat trait method, and term_get_userpass_input() to which some of
those implementations delegate). The only function that actually used
that bufchain, namely term_get_userpass_input(), now instead reads
from the ldisc's input queue via a couple of new Ldisc functions.

(Not _trivial_ functions, since input buffered by Ldisc can be a
mixture of raw bytes and session specials like SS_EOL! The input queue
inside Ldisc is a bufchain containing a fiddly binary encoding that
can represent an arbitrary interleaving of those things.)

This greatly simplifies the calls to seat_get_userpass_input in
backends, which now don't have to mess about with passing their own
user_input bufchain around, or toggling their want_user_input flag
back and forth to request data to put on to that bufchain.

But the flip side is that now there has to be some _other_ method for
notifying the terminal when there's more input to be consumed during
an interactive prompt, and for notifying the backend when prompt input
has finished so that it can proceed to the next stage of the protocol.
This is done by a pair of extra callbacks: when more data is put on to
Ldisc's input queue, it triggers a call to term_get_userpass_input,
and when term_get_userpass_input finishes, it calls a callback
function provided in the prompts_t.

Therefore, any use of a prompts_t which *might* be asynchronous must
fill in the latter callback when setting up the prompts_t. In SSH, the
callback is centralised into a common PPL helper function, which
reinvokes the same PPL's process_queue coroutine; in rlogin we have to
set it up ourselves.

I'm sorry for this large and sprawling patch: I tried fairly hard to
break it up into individually comprehensible sub-patches, but I just
couldn't tease out any part of it that would stand sensibly alone.

1 files changed, 231 insertions, 19 deletions

diff --git a/ldisc.c b/ldisc.c
index 8c985f06..b549c14a 100644
--- a/ldisc.c
+++ b/ldisc.c
@@ -18,6 +18,46 @@ struct Ldisc_tag {
     Seat *seat;
 
     /*
+     * When the backend is not reporting true from sendok(), terminal
+     * input that comes here is stored in this bufchain instead. When
+     * the backend later decides it wants session input, we empty the
+     * queue in ldisc_check_sendok_callback(), passing its contents on
+     * to the backend. Before then, we also provide data from this
+     * queue to term_get_userpass_input() via ldisc_get_input_token(),
+     * to be interpreted as user responses to username and password
+     * prompts during authentication.
+     *
+     * Unfortunately, the data stored in this queue is not all of the
+     * same type: our output to the backend consists of both raw bytes
+     * sent to backend_send(), and also session specials such as
+     * SS_EOL and SS_EC. So we have to encode our queued data in a way
+     * that can represent both.
+     *
+     * The encoding is private to this source file, so we can change
+     * it if necessary and only have to worry about the encode and
+     * decode functions here. Currently, it is:
+     *
+     *  - Bytes other than 0xFF are stored literally.
+     *  - The byte 0xFF itself is stored as 0xFF 0xFF.
+     *  - A session special (code, arg) is stored as 0xFF, followed by
+     *    a big-endian 4-byte integer containing code, followed by
+     *    another big-endian 4-byte integer containing arg.
+     *
+     * (This representation relies on session special codes being at
+     * most 0xFEFFFFFF when represented in 32 bits, so that the first
+     * byte of the 'code' integer can't be confused with the 0xFF
+     * followup byte indicating a literal 0xFF, But since session
+     * special codes are defined by an enum counting up from zero, and
+     * there are only a couple of dozen of them, that shouldn't be a
+     * problem! Even so, just in case, an assertion checks that at
+     * encode time.)
+     */
+    bufchain input_queue;
+
+    IdempotentCallback input_queue_callback;
+    prompts_t *prompts;
+
+    /*
      * Values cached out of conf.
      */
     bool telnet_keyboard, telnet_newline;
@@ -87,6 +127,8 @@ static void bsb(Ldisc *ldisc, int n)
         c_write(ldisc, "\010 \010", 3);
 }
 
+static void ldisc_input_queue_callback(void *ctx);
+
 #define CTRL(x) (x^'@')
 #define KCTRL(x) ((x^'@') | 0x100)
 
@@ -103,6 +145,14 @@ Ldisc *ldisc_create(Conf *conf, Terminal *term, Backend *backend, Seat *seat)
     ldisc->term = term;
     ldisc->seat = seat;
 
+    bufchain_init(&ldisc->input_queue);
+
+    ldisc->prompts = NULL;
+    ldisc->input_queue_callback.fn = ldisc_input_queue_callback;
+    ldisc->input_queue_callback.ctx = ldisc;
+    ldisc->input_queue_callback.queued = false;
+    bufchain_set_callback(&ldisc->input_queue, &ldisc->input_queue_callback);
+
     ldisc_configure(ldisc, conf);
 
     /* Link ourselves into the backend and the terminal */
@@ -125,12 +175,14 @@ void ldisc_configure(Ldisc *ldisc, Conf *conf)
 
 void ldisc_free(Ldisc *ldisc)
 {
+    bufchain_clear(&ldisc->input_queue);
     if (ldisc->term)
         ldisc->term->ldisc = NULL;
     if (ldisc->backend)
         backend_provide_ldisc(ldisc->backend, NULL);
     if (ldisc->buf)
         sfree(ldisc->buf);
+    delete_callbacks_for_context(ldisc);
     sfree(ldisc);
 }
 
@@ -139,8 +191,169 @@ void ldisc_echoedit_update(Ldisc *ldisc)
     seat_echoedit_update(ldisc->seat, ECHOING, EDITING);
 }
 
+void ldisc_enable_prompt_callback(Ldisc *ldisc, prompts_t *prompts)
+{
+    /*
+     * Called by the terminal to indicate that there's a prompts_t
+     * currently in flight, or to indicate that one has just finished
+     * (by passing NULL). When ldisc->prompts is not null, we notify
+     * the terminal whenever new data arrives in our input queue, so
+     * that it can continue the interactive prompting process.
+     */
+    ldisc->prompts = prompts;
+}
+
+static void ldisc_input_queue_callback(void *ctx)
+{
+    /*
+     * Toplevel callback that is triggered whenever the input queue
+     * lengthens. If we're currently processing an interactive prompt,
+     * we call back the Terminal to tell it to do some more stuff with
+     * that prompt based on the new input.
+     */
+    Ldisc *ldisc = (Ldisc *)ctx;
+    if (ldisc->term && ldisc->prompts) {
+        /*
+         * The integer return value from this call is discarded,
+         * because we have no channel to pass it on to the backend
+         * that originally wanted it. But that's OK, because if the
+         * return value is >= 0 (that is, the prompts are either
+         * completely filled in, or aborted by the user), then the
+         * terminal will notify the callback in the prompts_t, and
+         * when that calls term_get_userpass_input again, it will
+         * return the same answer again.
+         */
+        term_get_userpass_input(ldisc->term, ldisc->prompts);
+    }
+}
+
+static void ldisc_to_backend_raw(
+    Ldisc *ldisc, const void *vbuf, size_t len)
+{
+    if (backend_sendok(ldisc->backend)) {
+        backend_send(ldisc->backend, vbuf, len);
+    } else {
+        const char *buf = (const char *)vbuf;
+        while (len > 0) {
+            /*
+             * Encode raw data in input_queue, by storing large chunks
+             * as long as they don't include 0xFF, and pausing every
+             * time they do to escape it.
+             */
+            const char *ff = memchr(buf, '\xFF', len);
+            size_t this_len = ff ? ff - buf : len;
+            if (this_len > 0) {
+                bufchain_add(&ldisc->input_queue, buf, len);
+            } else {
+                bufchain_add(&ldisc->input_queue, "\xFF\xFF", 2);
+                this_len = 1;
+            }
+            buf += this_len;
+            len -= this_len;
+        }
+    }
+}
+
+static void ldisc_to_backend_special(
+    Ldisc *ldisc, SessionSpecialCode code, int arg)
+{
+    if (backend_sendok(ldisc->backend)) {
+        backend_special(ldisc->backend, code, arg);
+    } else {
+        /*
+         * Encode a session special in input_queue.
+         */
+        unsigned char data[9];
+        data[0] = 0xFF;
+        PUT_32BIT_MSB_FIRST(data+1, code);
+        PUT_32BIT_MSB_FIRST(data+5, arg);
+        assert(data[1] != 0xFF &&
+               "SessionSpecialCode encoding collides with FF FF escape");
+        bufchain_add(&ldisc->input_queue, data, 9);
+    }
+}
+
+bool ldisc_has_input_buffered(Ldisc *ldisc)
+{
+    return bufchain_size(&ldisc->input_queue) > 0;
+}
+
+LdiscInputToken ldisc_get_input_token(Ldisc *ldisc)
+{
+    assert(bufchain_size(&ldisc->input_queue) > 0 &&
+           "You're not supposed to call this unless there is buffered input!");
+
+    LdiscInputToken tok;
+
+    char c;
+    bufchain_fetch_consume(&ldisc->input_queue, &c, 1);
+    if (c != '\xFF') {
+        /* A literal non-FF byte */
+        tok.is_special = false;
+        tok.chr = c;
+        return tok;
+    } else {
+        char data[8];
+
+        /* See if the byte after the FF is also FF, indicating a literal FF */
+        bufchain_fetch_consume(&ldisc->input_queue, data, 1);
+        if (data[0] == '\xFF') {
+            tok.is_special = false;
+            tok.chr = '\xFF';
+            return tok;
+        }
+
+        /* If not, get the rest of an 8-byte chunk and decode a special */
+        bufchain_fetch_consume(&ldisc->input_queue, data+1, 7);
+        tok.is_special = true;
+        tok.code = GET_32BIT_MSB_FIRST(data);
+        tok.arg = toint(GET_32BIT_MSB_FIRST(data+4));
+        return tok;
+    }
+}
+
+static void ldisc_check_sendok_callback(void *ctx)
+{
+    Ldisc *ldisc = (Ldisc *)ctx;
+
+    if (!(ldisc->backend && backend_sendok(ldisc->backend)))
+        return;
+
+    /*
+     * Flush the ldisc input queue into the backend, which is now
+     * willing to receive the data.
+     */
+    while (bufchain_size(&ldisc->input_queue) > 0) {
+        /*
+         * Process either a chunk of non-special data, or an FF
+         * escape, depending on whether the first thing we see is an
+         * FF byte.
+         */
+        ptrlen data = bufchain_prefix(&ldisc->input_queue);
+        const char *ff = memchr(data.ptr, '\xFF', data.len);
+        if (ff != data.ptr) {
+            /* Send a maximal block of data not containing any
+             * difficult bytes. */
+            if (ff)
+                data.len = ff - (const char *)data.ptr;
+            backend_send(ldisc->backend, data.ptr, data.len);
+            bufchain_consume(&ldisc->input_queue, data.len);
+        } else {
+            /* Decode either a special or an escaped FF byte. The
+             * easiest way to do this is to reuse the decoding code
+             * already in ldisc_get_input_token. */
+            LdiscInputToken tok = ldisc_get_input_token(ldisc);
+            if (tok.is_special)
+                backend_special(ldisc->backend, tok.code, tok.arg);
+            else
+                backend_send(ldisc->backend, &tok.chr, 1);
+        }
+    }
+}
+
 void ldisc_check_sendok(Ldisc *ldisc)
 {
+    queue_toplevel_callback(ldisc_check_sendok_callback, ldisc);
 }
 
 void ldisc_send(Ldisc *ldisc, const void *vbuf, int len, bool interactive)
@@ -225,7 +438,7 @@ void ldisc_send(Ldisc *ldisc, const void *vbuf, int len, bool interactive)
                         bsb(ldisc, plen(ldisc, ldisc->buf[ldisc->buflen - 1]));
                     ldisc->buflen--;
                 }
-                backend_special(ldisc->backend, SS_EL, 0);
+                ldisc_to_backend_special(ldisc, SS_EL, 0);
                 /*
                  * We don't send IP, SUSP or ABORT if the user has
                  * configured telnet specials off! This breaks
@@ -234,11 +447,11 @@ void ldisc_send(Ldisc *ldisc, const void *vbuf, int len, bool interactive)
                 if (!ldisc->telnet_keyboard)
                     goto default_case;
                 if (c == CTRL('C'))
-                    backend_special(ldisc->backend, SS_IP, 0);
+                    ldisc_to_backend_special(ldisc, SS_IP, 0);
                 if (c == CTRL('Z'))
-                    backend_special(ldisc->backend, SS_SUSP, 0);
+                    ldisc_to_backend_special(ldisc, SS_SUSP, 0);
                 if (c == CTRL('\\'))
-                    backend_special(ldisc->backend, SS_ABORT, 0);
+                    ldisc_to_backend_special(ldisc, SS_ABORT, 0);
                 break;
               case CTRL('R'):          /* redraw line */
                 if (ECHOING) {
@@ -253,9 +466,9 @@ void ldisc_send(Ldisc *ldisc, const void *vbuf, int len, bool interactive)
                 break;
               case CTRL('D'):          /* logout or send */
                 if (ldisc->buflen == 0) {
-                    backend_special(ldisc->backend, SS_EOF, 0);
+                    ldisc_to_backend_special(ldisc, SS_EOF, 0);
                 } else {
-                    backend_send(ldisc->backend, ldisc->buf, ldisc->buflen);
+                    ldisc_to_backend_raw(ldisc, ldisc->buf, ldisc->buflen);
                     ldisc->buflen = 0;
                 }
                 break;
@@ -291,14 +504,13 @@ void ldisc_send(Ldisc *ldisc, const void *vbuf, int len, bool interactive)
                     /* FALLTHROUGH */
               case KCTRL('M'):         /* send with newline */
                     if (ldisc->buflen > 0)
-                        backend_send(ldisc->backend,
-                                     ldisc->buf, ldisc->buflen);
+                        ldisc_to_backend_raw(ldisc, ldisc->buf, ldisc->buflen);
                     if (ldisc->protocol == PROT_RAW)
-                        backend_send(ldisc->backend, "\r\n", 2);
+                        ldisc_to_backend_raw(ldisc, "\r\n", 2);
                     else if (ldisc->protocol == PROT_TELNET && ldisc->telnet_newline)
-                        backend_special(ldisc->backend, SS_EOL, 0);
+                        ldisc_to_backend_special(ldisc, SS_EOL, 0);
                     else
-                        backend_send(ldisc->backend, "\r", 1);
+                        ldisc_to_backend_raw(ldisc, "\r", 1);
                     if (ECHOING)
                         c_write(ldisc, "\r\n", 2);
                     ldisc->buflen = 0;
@@ -317,7 +529,7 @@ void ldisc_send(Ldisc *ldisc, const void *vbuf, int len, bool interactive)
         }
     } else {
         if (ldisc->buflen != 0) {
-            backend_send(ldisc->backend, ldisc->buf, ldisc->buflen);
+            ldisc_to_backend_raw(ldisc, ldisc->buf, ldisc->buflen);
             while (ldisc->buflen > 0) {
                 bsb(ldisc, plen(ldisc, ldisc->buf[ldisc->buflen - 1]));
                 ldisc->buflen--;
@@ -330,33 +542,33 @@ void ldisc_send(Ldisc *ldisc, const void *vbuf, int len, bool interactive)
                 switch (buf[0]) {
                   case CTRL('M'):
                     if (ldisc->protocol == PROT_TELNET && ldisc->telnet_newline)
-                        backend_special(ldisc->backend, SS_EOL, 0);
+                        ldisc_to_backend_special(ldisc, SS_EOL, 0);
                     else
-                        backend_send(ldisc->backend, "\r", 1);
+                        ldisc_to_backend_raw(ldisc, "\r", 1);
                     break;
                   case CTRL('?'):
                   case CTRL('H'):
                     if (ldisc->telnet_keyboard) {
-                        backend_special(ldisc->backend, SS_EC, 0);
+                        ldisc_to_backend_special(ldisc, SS_EC, 0);
                         break;
                     }
                   case CTRL('C'):
                     if (ldisc->telnet_keyboard) {
-                        backend_special(ldisc->backend, SS_IP, 0);
+                        ldisc_to_backend_special(ldisc, SS_IP, 0);
                         break;
                     }
                   case CTRL('Z'):
                     if (ldisc->telnet_keyboard) {
-                        backend_special(ldisc->backend, SS_SUSP, 0);
+                        ldisc_to_backend_special(ldisc, SS_SUSP, 0);
                         break;
                     }
 
                   default:
-                    backend_send(ldisc->backend, buf, len);
+                    ldisc_to_backend_raw(ldisc, buf, len);
                     break;
                 }
             } else
-                backend_send(ldisc->backend, buf, len);
+                ldisc_to_backend_raw(ldisc, buf, len);
         }
     }
 }