Reserve stack slot for luaL_error() during encode

Unlike "decode", encoding leaves both the key/value on the stack before
descending. This leaves no spare room for luaL_error() in case the depth
check or lua_checkstack() fails. Allocate an extra stack slot to ensure there is always room for
luaL_error() in json_check_encode_depth().

Note: this would not have caused a crash or fault due to the EXTRA_STACK
slot reserve, but it was a misuse of the Lua C API.

1 files changed, 11 insertions, 4 deletions

diff --git a/lua_cjson.c b/lua_cjson.c
index fea3ed5..f5af350 100644
--- a/lua_cjson.c
+++ b/lua_cjson.c
@@ -532,10 +532,17 @@ static int lua_array_length(lua_State *l, json_config_t *cfg, strbuf_t *json)
 static void json_check_encode_depth(lua_State *l, json_config_t *cfg,
                                     int current_depth, strbuf_t *json)
 {
-    /* Ensure there are enough slots free to traverse a table (key, value).
-     * luaL_error() and other Lua API functions use space from the
-     * "EXTRA_STACK" reserve. */
-    if (current_depth <= cfg->encode_max_depth && lua_checkstack(l, 2))
+    /* Ensure there are enough slots free to traverse a table (key,
+     * value) and push a string for a potential error message.
+     *
+     * Unlike "decode", the key and value are still on the stack when
+     * lua_checkstack() is called.  Hence an extra slot for luaL_error()
+     * below is required just in case the next check to lua_checkstack()
+     * fails.
+     *
+     * While this won't cause a crash due to the EXTRA_STACK reserve
+     * slots, it would still be an improper use of the API. */
+    if (current_depth <= cfg->encode_max_depth && lua_checkstack(l, 3))
         return;
 
     if (!cfg->encode_keep_buffer)