n-api: detect deadlocks in thread-safe function

We introduce status `napi_would_deadlock` to be used as a return status
by `napi_call_threadsafe_function` if the call is made with
`napi_tsfn_blocking` on the main thread and the queue is full.

Fixes: https://github.com/nodejs/node/issues/32615
Signed-off-by: Gabriel Schulhof <gabriel.schulhof@intel.com>
PR-URL: https://github.com/nodejs/node/pull/32860
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: Zeyu Yang <himself65@outlook.com>

1 files changed, 23 insertions, 0 deletions

diff --git a/src/node_api.cc b/src/node_api.cc
index fad9cf72a97..cb8bd4b4823 100644
--- a/src/node_api.cc
+++ b/src/node_api.cc
@@ -155,6 +155,29 @@ class ThreadSafeFunction : public node::AsyncResource {
       if (mode == napi_tsfn_nonblocking) {
         return napi_queue_full;
       }
+
+      // Here we check if there is a Node.js event loop running on this thread.
+      // If there is, and our queue is full, we return `napi_would_deadlock`. We
+      // do this for two reasons:
+      //
+      // 1. If this is the thread on which our own event loop runs then we
+      //    cannot wait here because that will prevent our event loop from
+      //    running and emptying the very queue on which we are waiting.
+      //
+      // 2. If this is not the thread on which our own event loop runs then we
+      //    still cannot wait here because that allows the following sequence of
+      //    events:
+      //
+      //    1. JSThread1 calls JSThread2 and blocks while its queue is full and
+      //       because JSThread2's queue is also full.
+      //
+      //    2. JSThread2 calls JSThread1 before it's had a chance to remove an
+      //       item from its own queue and blocks because JSThread1's queue is
+      //       also full.
+      v8::Isolate* isolate = v8::Isolate::GetCurrent();
+      if (isolate != nullptr && node::GetCurrentEventLoop(isolate) != nullptr)
+        return napi_would_deadlock;
+
       cond->Wait(lock);
     }