diff options
author | Aleksey Kliger (λgeek) <aleksey@lambdageek.org> | 2022-07-01 15:57:42 +0300 |
---|---|---|
committer | GitHub <noreply@github.com> | 2022-07-01 15:57:42 +0300 |
commit | 057722795c15ce46acdbddbb789e30f866eafcc3 (patch) | |
tree | ca4e56abf17e988282f6e6f5704e9e00bde10984 /src/mono/wasm | |
parent | 56966558c12523f82a0e27d5546ee4efba466848 (diff) |
[wasm-mt] Add MessageChannel between browser thread and new pthreads (#70908)
Add a mechanism for creating a communication channel between the browser thread and new pthreads (running in webworkers - note, Emscripten may recycle workers after a pthread exits).
This is done by adding our own event listener to the webworker (in the main thread) and to globalThis (in the worker). This conflicts with emscripten's message handlers (although it's considered a bug by Emscripten upstream that their event handler doesn't ignore unrecognized messages).
One potential problem here is that for any communication to happen, the worker must service its event loop. If it's just busy running a loop in wasm, it might never handle the messages. On the other hand, posting messages back to main should work.
Once we have our message handlers in place, the rest is straightforward, the worker creates a MessageChannel and transfers one of the ports to the browser thread.
With the browser-to-pthread channel established, we can build up cross-thread channels by asking the main thread to transfer ports on our behalf. This part isn't done yet.
---
Additionally in the worker, create an `EventTarget` that fires `dotnet:pthread:created` and `dotnet:pthread:attached` events whenever Emscripten begins running a new thread on one of its workers, and whenever that worker attaches to Mono. This lets runtime subsystems be notified on the JS side whenever threads come into existence or may potentially begin to run managed code.
---
Also re-organizes our `tsconfig.json` into `tsconfig.shared.json` (common flags), `tsconfig.worker.json` (uses the `esnext` and `worker` libs, so VS Code doesn't offer DOM completions and types, for example), and `tsconfig.json` (uses the `esnext` and `dom` libs). Subsystems with their own subdirectories (like `pthreads/worker` `pthreads/browser`, etc) can use the `tsconfig` `extends` property to include the appropriate root-directory config.
---
* outline of a dedicated channel between JS main thread and pthreads
* add JS entrypoints for pthread-channel
* wire up the MessageChannel to native thread attach
* debug printfs etc
* split up into pthread-channel module into worker, browser and shared
* pthreads modules
* add ENVIRONMENT_IS_PTHREAD; mono_wasm_pthread_worker_init
* Fixup names; call MessagePort.start; remove printfs
* remove whitespace and extra printfs
* Exclude threading exports in non-threaded runtime
Use the `USE_THREADS` emscripten library ifdef to prevent the entrypoints from being saved.
Use a new `MonoWasmThreads` rollup constant to remove JS roots.
Verified that a Release build single-threaded dotnet.js doesn't include any of the new pthread support code
* Add replacement for PThread.loadWasmModuleToWorker
This will allow us to install a message handler when the worker is created, before it has any pthreads assigned to it.
We can therefore simplify how we set up our own MessageChannel to simply send a single event from the worker to the browser thread, instead of having to lazily install the event handler on the main thread by queueing async work to the browser thread.
* Simplify the dedicated channel creation
now that we can add a message handler to a worker when Emscripten creates it, skip the complicated atomic notification process
* Don't forget the GC transition out to JS
* fix browser-eventpipe default import
* move mono_threads_wasm_on_thread_attached later in register_thread
Actually attach the thread to the runtime (so that GC transitions work) before calling out to JS
* also fix default import in browser-mt-eventpipe
* Add replacement for Module.PThread.threadInit
Use it to call mono_wasm_pthread_on_pthread_created
Rename the previous callback to mono_wasm_pthread_on_pthread_attached - it gets called but it's unused.
This is enough to get the diagnostic sever worker started up and pinging.
* Cleanup mono_wasm_pthread_on_pthread_created
* Share tsconfig parts using "extends" property
* Use an EventTarget and custom events for worker thread lifecycle notifications
* pass portToMain in ThreadEvents; update README
this lets pthread lifecycle event handlers post messages and setup listeners on the message port back to the main browser thread.
Also update the README to describe the current design
* make pthread/worker/events friendlier to tree shaking and node
* another approach to tree shaking
rollup doesn't understand fallthru. In the following (which is what `mono_assert` ammounts to) it retains `C`:
```
if (condition)
throw new Error (...);
// fallthru
return new C();
```
Solution is to use an if-then-else
```
if (condition)
throw new Error (...);
else
return new C(); // C is not retained if 'condition' is false
```
* fix annoying VSCode ESLint toast
Cannot read property 'loc' of undefined
See https://github.com/microsoft/vscode-eslint/issues/1149
and a proposed workaround in https://github.com/eslint/eslint/issues/14538#issuecomment-862280037
* Add Event and EventTarget polyfill for v8
* fix whitespace and comments
Diffstat (limited to 'src/mono/wasm')
20 files changed, 539 insertions, 33 deletions
diff --git a/src/mono/wasm/runtime/.eslintrc.cjs b/src/mono/wasm/runtime/.eslintrc.cjs index 5acfca7fed0..b69d519896a 100644 --- a/src/mono/wasm/runtime/.eslintrc.cjs +++ b/src/mono/wasm/runtime/.eslintrc.cjs @@ -29,7 +29,10 @@ module.exports = { "indent": [ "error", 4, - { SwitchCase: 1 } + { + SwitchCase: 1, + "ignoredNodes": ["VariableDeclaration[declarations.length=0]"] // fixes https://github.com/microsoft/vscode-eslint/issues/1149 + } ], "linebreak-style": "off", "quotes": [ diff --git a/src/mono/wasm/runtime/cjs/dotnet.cjs.lib.js b/src/mono/wasm/runtime/cjs/dotnet.cjs.lib.js index 1f06b5daa21..35da8f4d649 100644 --- a/src/mono/wasm/runtime/cjs/dotnet.cjs.lib.js +++ b/src/mono/wasm/runtime/cjs/dotnet.cjs.lib.js @@ -4,15 +4,28 @@ "use strict"; +#if USE_PTHREADS +const usePThreads = `true`; +const isPThread = `ENVIRONMENT_IS_PTHREAD`; +#else +const usePThreads = `false`; +const isPThread = `false`; +#endif + const DotnetSupportLib = { $DOTNET: {}, // these lines will be placed early on emscripten runtime creation, passing import and export objects into __dotnet_runtime IFFE // we replace implementation of readAsync and fetch // replacement of require is there for consistency with ES6 code $DOTNET__postset: ` -let __dotnet_replacements = {readAsync, fetch: globalThis.fetch, require, updateGlobalBufferAndViews}; +let __dotnet_replacement_PThread = ${usePThreads} ? {} : undefined; +if (${usePThreads}) { + __dotnet_replacement_PThread.loadWasmModuleToWorker = PThread.loadWasmModuleToWorker; + __dotnet_replacement_PThread.threadInit = PThread.threadInit; +} +let __dotnet_replacements = {readAsync, fetch: globalThis.fetch, require, updateGlobalBufferAndViews, pthreadReplacements: __dotnet_replacement_PThread}; let __dotnet_exportedAPI = __dotnet_runtime.__initializeImportsAndExports( - { isESM:false, isGlobal:ENVIRONMENT_IS_GLOBAL, isNode:ENVIRONMENT_IS_NODE, isWorker:ENVIRONMENT_IS_WORKER, isShell:ENVIRONMENT_IS_SHELL, isWeb:ENVIRONMENT_IS_WEB, locateFile, quit_, ExitStatus, requirePromise:Promise.resolve(require)}, + { isESM:false, isGlobal:ENVIRONMENT_IS_GLOBAL, isNode:ENVIRONMENT_IS_NODE, isWorker:ENVIRONMENT_IS_WORKER, isShell:ENVIRONMENT_IS_SHELL, isWeb:ENVIRONMENT_IS_WEB, isPThread:${isPThread}, locateFile, quit_, ExitStatus, requirePromise:Promise.resolve(require)}, { mono:MONO, binding:BINDING, internal:INTERNAL, module:Module, marshaled_exports: EXPORTS, marshaled_imports: IMPORTS }, __dotnet_replacements); updateGlobalBufferAndViews = __dotnet_replacements.updateGlobalBufferAndViews; @@ -20,6 +33,10 @@ readAsync = __dotnet_replacements.readAsync; var fetch = __dotnet_replacements.fetch; require = __dotnet_replacements.requireOut; var noExitRuntime = __dotnet_replacements.noExitRuntime; +if (${usePThreads}) { + PThread.loadWasmModuleToWorker = __dotnet_replacements.pthreadReplacements.loadWasmModuleToWorker; + PThread.threadInit = __dotnet_replacements.pthreadReplacements.threadInit; +} `, }; @@ -73,6 +90,11 @@ const linked_functions = [ "dotnet_browser_can_use_subtle_crypto_impl", "dotnet_browser_simple_digest_hash", "dotnet_browser_sign", + + /// mono-threads-wasm.c + #if USE_PTHREADS + "mono_wasm_pthread_on_pthread_attached", + #endif ]; // -- this javascript file is evaluated by emcc during compilation! -- diff --git a/src/mono/wasm/runtime/es6/dotnet.es6.lib.js b/src/mono/wasm/runtime/es6/dotnet.es6.lib.js index 8c338412395..c5851007de1 100644 --- a/src/mono/wasm/runtime/es6/dotnet.es6.lib.js +++ b/src/mono/wasm/runtime/es6/dotnet.es6.lib.js @@ -4,6 +4,14 @@ "use strict"; +#if USE_PTHREADS +const usePThreads = `true`; +const isPThread = `ENVIRONMENT_IS_PTHREAD`; +#else +const usePThreads = `false`; +const isPThread = `false`; +#endif + const DotnetSupportLib = { $DOTNET: {}, // this line will be placed early on emscripten runtime creation, passing import and export objects into __dotnet_runtime IFFE @@ -14,7 +22,12 @@ const DotnetSupportLib = { // Emscripten's getBinaryPromise is not async for NodeJs, but we would like to have it async, so we replace it. // We also replace implementation of readAsync and fetch $DOTNET__postset: ` -let __dotnet_replacements = {readAsync, fetch: globalThis.fetch, require, updateGlobalBufferAndViews}; +let __dotnet_replacement_PThread = ${usePThreads} ? {} : undefined; +if (${usePThreads}) { + __dotnet_replacement_PThread.loadWasmModuleToWorker = PThread.loadWasmModuleToWorker; + __dotnet_replacement_PThread.threadInit = PThread.threadInit; +} +let __dotnet_replacements = {readAsync, fetch: globalThis.fetch, require, updateGlobalBufferAndViews, pthreadReplacements: __dotnet_replacement_PThread}; if (ENVIRONMENT_IS_NODE) { __dotnet_replacements.requirePromise = import(/* webpackIgnore: true */'module').then(mod => { const require = mod.createRequire(import.meta.url); @@ -49,7 +62,7 @@ if (ENVIRONMENT_IS_NODE) { } } let __dotnet_exportedAPI = __dotnet_runtime.__initializeImportsAndExports( - { isESM:true, isGlobal:false, isNode:ENVIRONMENT_IS_NODE, isWorker:ENVIRONMENT_IS_WORKER, isShell:ENVIRONMENT_IS_SHELL, isWeb:ENVIRONMENT_IS_WEB, locateFile, quit_, ExitStatus, requirePromise:__dotnet_replacements.requirePromise }, + { isESM:true, isGlobal:false, isNode:ENVIRONMENT_IS_NODE, isWorker:ENVIRONMENT_IS_WORKER, isShell:ENVIRONMENT_IS_SHELL, isWeb:ENVIRONMENT_IS_WEB, isPThread:${isPThread}, locateFile, quit_, ExitStatus, requirePromise:__dotnet_replacements.requirePromise }, { mono:MONO, binding:BINDING, internal:INTERNAL, module:Module, marshaled_exports: EXPORTS, marshaled_imports: IMPORTS }, __dotnet_replacements); updateGlobalBufferAndViews = __dotnet_replacements.updateGlobalBufferAndViews; @@ -57,6 +70,10 @@ readAsync = __dotnet_replacements.readAsync; var fetch = __dotnet_replacements.fetch; require = __dotnet_replacements.requireOut; var noExitRuntime = __dotnet_replacements.noExitRuntime; +if (${usePThreads}) { + PThread.loadWasmModuleToWorker = __dotnet_replacements.pthreadReplacements.loadWasmModuleToWorker; + PThread.threadInit = __dotnet_replacements.pthreadReplacements.threadInit; +} `, }; @@ -110,6 +127,11 @@ const linked_functions = [ "dotnet_browser_can_use_subtle_crypto_impl", "dotnet_browser_simple_digest_hash", "dotnet_browser_sign", + + /// mono-threads-wasm.c + #if USE_PTHREADS + "mono_wasm_pthread_on_pthread_attached", + #endif ]; // -- this javascript file is evaluated by emcc during compilation! -- diff --git a/src/mono/wasm/runtime/exports.ts b/src/mono/wasm/runtime/exports.ts index 521fb4458a4..60ea7006d3d 100644 --- a/src/mono/wasm/runtime/exports.ts +++ b/src/mono/wasm/runtime/exports.ts @@ -3,6 +3,7 @@ import ProductVersion from "consts:productVersion"; import Configuration from "consts:configuration"; +import MonoWasmThreads from "consts:monoWasmThreads"; import { mono_wasm_new_root, mono_wasm_release_roots, mono_wasm_new_external_root, @@ -74,6 +75,8 @@ import { } from "./crypto-worker"; import { mono_wasm_cancel_promise_ref } from "./cancelable-promise"; import { mono_wasm_web_socket_open_ref, mono_wasm_web_socket_send, mono_wasm_web_socket_receive, mono_wasm_web_socket_close_ref, mono_wasm_web_socket_abort } from "./web-socket"; +import { mono_wasm_pthread_on_pthread_attached, afterThreadInit } from "./pthreads/worker"; +import { afterLoadWasmModuleToWorker } from "./pthreads/browser"; const MONO = { // current "public" MONO API @@ -184,14 +187,20 @@ export type BINDINGType = typeof BINDING; let exportedAPI: DotnetPublicAPI; +// We need to replace some of the methods in the Emscripten PThreads support with our own +type PThreadReplacements = { + loadWasmModuleToWorker: Function, + threadInit: Function +} + // this is executed early during load of emscripten runtime // it exports methods to global objects MONO, BINDING and Module in backward compatible way // At runtime this will be referred to as 'createDotnetRuntime' // eslint-disable-next-line @typescript-eslint/explicit-module-boundary-types function initializeImportsAndExports( - imports: { isESM: boolean, isGlobal: boolean, isNode: boolean, isWorker: boolean, isShell: boolean, isWeb: boolean, locateFile: Function, quit_: Function, ExitStatus: ExitStatusError, requirePromise: Promise<Function> }, + imports: { isESM: boolean, isGlobal: boolean, isNode: boolean, isWorker: boolean, isShell: boolean, isWeb: boolean, isPThread: boolean, locateFile: Function, quit_: Function, ExitStatus: ExitStatusError, requirePromise: Promise<Function> }, exports: { mono: any, binding: any, internal: any, module: any, marshaled_exports: any, marshaled_imports: any }, - replacements: { fetch: any, readAsync: any, require: any, requireOut: any, noExitRuntime: boolean, updateGlobalBufferAndViews: Function }, + replacements: { fetch: any, readAsync: any, require: any, requireOut: any, noExitRuntime: boolean, updateGlobalBufferAndViews: Function, pthreadReplacements: PThreadReplacements | undefined | null }, ): DotnetPublicAPI { const module = exports.module as DotnetModule; const globalThisAny = globalThis as any; @@ -258,6 +267,19 @@ function initializeImportsAndExports( replacements.noExitRuntime = ENVIRONMENT_IS_WEB; + if (replacements.pthreadReplacements) { + const originalLoadWasmModuleToWorker = replacements.pthreadReplacements.loadWasmModuleToWorker; + replacements.pthreadReplacements.loadWasmModuleToWorker = (worker: Worker, onFinishedLoading: Function): void => { + originalLoadWasmModuleToWorker(worker, onFinishedLoading); + afterLoadWasmModuleToWorker(worker); + }; + const originalThreadInit = replacements.pthreadReplacements.threadInit; + replacements.pthreadReplacements.threadInit = (): void => { + originalThreadInit(); + afterThreadInit(); + }; + } + if (typeof module.disableDotnet6Compatibility === "undefined") { module.disableDotnet6Compatibility = imports.isESM; } @@ -330,6 +352,13 @@ export const __initializeImportsAndExports: any = initializeImportsAndExports; / // the methods would be visible to EMCC linker // --- keep in sync with dotnet.cjs.lib.js --- +const mono_wasm_threads_exports = !MonoWasmThreads ? undefined : { + // mono-threads-wasm.c + mono_wasm_pthread_on_pthread_attached, +}; + +// the methods would be visible to EMCC linker +// --- keep in sync with dotnet.cjs.lib.js --- export const __linker_exports: any = { // mini-wasm.c mono_set_timeout, @@ -376,7 +405,10 @@ export const __linker_exports: any = { // pal_crypto_webworker.c dotnet_browser_can_use_subtle_crypto_impl, dotnet_browser_simple_digest_hash, - dotnet_browser_sign + dotnet_browser_sign, + + // threading exports, if threading is enabled + ...mono_wasm_threads_exports, }; const INTERNAL: any = { @@ -450,4 +482,4 @@ class RuntimeList { export function get_dotnet_instance(): DotnetPublicAPI { return exportedAPI; -}
\ No newline at end of file +} diff --git a/src/mono/wasm/runtime/imports.ts b/src/mono/wasm/runtime/imports.ts index 435a8f3c469..37daf4d7de3 100644 --- a/src/mono/wasm/runtime/imports.ts +++ b/src/mono/wasm/runtime/imports.ts @@ -21,6 +21,7 @@ export let ENVIRONMENT_IS_NODE: boolean; export let ENVIRONMENT_IS_SHELL: boolean; export let ENVIRONMENT_IS_WEB: boolean; export let ENVIRONMENT_IS_WORKER: boolean; +export let ENVIRONMENT_IS_PTHREAD: boolean; export let locateFile: Function; export let quit: Function; export let ExitStatus: ExitStatusError; @@ -33,7 +34,7 @@ export interface ExitStatusError { // eslint-disable-next-line @typescript-eslint/explicit-module-boundary-types export function setImportsAndExports( - imports: { isESM: boolean, isNode: boolean, isShell: boolean, isWeb: boolean, isWorker: boolean, locateFile: Function, ExitStatus: ExitStatusError, quit_: Function, requirePromise: Promise<Function> }, + imports: { isESM: boolean, isNode: boolean, isShell: boolean, isWeb: boolean, isWorker: boolean, isPThread: boolean, locateFile: Function, ExitStatus: ExitStatusError, quit_: Function, requirePromise: Promise<Function> }, exports: { mono: any, binding: any, internal: any, module: any, marshaled_exports: any, marshaled_imports: any }, ): void { MONO = exports.mono; @@ -49,6 +50,7 @@ export function setImportsAndExports( ENVIRONMENT_IS_SHELL = imports.isShell; ENVIRONMENT_IS_WEB = imports.isWeb; ENVIRONMENT_IS_WORKER = imports.isWorker; + ENVIRONMENT_IS_PTHREAD = imports.isPThread; locateFile = imports.locateFile; quit = imports.quit_; ExitStatus = imports.ExitStatus; diff --git a/src/mono/wasm/runtime/polyfills.ts b/src/mono/wasm/runtime/polyfills.ts index 257af61291c..474e40a6c73 100644 --- a/src/mono/wasm/runtime/polyfills.ts +++ b/src/mono/wasm/runtime/polyfills.ts @@ -1,3 +1,4 @@ +import MonoWasmThreads from "consts:monoWasmThreads"; import { ENVIRONMENT_IS_ESM, ENVIRONMENT_IS_NODE, Module, requirePromise } from "./imports"; let node_fs: any | undefined = undefined; @@ -29,6 +30,68 @@ export async function init_polyfills(): Promise<void> { } } as any; } + // v8 shell doesn't have Event and EventTarget + if (MonoWasmThreads && typeof globalThis.Event === "undefined") { + globalThis.Event = class Event { + readonly type: string; + constructor(type: string) { + this.type = type; + } + } as any; + } + if (MonoWasmThreads && typeof globalThis.EventTarget === "undefined") { + globalThis.EventTarget = class EventTarget { + private listeners = new Map<string, Array<EventListenerOrEventListenerObject>>(); + addEventListener(type: string, listener: EventListenerOrEventListenerObject | null, options?: boolean | AddEventListenerOptions) { + if (listener === undefined || listener == null) + return; + if (options !== undefined) + throw new Error("FIXME: addEventListener polyfill doesn't implement options"); + if (!this.listeners.has(type)) { + this.listeners.set(type, []); + } + const listeners = this.listeners.get(type); + if (listeners === undefined) { + throw new Error("can't happen"); + } + listeners.push(listener); + } + removeEventListener(type: string, listener: EventListenerOrEventListenerObject | null, options?: boolean | EventListenerOptions) { + if (listener === undefined || listener == null) + return; + if (options !== undefined) { + throw new Error("FIXME: removeEventListener polyfill doesn't implement options"); + } + if (!this.listeners.has(type)) { + return; + } + const listeners = this.listeners.get(type); + if (listeners === undefined) + return; + const index = listeners.indexOf(listener); + if (index > -1) { + listeners.splice(index, 1); + } + } + dispatchEvent(event: Event) { + if (!this.listeners.has(event.type)) { + return true; + } + const listeners = this.listeners.get(event.type); + if (listeners === undefined) { + return true; + } + for (const listener of listeners) { + if (typeof listener === "function") { + listener.call(this, event); + } else { + listener.handleEvent(event); + } + } + return true; + } + }; + } } export async function fetch_like(url: string): Promise<Response> { @@ -83,4 +146,4 @@ export function readAsync_like(url: string, onload: Function, onerror: Function) }).catch((err) => { onerror(err); }); -}
\ No newline at end of file +} diff --git a/src/mono/wasm/runtime/pthreads/README.md b/src/mono/wasm/runtime/pthreads/README.md new file mode 100644 index 00000000000..34f3508988c --- /dev/null +++ b/src/mono/wasm/runtime/pthreads/README.md @@ -0,0 +1,52 @@ +# Mono PThreads interface + +## Summary + +This is an internal API for the Mono runtime that provides some JS-side conveniences for working with pthreads. + +Currently this provides an API to register lifecycle event handlers that will run inside workers when Emscripten runs pthreads on the worker. + +Additionally, the API sets up a dedicated dotnet-specific channel for sending messages between the browser thread and individual pthreads. Unlike the WebWorker `worker.postMessage` `DedicatedWorkerGlobalScope.postMessage` these ports: + +1. Are not used by Emscripten or any other JS library. They are specific to dotnet. +2. Are tied to the lifetime of the *pthread* not of the worker. If Emscripten reuses a worker to start a new pthread, the runtime will get a new message channel between that new pthread and the browser thread. + +In the future, this may also provide APIs to establish a dedicated channel between any two arbitrary pthreads. +WebWorkers have a hierarchical relationship where the browser thread can talk to workers, but the workers cannot talk to each other. +On the other hand, pthreads in native code have a peer relationship: any two threads can talk to each other. The new API will help bridge the gap and provide a mechanism for the threads to communicate with each other in JS. + +## Main thread API + +In the main thread, `pthreads/browser` provides a `getThread` function that returns a `{ pthread_ptr: pthread_ptr, worker: Worker, port: MessagePort }` object that can be used to communicate with the worker thread. + +## Worker thread API + +In the worker threads, `pthread/worker` provides `currentWorkerThreadEvents` which is an [`EventTarget`](https://developer.mozilla.org/en-US/docs/Web/API/EventTarget) that fires `'dotnet:pthread:created'` and `'dotnet:pthread:attached'` events when a pthread is started on the worker, and when that pthread attaches to the Mono runtime. A good place to add event listeners is in `mono_wasm_pthread_worker_init` in `startup.ts`. +The events have a `portToMain` property which is a dotnet-specific `MessagePort` for posting messages to the main thread and for listening for messages from the main thread. + +## Implementation + + This is meant to provide a dedicated communication channel between a pthread and the main thread. + The Emscripten threading APIs don't provide a way to send [Transferable objects](https://developer.mozilla.org/en-US/docs/Glossary/Transferable_objects) + from one pthread to another. It is also not great for sending around JS objects in general. + + Instead, we hook a single custom message that gets called when a messsage is received from the pthread when it's created. + + This is how we set it up: + + 1. We replace emscripten's `PThread.loadWasmModuleToWorker` and `PThread.theadInit`(`threadInitTLS` in later emscripten versions) method with our own that call `afterLoadWasmModuleToWorker` and `afterThreadInit`, respectively. + 2. On the main browser thread `PThread.loadWasmModuleToWorker` is called to create workers for Emscripten's worker pool. It calls our `afterLoadWasmModuleToWorker`. + 3. `afterLoadWasmModuleToWorker` installs a `worker.AddEventListener("message", handler)` handler that watches for a custom mono "channel_created" message which receives a pthread id and a MessagePort whenever a thread is created on that worker. + 4. Something in the native code calls `pthread_create` + 5. A pthread is created and emscripten sends a command to a worker to create a pthread. + 6. The worker runs Emscripten's `PThread.threadInit` JS function which calls our `afterThreadInit` in the new thread running on the new worker + 7. the worker wakes posts the "channel_created" message to the main thread on the worker message event handler + 8. our custom message handler runs on the main thread and receives the MessagePort + 9. now the main thread and the worker have a dedicated communication channel + +Additionally, inside the worker we fire `'dotnet:pthread:created'` and `dotnet:pthread:attached'` events +when the worker begins running a new pthread, and when that pthread attaches to the Mono runtime, respectively. + +This could get better if the following things changed in Emscripten: + + 1. If we could have a way to avoid collisions with Emscripten's own message handlers entirely, we wouldn't need a MessageChannel at all, we could just piggyback on the normal worker communication. diff --git a/src/mono/wasm/runtime/pthreads/browser/index.ts b/src/mono/wasm/runtime/pthreads/browser/index.ts new file mode 100644 index 00000000000..cf2161cf236 --- /dev/null +++ b/src/mono/wasm/runtime/pthreads/browser/index.ts @@ -0,0 +1,87 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +import { Module } from "../../imports"; +import { pthread_ptr, MonoWorkerMessageChannelCreated, isMonoWorkerMessageChannelCreated, monoSymbol } from "../shared"; + +const threads: Map<pthread_ptr, Thread> = new Map(); + +export interface Thread { + readonly pthread_ptr: pthread_ptr; + readonly worker: Worker; + readonly port: MessagePort; +} + +function addThread(pthread_ptr: pthread_ptr, worker: Worker, port: MessagePort): Thread { + const thread = { pthread_ptr, worker, port }; + threads.set(pthread_ptr, thread); + return thread; +} + +function removeThread(pthread_ptr: pthread_ptr): void { + threads.delete(pthread_ptr); +} + +/// Given a thread id, return the thread object with the worker where the thread is running, and a message port. +export function getThread(pthread_ptr: pthread_ptr): Thread | undefined { + const thread = threads.get(pthread_ptr); + if (thread === undefined) { + return undefined; + } + // validate that the worker is still running pthread_ptr + const worker = thread.worker; + if (Internals.getThreadId(worker) !== pthread_ptr) { + removeThread(pthread_ptr); + thread.port.close(); + return undefined; + } + return thread; +} + +/// Returns all the threads we know about +export const getThreadIds = (): IterableIterator<pthread_ptr> => threads.keys(); + +function monoDedicatedChannelMessageFromWorkerToMain(event: MessageEvent<unknown>, thread: Thread): void { + // TODO: add callbacks that will be called from here + console.debug("got message from worker on the dedicated channel", event.data, thread); +} + +// handler that runs in the main thread when a message is received from a pthread worker +function monoWorkerMessageHandler(worker: Worker, ev: MessageEvent<MonoWorkerMessageChannelCreated<MessagePort>>): void { + /// N.B. important to ignore messages we don't recognize - Emscripten uses the message event to send internal messages + const data = ev.data; + if (isMonoWorkerMessageChannelCreated(data)) { + console.debug("received the channel created message", data, worker); + const port = data[monoSymbol].port; + const pthread_id = data[monoSymbol].thread_id; + const thread = addThread(pthread_id, worker, port); + port.addEventListener("message", (ev) => monoDedicatedChannelMessageFromWorkerToMain(ev, thread)); + port.start(); + } +} + +/// Called by Emscripten internals on the browser thread when a new pthread worker is created and added to the pthread worker pool. +/// At this point the worker doesn't have any pthread assigned to it, yet. +export function afterLoadWasmModuleToWorker(worker: Worker): void { + worker.addEventListener("message", (ev) => monoWorkerMessageHandler(worker, ev)); + console.debug("afterLoadWasmModuleToWorker added message event handler", worker); +} + +/// These utility functions dig into Emscripten internals +const Internals = { + getWorker: (pthread_ptr: pthread_ptr): Worker => { + // see https://github.com/emscripten-core/emscripten/pull/16239 + return (<any>Module).PThread.pthreads[pthread_ptr].worker; + }, + getThreadId: (worker: Worker): pthread_ptr | undefined => { + /// See library_pthread.js in Emscripten. + /// They hang a "pthread" object from the worker if the worker is running a thread, and remove it when the thread stops by doing `pthread_exit` or when it's joined using `pthread_join`. + const emscriptenThreadInfo = (<any>worker)["pthread"]; + if (emscriptenThreadInfo === undefined) { + return undefined; + } + return emscriptenThreadInfo.threadInfoStruct; + } +}; + + diff --git a/src/mono/wasm/runtime/pthreads/shared/index.ts b/src/mono/wasm/runtime/pthreads/shared/index.ts new file mode 100644 index 00000000000..d14013e04bf --- /dev/null +++ b/src/mono/wasm/runtime/pthreads/shared/index.ts @@ -0,0 +1,49 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +/// pthread_t in C +export type pthread_ptr = number; + +/// a symbol that we use as a key on messages on the global worker-to-main channel to identify our own messages +/// we can't use an actual JS Symbol because those don't transfer between workers. +export const monoSymbol = "__mono_message_please_dont_collide__"; //Symbol("mono"); + +/// Messages sent from the main thread using Worker.postMessage or from the worker using DedicatedWorkerGlobalScope.postMessage +/// should use this interface. The message event is also used by emscripten internals (and possibly by 3rd party libraries targeting Emscripten). +/// We should just use this to establish a dedicated MessagePort for Mono's uses. +export interface MonoWorkerMessage { + [monoSymbol]: object; +} + +/// The message sent early during pthread creation to set up a dedicated MessagePort for Mono between the main thread and the pthread. +export interface MonoWorkerMessageChannelCreated<TPort> extends MonoWorkerMessage { + [monoSymbol]: { + mono_cmd: "channel_created"; + thread_id: pthread_ptr; + port: TPort; + }; +} + +export function makeChannelCreatedMonoMessage<TPort>(thread_id: pthread_ptr, port: TPort): MonoWorkerMessageChannelCreated<TPort> { + return { + [monoSymbol]: { + mono_cmd: "channel_created", + thread_id, + port + } + }; +} + +export function isMonoWorkerMessage(message: unknown): message is MonoWorkerMessage { + return message !== undefined && typeof message === "object" && message !== null && monoSymbol in message; +} + +export function isMonoWorkerMessageChannelCreated<TPort>(message: MonoWorkerMessageChannelCreated<TPort>): message is MonoWorkerMessageChannelCreated<TPort> { + if (isMonoWorkerMessage(message)) { + const monoMessage = message[monoSymbol]; + if (monoMessage.mono_cmd === "channel_created") { + return true; + } + } + return false; +} diff --git a/src/mono/wasm/runtime/pthreads/shared/tsconfig.json b/src/mono/wasm/runtime/pthreads/shared/tsconfig.json new file mode 100644 index 00000000000..7b8ecd91fcc --- /dev/null +++ b/src/mono/wasm/runtime/pthreads/shared/tsconfig.json @@ -0,0 +1,3 @@ +{ + "extends": "../../tsconfig.shared.json" +} diff --git a/src/mono/wasm/runtime/pthreads/worker/events.ts b/src/mono/wasm/runtime/pthreads/worker/events.ts new file mode 100644 index 00000000000..933f72c065b --- /dev/null +++ b/src/mono/wasm/runtime/pthreads/worker/events.ts @@ -0,0 +1,47 @@ +import MonoWasmThreads from "consts:monoWasmThreads"; +import type { pthread_ptr } from "../shared"; + +export const dotnetPthreadCreated = "dotnet:pthread:created" as const; +export const dotnetPthreadAttached = "dotnet:pthread:attached" as const; + +/// Events emitted on the current worker when Emscripten uses it to run a pthread. +export interface WorkerThreadEventMap { + /// Emitted on the current worker when Emscripten first creates a pthread on the current worker. + /// May be fired multiple times because Emscripten reuses workers to run a new pthread after the current one is finished. + [dotnetPthreadCreated]: WorkerThreadEvent; + // Emitted on the current worker when a pthread attaches to Mono. + // Threads may attach and detach to Mono multiple times during their lifetime. + [dotnetPthreadAttached]: WorkerThreadEvent; +} + +export interface WorkerThreadEvent extends Event { + readonly pthread_ptr: pthread_ptr; + readonly portToMain: MessagePort; +} + +class WorkerThreadEventImpl extends Event implements WorkerThreadEvent { + readonly pthread_ptr: pthread_ptr; + readonly portToMain: MessagePort; + constructor(type: keyof WorkerThreadEventMap, pthread_ptr: pthread_ptr, portToMain: MessagePort) { + super(type); + this.pthread_ptr = pthread_ptr; + this.portToMain = portToMain; + } +} + + +export interface WorkerThreadEventTarget extends EventTarget { + dispatchEvent(event: WorkerThreadEvent): boolean; + + addEventListener<K extends keyof WorkerThreadEventMap>(type: K, listener: ((this: WorkerThreadEventTarget, ev: WorkerThreadEventMap[K]) => any) | null, options?: boolean | AddEventListenerOptions): void; + addEventListener(type: string, callback: EventListenerOrEventListenerObject | null, options?: boolean | AddEventListenerOptions): void; +} + +export function makeWorkerThreadEvent(type: keyof WorkerThreadEventMap, pthread_ptr: pthread_ptr, port: MessagePort): WorkerThreadEvent { + // this 'if' helps to tree-shake the WorkerThreadEventImpl class if threads are disabled. + if (MonoWasmThreads) { + return new WorkerThreadEventImpl(type, pthread_ptr, port); + } else { + throw new Error("threads support disabled"); + } +} diff --git a/src/mono/wasm/runtime/pthreads/worker/index.ts b/src/mono/wasm/runtime/pthreads/worker/index.ts new file mode 100644 index 00000000000..06ee53cea26 --- /dev/null +++ b/src/mono/wasm/runtime/pthreads/worker/index.ts @@ -0,0 +1,74 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. + +/// <reference lib="webworker" /> + +import MonoWasmThreads from "consts:monoWasmThreads"; +import { Module, ENVIRONMENT_IS_PTHREAD } from "../../imports"; +import { makeChannelCreatedMonoMessage, pthread_ptr } from "../shared"; +import { mono_assert, is_nullish } from "../../types"; +import { + makeWorkerThreadEvent, + dotnetPthreadCreated, + dotnetPthreadAttached, + WorkerThreadEventTarget +} from "./events"; + +// re-export some of the events types +export { + WorkerThreadEventMap, + dotnetPthreadAttached, + dotnetPthreadCreated, + WorkerThreadEvent, + WorkerThreadEventTarget, +} from "./events"; + +/// This is the "public internal" API for runtime subsystems that wish to be notified about +/// pthreads that are running on the current worker. +/// Example: +/// currentWorkerThreadEvents.addEventListener(dotnetPthreadCreated, (ev: WorkerThreadEvent) => { +/// console.debug ("thread created on worker with id", ev.pthread_ptr); +/// }); +export const currentWorkerThreadEvents: WorkerThreadEventTarget = + MonoWasmThreads ? new EventTarget() : null as any as WorkerThreadEventTarget; // treeshake if threads are disabled + +function monoDedicatedChannelMessageFromMainToWorker(event: MessageEvent<string>): void { + console.debug("got message from main on the dedicated channel", event.data); +} + +let portToMain: MessagePort | null = null; + +function setupChannelToMainThread(pthread_ptr: pthread_ptr): MessagePort { + console.debug("creating a channel", pthread_ptr); + const channel = new MessageChannel(); + const workerPort = channel.port1; + const mainPort = channel.port2; + workerPort.addEventListener("message", monoDedicatedChannelMessageFromMainToWorker); + workerPort.start(); + portToMain = workerPort; + self.postMessage(makeChannelCreatedMonoMessage(pthread_ptr, mainPort), [mainPort]); + return workerPort; +} + +/// This is an implementation detail function. +/// Called in the worker thread from mono when a pthread becomes attached to the mono runtime. +export function mono_wasm_pthread_on_pthread_attached(pthread_id: pthread_ptr): void { + const port = portToMain; + mono_assert(port !== null, "expected a port to the main thread"); + console.debug("attaching pthread to runtime", pthread_id); + currentWorkerThreadEvents.dispatchEvent(makeWorkerThreadEvent(dotnetPthreadAttached, pthread_id, port)); +} + +/// This is an implementation detail function. +/// Called by emscripten when a pthread is setup to run on a worker. Can be called multiple times +/// for the same worker, since emscripten can reuse workers. This is an implementation detail, that shouldn't be used directly. +export function afterThreadInit(): void { + // don't do this callback for the main thread + if (ENVIRONMENT_IS_PTHREAD) { + const pthread_ptr = (<any>Module)["_pthread_self"](); + mono_assert(!is_nullish(pthread_ptr), "pthread_self() returned null"); + console.debug("after thread init, pthread ptr", pthread_ptr); + const port = setupChannelToMainThread(pthread_ptr); + currentWorkerThreadEvents.dispatchEvent(makeWorkerThreadEvent(dotnetPthreadCreated, pthread_ptr, port)); + } +} diff --git a/src/mono/wasm/runtime/pthreads/worker/tsconfig.json b/src/mono/wasm/runtime/pthreads/worker/tsconfig.json new file mode 100644 index 00000000000..071a4d824c6 --- /dev/null +++ b/src/mono/wasm/runtime/pthreads/worker/tsconfig.json @@ -0,0 +1,7 @@ +{ + "extends": "../../tsconfig.worker.json", + "include": [ + "../../**/*.ts", + "../../**/*.d.ts" + ] +} diff --git a/src/mono/wasm/runtime/rollup.config.js b/src/mono/wasm/runtime/rollup.config.js index 74684132c0e..1bea33c8c9c 100644 --- a/src/mono/wasm/runtime/rollup.config.js +++ b/src/mono/wasm/runtime/rollup.config.js @@ -14,6 +14,7 @@ const configuration = process.env.Configuration; const isDebug = configuration !== "Release"; const productVersion = process.env.ProductVersion || "7.0.0-dev"; const nativeBinDir = process.env.NativeBinDir ? process.env.NativeBinDir.replace(/"/g, "") : "bin"; +const monoWasmThreads = process.env.MonoWasmThreads === "true" ? true : false; const terserConfig = { compress: { defaults: false,// too agressive minification breaks subsequent emcc compilation @@ -60,7 +61,7 @@ const inlineAssert = [ pattern: /^\s*mono_assert/gm, failure: "previous regexp didn't inline all mono_assert statements" }]; -const outputCodePlugins = [regexReplace(inlineAssert), consts({ productVersion, configuration }), typescript()]; +const outputCodePlugins = [regexReplace(inlineAssert), consts({ productVersion, configuration, monoWasmThreads }), typescript()]; const iffeConfig = { treeshake: !isDebug, diff --git a/src/mono/wasm/runtime/startup.ts b/src/mono/wasm/runtime/startup.ts index fb0b8d9e874..776a8eef168 100644 --- a/src/mono/wasm/runtime/startup.ts +++ b/src/mono/wasm/runtime/startup.ts @@ -1,8 +1,9 @@ // Licensed to the .NET Foundation under one or more agreements. // The .NET Foundation licenses this file to you under the MIT license. +import MonoWasmThreads from "consts:monoWasmThreads"; import { AllAssetEntryTypes, mono_assert, AssetEntry, CharPtrNull, DotnetModule, GlobalizationMode, MonoConfig, MonoConfigError, wasm_type_symbol, MonoObject } from "./types"; -import { ENVIRONMENT_IS_ESM, ENVIRONMENT_IS_NODE, ENVIRONMENT_IS_SHELL, INTERNAL, locateFile, Module, MONO, requirePromise, runtimeHelpers } from "./imports"; +import { ENVIRONMENT_IS_ESM, ENVIRONMENT_IS_NODE, ENVIRONMENT_IS_PTHREAD, ENVIRONMENT_IS_SHELL, INTERNAL, locateFile, Module, MONO, requirePromise, runtimeHelpers } from "./imports"; import cwraps from "./cwraps"; import { mono_wasm_raise_debug_event, mono_wasm_runtime_ready } from "./debug"; import GuardedPromise from "./guarded-promise"; @@ -18,6 +19,7 @@ import { mono_on_abort } from "./run"; import { mono_wasm_new_root } from "./roots"; import { init_crypto } from "./crypto-worker"; import { init_polyfills } from "./polyfills"; +import * as pthreads_worker from "./pthreads/worker"; export let runtime_is_initialized_resolve: () => void; export let runtime_is_initialized_reject: (reason?: any) => void; @@ -107,6 +109,10 @@ export function configure_emscripten_startup(module: DotnetModule, exportedAPI: } // Otherwise startup sequence is up to user code, like Blazor + if (MonoWasmThreads && ENVIRONMENT_IS_PTHREAD) { + mono_wasm_pthread_worker_init(); + } + if (!module.onAbort) { module.onAbort = () => mono_on_abort; } @@ -756,3 +762,17 @@ export type DownloadAssetsContext = { loaded_files: { url?: string, file: string }[], loaded_assets: { [id: string]: [VoidPtr, number] }, } + +/// Called when dotnet.worker.js receives an emscripten "load" event from the main thread. +/// +/// Notes: +/// 1. Emscripten skips a lot of initialization on the pthread workers, Module may not have everything you expect. +/// 2. Emscripten does not run the preInit or preRun functions in the workers. +/// 3. At the point when this executes there is no pthread assigned to the worker yet. +async function mono_wasm_pthread_worker_init(): Promise<void> { + // This is a good place for subsystems to attach listeners for pthreads_worker.currrentWorkerThreadEvents + console.debug("mono_wasm_pthread_worker_init"); + pthreads_worker.currentWorkerThreadEvents.addEventListener(pthreads_worker.dotnetPthreadCreated, (ev) => { + console.debug("thread created", ev.pthread_ptr); + }); +} diff --git a/src/mono/wasm/runtime/tsconfig.json b/src/mono/wasm/runtime/tsconfig.json index fd189dfb276..9423cf70a55 100644 --- a/src/mono/wasm/runtime/tsconfig.json +++ b/src/mono/wasm/runtime/tsconfig.json @@ -1,20 +1,10 @@ { - "compilerOptions": { - "noImplicitAny": true, - "noEmitOnError": true, - "removeComments": false, - "sourceMap": false, - "target": "ES2018", - "moduleResolution": "Node", - "lib": [ - "esnext", - "dom" - ], - "strict": true, + "extends": "./tsconfig.shared.json", + "compilerOptions": { + "lib": [ + "esnext", + "dom" + ], "outDir": "bin", - }, - "exclude": [ - "dotnet.d.ts", - "bin" - ] -}
\ No newline at end of file + } +} diff --git a/src/mono/wasm/runtime/tsconfig.shared.json b/src/mono/wasm/runtime/tsconfig.shared.json new file mode 100644 index 00000000000..a415a516ea3 --- /dev/null +++ b/src/mono/wasm/runtime/tsconfig.shared.json @@ -0,0 +1,18 @@ +{ + "compilerOptions": { + "noImplicitAny": true, + "noEmitOnError": true, + "removeComments": false, + "sourceMap": false, + "target": "ES2018", + "moduleResolution": "Node", + "strict": true, + "lib": [ + "esnext" + ], + }, + "exclude": [ + "dotnet.d.ts", + "bin" + ] +} diff --git a/src/mono/wasm/runtime/tsconfig.worker.json b/src/mono/wasm/runtime/tsconfig.worker.json new file mode 100644 index 00000000000..04108f647f2 --- /dev/null +++ b/src/mono/wasm/runtime/tsconfig.worker.json @@ -0,0 +1,9 @@ +{ + "extends": "./tsconfig.shared.json", + "compilerOptions": { + "lib": [ + "esnext", + "webworker" + ], + } +} diff --git a/src/mono/wasm/runtime/types/consts.d.ts b/src/mono/wasm/runtime/types/consts.d.ts index f7256df9373..c71e7862d5a 100644 --- a/src/mono/wasm/runtime/types/consts.d.ts +++ b/src/mono/wasm/runtime/types/consts.d.ts @@ -2,4 +2,9 @@ declare module "consts:*" { //Constant that will be inlined by Rollup and rollup-plugin-consts. const constant: any; export default constant; -}
\ No newline at end of file +} + +declare module "consts:monoWasmThreads" { + const constant: boolean; + export default constant; +} diff --git a/src/mono/wasm/wasm.proj b/src/mono/wasm/wasm.proj index 88acbbd551f..605d36b88b8 100644 --- a/src/mono/wasm/wasm.proj +++ b/src/mono/wasm/wasm.proj @@ -365,7 +365,7 @@ <RunWithEmSdkEnv Command="npm run lint" StandardOutputImportance="High" EmSdkPath="$(EMSDK_PATH)" WorkingDirectory="$(MonoProjectRoot)wasm/runtime/"/> <!-- compile typescript --> - <RunWithEmSdkEnv Command="npm run rollup -- --environment Configuration:$(Configuration),NativeBinDir:$(NativeBinDir),ProductVersion:$(ProductVersion)" EmSdkPath="$(EMSDK_PATH)" IgnoreStandardErrorWarningFormat="true" WorkingDirectory="$(MonoProjectRoot)wasm/runtime/"/> + <RunWithEmSdkEnv Command="npm run rollup -- --environment Configuration:$(Configuration),NativeBinDir:$(NativeBinDir),ProductVersion:$(ProductVersion),MonoWasmThreads:$(MonoWasmThreads)" EmSdkPath="$(EMSDK_PATH)" IgnoreStandardErrorWarningFormat="true" WorkingDirectory="$(MonoProjectRoot)wasm/runtime/"/> <Copy SourceFiles="runtime/package.json;" DestinationFolder="$(NativeBinDir)" |