| Age | Commit message (Collapse) | Author |
|---|
|
Node.js unofficially supports a shared library variant where the
main node executable is a thin wrapper around node.dll/libnode.so.
The key benefit of this is to support embedding Node.js in other
applications.
Since Node.js 12 there have been a number of issues preventing the
shared library build from working correctly, primarily on Windows:
* A number of functions used executables such as `mksnapshot` are
not exported from `libnode.dll` using a `NODE_EXTERN` attribute
* A dependency on the `Winmm` system library is missing
* Incorrect defines on executable targets leads to `node.exe`
claiming to export a number of functions that are actually in
`libnode.dll`
* Because `node.exe` attempts to export symbols, `node.lib` gets
generated causing native extensions to try to link against
`node.exe` not `libnode.dll`.
* Similarly, because `node.dll` was renamed to `libnode.dll`,
native extensions don't know to look for `libnode.lib` rather
than `node.lib`.
* On macOS an RPATH is added to find `libnode.dylib` relative to
`node` in the same folder. This works fine from the
`out/Release` folder but not from an installed prefix, where
`node` will be in `bin/` and `libnode.dylib` will be in `lib/`.
* Similarly on Linux, no RPATH is added so LD_LIBRARY_PATH needs
setting correctly for `bin/node` to find `lib/libnode.so`.
For the `libnode.lib` vs `node.lib` issue there are two possible
options:
1. Ensure `node.lib` from `node.exe` does not get generated, and
instead copy `libnode.lib` to `node.lib`. This means addons
compiled when referencing the correct `node.lib` file will
correctly depend on `libnode.dll`. The down side is that
native addons compiled with stock Node.js will still try to
resolve symbols against node.exe rather than libnode.dll.
2. After building `libnode.dll`, dump the exports using `dumpbin`,
and process this to generate a `node.def` file to be linked into
`node.exe` with the `/DEF:node.def` flag. The export entries
in `node.def` will all read
```
my_symbol=libnode.my_symbol
```
so that `node.exe` will redirect all exported symbols back to
`libnode.dll`. This has the benefit that addons compiled with
stock Node.js will load correctly into `node.exe` from a shared
library build, but means that every embedding executable also
needs to perform this same trick.
I went with the first option as it is the cleaner of the two
solutions in my opinion. Projects wishing to generate a shared
library variant of Node.js can now, for example,
```
.\vcbuild dll package vs
```
to generate a full node installation including `libnode.dll`,
`Release\node.lib`, and all the necessary headers. Native addons
can then be built against the shared library build easily by
specifying the correct `nodedir` option.
For example
```
>npx node-gyp configure --nodedir
C:\Users\User\node\Release\node-v18.0.0-win-x64
...
>npx node-gyp build
...
>dumpbin /dependents build\Release\binding.node
Microsoft (R) COFF/PE Dumper Version 14.29.30136.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file build\Release\binding.node
File Type: DLL
Image has the following dependencies:
KERNEL32.dll
libnode.dll
VCRUNTIME140.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
...
```
PR-URL: https://github.com/nodejs/node/pull/41850
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Beth Griggs <bgriggs@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
|
|
Allow passing a string as the main module rather than using
the callback variant.
PR-URL: https://github.com/nodejs/node/pull/30467
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Gireesh Punathil <gpunathi@in.ibm.com>
|
|
PR-URL: https://github.com/nodejs/node/pull/31321
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Tobias Nießen <tniessen@tnie.de>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
|
|
PR-URL: https://github.com/nodejs/node/pull/27980
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
|
|
PR-URL: https://github.com/nodejs/node/pull/27509
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
|
|
This patch splits `NativeModuleLoader` into two parts - a singleton
that only relies on v8 and `node::Mutex` and a proxy class for
the singleton (`NativeModuleEnv`) that provides limited access to
the singleton as well as C++ bindings for the Node.js binary.
`NativeModuleLoader` is then no longer aware of `Environment`.
PR-URL: https://github.com/nodejs/node/pull/27160
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Richard Lau <riclau@uk.ibm.com>
|
|
This allows us to query the categories of modules in C++
so we can implement the code cache generator in C++ that
does not depend on a Node.js binary.
PR-URL: https://github.com/nodejs/node/pull/27046
Refs: https://github.com/nodejs/node/issues/21563
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
|
|
This patch splits the execution mode selection from the environment
setup in `lib/internal/bootstrap/node.js`, and split the entry point
of different execution mode into main scripts under
`lib/internal/main`:
- `check_syntax.js`: used when `-c`/`--check` which only checks the
syntax of the input instead of executing it.
- `eval_stdin.js`: used when `-e` is passed without value and stdin
is not a TTY (e.g. something is piped).
- `eval_string`: used when `-e` is passed along with a string argument
- `inspect.js`: for `node inspect`/`node debug`
- `print_bash_completion.js`: for `--completion-bash`
- `print_help.js`: for `--help`
- `prof_process.js`: for `--prof-process`
- `repl.js`: for the REPL
- `run_main_module.js`: used when a main module is passed
- `run_third_party_main.js`: for the legacy `_third_party_main.js`
support
- `worker_thread.js`: for workers
This makes the entry points easier to navigate and paves the way
for customized v8 snapshots (that do not need to deserialize
execution mode setup) and better embedder APIs.
As an example, after this patch, for the most common case where
Node.js executes a user module as an entry point, it essentially
goes through:
- `lib/internal/per_context.js` to setup the v8 Context (which is
also run when setting up contexts for the `vm` module)
- `lib/internal/bootstrap/loaders.js` to set up internal binding
and builtin module loaders (that are separate from the loaders
accessible in the user land).
- `lib/internal/bootstrap/node.js`: to set up the rest of the
environment, including various globals and the process object
- `lib/internal/main/run_main_module.js`: which is selected from
C++ to prepare execution of the user module.
This patch also removes `NativeModuleLoader::CompileAndCall` and
exposes `NativeModuleLoader::LookupAndCompile` directly so that
we can handle syntax errors and runtime errors of bootstrap
scripts differently.
PR-URL: https://github.com/nodejs/node/pull/25667
Reviewed-By: Anna Henningsen <anna@addaleax.net>
|
|
The idea is to allow the C++ layer to run arbitrary scripts
as the main script. This paves the way for
- cctest of the execution of Node.js instances
- Earlier handling of per-process CLI options that affect
execution modes (those usually do not make sense for the
embedders).
- Targets like mkcodecache or mksnapshot.
Also moves the handling of `_third_party_main.js` into C++.
PR-URL: https://github.com/nodejs/node/pull/25474
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Minwoo Jung <minwoo@nodesource.com>
|
|
- Remove `NativeModule._source` - the compilation is now entirely
done in C++ and `process.binding('natives')` is implemented
directly in the binding loader so there is no need to store
additional source code strings.
- Instead of using an object as `NativeModule._cached` and insert
into it after compilation of each native module, simply prebuild
a JS map filled with all the native modules and infer the
state of compilation through `mod.loading`/`mod.loaded`.
- Rename `NativeModule.nonInternalExists` to
`NativeModule.canBeRequiredByUsers` and precompute that
property for all the native modules during bootstrap instead
of branching in every require call during runtime. This also fixes
the bug where `worker_threads` can be made available with
`--expose-internals`.
- Rename `NativeModule.requireForDeps` to
`NativeModule.requireWithFallbackInDeps`.
- Add a test to make sure we do not accidentally leak any module
to the global namespace.
PR-URL: https://github.com/nodejs/node/pull/25352
Reviewed-By: Anna Henningsen <anna@addaleax.net>
|
|
This patch changes the NativeModuleLoader to always try to find
code cache for native modules when it compiles them, and always
produce and store the code cache after compilation. The cache
map is protected by a mutex and can be accessed by different
threads - including the worker threads and the main thread. Hence any
thread can reuse the code cache if the native module has already
been compiled by another thread - in particular the cache of the
bootstrappers and per_context.js will always be hit when a new thread
is spun.
This results in a ~6% startup overhead in the worst case
(when only the main thread is launched without requiring any additional
native module - it now needs to do the extra work of finding and
storing caches), which balances out the recent improvements by moving
the compilation to C++, but it also leads to a ~60% improvement in
the best case (when a worker thread is spun and requires a lot of native
modules thus hitting the cache compiled by the main thread).
PR-URL: https://github.com/nodejs/node/pull/24950
Reviewed-By: Anna Henningsen <anna@addaleax.net>
|
|
In preparation of sharing code cache among different threads -
we simply rely on v8 to reject invalid cache, since there isn't
any serious consequence when the cache is invalid anyway.
PR-URL: https://github.com/nodejs/node/pull/24950
Reviewed-By: Anna Henningsen <anna@addaleax.net>
|
|
Instead of treating config.gypi as a JavaScript file, specialize
the processing in js2c and make the serialized result a real JSON
string (with 'true' and 'false' converted to boolean values) so
we don't have to use a custom deserializer during bootstrap.
In addition, store the JSON string separately in NativeModuleLoader,
and keep it separate from the map of the builtin source code, so
we don't have to put it onto `NativeModule._source` and delete it
later, though we still preserve it in `process.binding('natives')`,
which we don't use anymore.
This patch also makes the map of builtin source code and the
config.gypi string available through side-effect-free getters
in C++.
PR-URL: https://github.com/nodejs/node/pull/24816
Reviewed-By: Gus Caplan <me@gus.host>
|
|
Registration initialization functions are expected to have a 4th
argument, a void*, so add them where necessary to fix the warnings.
PR-URL: https://github.com/nodejs/node/pull/24737
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
|
|
This patch introduces a NativeModuleLoader::CompileAndCall that
can run a JS script under `lib/` as a function called
with a null receiver and arguments specified from the C++ layer.
Since all our bootstrappers are wrapped in functions in the
source to avoid leaking variables into the global scope anyway,
this allows us to remove that extra indentation in the JS source code.
As a start we move the compilation and execution of per_context.js
to NativeModuleLoader::CompileAndCall(). This patch also changes the
return value of NativeModuleLoader::LookupAndCompile() to a MaybeLocal
since the caller has to take care of the result being empty
anyway.
This patch reverts the previous design of having the
NativeModuleLoader::Compile() method magically know about the
parameters of the function - until we have tooling
in-place to guess the parameter names in the source with some
annotation, it's more readable to allow the caller to specify
the parameters along with the arguments values.
PR-URL: https://github.com/nodejs/node/pull/24660
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Gus Caplan <me@gus.host>
|
|
Instead of putting the source code and the cache in v8::Objects,
put them in per-process std::maps. This has the following benefits:
- It's slightly lighter in weight compared to storing things on the
v8 heap. Also it may be slightly faster since the preivous v8::Object
is already in dictionary mode - though the difference is very small
given the number of native modules is limited.
- The source and code cache generation templates are now much simpler
since they just initialize static arrays and manipulate STL
constructs.
- The static native module data can be accessed independently of any
Environment or Isolate, and it's easy to look them up from the
C++'s side.
- It's now impossible to mutate the source code used to compile
native modules from the JS land since it's completely separate
from the v8 heap. We can still get the constant strings from
process.binding('natives') but that's all.
A few drive-by fixes:
- Remove DecorateErrorStack in LookupAndCompile - We don't need to
capture the exception to decorate when we encounter
errors during native module compilation, as those errors should be
syntax errors and v8 is able to decorate them well. We use
CompileFunctionInContext so there is no need to worry about
wrappers either.
- The code cache could be rejected when node is started with v8 flags.
Instead of aborting in that case, simply keep a record in the
native_module_without_cache set.
- Refactor js2c.py a bit, reduce code duplication and inline Render()
to make the one-byte/two-byte special treatment easier to read.
PR-URL: https://github.com/nodejs/node/pull/24384
Fixes: https://github.com/Remove
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
|
|
This patch refactors out a part of NativeModule.prototype.compile
(in JS land) into a C++ NativeModule class, this enables a
couple of possibilities:
1. By moving the code to the C++ land, we have more opportunity
to specialize the compilation process of the native modules
(e.g. compilation options, code cache) that is orthogonal to
how user land modules are compiled
2. We can reuse the code to compile bootstrappers and context
fixers and enable them to be compiled with the code cache later,
since they are not loaded by NativeModule in the JS land their
caching must be done in C++.
3. Since there is no need to pass the static data to JS for
compilation anymore, this enables us to use
(std::map<std::string, const char*>) in the generated
node_code_cache.cc and node_javascript.cc later, and scope
every actual access to the source of native modules to a
std::map lookup instead of a lookup on a v8::Object in
dictionary mode.
This patch also refactor the code cache generator and tests
a bit and trace the `withCodeCache` and `withoutCodeCache`
in a Set instead of an Array, and makes sure that all the cachable
builtins are tested.
PR-URL: https://github.com/nodejs/node/pull/24221
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
|
