diff options
author | William Marlow <william.marlow@ibm.com> | 2022-02-04 13:12:57 +0300 |
---|---|---|
committer | RafaelGSS <rafael.nunu@hotmail.com> | 2022-05-10 15:13:19 +0300 |
commit | 0eb32ed976296b5082c7c5e1e13f32e91cb23400 (patch) | |
tree | 90d2389881106e04696c40dce513feab6b7b15b3 /tools | |
parent | 840e61e745e37448c9b23a0d44892b0bc50e1f24 (diff) |
build: fix various shared library build issues
Node.js unofficially supports a shared library variant where the
main node executable is a thin wrapper around node.dll/libnode.so.
The key benefit of this is to support embedding Node.js in other
applications.
Since Node.js 12 there have been a number of issues preventing the
shared library build from working correctly, primarily on Windows:
* A number of functions used executables such as `mksnapshot` are
not exported from `libnode.dll` using a `NODE_EXTERN` attribute
* A dependency on the `Winmm` system library is missing
* Incorrect defines on executable targets leads to `node.exe`
claiming to export a number of functions that are actually in
`libnode.dll`
* Because `node.exe` attempts to export symbols, `node.lib` gets
generated causing native extensions to try to link against
`node.exe` not `libnode.dll`.
* Similarly, because `node.dll` was renamed to `libnode.dll`,
native extensions don't know to look for `libnode.lib` rather
than `node.lib`.
* On macOS an RPATH is added to find `libnode.dylib` relative to
`node` in the same folder. This works fine from the
`out/Release` folder but not from an installed prefix, where
`node` will be in `bin/` and `libnode.dylib` will be in `lib/`.
* Similarly on Linux, no RPATH is added so LD_LIBRARY_PATH needs
setting correctly for `bin/node` to find `lib/libnode.so`.
For the `libnode.lib` vs `node.lib` issue there are two possible
options:
1. Ensure `node.lib` from `node.exe` does not get generated, and
instead copy `libnode.lib` to `node.lib`. This means addons
compiled when referencing the correct `node.lib` file will
correctly depend on `libnode.dll`. The down side is that
native addons compiled with stock Node.js will still try to
resolve symbols against node.exe rather than libnode.dll.
2. After building `libnode.dll`, dump the exports using `dumpbin`,
and process this to generate a `node.def` file to be linked into
`node.exe` with the `/DEF:node.def` flag. The export entries
in `node.def` will all read
```
my_symbol=libnode.my_symbol
```
so that `node.exe` will redirect all exported symbols back to
`libnode.dll`. This has the benefit that addons compiled with
stock Node.js will load correctly into `node.exe` from a shared
library build, but means that every embedding executable also
needs to perform this same trick.
I went with the first option as it is the cleaner of the two
solutions in my opinion. Projects wishing to generate a shared
library variant of Node.js can now, for example,
```
.\vcbuild dll package vs
```
to generate a full node installation including `libnode.dll`,
`Release\node.lib`, and all the necessary headers. Native addons
can then be built against the shared library build easily by
specifying the correct `nodedir` option.
For example
```
>npx node-gyp configure --nodedir
C:\Users\User\node\Release\node-v18.0.0-win-x64
...
>npx node-gyp build
...
>dumpbin /dependents build\Release\binding.node
Microsoft (R) COFF/PE Dumper Version 14.29.30136.0
Copyright (C) Microsoft Corporation. All rights reserved.
Dump of file build\Release\binding.node
File Type: DLL
Image has the following dependencies:
KERNEL32.dll
libnode.dll
VCRUNTIME140.dll
api-ms-win-crt-string-l1-1-0.dll
api-ms-win-crt-stdio-l1-1-0.dll
api-ms-win-crt-runtime-l1-1-0.dll
...
```
PR-URL: https://github.com/nodejs/node/pull/41850
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Beth Griggs <bgriggs@redhat.com>
Reviewed-By: Richard Lau <rlau@redhat.com>
Diffstat (limited to 'tools')
-rw-r--r-- | tools/gen_node_def.cc | 197 | ||||
-rwxr-xr-x | tools/install.py | 21 |
2 files changed, 206 insertions, 12 deletions
diff --git a/tools/gen_node_def.cc b/tools/gen_node_def.cc new file mode 100644 index 00000000000..f4cabbd84c1 --- /dev/null +++ b/tools/gen_node_def.cc @@ -0,0 +1,197 @@ +#include <Windows.h> +#include <algorithm> +#include <cstdint> +#include <fstream> +#include <iostream> +#include <memory> +#include <vector> + +// This executable takes a Windows DLL and uses it to generate +// a module-definition file [1] which forwards all the exported +// symbols from the DLL and redirects them back to the DLL. +// This allows node.exe to export the same symbols as libnode.dll +// when building Node.js as a shared library. This is conceptually +// similary to the create_expfile.sh script used on AIX. +// +// Generating this .def file requires parsing data out of the +// PE32/PE32+ file format. Helper structs are defined in <Windows.h> +// hence why this is an executable and not a script. See [2] for +// details on the PE format. +// +// [1]: https://docs.microsoft.com/en-us/cpp/build/reference/module-definition-dot-def-files +// [2]: https://docs.microsoft.com/en-us/windows/win32/debug/pe-format + +// The PE32 format encodes pointers as Relative Virtual Addresses +// which are 32 bit offsets from the start of the image. This helper +// class hides the mess of the pointer arithmetic +struct RelativeAddress { + uintptr_t root; + uintptr_t offset = 0; + + RelativeAddress(HMODULE handle) noexcept + : root(reinterpret_cast<uintptr_t>(handle)) {} + + RelativeAddress(HMODULE handle, uintptr_t offset) noexcept + : root(reinterpret_cast<uintptr_t>(handle)), offset(offset) {} + + RelativeAddress(uintptr_t root, uintptr_t offset) noexcept + : root(root), offset(offset) {} + + template <typename T> + const T* AsPtrTo() const noexcept { + return reinterpret_cast<const T*>(root + offset); + } + + template <typename T> + T Read() const noexcept { + return *AsPtrTo<T>(); + } + + RelativeAddress AtOffset(uintptr_t amount) const noexcept { + return {root, offset + amount}; + } + + RelativeAddress operator+(uintptr_t amount) const noexcept { + return {root, offset + amount}; + } + + RelativeAddress ReadRelativeAddress() const noexcept { + return {root, Read<uint32_t>()}; + } +}; + +// A wrapper around a dynamically loaded Windows DLL. This steps through the +// PE file structure to find the export directory and pulls out a list of +// all the exported symbol names. +struct Library { + HMODULE library; + std::string libraryName; + std::vector<std::string> exportedSymbols; + + Library(HMODULE library) : library(library) { + auto libnode = RelativeAddress(library); + + // At relative offset 0x3C is a 32 bit offset to the COFF signature, 4 bytes + // after that is the start of the COFF header. + auto coffHeaderPtr = + libnode.AtOffset(0x3C).ReadRelativeAddress().AtOffset(4); + auto coffHeader = coffHeaderPtr.AsPtrTo<IMAGE_FILE_HEADER>(); + + // After the coff header is the Optional Header (which is not optional). We + // don't know what type of optional header we have without examining the + // magic number + auto optionalHeaderPtr = coffHeaderPtr.AtOffset(sizeof(IMAGE_FILE_HEADER)); + auto optionalHeader = optionalHeaderPtr.AsPtrTo<IMAGE_OPTIONAL_HEADER>(); + + auto exportDirectory = + (optionalHeader->Magic == 0x20b) ? optionalHeaderPtr.AsPtrTo<IMAGE_OPTIONAL_HEADER64>() + ->DataDirectory[IMAGE_DIRECTORY_ENTRY_EXPORT] + : optionalHeaderPtr.AsPtrTo<IMAGE_OPTIONAL_HEADER32>() + ->DataDirectory[IMAGE_DIRECTORY_ENTRY_EXPORT]; + + auto exportTable = libnode.AtOffset(exportDirectory.VirtualAddress) + .AsPtrTo<IMAGE_EXPORT_DIRECTORY>(); + + // This is the name of the library without the suffix, this is more robust + // than parsing the filename as this is what the linker uses. + libraryName = libnode.AtOffset(exportTable->Name).AsPtrTo<char>(); + libraryName = libraryName.substr(0, libraryName.size() - 4); + + const uint32_t* functionNameTable = + libnode.AtOffset(exportTable->AddressOfNames).AsPtrTo<uint32_t>(); + + // Given an RVA, parse it as a std::string. The resulting string is empty + // if the symbol does not have a name (i.e. it is ordinal only). + auto nameRvaToName = [&](uint32_t rva) -> std::string { + auto namePtr = libnode.AtOffset(rva).AsPtrTo<char>(); + if (namePtr == nullptr) return {}; + return {namePtr}; + }; + std::transform(functionNameTable, + functionNameTable + exportTable->NumberOfNames, + std::back_inserter(exportedSymbols), + nameRvaToName); + } + + ~Library() { FreeLibrary(library); } +}; + +bool IsPageExecutable(void* address) { + MEMORY_BASIC_INFORMATION memoryInformation; + size_t rc = VirtualQuery( + address, &memoryInformation, sizeof(MEMORY_BASIC_INFORMATION)); + + if (rc != 0 && memoryInformation.Protect != 0) { + return memoryInformation.Protect == PAGE_EXECUTE || + memoryInformation.Protect == PAGE_EXECUTE_READ || + memoryInformation.Protect == PAGE_EXECUTE_READWRITE || + memoryInformation.Protect == PAGE_EXECUTE_WRITECOPY; + } + return false; +} + +Library LoadLibraryOrExit(const char* dllPath) { + auto library = LoadLibrary(dllPath); + if (library != nullptr) return library; + + auto error = GetLastError(); + std::cerr << "ERROR: Failed to load " << dllPath << std::endl; + LPCSTR buffer = nullptr; + auto rc = FormatMessageA( + FORMAT_MESSAGE_ALLOCATE_BUFFER | FORMAT_MESSAGE_FROM_SYSTEM, + nullptr, + error, + LANG_USER_DEFAULT, + (LPSTR)&buffer, + 0, + nullptr); + if (rc != 0) { + std::cerr << buffer << std::endl; + LocalFree((HLOCAL)buffer); + } + exit(1); +} + +int main(int argc, char** argv) { + if (argc != 3) { + std::cerr << "Usage: " << argv[0] + << " path\\to\\libnode.dll path\\to\\node.def" << std::endl; + return 1; + } + + auto libnode = LoadLibraryOrExit(argv[1]); + auto defFile = std::ofstream(argv[2]); + defFile << "EXPORTS" << std::endl; + + for (const std::string& functionName : libnode.exportedSymbols) { + // If a symbol doesn't have a name then it has been exported as an + // ordinal only. We assume that only named symbols are exported. + if (functionName.empty()) continue; + + // Every name in the exported symbols table should be resolvable + // to an address because we have actually loaded the library into + // our address space. + auto address = GetProcAddress(libnode.library, functionName.c_str()); + if (address == nullptr) { + std::cerr << "WARNING: " << functionName + << " appears in export table but is not a valid symbol" + << std::endl; + continue; + } + + defFile << " " << functionName << " = " << libnode.libraryName << "." + << functionName; + + // Nothing distinguishes exported global data from exported functions + // with C linkage. If we do not specify the DATA keyword for such symbols + // then consumers of the .def file will get a linker error. This manifests + // as nodedbg_ symbols not being found. We assert that if the symbol is in + // an executable page in this process then it is a function, not data. + if (!IsPageExecutable(address)) { + defFile << " DATA"; + } + defFile << std::endl; + } + + return 0; +} diff --git a/tools/install.py b/tools/install.py index e2440a082f2..47e9d8bd7a1 100755 --- a/tools/install.py +++ b/tools/install.py @@ -133,20 +133,17 @@ def files(action): output_file = 'node' output_prefix = 'out/Release/' - if 'false' == variables.get('node_shared'): - if is_windows: - output_file += '.exe' - else: + if is_windows: + output_file += '.exe' + action([output_prefix + output_file], 'bin/' + output_file) + + if 'true' == variables.get('node_shared'): if is_windows: - output_file += '.dll' + action([output_prefix + 'libnode.dll'], 'bin/libnode.dll') + action([output_prefix + 'libnode.lib'], 'lib/libnode.lib') else: - output_file = 'lib' + output_file + '.' + variables.get('shlib_suffix') - - if 'false' == variables.get('node_shared'): - action([output_prefix + output_file], 'bin/' + output_file) - else: - action([output_prefix + output_file], 'lib/' + output_file) - + output_lib = 'libnode.' + variables.get('shlib_suffix') + action([output_prefix + output_lib], 'lib/' + output_lib) if 'true' == variables.get('node_use_dtrace'): action(['out/Release/node.d'], 'lib/dtrace/node.d') |