Repository navigation
Option to have a shared V8 library? #53509
Description
Activity
@nodejs/v8 this sounds like a good idea though probably not super practical due to the fact we float patches over v8
If the need is to not have to rebuild V8, it seems the goal overlaps with the (very old) effort of linking Node.js with
v8_monolith.ainstead, but no one has been volunteering in making it work AFAIK. It just needs someone to wrestle with the gyp configs to figure out how to do it. Then it would become possible to cache thev8_monolith.aand reuse it in different Node.js builds that don't change anything indeps/v8.Reacted by ExE Boss and Sambut no one has been volunteering in making it work
/me raises hand
I plan on working on it, provided it can be done in a reasonable time frame. If anyone has concerns, now would be a good time to voice them.
@bnoordhuis are you still interested in working on this proposal? Or perhaps, could you give a few pointers so that I can try exploring this?
I haven't started yet so you're welcome to it. I'd start with the monolith build. Once that works, a shared library should be relatively easy.
In terms of the side quest of making a super easy way to start development without having to build everything from scratch, an approach using Docker containers seems like it would work. @billywhizz and I have been playing with this internally and it seems viable - the image we built is 2.44GB which is not that bad to pull on a fast internet connection. We found https://github.com/nodejs/devcontainer which seems to have been abandoned, but embodies the same idea?
I think I should've phrased the issue properly, @alexweej .
In Guix/NixOS lingua, packages are dependent on other packages, called inputs. Packages are going to be built in a reproducible, isolated environment with highly-restricted network - so Docker cannot be used here. The only way to provide those packages are either through in-source dependencies or inputs.
I wish to be provided an option to remove in-source dependency contamination in NodeJS - by packing in-source dependencies as packages instead. Packages would be built only once (in both dev and build environment) and cached locally or on build farms. This would ensure that future builds would take constant time for NodeJS.
But as it turns out, NodeJS V8 is a customized fork. It is also not maintained separately, but as a in-source dependency for NodeJS, which makes it even more harder.
Hey, IMO you did phrase it properly but perhaps were unclear on the "AS A __ I WANT __ SO THAT __".
AS A Node.js contributor
I WANT a ready-made working copy of Node.js to make quick edits and run the tests
SO THAT the barrier to entry is reduced for contributionsThis seemed to be what @joyeecheung was referring to in #53509 (comment) above, but maybe I misunderstood.
But what you're asking is:
AS A software distributor (Nixpkgs/Debian/Red Hat)
I WANT the Node.js package build to not require an embedded vendored build of V8
SO THAT I can reduce my build compute bill and reuse the V8 build for more than just Node.jsIs that correct?
AS A software distributor (Nixpkgs/Debian/Red Hat)
I WANT the Node.js package build to not require an embedded vendored build of V8
SO THAT I can reduce my build compute bill and reuse the V8 build for more than just Node.jsI think that this sums it perfectly.
Reacted by Alexander “weej” Jones, Aleksandar Fabijanic and yufeiyThis new feature will also benefit windows on ARM devices. For popular X64 Apps developed with Node.js running on Windows on ARM device, which suffer from poor performance due to emulation instruction translation. Once this feature implemented, simply replacing V8 shared library with ARM64EC could result in a 2-3x improvement in JavaScript Performance.
Reacted by Ashvith10, XuSai and yufeiyHi all,
I am trying to integrate v8 as a shared lib into node. However, I encounter an error
[2/6985] ACTION //v8:run_mksnapshot_default(//build/toolchain/win:win_clang_x64) FAILED: gen/v8/embedded.S snapshot_blob.bin C:/Users/xxxxxx/software/depot_tools/bootstrap-2@3_11_8_chromium_35_bin/python3/bin/python3.exe ../../v8/tools/run.py ./mksnapshot --turbo_instruction_scheduling --stress-turbo-late-spilling --target_os=win --target_arch=x64 --embedded_src gen/v8/embedded.S --predictable --no-use-ic --turbo-elide-frames --embedded_variant Default --random-seed 314159265 --startup_blob snapshot_blob.bin --no-native-code-counters --concurrent-builtin-generation --concurrent-turbofan-max-threads=0 Return code is 2147483651
This is odd, has anyone experienced a similar issue and could give me some advice ?
Platform: Windows
Compiler: clang-cl
Node version: 22.17.1 LTS
V8 version: 14.0.241There has been no activity on this feature request for 5 months. To help maintain relevant open issues, please add the never-stale
Issues and PRs exempt from automated stale handling. label or close this issue if it should be closed. If not, the issue will be automatically closed 6 months after the last non-automated comment.
For more information on how the project manages feature requests, please consult the feature request management document.Now that I got ping by this issue, as a related update: I have done some work recently to revive https://github.com/nodejs/devcontainer and @aduh95 recently submitted nodejs/devcontainer#22 to make the devcontainer use Nix. There is a devcontainer configuration in core to pair with the published docker images https://github.com/nodejs/node/blob/main/.devcontainer/base/devcontainer.json - I still don't know if it would be possible one day to make that image use a v8 package - as mentioned above Node.js uses a fork of v8, so the best you could do is to make a node-v8 package instead. But for the purpose of hacking on Node.js without having to build everything, the docker images allow just that.
There is a guide on how to use the current devcontainer configurations with VSCode https://github.com/nodejs/node/blob/main/doc/contributing/using-devcontainer.md , or you could also try GitHub code spaces to hack on a remote container in the browser if, say, you are away from desk. There are some screenshots on how to do this here: #60472 (comment)
I think all of this is entirely possible. I've started looking at the code, and I'll see what I can come up from there. Right now, Node has
--without-bundled-v8but it's incomplete with a hand-rolledpkg_config("v8")and includes no overrides. So that's part of the equation, but not likely part of the solution.The comment by @benjamingr is important, but definitely not a blocker. The concern is valid but the conclusion is wrong. The patches represent Node requirements on V8's behavior. Those requirements don't vanish when V8 is external, they must be satisfied by whatever V8 is linked, whether that's a system-packaged V8, a distributor-built V8, or an alternative engine with v8 API Shims. But, and this is key for this, "satisfied" can't mean "hope the distributor figured it out." Node would still take responsibility for its own requirements, just through three mechanisms:
- Validate: hard-error at configure time for ABI requirements that cannot be worked around (pointer compression, sandbox, version match)
- Adapt: provide fallback code paths in Node for features that have both a fast path and a correct slow path.
- Document: Publish a concrete V8 Build Configuration Spec so distributors and alternative engines know exactly what to provide and why.
This works out well when you look at the nature of the floating patches.
-
Build Configuration Flags: Flags that already exist within V8, Node just sets a different default. like
v8_promise_internal_field_countis set to 1 where the package default is 0. -
MSVC compiler compatibility patches: For the purposes of any PR, this would be irrelevant as the external V8 is already compiled. But Node can absolutely handle header warnings in node.gypi when using a shared V8.
-
V8 Embedder API requirements. These aren't code patches either, they're upstream v8 API that any v8-api-compatible engine must provide like
V8::Context::SetPromiseHooks()orv8::Isolate::SetPromiseHook()and a few others. So that's where the documentation comes into play, which is the V8 API contract. Any V8 Compliant Engine has them already.
Overall, at first glance, this is entirely doable and I don't think it'll be very difficult to actually achieve. I'll spend a bit of time seeing what I can come up with and report back.
This issue has been marked as stale due to 90 days of inactivity.
It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.The take away from our experience of using
--without-bundled-v8in GHA is that it would be really worth it to support this officially, what it would take is probably Node.js distributing its own flavor of V8 (i.e. the source with the patches we're using)
What is the problem this feature will solve?
In distros like NixOS and GuixSD, building NodeJS from scratch is a heavy computation. Building vendored V8 NodeJS is not only computationally heavy, but it also makes caching the good parts of the library impossible for a simple failed test. It is painful having to wait for a day to have a package build, only for the tests to fail for the package maintainer to disable or patch them one by one per build.
What is the feature you are proposing to solve the problem?
With V8 as a shared library, it would be possible to not only cache the dependency in a functional package manager, but also use it with other packages that require them.
What alternatives have you considered?
None, as I've read somewhere than v8 dependencies are patched, which would make it impossible to use vanilla v8 alongside.