Repository navigation
bytecode-only SEA caching? #73
Description
Activity
bytenode exists, but it's focused on obfuscation use cases, and not faster loading. Because of this it doesn't work in some cases, which makes it difficult to recommend as a general solution.
To be clear, the cases you're referring to where bytenode doesn't work are the ones that are listed in https://github.com/bytenode/bytenode#known-issues-and-limitations, right?
bytenode exists, but it's focused on obfuscation use cases, and not faster loading. Because of this it doesn't work in some cases, which makes it difficult to recommend as a general solution.
To be clear, the cases you're referring to where bytenode doesn't work are the ones that are listed in https://github.com/bytenode/bytenode#known-issues-and-limitations, right?
Yes. I don't know of any other cases where bytenode fails, but it's possible they exist.
Reacted by Darshan SenHere's a benchmark for the runtime of
yarn helpfor various compilation strategies:yarn-3.5.1.cjs: cjs bundle of all of yarn's source codecached_yarn.js: uses a Node.js native module which has yarn's cjs bundled source code & a code cache (with kEagerCompile) embedded in the shared library.yarn: non-bundled yarn source code. (This usesv8-compile-cache.)bytenode yarn-3.5.1.jsc: executed the result ofbytenode -con the cjs bundleyarn-3.5.1.cjs
$ hyperfine -N -w 30 -L cmd 'node yarn-3.5.1.cjs help','node cached_yarn.js help','yarn help','bytenode yarn-3.5.1.jsc help' '{cmd}' Benchmark 1: node yarn-3.5.1.cjs help Time (mean ± σ): 144.2 ms ± 1.4 ms [User: 168.3 ms, System: 12.5 ms] Range (min … max): 142.9 ms … 147.5 ms 20 runs Benchmark 2: node cached_yarn.js help Time (mean ± σ): 95.8 ms ± 1.3 ms [User: 123.0 ms, System: 10.8 ms] Range (min … max): 93.7 ms … 99.4 ms 31 runs Benchmark 3: yarn help Time (mean ± σ): 110.0 ms ± 1.6 ms [User: 88.9 ms, System: 14.4 ms] Range (min … max): 107.8 ms … 114.0 ms 27 runs Benchmark 4: bytenode yarn-3.5.1-patched.jsc help Time (mean ± σ): 73.1 ms ± 1.5 ms [User: 53.0 ms, System: 10.4 ms] Range (min … max): 69.5 ms … 77.7 ms 41 runs Summary 'bytenode yarn-3.5.1-patched.jsc help' ran 1.31 ± 0.03 times faster than 'node cached_yarn.js help' 1.50 ± 0.04 times faster than 'yarn help' 1.97 ± 0.04 times faster than 'node yarn-3.5.1.cjs help'
so bytenode is around 50-97% faster than regular yarn, which is pretty impressive. Replacing the source code with "fake" code avoids a lot of parsing overhead, which means that it's even faster than actually using the source code.
This also made me look more at bytenode, which made me see that it uses the V8 flags
--no-lazy/--no-flush-bytecode. This will increase memory usage but that's probably acceptable for many use cases. It also means it will stop working if V8 removes either of those flags, which is not great from a stability perspective.Anyway, SEA-style bytecode caching would give a solid 13-33% startup improvement, depending on exactly what your baseline is. I may try to see how fast snapshotting would be as a next step, but so far it seems pretty difficult to get yarn into a snapshottable form.
Reacted by Darshan Sen@kvakil thanks for sharing the numbers! I've submitted a PR for this - nodejs/node#48191.
- added a commit that references this issue
on Jul 21, 2023 - added a commit that references this issue
on Jul 26, 2023 - added a commit that references this issue
on Jul 27, 2023 - added 2 commits that reference this issue
on Aug 14, 2023 - added a commit that references this issue
on Aug 14, 2023 - added 2 commits that reference this issue
on Aug 15, 2023
Currently SEA is focused on snapshotting. However it can be difficult to make a snapshot because (1) not everything is supported and (2) it can be hard for the developer to disentangle the various stages of startup into "snapshottable" and "not snapshottable" parts.
On the other hand V8 bytecode caching always works. In my testing bytecode caching can speedup initialization of some common Node executables (like
yarn,npm) by ~10%. This is a pretty modest improvement but it's nearly free.I think it would be interesting if SEA could also support "bytecode-only" caching, where:
This also has minor benefits for memory usage as the source code memory can be shared across different executables.
Alternatives:
require) and also is slower than it could be (since user-spacerequireis pretty slow).__pycache__, but that's sort of disjoint from this.)