镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

Re-enabling V8 snapshots #14171

Description

@ofrobots

As part of the July 11 2017 security release we disabled V8 snapshots to mitigate hash flooding attacks against Node servers. The problem is that the snapshot is built at build time – and whatever the hash seed got used at that time gets baked into that particular Node.js binary.

Disabling snapshots has some negative performance & memory footprint consequences for code that heavily relies on creating lots of V8 contexts (e.g. via vm.runInNewContext). Startup time might also be negatively affected (although this should not be substantial).

This issue is for discussing a way to getting V8 snapshots enabled back again. There are some alternatives that have been proposed:

  1. When the snapshot is deserialized, generate a new hash seed and then rehash all hash tables in the snapshot (including all dictionary mode objects).
  2. Hook into the Node.js install or startup process to periodically 'refresh' the snapshot blob on disk. This has ergonomic issues – requires modification of installers, and it may not be possible to write to disk in all environments.
  3. Generate a new snapshot on each startup and re-use it for future contexts. This will help address performance of vm.runInNewContext but will not help with the default startup time.
  4. Modify the V8 object model so that each hash table has its own seed. This will have performance consequences even for code that doesn't need multiple contexts.

/cc @nodejs/v8 @nodejs/ctc

Activity

  1. refack commented on Jul 11, 2017

    @refack
    Contributor

    IMHO (1) seems to make most sense.
    (2) breaks many installation stories that don't depend on install/version manager (specifically mine 😞, that is to just download the exe)
    (3) seems like a compromised version of (1)
    (4) might make sense anyway, but AFAICT on it's own does not mitigate the vulnerability, since all tables stored in the snapshot will still have known seeds.

  2. ofrobots commented on Jul 11, 2017

    @ofrobots
    ContributorAuthor

    (4) might make sense anyway, but AFAICT on it's own does not mitigate the vulnerability, since all tables stored in the snapshot will still have known seeds.

    To clarify, I am proposing that each hash table has its own hash seed rather than having a global hash seed that gets used for all hash tables. This will address vulnerability; in fact, it would provide even stronger protection against hash flooding attacks. This would be quite a substantial undertaking on the VM side though.

  3. hashseed commented on Jul 11, 2017

    @hashseed
    Member

    With (4) every hash table instance would have its own hash seed, which is random. This local hash seed is not constant, and knowing the random seed of one particular instance does not affect other instances.

  4. hashseed commented on Jul 11, 2017

    @hashseed
    Member

    (4) is also least likely to be backportable. I also favor (1).

  5. refack commented on Jul 11, 2017

    @refack
    Contributor

    To clarify, I am proposing that each hash table has its own hash seed rather than having a global hash seed that gets used for all hash tables. This will address vulnerability; in fact, it would provide even stronger protection against hash flooding attacks. This would be quite a substantial undertaking on the VM side though.

    With (4) every hash table instance would have its own hash seed, which is random. This local hash seed is not constant, and knowing the random seed of one particular instance does not affect other instances.

    But it will leave all the tables that are "frozen" in the snapshot with knowable seeds thus compromisable. If none of them are important, theoretically you could just have two seeds: one for the defrosted tables and a fresh one for everything else

  6. refack commented on Jul 11, 2017

    @refack
    Contributor

    @hashseed your username has never been so relevant 😄

  7. hashseed commented on Jul 11, 2017

    @hashseed
    Member

    :)

    You are right in that the deserialized tables would be compromised. We probably don't want that.

  8. ofrobots commented on Jul 11, 2017

    @ofrobots
    ContributorAuthor

    It sounds like for 4) we need 1) regardless.

  9. added
    discussIssues opened for discussion and feedback.
    v8 engineIssues and PRs related to the V8 dependency.
    on Jul 11, 2017
  10. targos commented on Jul 11, 2017

    @targos
    Member

    How does the performance hit of rehashing the tables at startup compares to the current workaround?

  11. hashseed commented on Jul 11, 2017

    @hashseed
    Member

    Rehashing would be a lot faster than bootstrapping from scratch.

    A variation of that could be lazily rehashing: upon deserialization, we mark all hash tables. Marked hash tables use the old hash seed. Once a table expands, it is rehashed anyways (iirc). That's when we would use the new hash seed.
    That should work as mitigation because DOS attack is based on adding entries to the hash table to provoke hash collisions, which eventually would lead to expanding the table.

  12. refack commented on Jul 11, 2017

    @refack
    Contributor

    startup compares to the current workaround?

    Just some quick anecdotal numbers from Windows x64
    50 X node813 -e "process._rawDebug('.')" => 9.57s => ~0.2s per instance
    50 X node814 -e "process._rawDebug('.')" => 19.17s => ~0.6s per instance
    ~200% slowdown (but still relatively fast)

  13. hashseed commented on Jul 11, 2017

    @hashseed
    Member

    That's probably helped by the fact that we have migrated a lot of the builtins away from being self hosted in JS. That reduces the time spent in bootstrapping a context from scratch.

  14. mhdawson commented on Jul 11, 2017

    @mhdawson
    Member

    The lazily rehashing sounds like a good way to only pay the price or rehashing when its actually needed.

  15. 32 remaining items

  16. BorisKozo commented on Jul 31, 2017

    @BorisKozo

    @ofrobots Hi, our system (not internet facing) creates lots of V8 contexts :).
    We recently upgraded from 8.0.0 to 8.1.4 and now the initialization phase of our system, where those contexts are created, changed from ~2 seconds to ~8 minutes. Is there some workaround to creating the snapshots we can use in the meantime because what we are doing now is compiling 8.1.4 without your change and I don't think this is a good long term solution.

    We upgraded because we had some issues that were resolved in 8.1 so I guess we can try and use 8.1.3 but since we are doing some low level v8 things we would rather to stay updated with the versions and not stay on a particular version or compile our own fork.

  17. hashseed commented on Jul 31, 2017

    @hashseed
    Member

    @BorisKozo if you don't expect hash flooding attacks to be an issue in your use case, that's a valid fix. Otherwise you could upgrade to 8.1.4 and cherry-pick this fix in addition.

  18. refack commented on Jul 31, 2017

    @refack
    Contributor

    We upgraded because we had some issues that were resolved in 8.1 so I guess we can try and use 8.1.3 but since we are doing some low level v8 things we would rather to stay updated with the versions and not stay on a particular version or compile our own fork.

    @BorisKozo, I'm with @hashseed, using 8.1.3 until 8.3.0 goes out seems like a valid solution.
    Another solution would be using the latest release source tarball but building it with ./configure --with-snapshots, so it's not a fork, just a tweaked build (we try to test with snapshots to make sure there are no regressions).

  19. BorisKozo commented on Jul 31, 2017

    @BorisKozo

    Thanks for the replies! We can wait for 8.3.0 if you can confirm that there is a fix there.

  20. refack commented on Aug 1, 2017

    @refack
    Contributor

    Thanks for the replies! We can wait for 8.3.0 if you can confirm that there is a fix there.

    @BorisKozo we try not to commit to the content of specific releases. At the moment I see no reason it won't make it in (this is the PR #14345) but things may change. It's high probability that will make it out in the next few weeks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    discussIssues opened for discussion and feedback.v8 engineIssues and PRs related to the V8 dependency.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions