镜像站点 · 本页由第三方 GitHub 只读镜像提供,非 GitHub 官方站点,不接受任何登录或凭据输入。前往 github.com
Skip to content

docs: writable streams from async iterators example issue #31222

Description

@ronag

Given the example from https://nodejs.org/api/stream.html#stream_piping_to_writable_streams_from_async_iterators.

There is a problem where this can cause an unhandled exception if the writable emits an 'error' before the first chunk arrives from the iterator, e.g. if writable is a fs stream and it fails while opening the file.

  const writable = fs.createWriteStream('./file');

  for await (const chunk of iterator) {
    // Handle backpressure on write().
    if (!writable.write(chunk))
      await once(writable, 'drain');
  }
  writable.end();
  // Ensure completion without errors.
  await finished(writable);

Activity

  1. ronag commented on Jan 6, 2020

    @ronag
    MemberAuthor

    @mcollina / @nodejs/streams: I can't think of a elegant way to resolve this. Any ideas?

    Otherwise, I would suggest we remove this example until we find a better way in order to avoid people using this pattern and thinking it's safe.

  2. ronag commented on Jan 6, 2020

    @ronag
    MemberAuthor

    Maybe?

    EDIT: Or rather maybe not :D

      const writable = fs.createWriteStream('./file');
    
      // catch 'error' while not writing
      // EDIT: add a noop .catch to avoid unhandledRejection before awaiting.
      const finishedPromise = finished(writable).catch(() => {}); 
      for await (const chunk of iterator) {
        if (writable.destroyed) // assumes destroy() on 'error'
          break;
    
        // Handle backpressure on write().
        if (!writable.write(chunk))
          await once(writable, 'drain');
      }
      writable.end();
      // Ensure completion without errors.
      await finishedPromise;
  3. himself65 commented on Jan 6, 2020

    @himself65
    Member

    what is the const finishedPromise = finished(writable); meaning? 🤔

  4. ronag commented on Jan 6, 2020

    @ronag
    MemberAuthor

    what is the const finishedPromise = finished(writable); meaning? 🤔

    it returns a promise that will resolve or reject when the writable finishes or errors.

    Actually, that won't work either since we currently do unhandled rejection for rejected promises without a catch.

  5. ronag commented on Jan 6, 2020

    @ronag
    MemberAuthor

    @himself65: "Fixed". It's not good though. Maybe not that one.

  6. ronag commented on Jan 6, 2020

    @ronag
    MemberAuthor

    Another try. What if we use some kind of event that signals that no error can occur until a write is issued?

      const writable = fs.createWriteStream('./file');
    
      await once(writable, 'ready');
      // No error possible until writable.write
    
      for await (const chunk of iterator) {
        // Handle backpressure on write().
        if (!writable.write(chunk))
          await once(writable, 'drain');
      }
      writable.end();
      // Ensure completion without errors.
      await finished(writable).catch(() => {}); 

    Though unfortunately, that does not exist currently. Probably also not possible to enforce since one can call .destroy(err) at any time. I don't like this either...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions