Add toUint8Array to allow transferable data response e.g. between worker threads

Feature request

What are you trying to achieve?

I'm working in a multi-threaded Node.js application where sharp().toBuffer() output is passed to worker_threads via postMessage. To avoid copying overhead, I'd like to transfer the backing ArrayBuffer (i.e., use zero-copy transfers).

However, the current Buffer returned by toBuffer() may point to a shared or larger memory pool (via node::Buffer::New(...) with internal backing), making the .buffer potentially non-transferrable or containing unrelated memory slices.

Specifically you can test this by calling require('worker_threads'). isMarkedAsUntransferable(buffer) which today returns true.

When you searched for similar feature requests, what did you find that might be related?

Didn't find anything relevant to transferrable ArrayBuffer as a backing store.

What would you expect the API to look like?

Ideally, the existing toBuffer() call would already return a Buffer backed by a tightly sliced ArrayBuffer that is safe to transfer.

If not feasible as the default, a new option like:

sharp(...).toBuffer({ transferrable: true });

Images returned by sharp as Buffer objects do not come from the shared pool so should already be transferable.

nodejs/node#55593 suggests this might be a recent Node.js change/feature/bug.

Can you point me to the place where those buffers are created in sharp? Also looked at this comment which may indicate that there is a pooling behavior: nodejs/node#55593 (comment)

libvips (or one of its dependencies) allocates the memory, sharp uses NewOrCopy to wrap it as a Buffer instance.

sharp/src/pipeline.cc

Line 1355 in 031c808

    
           Napi::Buffer<char> data = Napi::Buffer<char>::NewOrCopy(env, static_cast<char*>(baton->bufferOut),

For Node.js this is a zero-copy operation. For recent versions of Electron that enable its memory cage feature, this involves a copy, which I guess might potentially involve an allocation from the pool.

Thanks for sharing the code sample. While it may be zero-copy for Node.js, the resulting buffer is still marked as untransferable.

To address this without introducing a copy, we could allocate the memory ourselves and wrap it with a shared_ptr:

std::shared_ptr<char> dataPtr(new char[baton->bufferOutLength], [](char* data) {
  delete[] data;
});
baton->bufferOut = dataPtr.get();

Napi::Buffer<char> data = Napi::Buffer<char>::New(
  env,
  dataPtr.get(),
  baton->bufferOutLength,
  [](Napi::Env, char*, std::shared_ptr<char>* hint) {
    delete hint;
  },
  new std::shared_ptr<char>(std::move(dataPtr))
);

(note: I haven't used Napi myself, but a similar approach works for Nan)

This avoids pooled memory, ensures transferability, and retains zero-copy. It would be great to have this as the default behavior.

Not sure how this behaves under Electron, so I’m open to thoughts there. Also, I believe the enhancement label should stay—this is a real inefficiency for multi-threaded applications.

This still feels like something that needs to be corrected at the Node.js and/or Node-API level. If these are marking externally-allocated and therefore transferable Buffers as untransferable then I think that's a better place to deal with the cause, rather than attempt to deal with the effects everywhere else.

I think Node.JS behavior is both correct and expected — digging into node_buffer.cc, I think the behavior here does relate directly to how memory ownership is tracked for external ArrayBuffer backing stores.

In particular, whether a buffer is transferable or not depends on whether Node can prove that its memory is externally owned and properly finalizable.

In CallbackInfo::CreateTrackedArrayBuffer() (see here), Node constructs the BackingStore using:

std::unique_ptr<BackingStore> bs =
    ArrayBuffer::NewBackingStore(data, length,
        [](void*, size_t, void* arg) {
            static_cast<CallbackInfo*>(arg)->OnBackingStoreFree();
        }, self);

What’s key here is that Node avoids marking such ArrayBuffers as untransferable only if:
1. The backing store is created with a proper finalizer.
2. Node has full control over its deallocation via the CallbackInfo object.

If instead a Buffer is created with pooled memory or shared allocator slabs (like NewOrCopy() often does in sharp), Node can’t guarantee memory safety across threads — and (I believe) V8 errs on the side of caution, marking .buffer as untransferable.

From what I can tell, Node.js always marks all externally-allocated Buffer instances as untransferable:

https://github.com/nodejs/node/blob/004ecc12eb43c1253274728209c0790f06820437/src/node_buffer.cc#L460-L464

The call chain from sharp to here is Napi::Buffer::NewOrCopy -> Napi::Buffer::New -> napi_create_external_buffer -> node::Buffer::New. A finalizer callback is provided all the way upstream.

Perhaps a new "this data is transferable" option exposed at the napi_create_external_buffer level (and then exposed downstream) would help solve this?

Hi all,

I've also encountered this issue and created a minimal repository to reproduce the problem of transferring sharp's Buffer (specifically, its underlying ArrayBuffer) between threads.

Repo: https://github.com/shuizhongyueming/piscina-and-sharp

In my tests:

On Node.js v20.x, the ArrayBuffer from sharp().toBuffer() can be transferred successfully.
On Node.js v22.x, attempting the same transfer results in a DataCloneError: Cannot transfer object of unsupported type.

The repository includes examples using both plain worker_threads and Piscina to demonstrate this. Hope this helps with diagnosing the issue!

Add toArrayBuffer to generate transferable output #4355

I've investigated this more deeply and from what I can tell, Node.js' Buffer instances are always going to have the untransferable flag set, regardless of where the underlying data came from. However, by providing data as more portable ECMAScript ArrayBuffer instances, this problem appears to go away.

Ultimately I think moving away from, or at least a soft-deprecation of, the use of Buffer towards providing data as both/either ArrayBuffer and SharedArrayBuffer will be the most future-proof approach.

UPDATE: Thinking about this more, if we're going to add new API then I'd like to go a bit further and provide TypedArray instances. Most scenarios will result in Uint8Array, except for raw() output where the type will match the bit depth. So this means providing something like toUint8Array() and toTypedArray<T>(). This approach will unlock features like easy access to the #Uint8Array.toBase64() method, and I suspect works better with fetch() etc.

Commit dbcb7e6 adds toUint8Array(), which returns a TypedArray backed by a transferable ArrayBuffer.

Add toUint8Array for output backed by transferable ArrayBuffer #4355

A pre-release with the new toUint8Array() feature is now available should anyone like to test this.

npm i sharp@0.35.0-rc.0

just tested toUint8Array() and 0.35.0-rc.0. getting

#
# Fatal error in HandleScope::HandleScope
# Entering the V8 API without proper locking in place
#

in our E2E tests.

with toBuffer and stable sharp, no issue,

Thanks for testing, that's an interesting error, are you able to share a minimal code sample that reliably reproduces this?

I think this relates to function callbacks from C++ to JS, and there's been some back and forth on the right approach for these; I suspect MakeCallback is worth re-testing.

Sorry to bother but is there any updates on this? I just moved from piping ImageMagick through execSync and so far this is the only minor hurdle.

Latest release candidate now available for testing purposes:

npm install sharp@0.35.0-rc.2

v0.35.0 now available with toUint8Array()

https://sharp.pixelplumbing.com/api-output/#touint8array

chore(deps): update dependency sharp to ^0.35.5 (#9)

Navigation Menu

Repository navigation

Add toUint8Array to allow transferable data response e.g. between worker threads #4355

Feature request

What are you trying to achieve?

When you searched for similar feature requests, what did you find that might be related?

What would you expect the API to look like?

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Add toUint8Array to allow transferable data response e.g. between worker threads #4355

Description

Feature request

What are you trying to achieve?

When you searched for similar feature requests, what did you find that might be related?

What would you expect the API to look like?

Activity

lovell commented on Mar 28, 2025

Rush commented on Mar 28, 2025

lovell commented on Mar 29, 2025

Rush commented on Apr 13, 2025

lovell commented on Apr 13, 2025

Rush commented on Apr 14, 2025

lovell commented on Apr 14, 2025

shuizhongyueming commented on May 13, 2025

lovell commented on Dec 22, 2025

lovell commented on Jan 4, 2026

Rush commented on Feb 13, 2026

lovell commented on Feb 13, 2026

james-pre commented on Apr 3, 2026

lovell commented on Apr 6, 2026

lovell commented on Jun 10, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions