Interoperability between the v8::External and Napi::External is broken #58480

Version

20.x 22.x

Platform

any

Subsystem

No response

What steps will reproduce the bug?

With the merge of PR-51149 the interoperability between the v8::External and the Napi::External API usage is broken.
Create a v8::External value using the v8-API (e.g. from the NodeJS embedder C++ API) and later retrieve the value from an native addon API base Napi::External will not result in the same native pointer.
This way it is no longer possible to provide any native data reference from the embedder API to a native addon function.

How often does it reproduce? Is there a required condition?

It is always happens. Simply create a External value based on one of the APIs and retrieve it using the other API.
E.g. create a External value from the embedder functions:

  auto cpp_object_key = v8::String::NewFromUtf8(isolate, "__my_cpp_object").ToLocalChecked();
  if (!global->Set(setup_->context(), cpp_object_key, v8::External::New(isolate, this)).FromMaybe(false)) {
    // Error handling
  }

Retrieve the pointer later from a native addon function:

  Script *script =
      env.Global().Get("__my_cpp_object")).As<Napi::External<Script>>().Data();
  if (script == nullptr) {
    // error handling
  }

What is the expected behavior? Why is that the expected behavior?

The expected behavior would be the same native pointer is given back. This worked until Node version 18.x.

With the current implementation it is no longer possible to exchange a nativ C++ pointer between code written based on the v8 API and node addon API.

In my personal view the design issue starts with the v8::External being a simple value in the V8-API. And being a TypeTaggable derived type in the addon API. With the current implementation there is no counterpart to the v8::External value type in the addon API anymore.

What do you see instead?

Creating a v8::External value and retrieve it based on the Napi::External we get a totally different pointer. The Napi::External expects the pointer to point to the internal class Napi::ExternalWrapper. But the v8::External API does not create the Napi::ExternalWrapper instance.

Additional information

We use the the Nodejs embedder API to embedd a nodejs engine into our C++ based application. As there is no node addon embedder API we have to use the node/v8 api in setting up the embedded NodeJS engine.
Additionally we provide some native addon functions to provide access to to data from the surrounding process. As we have multiple NodeJS engines running inside our process we need to find a way to provide access to the inividual C++ object creating a nodeJS instance each. Until node 18.x we used an v8::External value created in the global from the embedder function and were able to retrieve this pointer from within the addon API functions.

Workaround we use now:
The only workaround we found now is to create an v8::ArrayBuffer with the size of the native pointer, Copy the pointer into the arrray buffer. Create a v8::Uint8Array and add this array to global. In this code we have our class Script creating the embedded NodeJS engine.

  auto array_buffer = v8::ArrayBuffer::New(isolate, sizeof(Script *));
  Script *this_ptr{this};
  memcpy(array_buffer->Data(), &this_ptr, sizeof(Script *));

  auto global = setup_->context()->Global();
  auto cpp_object_key = v8::String::NewFromUtf8(isolate, "__my_cpp_script_object").ToLocalChecked();
  auto uint8_array = v8::Uint8Array::New(array_buffer, 0, sizeof(Script *));
  if (!global->Set(setup_->context(), cpp_object_key, uint8_array).FromMaybe(false)) {
    // Error handling
  }

And in the native addon function copy the pointer back from the Array to get access to the C++ Script instance:

  auto global = env.Global();
  
  auto uint8_array = global.Get("__my_cpp_script_object").As<Napi::Uint8Array>();
  Script *script{nullptr};
  memcpy(&script, uint8_array.Data(), sizeof(Script *));