Promise.resolve('import("node:fs")').then(eval) throws TypeError: Invalid host defined options

Version

v20.7.0

Platform

Linux PIG-2016 5.15.90.1-microsoft-standard-WSL2 #1 SMP Fri Jan 27 02:56:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

esm

What steps will reproduce the bug?

node -e 'Promise.resolve(`import("node:fs")`).then(eval)'

How often does it reproduce? Is there a required condition?

No response

What is the expected behavior? Why is that the expected behavior?

it shouldn't throw.

What do you see instead?

these DONT throw

node -e 'Promise.resolve(`import("node:fs")`).then(x=>globalThis.eval(x))'
node -e 'Promise.resolve(`import("node:fs")`).then(x=>(0,eval)(x))'
node -e 'Promise.resolve(`34`).then(eval)'
node -e 'Promise.resolve().then(eval)'
node -e 'eval(`import("node:fs")`)'
node -e 'globalThis.eval(`import("node:fs")`)'
node -e '(0,eval)(`import("node:fs")`)'
node -e '`import("node:fs")`.replace(/^.*$/s,eval)'

these DO throw

node -e 'Promise.resolve(`import("node:fs")`).then(eval)'
node -e 'Promise.resolve(`import("node:fs")`).then((0,eval))'
node -e 'Promise.resolve(`import("node:fs")`).then(globalThis.eval)'

jcbhmr@PIG-2016:~$ node
Welcome to Node.js v20.7.0.
Type ".help" for more information.
> Promise.resolve('import("node:fs")').then(eval)
Promise {
  <pending>,
  [Symbol(async_id_symbol)]: 234,
  [Symbol(trigger_async_id_symbol)]: 233
}
> Uncaught TypeError: Invalid host defined options
    at eval (eval at processTicksAndRejections (node:internal/process/task_queues:95:5), <anonymous>:1:1)
    at eval (<anonymous>)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
>

Additional information

deno version of the same thing DOESN'T throw:

jcbhmr@PIG-2016:~$ deno
Deno 1.36.3
exit using ctrl+d, ctrl+c, or close()
REPL is running with all permissions allowed.
To specify permissions, run `deno repl` with allow flags.
> Promise.resolve('import("node:fs")').then(eval)
Promise {
  [Module: null prototype] {
    Dir: [class Dir],
    Dirent: [class Dirent],
    F_OK: 0,
    O_APPEND: 8,
    O_CREAT: 512,
    O_DIRECTORY: 1048576,
    O_DSYNC: 4194304,
    O_EXCL: 2048,
    O_NOCTTY: 131072,
...

Can you copy paste the output of your console instead of sending screen shots? Screenshots can be very unhelpful for e.g. folks using screen readers and/or folks with visual disabilities.

The following works on most¹ runtimes but Node.js:

Promise.resolve(`import("data:text/javascript,")`).then(eval).then(console.log, console.error)

I'd like to say all, but there are so many runtimes these days I surely missed at least one ↩

The two spots I could find from a GitHub search for "Invalid host defined options" are these:

node/src/module_wrap.cc

Lines 572 to 576 in b64f620

    
           resolver 
        
               ->Reject(context, 
        
                        v8::Exception::TypeError(FIXED_ONE_BYTE_STRING( 
        
                            context->GetIsolate(), "Invalid host defined options"))) 
        
               .ToChecked();

node/deps/v8/src/d8/d8.cc

Lines 1252 to 1255 in b64f620

    
           resolver 
        
               ->Reject(context, v8::Exception::TypeError(String::NewFromUtf8Literal( 
        
                                     isolate, "Invalid host defined options"))) 
        
               .ToChecked();

The following check is failing (options->Length() is supposed to be 9, it's 0 here):

node/src/module_wrap.cc

Lines 568 to 569 in 26c8858

    
           Local<FixedArray> options = host_defined_options.As<FixedArray>(); 
        
           if (options->Length() != HostDefinedOptions::kLength) {

Here's a backtrace (maybe I need a debug build):

$ echo 'setImmediate(eval.bind(null, `import(null)`)' > entry.js
$ lldb ./node entry.js
(lldb) b module_wrap.cc:568
Breakpoint 1: where = node`node::loader::ImportModuleDynamically(v8::Local<v8::Context>, v8::Local<v8::Data>, v8::Local<v8::Value>, v8::Local<v8::String>, v8::Local<v8::FixedArray>) + 160 at module_wrap.cc:568:52, address = 0x00000001001cb534
(lldb) r
Process 88426 launched: '/Users/duhamean/Documents/node/node' (arm64)
Process 88426 stopped
Target 0: (node) stopped.
(lldb) n
Process 88426 stopped
(lldb) call options->Length()
(int) $0 = 0
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
  * frame #0: 0x00000001001cb534 node`node::loader::ImportModuleDynamically(context=(val_ = 0x000000010600ff38), host_defined_options=(val_ = 0x000000010600ff40), resource_name=(val_ = 0x000000010600ff48), specifier=(val_ = 0x000000016fdfdeb0), import_assertions=(val_ = 0x00000001080082d8)) at module_wrap.cc:568:52
    frame #1: 0x000000010077373c node`v8::internal::Isolate::RunHostImportModuleDynamicallyCallback(this=0x0000000108008000, maybe_referrer=<unavailable>, specifier=<unavailable>, maybe_import_assertions_argument=<unavailable>) at isolate.cc:5123:5 [opt]
    frame #2: 0x0000000100bd02ac node`v8::internal::Runtime_DynamicImportCall(int, unsigned long*, v8::internal::Isolate*) at runtime-module.cc:38:3 [opt]
    frame #3: 0x0000000100bd01d8 node`v8::internal::Runtime_DynamicImportCall(args_length=2, args_object=<unavailable>, isolate=0x0000000108008000) at runtime-module.cc:24:1 [opt]
    frame #4: 0x0000000100f3ca1c node`Builtins_DeoptimizationEntry_Eager + 608796
    frame #5: 0x0000000100fefc3c node`Builtins_DeoptimizationEntry_Eager + 1342524
    frame #6: 0x0000000100eb43e4 node`Builtins_DeoptimizationEntry_Eager + 50148
    frame #7: 0x0000000100eb250c node`Builtins_DeoptimizationEntry_Eager + 42252
    frame #8: 0x0000000100eb21f4 node`Builtins_DeoptimizationEntry_Eager + 41460
    frame #9: 0x0000000100756e38 node`v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [inlined] v8::internal::GeneratedCode<unsigned long, unsigned long, unsigned long, unsigned long, unsigned long, long, unsigned long**>::Call(this=<unavailable>, args=<unavailable>, args=<unavailable>, args=<unavailable>, args=<unavailable>, args=<unavailable>, args=<unavailable>) at simulator.h:154:12 [opt]
    frame #10: 0x0000000100756e34 node`v8::internal::(anonymous namespace)::Invoke(isolate=<unavailable>, params=<unavailable>) at execution.cc:427:33 [opt]
    frame #11: 0x00000001007566c8 node`v8::internal::Execution::Call(isolate=0x0000000108008000, callable=<unavailable>, receiver=<unavailable>, argc=0, argv=0x0000000000000000) at execution.cc:529:10 [opt]
    frame #12: 0x000000010066b5e0 node`v8::internal::Builtin_GlobalEval(int, unsigned long*, v8::internal::Isolate*) at builtins-global.cc:110:3 [opt]
    frame #13: 0x000000010066b51c node`v8::internal::Builtin_GlobalEval(args_length=<unavailable>, args_object=<unavailable>, isolate=0x0000000108008000) at builtins-global.cc:84:1 [opt]
    frame #14: 0x0000000100f3cb24 node`Builtins_DeoptimizationEntry_Eager + 609060
    frame #15: 0x0000000100f98fb8 node`Builtins_DeoptimizationEntry_Eager + 987064
    frame #16: 0x0000000100edab94 node`Builtins_DeoptimizationEntry_Eager + 207764
    frame #17: 0x0000000100eb23f4 node`Builtins_DeoptimizationEntry_Eager + 41972
    frame #18: 0x0000000100756df0 node`v8::internal::(anonymous namespace)::Invoke(v8::internal::Isolate*, v8::internal::(anonymous namespace)::InvokeParams const&) [inlined] v8::internal::GeneratedCode<unsigned long, unsigned long, v8::internal::MicrotaskQueue*>::Call(this=<unavailable>, args=<unavailable>, args=<unavailable>) at simulator.h:154:12 [opt]
    frame #19: 0x0000000100756dec node`v8::internal::(anonymous namespace)::Invoke(isolate=<unavailable>, params=<unavailable>) at execution.cc:443:33 [opt]
    frame #20: 0x0000000100757738 node`v8::internal::(anonymous namespace)::InvokeWithTryCatch(isolate=0x0000000108008000, params=0x000000016fdfe5f0) at execution.cc:490:20 [opt]
    frame #21: 0x0000000100757924 node`v8::internal::Execution::TryRunMicrotasks(isolate=<unavailable>, microtask_queue=<unavailable>) at execution.cc:601:10 [opt]
    frame #22: 0x000000010078110c node`v8::internal::MicrotaskQueue::RunMicrotasks(this=0x0000000104a07400, isolate=0x0000000108008000) at microtask-queue.cc:174:22 [opt]
    frame #23: 0x000000010078190c node`v8::internal::MicrotaskQueue::PerformCheckpoint(v8::Isolate*) [inlined] v8::internal::MicrotaskQueue::PerformCheckpointInternal(this=0x0000000104a07400, v8_isolate=0x0000000108008000) at microtask-queue.cc:126:3 [opt]
    frame #24: 0x00000001007818d0 node`v8::internal::MicrotaskQueue::PerformCheckpoint(this=0x0000000104a07400, isolate=0x0000000108008000) at microtask-queue.h:46:5 [opt]
    frame #25: 0x000000010006f1b0 node`node::InternalCallbackScope::Close(this=0x000000016fdfea90) at callback.cc:137:35
    frame #26: 0x000000010006efcc node`node::InternalCallbackScope::~InternalCallbackScope(this=0x000000016fdfea90) at callback.cc:92:3
    frame #27: 0x000000010006eae4 node`node::InternalCallbackScope::~InternalCallbackScope(this=0x000000016fdfea90) at callback.cc:91:49
    frame #28: 0x00000001001d5cbc node`node::StartExecution(env=0x0000000105815e00, cb=node::StartExecutionCallback @ 0x000000016fdfeb28) at node.cc:372:1
    frame #29: 0x000000010007d478 node`node::LoadEnvironment(env=0x0000000105815e00, cb=node::StartExecutionCallback @ 0x000000016fdfec28) at environment.cc:547:10
    frame #30: 0x00000001002e3c30 node`node::NodeMainInstance::Run(this=0x000000016fdfed60, exit_code=0x000000016fdfecb4, env=0x0000000105815e00) at node_main_instance.cc:108:7
    frame #31: 0x00000001002e386c node`node::NodeMainInstance::Run(this=0x000000016fdfed60) at node_main_instance.cc:88:3
    frame #32: 0x00000001001d84b0 node`node::StartInternal(argc=2, argv=0x00000001049286c0) at node.cc:1369:24
    frame #33: 0x00000001001d80f8 node`node::Start(argc=2, argv=0x000000016fdff170) at node.cc:1376:27
    frame #34: 0x00000001012831c4 node`main(argc=2, argv=0x000000016fdff170) at node_main.cc:97:10
    frame #35: 0x00000001a3663f28 dyld`start + 2236
(lldb) c
TypeError: Invalid host defined options
    at eval (eval at <anonymous> (unknown source), <anonymous>:1:1)
    at eval (<anonymous>)
Process 88426 exited with status = 0 (0x00000000)

host_defined_options is empty because of this condition:

node/deps/v8/src/execution/isolate.cc

Lines 5113 to 5116 in c2cd744

    
           if (maybe_referrer.is_null()) { 
        
             host_defined_options = factory()->empty_fixed_array(); 
        
             resource_name = factory()->null_value(); 
        
           } else {

maybe_referrer comes from here:

node/deps/v8/src/runtime/runtime-module.cc

Lines 36 to 37 in c2cd744

    
           Handle<Script> referrer_script = 
        
               GetEvalOrigin(isolate, Script::cast(function->shared().script()));

This is likely a V8 issue, I can also reproduce it with d8. The problem probably comes from that V8 is unable to retrieve the referrer/origin when code is eval'ed directly from a microtask - if you wrap it in an anonymous function V8 is going to generate bytecode that passes the closure to a runtime call so that it can figure out the script and then the referrer/origin. If it's just eval directly with no JS frames above V8 uses an empty script as the wrapper and cannot infer host-defined options.

Also this does not only affect host-defined options, for example V8 would not be able to show the script name if an error is thrown from code being eval'ed directly from a microstask because of the same cause. I'll open an upstream issue.

Upstream issue https://bugs.chromium.org/p/v8/issues/detail?id=14336

@joyeecheung

@joyeecheung naive question: if it's a V8 problem then why doesn't Deno (also V8-based) have this issue? 🤔 is deno doing some magic or something?

jcbhmr@PIG-2016:~$ deno
Deno 1.37.0
exit using ctrl+d, ctrl+c, or close()
REPL is running with all permissions allowed.
To specify permissions, run `deno repl` with allow flags.
> Promise.resolve('import("node:fs")').then(eval).then(console.log)
[Module: null prototype] {
  Dir: [class Dir],
  Dirent: [class Dirent],
...

FWIW Chromium is also not affected. Even if it is not a V8 issue in the end, the patch that will fix d8 will probably contain all the needed info to fix it in Node.js, so I think our best course of action is to wait for V8 team for a fix.

I think the answer to whether we should fallback comes down to:

Do we think falling back should be by-design - at least as far as what the spec currently says, that would not be non-conformant. On the other hand, throwing is probably not non-conformant either. This is ultimately implementation-defined.
If we should fall back, what is the base we should fallback to?
Should scripts/modules compiled using vm methods + importModuleDynamically fallback in this case too? That would make escaping the configured hook possible, so at least warrants a note in the doc. This may also apply to custom loaders @nodejs/loaders

On the other hand, eval function call from a microtask sets the current execution context to be null

Actually I think the editor notes does not apply to microtasks, from https://262.ecma-international.org/14.0/#sec-hostenqueuepromisejob the implementation should restore the active script from when the microtask is queued

Let scriptOrModule be GetActiveScriptOrModule() at the time HostEnqueuePromiseJob is invoked. If realm is not null, each time job is invoked the implementation must perform implementation-defined steps such that scriptOrModule is the active script or module at the time of job's invocation.

@joyeecheung

On the other hand, eval function call from a microtask sets the current execution context to be null

Actually I think the editor notes does not apply to microtasks, from https://262.ecma-international.org/14.0/#sec-hostenqueuepromisejob the implementation should restore the active script from when the microtask is queued

Let scriptOrModule be GetActiveScriptOrModule() at the time HostEnqueuePromiseJob is invoked. If realm is not null, each time job is invoked the implementation must perform implementation-defined steps such that scriptOrModule is the active script or module at the time of job's invocation.

@joyeecheung this is interesting. I tested on Chrome 117, Firefox 119, and Safari 17, none of them respected this statement. They resolve eval import calls relative to the origin of the document, rather than the initiating module. I believe either the ecma262 has to be corrected to reflect the web reality, or the browsers have to fix the behavior.

I’m confused; why would anyone expect this to work? This would attempt to eval the toString of the module namespace object, and you simply can’t assume eval of any function will be portable.

@ljharb

@ljharb it's not about eval-ing the toString result of a module namespace. The problem here is how to resolve the specifier when the eval function is called with a string script containing dynamic imports and the eval function is invoked by built-in callbacks directly.

I just noticed the HTML spec deals with this exact situation in https://html.spec.whatwg.org/multipage/webappapis.html#hostmakejobcallback

Interesting, apparently WHATWG spec is explicitly saying that in this case the referrer should be the original script (and, on the contrary, if it's from a onclick handler, there's no original script), while browsers don't actually implement that behavior..

@joyeecheung

@joyeecheung they took this to TC39 at one point but TC39 members had some concerns about the behavior because it could lead to an exploit by giving extra permissions to the eval'd string

From July 2020: https://docs.google.com/presentation/d/19S97ZqhibJABqzeP5ZU6Flk6TVgWzXuvJWFbNTTfpWs/edit#slide=id.p

Submitted tc39/ecma262#3195 to propagate active scripts with arbitrary microtask callbacks (like FinalizationRegistry callbacks) in ecma262.

vm: allow dynamic import with a referrer realm

Navigation Menu

Repository navigation

`Promise.resolve('import("node:fs")').then(eval)` throws `TypeError: Invalid host defined options` #49726

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior? Why is that the expected behavior?

What do you see instead?

Additional information

6 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	resolver
	->Reject(context,
	v8::Exception::TypeError(FIXED_ONE_BYTE_STRING(
	context->GetIsolate(), "Invalid host defined options")))
	.ToChecked();

	resolver
	->Reject(context, v8::Exception::TypeError(String::NewFromUtf8Literal(
	isolate, "Invalid host defined options")))
	.ToChecked();

	Local<FixedArray> options = host_defined_options.As<FixedArray>();
	if (options->Length() != HostDefinedOptions::kLength) {

	if (maybe_referrer.is_null()) {
	host_defined_options = factory()->empty_fixed_array();
	resource_name = factory()->null_value();
	} else {

	Handle<Script> referrer_script =
	GetEvalOrigin(isolate, Script::cast(function->shared().script()));

Promise.resolve('import("node:fs")').then(eval) throws TypeError: Invalid host defined options #49726

Description

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior? Why is that the expected behavior?

What do you see instead?

Additional information

Activity

aduh95 commented on Sep 20, 2023

Footnotes

jcbhmr commented on Sep 20, 2023

jcbhmr commented on Sep 20, 2023

jcbhmr commented on Sep 20, 2023

aduh95 commented on Sep 21, 2023

targos commented on Sep 22, 2023

joyeecheung commented on Sep 22, 2023

joyeecheung commented on Sep 22, 2023

joyeecheung commented on Sep 22, 2023

jcbhmr commented on Sep 22, 2023

aduh95 commented on Sep 22, 2023

6 remaining items

joyeecheung commented on Sep 26, 2023

joyeecheung commented on Sep 27, 2023

legendecas commented on Oct 9, 2023

ljharb commented on Oct 9, 2023

legendecas commented on Oct 9, 2023

aduh95 commented on Oct 10, 2023

joyeecheung commented on Oct 11, 2023

bmeck commented on Oct 11, 2023

legendecas commented on Oct 13, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

`Promise.resolve('import("node:fs")').then(eval)` throws `TypeError: Invalid host defined options` #49726