node should execute files without extension as ESM if "type": "module", is specified

Version: v14.4.0
Platform: The node:14 docker container

What steps will reproduce the bug?

I'm creating a file called foo (without extension) at /app/bin/foo. I have a /app/package.json file with "type": "module",.
The foo file has a #!/usr/bin/env node shebang and /app/bin is in the PATH.

Now I want to run foo.

What is the expected behavior?

I would expect node to execute the file as ESM because of the "type": "module", in the parents folder package.json.

What do you see instead?

I get the following error:

internal/modules/run_main.js:54
    internalBinding('errors').triggerUncaughtException(
                              ^

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension "" for /app/bin/foo
    at Loader.defaultGetFormat [as _getFormat] (internal/modules/esm/get_format.js:65:15)
    at Loader.getFormat (internal/modules/esm/loader.js:113:42)
    at Loader.getModuleJob (internal/modules/esm/loader.js:243:31)
    at async Loader.import (internal/modules/esm/loader.js:177:17) {
  code: 'ERR_UNKNOWN_FILE_EXTENSION'
}

I assume node wants to check the file extension to see if its a .cjs extension and would require to execute it as common js. However IMHO if there is no file extension it should just respect the package.json in the parent folder and run it as ESM.

Use a dummy package.json in git hooks folder

I can't believe it's been a year and this is still not fixed. How are you supposed to use simple executable with a shebang that you want to add to $PATH? You don't want to use myprogram.js in your terminal.

Not even --input-type works (not that it would be useful for $PATH executables):

$ node --input-type module  bin/program
internal/process/esm_loader.js:74
    internalBinding('errors').triggerUncaughtException(
                              ^

Error [ERR_INPUT_TYPE_NOT_ALLOWED]: --input-type can only be used with string input via --eval, --print, or STDIN
←[90m    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:804:13)←[39m
←[90m    at Loader.resolve (internal/modules/esm/loader.js:86:40)←[39m
←[90m    at Loader.getModuleJob (internal/modules/esm/loader.js:230:28)←[39m
←[90m    at Loader.import (internal/modules/esm/loader.js:165:28)←[39m
←[90m    at internal/modules/run_main.js:46:28←[39m
←[90m    at Object.loadESM (internal/process/esm_loader.js:68:11)←[39m {
  code: ←[32m'ERR_INPUT_TYPE_NOT_ALLOWED'←[39m
}
``

I can't believe it's been a year and this is still not fixed. How are you supposed to use simple executable with a shebang that you want to add to $PATH? You don't want to use myprogram.js in your terminal.

Not even --input-type works (not that it would be useful for $PATH executables):

$ node --input-type module  bin/program
internal/process/esm_loader.js:74
    internalBinding('errors').triggerUncaughtException(
                              ^

Error [ERR_INPUT_TYPE_NOT_ALLOWED]: --input-type can only be used with string input via --eval, --print, or STDIN
←[90m    at Loader.defaultResolve [as _resolve] (internal/modules/esm/resolve.js:804:13)←[39m
←[90m    at Loader.resolve (internal/modules/esm/loader.js:86:40)←[39m
←[90m    at Loader.getModuleJob (internal/modules/esm/loader.js:230:28)←[39m
←[90m    at Loader.import (internal/modules/esm/loader.js:165:28)←[39m
←[90m    at internal/modules/run_main.js:46:28←[39m
←[90m    at Object.loadESM (internal/process/esm_loader.js:68:11)←[39m {
  code: ←[32m'ERR_INPUT_TYPE_NOT_ALLOWED'←[39m
}
``

I had exactly the same problem, try deleting repository folder and cloning it again, it helped me, or as a last resort, you can try wsl

@wsehl

@wsehl I don't think proposed solutions would help.

In an empty folder this works:

touch empty && node empty

But not in a folder with package.json containing type: module which is worse since it's explicit

$ echo '{"type": "module"}' > package.json && touch empty && node empty
internal/process/esm_loader.js:74
    internalBinding('errors').triggerUncaughtException(
                              ^

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension "" for ...\empty
←[90m    at Loader.defaultGetFormat [as _getFormat] (internal/modules/esm/get_format.js:71:15)←[39m
←[90m    at Loader.getFormat (internal/modules/esm/loader.js:102:42)←[39m
←[90m    at Loader.getModuleJob (internal/modules/esm/loader.js:231:31)←[39m
←[90m    at async Loader.import (internal/modules/esm/loader.js:165:17)←[39m
←[90m    at async Object.loadESM (internal/process/esm_loader.js:68:5)←[39m {
  code: ←[32m'ERR_UNKNOWN_FILE_EXTENSION'←[39m
}

How are you supposed to use simple executable with a shebang that you want to add to $PATH?

Here's a workaround assuming bin/program is the ES module you want executed:

cd bin
mv program program.mjs
echo '{ "type": "commonjs" }' > package.json
printf '#!/usr/bin/env node\n"use strict";import("./program.mjs");\n' > program

Then you can add it to your $PATH and don't have to use the extension in your terminal – I think the only difference is you won't get exit code 13 in case of unfinished Top-Level Await (because you can't use TLA in CJS), other than that it should behave as if bin/program.mjs was executed directly.

@aduh95

@aduh95 Thank you for the workaround. It worked 🎉

I had to add manual handling of rejections to make TLA errors in bin/program result in exit code 1:

#!/usr/bin/node

import('../src/myprogram.js').catch((err) => {
  console.error(err);
  process.exit(1);
});

However it's very sad to have to use this workaround in node and it's going to make backwards compatibility with already released node versions hard even if it's fixed.

I hope that node team will fix this soon before everyone starts using this workaround although some package maintainers that care about being compatible with all node versions will always have to.

I also encountered this problem, but I was able to work around it by adding .js. The type field of package.json is still module.

mizdra/eslint-interactive@a932cb1

(This comment is more fitting for #37512, but that one was closed as a duplicate of this, so posting it here.)

Perhaps it is best to repost this as a new issue, since I haven't since any serious discussion about the plethora of bogus workaround suggestions.

I'm surprised at the fact that writing an esm script is still such a huge pita. But this is not new, since it seems that everyone and their cats are surprised at this too.

What's also surprising is the constant flood of "workaround" hacks that are fundamentally and utterly broken (like the above, or that one). Here's a list of what I can remember:

Rename your file as *.mjs. This is not an option, since a script name should absolutely not include the language in its extension. If I give you a script called foo which is written in bash, I can later reimplement it in perl and you won't need to change your use. It should be possible to do the same with a node script.
Add a "type": "module" to your package.json. This is not an option in the same scenario: now instead of dropping my script in your ~/bin or wherever, I give you too files?? But more than that: what if you want to drop my script into a directory that already has an existing package.json? Or if my ~/bin already has some existing node scripts, I certainly don't want to break them all to run the new script.
Use --input-type=module < "$0". When you do this, it looks like it works. At some point you'll notice that error messages all refer to [eval1], but it's a small price, right? The you do a script that -- gasp -- actually uses stdin for whatever, and it goes down in flames.
The hack that I settled on (see below) is also broken: a script should not create new files or symlinks. For example, it should work from a read-only directory.

A proper way (which some imaginary node-esm binary should do) to run a script should not depend on the filename of the script in any way. It should work for foo, foo.js, or foo bar.py, and still allow me to write esm code. It should also not require any other files --- specifically, not any package.json files that affect more than just my script. And it should certainly not do what my hack does: create new files when you run them.

For reference, here's the only sane prefix that I found so far. It's pretty horrible, but it works. I'd be very happy if there's a better way to make stuff run.

#!/usr/bin/env bash
/*.................................................... 2>/dev/null # -*- js -*-
[[ -h "$0" || "$0" = *.mjs ]] && exec node "$0" "$@"
[[ -e "$0.mjs" ]] && { echo "error: $0.mjs exists" 1>&2; exit 1; }
mv "$0" "$0.mjs"
ln -s "$0.mjs" "$0"
exec node "$0.mjs" "$@"
*/

FWIW you could achieve the same thing with a smaller boilerplate using the experimental loader API (Node.js 16.12+):

#!/bin/sh
/*.................................................... 2>/dev/null # -*- js -*-
exec node --experimental-loader 'data:text/javascript,let%20t%3D!0%3Bexport%20async%20function%20resolve(e%2Co%2Cn)%7Bconst%20r%3Dawait%20n(e%2Co)%3Breturn%20t%26%26(r.format%3D%22module%22%2Ct%3D!1)%2Cr%7D' "$0" "$@"
*/

Un-minified loader code

let entryPoint = true;
export async function resolve(url, context, next) {
  const nextResolve = await next(url, context);
  if (entryPoint) {
    nextResolve.format = 'module';
    entryPoint = false;
  }
  return nextResolve;
}

A proper way (which some imaginary node-esm binary should do)

Having a separate binary comes with its own challenges and drawbacks, we wouldn't want to end up in a Python 2 / Python 3 scenario. If you feel strongly about this, you could create that binary yourself (which could be a simple Bash script that calls node with the above loader CLI flag), and either make it available on the npm registry and ask your users to npm install -g node-esm, either open a PR to this repo that makes such script auto-install alongside node.

@aduh95

@aduh95: thanks for the suggestion. I saw this mentioned, but ignored it since it's very clearly experimental.

But it still fails:

It spits a warning. Is there a way to silence just this one?
More importantly, it also fails with Unknown file extension "".

BTW, the imaginary node-esm point is mostly irrelevant. Yes, if there's a script then such a wrapper can be made in the same way. However, as long as it relies on experimental features, it should live in node itself, as part of the experiment. The inescapable point here is that there should be some sane way to achieve it, something that doesn't involve experimental features or obscure incantations. (But for my needs, I'll be happy to still try them out.)

@aduh95

@aduh95, Update: looks like the value is a promise rather than a value, so the following is more likely what you wanted to do (I added the node: test just in case, since it seemed to make it angry). The only bit that is missing now is silencing the warning. Any idea how to do that?

#!/usr/bin/env bash
/*.................................................... 2>/dev/null # -*- js -*-
exec node --experimental-loader='data:text/javascript,
let fst = true;
export function resolve(url, context, next) {
  const res = next(url, context);
  if (url.startsWith("node:") || !fst) return res;
  fst = false;
  return res.then(r => (r.format="module", r));
}' "$0" "$@"
*/

It spits a warning. Is there a way to silence just this one?

I guess you could add the --no-warnings flag, but that would silence all warnings, not just this one.

2. More importantly, it also fails with Unknown file extension "".

My bad, I forgot the await as you said in your other comment. Here's a snippet that works (I've tested with v16.14.2, and v17.9.0):

#!/bin/sh
/*.................................................... 2>/dev/null # -*- js -*-
exec node --experimental-loader 'data:text/javascript,let%20t%3D!0%3Bexport%20async%20function%20resolve(e%2Co%2Cn)%7Bconst%20r%3Dawait%20n(e%2Co)%3Breturn%20t%26%26(r.format%3D%22module%22%2Ct%3D!1)%2Cr%7D' "$0" "$@"
*/

something that doesn't involve experimental features

The loader API is your best hope to make node behaves as you want it to behave with ESM content. It's still experimental because there are not enough contributors to do the work; if you want this to change, please contribute or convince a big tech company to invest in this part of Node.js. (to be clear, I think your comments are helpful because you share a (hacky) workaround that can help others, and thank you for that. However pestering about the current situation is not welcome if you are not willing to make something to improve that situation.)

@elibarzilay

@elibarzilay I admire your persistence. I have lost way too much time due to this issue. I'm disappointed that there needs to be a discussion about this after such a long time and it was not addressed from day one of es6 modules introduction in node. Seems such a basic requirement to consider for introducing modules in node.

@aduh95 Clever but still a workaround. We should provide a workaround in the meantime but most importantly we should actually fix this issue.

We should be able to have an isolated es6 script in a system bin folder work out of the box without a file extension as it works on most systems. You don't type cp.exe or composer.php in a terminal to invoke them. Also package.json should not be a requirement since a package.json is not required to make a node program work.

Some long term solutions that I see:

do syntax detection for files without extension
some type of explicit statement that enables parsing files as esm, maybe only usable in files without extension. Think of debugger; or 'use strict'; or require.enableModules()
respect the type in package.json module type if a package.json is present for files without extension
introduce another explicit executable for node and use different shebang #!/usr/bin/env node-esm (probably not cleanest but would work)

The main value is simple for me: provide a very straightforward way to use es6 modules as first class citizen in the future of js world and keep the scripting standards from python, perl, bash etc. of making executable files that can change language without needing to rewrite all the scripts that call those executables. Also make sure that stdin, stdout, stderr, signals and exit code are handled transparently.

It would be such a pity to have to write a bash wrapper for every executable exposed by a packages for the years to come 😢 Some projects have many executable entry points.

Of course the best solution should be discussed but I would love to see a focus on fixing the root problem once and for all the years to come.

@aduh95

@aduh95: Bah. I tried an await, but forgot to try and make the function async, and instead assumed that the loader thing should be defined as a simple function. So if only there's a way to silence only a specific warning I'd be happy to even make it an npm package or whatever (though I'd need to test it beyond my simple need, like whether it withstands running in a directory with a random package.json etc).

And I think that you misunderstood me mumbling about an experimental feature. The thing is that as long as it's not publicly available (without an "experimental" name and without a warning), then it's essentially a private api, and therefore should not be used by external code. But it could be part of node somehow (some contrib script maybe), since then using private functionality would be perfectly fine.

@aalexgabi, FWIW, I do see the argument for avoiding a second binary, even if it's as mild as some node-esm symlink to node that simply has the esm default if it's invoked under the symlinked name. Just an example; I'm not really suggesting a symlink. Instead, if this thing works fine, and if it indeed doing the job of changing the default only for the "main" file on the command line, then just make --input-type do this too (or add a similar flag) and that will make writing esm scripts easy enough that there won't be any need for a second binary.

@GeoffreyBooth

@GeoffreyBooth can you please tell the full story:

We are working on solving this feature request via #49432 and #49629.

Resolved by #49974.

Navigation Menu

Repository navigation

node should execute files without extension as ESM if `"type": "module",` is specified #34049

What steps will reproduce the bug?

What is the expected behavior?

What do you see instead?

17 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

node should execute files without extension as ESM if "type": "module", is specified #34049

Description

What steps will reproduce the bug?

What is the expected behavior?

What do you see instead?

Activity

aalexgabi commented on Oct 7, 2021

wsehl commented on Oct 8, 2021

aalexgabi commented on Oct 11, 2021

aduh95 commented on Oct 11, 2021

aalexgabi commented on Oct 11, 2021

mizdra commented on Dec 30, 2021

elibarzilay commented on Apr 18, 2022

aduh95 commented on Apr 18, 2022

elibarzilay commented on Apr 18, 2022

elibarzilay commented on Apr 18, 2022

aduh95 commented on Apr 18, 2022

aalexgabi commented on Apr 18, 2022

elibarzilay commented on Apr 18, 2022

17 remaining items

GeoffreyBooth commented on Sep 14, 2023

andrew-aladjev commented on Sep 14, 2023

GeoffreyBooth commented on Sep 14, 2023

andrew-aladjev commented on Sep 15, 2023

GeoffreyBooth commented on Sep 15, 2023

andrew-aladjev commented on Sep 15, 2023

benjamingr commented on Sep 15, 2023

andrew-aladjev commented on Sep 16, 2023

benjamingr commented on Sep 16, 2023

andrew-aladjev commented on Sep 16, 2023

benjamingr commented on Sep 16, 2023

andrew-aladjev commented on Sep 16, 2023

aduh95 commented on Sep 16, 2023

GeoffreyBooth commented on Oct 9, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

node should execute files without extension as ESM if `"type": "module",` is specified #34049