Repository navigation
Support .mjs output #18442
Description
Activity
Kitson Kelly (@kitsonk)
Thanks for posting these link but I am not sure if they are relevant, could you explain them? I found these while checking if the issue was already opened: most of the issues you listed are about input files. This issue is about compilingtsfiles tomjsinstead ofjs(adding an options to change the compiler output).Reacted by Escape0707, Kyle Culp, Paul Draper, Albin Larsson, Andrey Sakharov, Andriy Komm, Johan, Troy Rhinehart and Abhijeet SinghBasically those say that TypeScript does not want to concern itself with managing extensions like that, as it is already complicated enough. In particular the output extensions run afoul of the TypeScript non-goal of:
- Provide an end-to-end build pipeline. Instead, make the system extensible so that external tools can use the compiler for more complex build workflows.
If you want
.mjsfiles, it would be best to do something like:$ npm install renamer -g $ renamer -regex --find '\.js^' --replace '.mjs' './outDir/**/*.js'Reacted by Zev Spitz, Damian Piątkowski, Nischit Ranganath, John Johnson, Wenfang Du, Levent Oz, Jason Walton, Shantanu Sharma, Eliaz Bobadilla, Enes SOYLU and 1 moreReacted by Alexey Svetliakov, Adam, dcharbonnier, Emil Ekberg, Wang Zishi, Amit Portnoy, Saleh Abdel Motaal, Trotyl Yu, Jonny Eskew, Nathan Mc Grath and 207 moreReacted by Josh Pike, Daniel K., Tyler Young, arianrhodsandlot, Egor Andrianov, Escape0707, Elnar Khisamov, S. B. Tam, ThibaudAV, fartwhif and 2 moreReacted by Dario Farzati, OKUNOKENTARO, Mark Bjerke, Павел Богатырёв, Aldwin Vlasblom, ixcviw7bw, desmap, David Rodrigues, Brian Takita, Tim Morris and 2 moreThank you very much for clarifying your comment.
I understand that adding any new feature is a burden since it means that it then must be supported for a long time, but I believe that the benefits of themjssupport are worth it: it won't turntscin a complex build tool but help TS users.The long term goal of the Node team is to allow users to author their code using ES modules without paying a cost due to the commonJs modules and script target. This is one of the reasons why the proposals to require
"use module"or anexportstatement (even an empty one) were dropped in favour of using a new extension. See the Node EPS discussions in nodejs/node-eps#57 and nodejs/node-eps#60. If a post-compilation step to rename the files is still required in two years only because.mjswas introduced later then it will just add another burden to remember for years (like the BOM or the different line endings)... Even if there are some discussions to support ES modules with the.jsextension ("module"property), the current position is that - if such support is added - it should be used as a fallback for old tooling..mjsis promoted for new code.Complex build tools have their place for assets management, dead-code elimination, bundling, minification, etc. These all fall out of scope for TS, but being able to produce runnable code does not. Big projects have their own workflows and use the TS library (either directly or through plugins for other task runners), adding the renaming step is not a big deal for them. On the other hand, a sizeable chunk of projects using Typescript are small/medium sized libraries. These libraries usually only need a single tool:
tsc. If they want to support native ES modules, they'll currently have to come up with a command similar to the one posted above by Kitson Kelly (@kitsonk). The problems are that it raises the barrier to entry, causes duplicate effort and makes the build more expensive.It raises the barrier to entry because newcomers can no longer simply use
tsc -p && node index.mjs(--experimental-moduleswill no longer be required when the use-case for this issue will be relevant) and run their code to try out Typescript. For existing projects, the "build and run" command is usually already defined as an npm script or gulp/grunt/webpack/whatever task, writing this command (or understanding it) will be more difficult: for some persons it will be trivial, for others it'll require a few hours of research. The rate of change in the JS ecosystem is already pretty high, let's try to not worsen it.
Now, a related problem is that it may cause some inconsistencies and duplication: different projects will use different ways to rename their file. I discoveredrenamerthanks to the message above, but I would have written a gulp task otherwise. Someone else would have used a POSIX-only shell command with broken edge cases or rolled their own helper Node script. To be honest, I think that most good projects would have a good implementation but this lack of standard way to deal with file renames when no build system is already there may increase the "barrier to entry" problem.
Finally, once the manual renaming is configured, it still represents an additional cost. I don't have any measures but for medium projects and using Node to rename the files, it may be longer to start the VM than running the file rename: why not doing it in the same process as TS? Also: what about--watch? How do I make it play well with my custom command? Do you have to have to watch the build directory to rename as the files are emitted?So far, most of my arguments were about the build complexity: having it done once in TS is better than having each project coming up with its own solution. There is another important reason to have ES modules working out of the box, and I think that it aligns with the goals of TS:
- Align with current and future ECMAScript proposals.
- Preserve runtime behavior of all JavaScript code.
ES modules are part of the spec and the TS syntax uses them. These modules have a specified runtime behaviour that cannot be fully replicated in commonJs. It affects among other things circular dependencies, early errors, mutation of exported namespace, etc. Many people won't care, until it bites them. When setting the
moduleoption tocommonjs, TS does a pretty good job of generating an output with the expected behaviour but it does not trump using real ES modules. So we are in a situation where we can either set an option and have a good approximation using commonJs but if we want the exact spec and runtime behavior we have to jump through hoops and loops.One last argument is that even if browsers support native ES modules without requiring
.mjs, real-world usage of native ES modules will start server-side with Node because you can control its version. Node is important for the whole ecosystem so it would be damageable to just ignore that it uses.mjsfor ES modules. You can't just treat it as a "platform detail" when this platform is used by the majority of the TS audience.To summarize, here are my main points:
- Now that native support exists for ES modules, people will want to use it out-of-the-box.
- Node settled on
.mjsand hopes that it'll gain traction..mjsis Node-specific, but Node is important - Many libraries only need
tsc, requiring a second tool for renaming increases barrier to entry and causes duplicated effort (of varying quality). - Using a post-compilation command leads to higher build times and does not play well with the watch mode.
- Using native ES modules is the only way to match the spec exactly, TS should not make it hard to use
Reacted by Felix Schröter, Max Desiatov, dcharbonnier, Wang Zishi, Jonny Eskew, Saleh Abdel Motaal, Amit Portnoy, Steven, Nathan Mc Grath, Tom Yaxley and 92 moreReacted by Alberto Fiori and Akshar PatelJust an Opinion
As a very strong fan of TypeScript, and obviously a user of node (I guess even TypeScript is) I would really like to see some agreement between them on an issue that has had so much disagreements over many years of rational and sometimes awkward discussions.
I would have loved it if node had a transition phase to make common js (the non-standard) become .cjs over a period of two years.
That said, mjs seems to be the current direction, obviously that non-js file extension (either one) itself does not matter, but the ability for TypeScript to provide a way to control this inline as the originator of the transpiled files is something that really falls to TypeScript.
That said (again), mjs as a locked extension is kind of very opinionated, someone must have realized that forcing js was okay, it was already called js, but forcing mjs, come on, even my spelling checker is nagging me on that one. I hope nodejs will become a little more flexible on that one.
Proposal
Please make it possible to specify mjs as a module format that simply means compiling es2015 modules and calling them mjs :)
Reacted by Amit Portnoy, Mark Bjerke, James Bromwell, Scott Gibson, Markus Gachnang, GoToLoop, Roy Ivy III, Chen An, sripberger, CEbbinghaus and 4 moreIt's important to note here that
.mjsisn't only about file extensions. running node with the
--experimental-modulesflag also means that ES6 module specifiers must be valid urls, so the extension has to be included in the import statement, e.g.import x from './x.mjsIf I'm understanding this right, renaming the files isn't a solution, TS would also need to compile module specifiers with the
.mjsextension in the ES2015+/mjs mode. We're keen to support newer versions of node by publishing non-downleveled iterators as es2015+ modules for IxJS, so it would be great to get an answer on this soon.Reacted by Phips Peter, Zheeeng, Franklin Yu, Cody Swartz, Homa Wong, luvies, Paul Draper, Linus Unnebäck, Hu Kun, Matthew Gamble and 9 moreTypeScript already allows you to include a
.jsextension on your imports (even when you're actually importing.tsTypeScript files). The change needed would be to expand that to also allow.mjs.Reacted by Frank Lemanschik, Linus Unnebäck, Paolo Denti, Sanath Kumar U, Alberto Fiori, Steve Fenton, Abhijeet Singh and pc-erinIt's not about the source files including the extension, it's about the compiled files.
If I compile
import x from './x'into ES5/CommonJS, TS emits:var x = require("./x");
If I compile it to ES2015/ESModules with an
.mjsextension, TS should emit:import x from "./x.mjs"
If a library is trying to support both old and new node with CommonJS and ESM side-by-side (as
*.jsand*.mjsextensions respectively), node only imports the.mjsfiles if the extension is included in the module specifier.Reacted by Bazyli Brzóska, Lydia Schow, Phips Peter, Franklin Yu, Gaubee, Cody Swartz, Ronny Hanssen, Ant Stanley, Max J. Polster, Homa Wong and 48 moreThere are some potential updates in the works… including a --loader option that would you to specify a file with a resolve hook that will take the specifier and parent module path, then it could handle extension prioritization as needed. It seems to be very flexible but their goal is to make it declarative and not have people overloading the actual loader functions.
In terms of picking mjs over js, that is only the case in the CJS loader system which uses the now legacy Module prototype with it's hooks, the ones that everyone likes override all the time.
In the current --experimental-modules release, the ESM loader is hard coded to '.mjs' or falls back to the CJS loader.
What I've seen so far over the past few days makes me believe that the ESM loading system will take over with pluggable CJS loading (unless opting to use the legacy-modules by flag or based on the main file).
Paul Taylor (@trxcllnt) I guess regarding the part about baking the extension right into the import… this applies to import/export everywhere now as part of loader specs.
When ES2015 modules were a spec'd, there were no loader specs, the notion that the extension can change from source file to .js made it a more natural way to go and everyone got too comfortable there. But all loaders (especially web browsers) realized that no-extension means potential security and not to mention load-time drawbacks.
So if TypeScript would continue to work without third-party tooling, it will need to find a strategy to write out standard (not ISO but platform-specific) out-of-the-"compiler"-box projects that will just work in either node or browsers at least depending on the compilerOptions intent specified by the user, and do so without asking the user to mockup some hack to get it to work.
This is just an opinion, but honestly, it feels like it for TypeScript to address.
Checkout this MDN reference but make sure you notice the part where it first said "excluding the .js extension" then in the examples included the extension anyway.
I'd like to include here that the IANA has an Internet Draft which specifically adds
.mjsfor COMMON usage that represents the Module goal of ECMAScript ..mjsis not purely a Node.js concern and is even included in an example for browser specs and is supported by a variety MIME DBs already like shared-mime-info.Saleh Abdel Motaal (@daflair) can you expand on what
(unless opting to use the legacy-modules by flag or based on the main file).
means?
The next step after that PR is to setup per package loaders to be able to guard against global loader mutation. So, it might be going the direction you think.
When ES2015 modules were a spec'd, there were no loader specs, the notion that the extension can change from source file to .js made it a more natural way to go and everyone got too comfortable there. But all loaders (especially web browsers) realized that no-extension means potential security and not to mention load-time drawbacks.
Interestingly the WHATWG Loader Spec had a very early version in the ECMAScript spec that was removed at the last minute before ES2015! There are interesting other things like the original CommonJS Spec which mandated not to have extensions. We should probably avoid dwelling on the past so much since the different loaders vary so much on these opinions.
Node's EP specced a superset of the WHATWG resolve algorithm that does do various Node idioms like file extension completion. However, the browser has a subset that is safe to use in Node. Still, even with that subset there are problems with dependency trees since "bare" imports are waiting on userland feedback (you can get involved in the hook here), but is mostly left up to intelligent servers and service workers for now.
I'd recommend trying to compile down to the WHATWG compatible specifiers except for bare specifiers for now.
Bradley Farias (@bmeck) what I meant by:
unless opting to use the legacy-modules by flag or based on the main file
I was making an assumption that at some point you will have to use something like --disable-es-modules or --legacy-modules which is really an option that simply does the opposite of --experimental-modules (when this flag is the default behaviour) to resort the current loader system and completely bypass anything related to the new loader (ie legacy applications that simply find ways to be incompatible with this existential change to their eco system).
But now that I dug in a little deeper in your recent PR's this might not be your intent…
So it is best to ask you about your intent here 😉?
Saleh Abdel Motaal (@daflair) I'm still not understanding. ESM support in
--experimental-modulescompletely avoids touching CJS except by overtaking the defaulting of.mjsto CJS (it now throws). I can't think of a reason to introduce such a flag. Also, it would be a very hard sell to change the default behavior from CJS to ESM or vice versa and I doubt that will ever happen.200 remaining items
adding .mts etc doesn't really help at all for package maintainers who want to provide .mjs and .cjs exports to be consumed by either audience
Node is the only platform which cares about using the mjs (or cjs) extension (optionally). For anywhere else, just using .js is fine. In node, you should almost never compile your package into both formats; this causes two copies of your package (potentially) to get loaded into node at runtime, which, in addition to being performance inefficient, can cause issues with state, class nominality, and unique symbols. Instead, in node, you should really just provide a format-specific convenience wrapper around a single source-of-truth package format (eg, a handwritten esm wrapper that imports the cjs core). Ergo, you really shouldn't need to compile the same source to both a
.jscjs file and a.mjsesm file - doing so likely means you're quietly breaking something in node. (Now, if you're building for node and another platform, that's another story, but then you don't need the mjs extension anymore, since it's only used for format disambiguation in node.)Worst case, if you really really badly want to make node load two full, separate copies of your library, you could always compile once into a
lib/cjsdir and once into alib/esmdir, and drop apackage.jsonwith"type": "module"intolib/esm. Still don't actually need.mjsat all, let alone output extension overrides for it. It is actually incredibly valuable to know that.tsalways maps to.js- it just makes resolution logic so much more easy and reliable.Reacted by Linus Unnebäck, Paul Fidika, pfdgithub and csr632Hm, if you can just have a package.json in a dist-esm folder with just
{"type":"module"}inside that doesn't break things in weird and wonderful new ways then that neatly solves any objections I have.. the situation I'm thinking about is indeed the package maintainers that need to build 3 versions of the package to distribute on npm because they want to support consumption from node cjs, node esm, webpack, roll-up, <script type=module>, ie11 all at the same time with a single package...Reacted by Frank Lemanschik, ZHAO Jin-Xiang and Abhijeet SinghReacted by Frank Lemanschik and ZHAO Jin-Xiangjamie-pate thats exactly my observation i am researcher in that fild and member of the nodejs package-maintainance group. We Really did break evolution at some points. But the Good thing is when everything would be offered ESM Only with .js extension even without a package.json we would make the whole ecosystem healty again.
when using NPM packages you should dist only one flavor -esm -cjs -webpack everything a own package and consider even sharing packages the esm code for example could when it is a really old package you could start offering a ESM package that loads and depends on the CJS package do not even transpil that to ESM you gain nothing from that.
to load a single file CJS package inside ESM it would be enough to know that both use the same globals so we can shim like this
const resetLoader = () => { globalThis.exports = {}; globalThis.module = { exports: globalThis.exports }; } resetLoader() await import('singleFile.cjs') // i do not want to over complicate it const myCjsModule = globalThis.module.exports; resetLoader()
there is ongoing work on the import.meta.resolve feature that will complet the above shim with resolving capabilitys
Trying to give a practical example here: I have been working on a deep dependency in our code that has been written in CommonJs a long time ago. I used it as a test to see if I could update it to be a package that is written in Typescript and supports both to be required as cjs (to not break when I drop it in the greater application) and to offer esm for the whole system to be step-by-step updated to ESM-only code. → @tradle/typeforce
My plan was to use
tscto compile teach variant of thesrcfolder into acjsandmjsfolder with source-maps for better debugging experiences.|- cjs | \- index.js |- mjs | \- index.js \- src \- index.tsIn the
package.jsonI defined both themainandmodulefield with the respective.jsfile. Using two differenttsconfig.jsonvariants (1, 2) as previously suggested with a shared base file.In order to be able to test the generated files I wanted to be able to load them directly in the CLI.
import('./mjs/index.js')will fail with the obvious(node:12398) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.error message. So using a script I fixed this to rename all.jsfiles to.mjs. This script needed to be expanded as with the change to.mjsalso I better change the type definitions to.d.mtsand the source maps to.mjs.mapfor consistency. The ties between the source and the source-maps had both to be updated as well. A pretty interesting problem that I found was that a.mjsfile can not import a relative.mjsfile.(node:12628) UnhandledPromiseRejectionWarning: Error [ERR_UNSUPPORTED_DIR_IMPORT]Of course in typescript I can not change
import { compile } from './compile'tofrom './compile.tsas I would be running intots(2691): An import path cannot end with a '.ts' extension.so I had to add some additional fixture in the rename script.The initial tests were written with
tapeand to run the tests I wanted to be able to use just run the tests usingts-node test/index.ts. In order for this to work I set the./tsconfig.jsonto havemodule = commonjsasts-nodedoesn't support ESM yet(?).To make sure that all code works same I originally I tried to put the tests in
src/testand compile it for each variant and run them in that variant but I havn't managed to compile the tests with the external commonjs libary (fresh-tape) asmodule = nodejs, which is why I put it in thetestfolder.I think it would help quite a bit if typescript would build node esm modules with a
.mjsending and also replace the directory import statements with direct.mjslinks.Reacted by Alex Rattray, Loris Leiva, pfdgithub, Phil Léger, Adam Blaszkiewicz and NataliaIn order to be able to test the generated files I wanted to be able to load them directly in the CLI. import('./mjs/index.js') will fail with the obvious (node:12398) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension
Martin Heidegger (@martinheidegger) check out how https://github.com/jwalton/solox does this. The build process for solox is very similar to what you proposed with an esm tsconfig.json and a cjs tsconfig.json. These get used to compile the source into ./dist/esm and ./dist/cjs. There's a post-build step which adds a package.json to each of these folders; in ./dist/esm we write a package.json that contains
{"type": "module"}, and in the cjs folder it's{"type": "commonjs"}. Conditional exports in the top level package.json ensure that the correct version is picked up based on whether the module is being imported or required.The result is that all files are ".js" files, but if you clone and build this project you'll find that
await import('./dist/esm/index.js')from the node REPL works exactly as you'd expect, as doesrequire('./dist/cjs/index.js'). No rename scripts are required.My tests are in /test, but they import from ../src, and tests run fine out of the test folder using jest and ts-jest. (If you run into problems with tests, you might want to flip things so the tsconfig.json is commonjs, and have a tsconfig.esm.json that's
"module": "es6".)Note that one potential downside of this approach (and your original approach) is that it's possible for projects to end up with two copies of your code in memory (if they both import your code and require it in different places). This approach should definitely be avoided if you have singleton instances or other global state.
Reacted by Loris Leiva, pfdgithub, csr632, daiwawawa, Dylan Bulmer, Moe Jangda, Natalia and Armando FazReacted by Loris Leiva, Brady Holt, Adam Blaszkiewicz, Benjamin Tamasi, KaKa and Moe JangdaThank you Jason Walton (@jwalton) for sharing notes.
There's a post-build step which adds a package.json to each of these folders; in ./dist/esm we write a package.json that contains {"type": "module"}
Oh! Thank you for pointing this out. This will indeed make my solution a lot simpler. Wouldn't this be a good feature to add to the typescript compile process?
Conditional exports in the top level package.json ensure that the correct version is picked up based on whether the module is being imported or required.
Actually, I also was under the requirement to support deep imports like
import * from 'typeforce/async'as part of backwards compatibility and I needed to conditional exports per file. (manual task atm.) It would be nice to have a good automation for that as well.In a simple experiment with solox (found here martinheidegger/solox@e0a5edd) I tried to add a simple dependency and ran into a catch-22 where lint would break without a
.tsdeclaration and compile now breaks with. 😅In a simple experiment with solox (found here martinheidegger/solox@e0a5edd) I tried to add a simple dependency and ran into a catch-22 where lint would break without a .ts declaration and compile now breaks with. 😅
This is a idiosyncracy of typescript, and it's not very intuative, but try making this change:
-import foo from './test.ts'; +import foo from './test.js';
(i.e. change the "ts" to a "js" in your import) and your experiment should build and run correctly.
Reacted by Martin HeideggerFor deep imports in nodejs you can use the exports property of package.json
https://nodejs.org/api/packages.html#exports"exports": { "./*": "./dist/*.js", "./*/*": "./dist/*/*.js" },
NodeJS does not recognize the "modules" property, only bundlers will recognize that.
I'm using that with the same
import 'test.js';approach in typescript and the{"type":"module"}micro package.json approach as shared above, and finally after a week of futzing i'm pretty happy with what I've got. (except that the webpack/angular part of the project still usesimport 'test'because they haven't agreed on the manditory import extensions idea 😭 )Reacted by Martin Heidegger, fartwhif, Joseph Garrone and Hunter Beast(i.e. change the "ts" to a "js" in your import) and your experiment should build and run correctly.
Interesting. This seems to be not yet supported by
ts-node, though this is actively being worked on TypeStrong/ts-node#1361 I will revisit this and post an update using yours and jamie-pate input when that landed.I use
swcto transpile mytsfiles tomjs.yarn add -D @swc/cli @swc/core yarn run swc next.config.ts -o next.config.mjs -C module.type=es6 -C sourceMaps=inline
Reacted by xiangnan, Bjørn Gilstad, JH.Lee, Michele Riva, Lucas Vazquez, Merlin, Hunter Beast and Muhammad AfifudinReacted by Abhijeet Singh, electrovir and AlexIn order to be able to test the generated files I wanted to be able to load them directly in the CLI. import('./mjs/index.js') will fail with the obvious (node:12398) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension
Martin Heidegger (@martinheidegger) check out how https://github.com/jwalton/solox does this. The build process for solox is very similar to what you proposed with an esm tsconfig.json and a cjs tsconfig.json. These get used to compile the source into ./dist/esm and ./dist/cjs. There's a post-build step which adds a package.json to each of these folders; in ./dist/esm we write a package.json that contains
{"type": "module"}, and in the cjs folder it's{"type": "commonjs"}. Conditional exports in the top level package.json ensure that the correct version is picked up based on whether the module is being imported or required.The result is that all files are ".js" files, but if you clone and build this project you'll find that
await import('./dist/esm/index.js')from the node REPL works exactly as you'd expect, as doesrequire('./dist/cjs/index.js'). No rename scripts are required.My tests are in /test, but they import from ../src, and tests run fine out of the test folder using jest and ts-jest. (If you run into problems with tests, you might want to flip things so the tsconfig.json is commonjs, and have a tsconfig.esm.json that's
"module": "es6".)Note that one potential downside of this approach (and your original approach) is that it's possible for projects to end up with two copies of your code in memory (if they both import your code and require it in different places). This approach should definitely be avoided if you have singleton instances or other global state.
Jason Walton (@jwalton) This proposal looks like a duplicate of probably the most insightful post on the topic to date cc Michael O'Brien (@mobsense) : https://www.sensedeep.com/blog/posts/2021/how-to-create-single-source-npm-module.html
By the way, for those who may be interested in having a look at the rewrite strategy path, it also seems worth considering the 2 packages below :
• tsukuru
Reacted by ItsJustRubyGuillaume FORTAINE (@gfortaine) Excellent suggestion. By placing a
{ "type": "module" }into my/dist/esmfolder I no longer have to have the"type": "module"at my root package.json but my ESM is now properly importing, assuming I manually specify the .js extensions, but this is close.Reacted by Guillaume FORTAINE, Arnaud Buchholz, akitaSummer and Brett ZamirReacted by Gergely Borgulya- Reacted by Lucas Vazquez
- added a commit that references this issue
on Apr 19, 2023 - added a commit that references this issue
on Sep 12, 2023
Experimental support for ES modules just landed in Node 8.5 (changelog, PR).
Since ES modules have some parsing and semantic differences, Node decided to use the
mjsextension for ES modules (whilejsis for the "script" target and commonjs modules).The current Typescript version (
2.5.2) supports ES modules emission but uses thejsextension by default. It means that to use it with Node, a post-compilation step is required to change the extension fromjstomjs. This adds undesirable complexity to use native ES modules support with Node.A solution would be to add a compiler option to output
*.mjsfiles when emitting ES modules.Edit (2018-03-22): The propositions below are a bit outdated. I recommend reading the issue to see the progression. See this comment for my current proposition.
Notes:
*.jsextension, many tools rely on thejsextension.*.mtsfiles would compile to*.mjs, this would be similar to*.tsxand*.jsx.