Repository navigation
Utility method for exposing Node’s type resolution #49446
Description
Activity
i guess this is really more like a
canRequireapi right? sinceimport()can handle everything.i guess this is really more like a
canRequireapi right? sinceimport()can handle everything.Well first, can
import()expressions handle everything thatimportstatements can? I assume yes? But we still needrequirefor.jsonand.node.For this particular use case, sure, knowing whether or not
requirecan be used solves the need: ifcanRequire, thenrequirethe file, else usefs.readFileSyncto get it as a string and thenesmto transpile it and thenModule._compileto load it (like with https://github.com/floatdrop/require-from-string). This is for tools that can’t or won’t refactor to use async functions and therefore useimport().But that’s not as useful as just knowing whether Node would treat a particular
.jsfile/path as CommonJS or as ESM. Knowing that would also solve this use case, and open up other possibilities like knowing how to parse the file if a tool wanted to do anything with it (like transpile it etc.). Also it would be more future-proof, for example ifrequireof ESM ever happens—since if it does, wouldrequirestill be synchronous or would it return a promise likeimport()?Also @devsnek please see eslint/eslint#12333 (comment) for a pretty extensive list of gotchas that one user ran into when trying to replace
requirewithimport(). Unless there are workarounds or answers for all of those cases, that user is going to need to know when to userequireinstead ofimport(). (And if you don’t mind commenting on that thread and answering those questions, I’d appreciate it.)after all is said and done we should be able to load json and native add-ons with import too. the process of migration is always going to be tough.
I’m confused...
import()can load a CommonJS file? Maybe I’m misreading the docs.I need to load a
.jsfile, from a CommonJS file, and I don’t know what it is, so I don’t know whether to useimport()orrequire(). Ideally, I could give a function a filepath and it could tell me the type of that file. Or, even better, it could just do the right thing (and return aPromise, I suppose)import()can load a CommonJS fileYes,
import()andimportcan each load both CommonJS and ES module files.Duplicate of #30514
- marked this as a duplicate of esm: The
getPackageTypeutility function is not exposed #30514on Nov 25, 2019 @GeoffreyBooth, I presume needs are met for this issue as well. Should it be closed?
I don't think so. The other issue was specific to the context of loaders, where the needs are met; but the issue here is more general, and covers user application code (like a build tool needing to know how to load a file). The
defaultGetFormatfunction mentioned in the other issue is only available to loaders, not application or library code.Reacted by Derek Lewis and ExE BossJest would very much use such a function, and I'd love to not have to implement the logic manually. We implement our own
require(for mocking and dependency tracking, and to run/evaluate them in differentvm.Contexts) and will use custom linking etc for ESM support when we get that far (see jestjs/jest#9430). We'd need to know to usevm.Scriptorvm.Modulewhen loading a file - both from user space and fromnode_modules. A utility function from node that can figure that out for us would simplify things for us.@SimenB I believe what you need is not to figure out whether it's esm or cjs, but rather what does node's module resolution think the file is, according to its module resolution algorithm.
For example, let's say you have a test.js file that has no imports and is in a folder with a package.json with type:module. What node would do is treat it as esm. Even if it has a "require", node would treat it as esm. And what we would expect from jest is to treat it as esm and "import" it and not "require" (or the equivalent you will use with "vm")
I added esm support for Mocha, and the trick I used to figure that out was to require the module, and if I got an error from node that says that it's an esm, then I import it. A hack, but given that there is currently no way to figure it out (besides duplicating nodes algorithm), then that is the only solution I could find.
Reacted by Evan Plaice and ExE BossI added esm support for Mocha, and the trick I used to figure that out was to require the module, and if I got an error from node that says that it's an esm, then I import it.
Why not do it the other way around?
import()can accept either ESM or CommonJS, so it should just work on the first try without you needing to catch an error and fall back to an alternative.Why not do it the other way around?
Not who you asked but I assume because it would break synchronous consumers that may not support async test suite setup. Starting with require means the change in behavior only affects people who actively use modules.
P.S.: But I agree that in the general case starting with ESM will be safer, especially in a future where loaders may make it impossible ("harder than reasonably implementable") for CJS to fail when ESM is
required.Reacted by snek, Jordan Harband, Evan Plaice and ExE BossYeah, I'm hoping the function discussed in the OP will allow us to not have to implement guessing or fallbacks. The use case of loading config presented in the OP is not my main use case, but rather custom implementation of
requireandimport. We can of course look at file extensions and use https://github.com/sindresorhus/read-pkg-up and inspect thetypefield ourselves, but since node already implements this logic it'd be wonderful to not have to roll our own 🙂Reacted by Gil Tayar and Bernard@GeoffreyBooth - @jkrems should have been correct, but since we have two paths currently in Mocha—an async one that supports both esm and cjs, and a sync one that supports only cjs and is there for api backward compatibility, then that wasn't a problem for us.
No, the real reason is that I was chicken. 😀Mocha is used by millions of tests around the world, and just thinking that suddenly they're all going to pass through the newly deployed
importmade me, frankly, a bit worried. We will probably switch to using import, as you said we should, but we really want the ESM mechanism to get some real time in production!Reacted by Jordan Harband and Evan Plaice9 remaining items
There has been no activity on this feature request for 5 months and it is unlikely to be implemented. It will be closed 6 months after the last non-automated comment.
For more information on how the project manages feature requests, please consult the feature request management document.
I still definitely want this, but the implementation baked into
jest-resolveseems to be correct (I haven't gotten any bug reports on it, at least).@SimenB @GeoffreyBooth as the issue got closed: see: eslint/eslint#12333 (comment)
but without a result i want to supply some impotent internal insights.
- The ESM System has a internal module cache that is not exposed by design. It is Garbage Collected fully system internal managed.
- A ESModule gets loaded only once and then stays in memory thats why the tests did not re-run to re run them you need to invalidate the cache this is done via url patterns as in the browser
import('./my.js?cache=xxxx')this is a example using a server query string but you can also use simple url hashesimport('./my.js#1')import('./my.js#2')import(`./my.js#${new Date().toISOString()}`) - The only way to clear the v8 Module cache is to throw away the whole isolate
- for frontend module loaders where i want explicit garbage collection control i need to create a whole new context that i can throw away and connect the scripts via postMessage or other internal messaging.
High Level Picture
The Browser Platform Manages the life cycle of the isolates that loaded the modules eg: workers lifecycle design which inherent
schedules regular termination of the worker when not needed. Or Webpages that are not in focus for a longer time. As the internal code is not aware of the outer Embedder the design choice was to not expose the cache.Saw the resolution part to late like the experimental flag for auto module types
parsing the file for import as static syntax and export as static syntax is the only way to say if something is a module or not
// is a module if it contains import or export keyword not followed by ( as next char to it no matter how much spaces or new lines are inbetweenhttps://nodejs.org/dist/latest-v21.x/docs/api/cli.html#--experimental-detect-module
the
--experimental-detect-modulesimple uses the same that typescript uses in classic resolve so both systems would have same behavior.There has been no activity on this feature request for 5 months. To help maintain relevant open issues, please add the never-stale
Issues and PRs exempt from automated stale handling. label or close this issue if it should be closed. If not, the issue will be automatically closed 6 months after the last non-automated comment.
For more information on how the project manages feature requests, please consult the feature request management document.There has been no activity on this feature request and it is being closed. If you feel closing this issue is not the right thing to do, please leave a comment.
For more information on how the project manages feature requests, please consult the feature request management document.
Split off from nodejs/modules#389 (comment). I think it would be useful to have a function that returns how Node would try to interpret a file, in particular whether Node would try to parse a path to a
.jsfile as CommonJS or as ESM. This would spare tools from needing to reimplement Node’s “find the nearest parentpackage.jsonand see if it has a"type"field” algorithm.Whether the file would be interpreted successfully as that type is irrelevant; it could be a zero-byte file for all this API cares. One use case for this is for tools loading
.jsconfig files to know whether to try to load the files viarequireorimport().