ES Modules are over 9 years old, and they’re still such a hodgepodge half-assed implementation Node.js. That was (and still is) a really botched migration.
Is there a good write up anywhere on CommonJS and ESM history and where the ecosystem is going?
I feel like I went away from JS for 10 years and came back to find the package ecosystem had forked into two incompatible camps, which seems very unfortunate. Now when I - rarely - work on JS it’s not clear which camp I should hitch my wagon to
Going all in on ESM-only for packages you publish is basically impossible.
This makes ESM-only packages a tiny minority, making it less urgent to adopt ESM. A vicious feedback cycle hindering adoption.
Living in a world with both module systems has caused a lot of pain and suffering.
Due to the excellent work of Joyee Cheung, there’s now light at the end of the tunnel. OLD code that uses require() will be able to use NEW ESM code. Once Node.js 23 is widely adopted, people have a path to reasonably publishing ESM-only packages.
Node.js 23 just came out so this advice is not battle tested, but people are hoping that the future will be:
publish esm-only packages
enforce Node.js 23 in the package.json “engines” field
And for those that can’t enforce a cutting edge Node version, there’s the exports.module condition, but more likely these people will just wait and let things shake out.
Node 23 still hasn’t actually fixed the intractable incompatibility between CJS and ESM, namely that module loading is asynchronous in ESM and synchronous in CJS. None of the runtimes have made loading an ESM module that uses asynchrony in its initialization (aka top-level await) work when loaded from a CJS module.
So you can write your application with ESM, because ESM loading CJS is fine. If you require Node 23 (feasible in a few years perhaps), you can publish an ESM-only package and CJS users can use it — as long as you have no top-level await. Publishing a package using ESM “freely” still has no visible route to feasibility that I know of.
The reason they added require(ESM) was the critical insight that only a minority of ESM packages used top level await. That’s not as big a problem as you might think.
Living in a world with both module systems has caused a lot of pain and suffering.
I suffered a lot because my brain refuses to learn things that are wildly inconsistent.
publish esm-only packages
I’d prefer going Typescript only and then letting the engine underneath figure out where code is sourced from, if that would be even remotely possible.
I’ve been TS only for over a decade. It doesn’t help shield you from most of this. For example, ESM requires extensions on relative imports, and TS doesn’t, but the second you publish your package you have to think about how your code is going to get those extensions. Given certain configurations, which some packages I publish use, you need to add js extensions to relative imports FROM TYPESCRIPT. Even though there is no such file on disk. Imagine explaining that to a new developer.
I know. Our TS stuff builds both kinds of modules for reason only 1-2 people in the company can half explain.
I’m awkwardly pointing at full TS runtimes where we don’t build the code to ship anymore. TS end to end. I’ve heard this stuff theoretically exists but there should be a lot more energy spent on it.
For bundled browser libraries that have no external dependencies (and therefore have no imports or requires) going ESM-only is usually fine. These libraries rarely care how they are loaded. I think? I don’t have a ton of experience with publishing such libraries.
Publishing anything else as ESM-only is impractical. There becomes a hard wall between your package and everything else that has ever existed.
Even for an unbundled browser library meant to be consumed by a bundler during a compile step, being ESM-only will cause issues.
I hope people in the future will study the great javascript module cliff the same way they will study the python 3 cliff. There are some valuable lessons in how (and if) to do world-breaking changes the we seem unable to internalize yet.
What are the advantages to using a top-level await in a module? I still do not understand the upside of ESM over CommonJS, and I still write my new JavaScript using the old style. These days, I mostly write Typescript, however, which I think can compile to either.
I have a config module that can read AWS secrets to get the config data, which is exported as an object. There is no synchronous way to get an AWS secret, so that is a top-level await.
I also have a module that initializes a DB connection object, which isn’t asynchronous, but needs the DB password config.dbPassword.
I fixed this with a startup script that copies the secret into a file before starting Node, but it’s quite clunky.
Thank you for the specific examples! In the past, when I’ve had to do such things, I’ve wrapped them in a function call that could be awaited. For example, the former might have a line like const cfg = await getConfig(), or maybe instead I would have cfg as a global, but make it invalid until the cfg.init() method had been awaited.
Trouble is, some modules have non-async functions that need the config. But they can’t do a top-level await getConfig() to ensure it’s there first. You could restructure everything that needs config to be async, but that would have been fragile and more effort than just giving up and generating a .env file from the secret before starting Node.
The reason you might want to use a top-level await is that it lets you run (asynchronously) run initialization code before exporting something. For example, you could export a database connection but first make sure it’s fully initialized.
I suppose this is subjective preference, but I like my module imports like I like my coffee: without TCP sockets being opened implicitly.
Though having said that, I suppose it isn’t uncommon in other languages to do something like load a file at module import, some lookup table or binary blob, and I guess in JS land that would need to be an async call.
The problem is not limited to IO. Loading WASM is an asynchronous operation for example.
If you have any form of touching of an async API during init you have this problem. And if you look at most JS APIs…. Well they tend towards async only
ES Modules are over 9 years old, and they’re still such a hodgepodge half-assed implementation Node.js. That was (and still is) a really botched migration.
It’s wild. People would be complaining about the Python3 migration and Python packaging, while this situation is just standing there.
Is there a good write up anywhere on CommonJS and ESM history and where the ecosystem is going?
I feel like I went away from JS for 10 years and came back to find the package ecosystem had forked into two incompatible camps, which seems very unfortunate. Now when I - rarely - work on JS it’s not clear which camp I should hitch my wagon to
ESM in a nutshell:
Node.js 23 just came out so this advice is not battle tested, but people are hoping that the future will be:
And for those that can’t enforce a cutting edge Node version, there’s the exports.module condition, but more likely these people will just wait and let things shake out.
Node 23 still hasn’t actually fixed the intractable incompatibility between CJS and ESM, namely that module loading is asynchronous in ESM and synchronous in CJS. None of the runtimes have made loading an ESM module that uses asynchrony in its initialization (aka top-level await) work when loaded from a CJS module.
So you can write your application with ESM, because ESM loading CJS is fine. If you require Node 23 (feasible in a few years perhaps), you can publish an ESM-only package and CJS users can use it — as long as you have no top-level await. Publishing a package using ESM “freely” still has no visible route to feasibility that I know of.
The reason they added require(ESM) was the critical insight that only a minority of ESM packages used top level await. That’s not as big a problem as you might think.
I suffered a lot because my brain refuses to learn things that are wildly inconsistent.
I’d prefer going Typescript only and then letting the engine underneath figure out where code is sourced from, if that would be even remotely possible.
I’ve been TS only for over a decade. It doesn’t help shield you from most of this. For example, ESM requires extensions on relative imports, and TS doesn’t, but the second you publish your package you have to think about how your code is going to get those extensions. Given certain configurations, which some packages I publish use, you need to add js extensions to relative imports FROM TYPESCRIPT. Even though there is no such file on disk. Imagine explaining that to a new developer.
I know. Our TS stuff builds both kinds of modules for reason only 1-2 people in the company can half explain.
I’m awkwardly pointing at full TS runtimes where we don’t build the code to ship anymore. TS end to end. I’ve heard this stuff theoretically exists but there should be a lot more energy spent on it.
Ah, right, makes sense. No such experience myself but checking out Deno is on my todo list!
Do you mean for Node.js packages or does your statement there apply to packages-that-run-in-browsers too?
I’ve got very interested in ESM for browser code over the past couple of years, now that it’s effectively universally supported.
For bundled browser libraries that have no external dependencies (and therefore have no imports or requires) going ESM-only is usually fine. These libraries rarely care how they are loaded. I think? I don’t have a ton of experience with publishing such libraries.
Publishing anything else as ESM-only is impractical. There becomes a hard wall between your package and everything else that has ever existed.
Even for an unbundled browser library meant to be consumed by a bundler during a compile step, being ESM-only will cause issues.
They are mostly compatible though. I think it can be a bit painful to import from one to the other, but it reliably works once you know how.
[Comment removed by author]
[Comment removed by author]
I hope people in the future will study the great javascript module cliff the same way they will study the python 3 cliff. There are some valuable lessons in how (and if) to do world-breaking changes the we seem unable to internalize yet.
What are the advantages to using a top-level await in a module? I still do not understand the upside of ESM over CommonJS, and I still write my new JavaScript using the old style. These days, I mostly write Typescript, however, which I think can compile to either.
I have a config module that can read AWS secrets to get the config data, which is exported as an object. There is no synchronous way to get an AWS secret, so that is a top-level await.
I also have a module that initializes a DB connection object, which isn’t asynchronous, but needs the DB password
config.dbPassword.I fixed this with a startup script that copies the secret into a file before starting Node, but it’s quite clunky.
Thank you for the specific examples! In the past, when I’ve had to do such things, I’ve wrapped them in a function call that could be awaited. For example, the former might have a line like
const cfg = await getConfig(), or maybe instead I would have cfg as a global, but make it invalid until thecfg.init()method had been awaited.Trouble is, some modules have non-async functions that need the config. But they can’t do a top-level
await getConfig()to ensure it’s there first. You could restructure everything that needs config to be async, but that would have been fragile and more effort than just giving up and generating a.envfile from the secret before starting Node.The reason you might want to use a top-level await is that it lets you run (asynchronously) run initialization code before exporting something. For example, you could export a database connection but first make sure it’s fully initialized.
I suppose this is subjective preference, but I like my module imports like I like my coffee: without TCP sockets being opened implicitly.
Though having said that, I suppose it isn’t uncommon in other languages to do something like load a file at module import, some lookup table or binary blob, and I guess in JS land that would need to be an async call.
The problem is not limited to IO. Loading WASM is an asynchronous operation for example.
If you have any form of touching of an async API during init you have this problem. And if you look at most JS APIs…. Well they tend towards async only