ESM loading fails for native [email protected] loader

[hbarcelos]

Description
I'm trying to import xstate using Node@13 native support for ES Modules.

Expected Result
Should be able to import as specified is the docs

Actual Result

The requested module 'xstate' does not provide an export named 'Machine'

Reproduction

Simply run the following code after installing xstate and run it with Node >= 13.0.0

import { Machine } from 'xstate';

Additional context
I've tried both setting the "type": "module" in my package.json and renaming the script file to *.mjs, but the error still persists.

Out of curiosity, I decided to use the import-star syntax to see what happened:

import * as xs from 'xstate';

console.log(xs);

It outputs:

[Module] {
  default: {
    matchesState: [Function: matchesState],
    mapState: [Function: mapState],
    StateNode: [Function: StateNode],
    State: [Function: State] {
      from: [Function (anonymous)],
      create: [Function (anonymous)],
      inert: [Function (anonymous)]
    },
    Machine: [Function: Machine],
    createMachine: [Function: createMachine],
    send: [Function: send],
    sendParent: [Function: sendParent],
    sendUpdate: [Function: sendUpdate],
    assign: [Function (anonymous)],
    doneInvoke: [Function: doneInvoke],
    forwardTo: [Function: forwardTo],
    interpret: [Function: interpret],
    Interpreter: [Function: Interpreter] {
      defaultOptions: [Object],
      interpret: [Function: interpret]
    },
    spawn: [Function: spawn],
    matchState: [Function: matchState],
    actions: {
      raise: [Function: raise],
      send: [Function: send],
      sendParent: [Function: sendParent],
      sendUpdate: [Function: sendUpdate],
      log: [Function: log],
      cancel: [Function (anonymous)],
      start: [Function: start],
      stop: [Function: stop],
      assign: [Function (anonymous)],
      after: [Function: after],
      done: [Function: done],
      respond: [Function: respond],
      forwardTo: [Function: forwardTo],
      escalate: [Function: escalate]
    },
    ActionTypes: {
      Start: 'xstate.start',
      Stop: 'xstate.stop',
      Raise: 'xstate.raise',
      Send: 'xstate.send',
      Cancel: 'xstate.cancel',
      NullEvent: '',
      Assign: 'xstate.assign',
      After: 'xstate.after',
      DoneState: 'done.state',
      DoneInvoke: 'done.invoke',
      Log: 'xstate.log',
      Init: 'xstate.init',
      Invoke: 'xstate.invoke',
      ErrorExecution: 'error.execution',
      ErrorCommunication: 'error.communication',
      ErrorPlatform: 'error.platform',
      ErrorCustom: 'xstate.error',
      Update: 'xstate.update',
      Pure: 'xstate.pure'
    },
    SpecialTargets: { Parent: '#_parent', Internal: '#_internal' }
  }
}

So it looks like even though state has an ESM target, it doesn't seem it's being properly loaded, because this looks like the CommonJS exported module.

For reference, here's my package.json:

{
  "name": "xstate-labs",
  "version": "1.0.0",
  "module": "index.js",
  "license": "MIT",
  "type": "module",
  "dependencies": {
    "xstate": "^4.7.7"
  }
}

Tested with [email protected].

[Andarist]

Thanks for the report - I will take a look later to see what's the best way to support this.

[hbarcelos]

Just a little more context:

I tried to pair it up with typescript@3.7.5 to see if I could get rid of the error.

However, after running ts-node index.ts with node@13.7.0, I got the following error:

(node:4678) ExperimentalWarning: The ESM module loader is experimental.
TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /home/henrique/labs/xstate-labs/src/index.ts
    at Loader.defaultGetFormat [as _getFormat] (internal/modules/esm/get_format.js:71:15)
    at Loader.resolve (internal/modules/esm/loader.js:98:42)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at Loader.getModuleJob (internal/modules/esm/loader.js:188:29)
    at Loader.import (internal/modules/esm/loader.js:163:17)

Then I downgraded to node@12.14.1 it works just fine with TypeScript.

I'm guessing this is more of a general problem with node@13 ESM loader.

[Andarist]

I've educated myself about ESM support in node@13 and I think I understand its intricacies now. It's not as easy to support it as I have thought - when you also want to support older nodes, bundlers etc (and we do).

I will explore making it work in v5, but at the moment I don't think it's worth it to include this in v4.

If you want to patch your node_modules and xstate in it then I think this should work if you add this to pkg.json

"type": "module",
"exports": "./es/index.js"

[jrogado]

I get the same problem with node v13.9.0 and xstate 4.8.0.

SyntaxError: The requested module 'xstate' does not provide an export named 'interpret'

Any developments on this issue?

[Andarist]

Literally the previous comment says that we wont work on this right now and that we are going to tackle this in upcoming v5

[davidkpiano]

With that said, I'd accept a PR from someone who wants to fix this for the current version.

[Andarist]

Problem is that this would be a breaking change for all of our ESM-node consumers - im not sure if it would be a good thing to “fix” this within v4 for that reason because we know that this would break all of those consumers immediately.

[jrogado]

So, in the meantime what is your advice to keep things gong? Revert to a previous version of node.js? Which, by the way?

[jrogado]

In the meantime, found this workaround:

npm install esm
node -r esm xstateTest.js

Also added this line to launch.json node xstateTest.js launch configuration:
"runtimeArgs": ["-r", "esm"]

[Andarist]

Sorry for not responding sooner, I've missed your reply.

So, in the meantime what is your advice to keep things gong?

1. use CJS
2. or use default export for now
3. or use esm loader like you have done.

[Andarist]

Just an update - we'll be providing "exports" field in the future in the XState 5. It's not yet implemented and nor v5 is ready to be shipped even as a prerelease but at least I have a plan now to make this happen. At this point in time, pkg.json#exports semantics seem to be pretty stable as webpack is going to ship support for it soon-ish, so it should be safe for us to start using that in the future.

[horacioh]

I know this is not going to be fixed for this version (I read the older comments), but just as a question: does this it not fixed by either using the .mjs extension or adding the "type": "module" to the package.json?

trying to learn about these issues myself, since I have a full-ESM setup even for testing (using vitest)

[Andarist]

.mjs extension might break some older tools that might work with XState today.

I feel super bad about issues like this - I understand the pain of the users trying to utilize this. I've wasted dozens of hours maneuvering through that ESM mess, trying to figure out a solution that would work everywhere, with every tool, for every package. This has been proven to be just impossible.

Some solutions work for some packages while they might not work for some other packages. Trying to figure out a perfect combination of things that can be done to address this in a particular package is exhausting and is always associated with a risk that we won't get it right for the first time (for the record - I've been going through this with Nicolo from the Babel team and it took him like 5 patch releases of the @babel/runtime to land on something that has worked in all of the supported contexts. The situation with XState would probably be simpler but still...).

We also can't just use the type: module or .mjs because that would make it impossible to require XState in node and there is surely code out there that needs this to work.

However, I think that we are only using named exports and currently node should be able to load CJS files with named exports from an ESM file - if only those named exports are declared in a certain way (to be statically analyzable). If you prepare a runnable repro case of the problem I might take a look at this to assess what's going on. Note that this was not true at the beginning and there are node versions out there that can't do this - but AFAIK all of the latest versions in any major line of node with ESM support should already be able to handle this.

[horacioh]

I have a reproduction repo I can push tonight! I manage to fix it by setting a config on vitest side.

one solution that could work is basically having a separate esm version of the library. since this could be just a different build config it could be generated so we can avoid code duplication and inconsistencies. something like:

import { createModel } from 'xstate/esm'

not sure if this is important for this version or maybe v5!

thanks for your response @Andarist !

[horacioh]

adding some references of issues that happens with other packages with similar configurations as xstate:

vitejs/vite#6395

https://github.com/horacioh/vitest-radix-demo

[Andarist]

I've verified that node can load XState just fine, using the code like this:

import { createMachine } from 'xstate'
import { createModel } from 'xstate/lib/model.js'

console.log({
    createMachine,
    createModel
})

If the issue is about Vite/Vitest... well, this is kinda their issue, right? 😉 I mean, especially for Vitests, this sounds like an infancy problem that hopefully should be fixed somewhat soon. Fixing this anyhow on our side would be rather counter-productive because that would only fix compatibility with this specific tool for a single package (XState). The proper fix to fix compatibility with valid packages should be pursued on their side.

[horacioh]

@Andarist yes you are right, there's no real issue withing XState in order to use it with Vite nor Vitest. I was commenting the issues I have related to libraries as XState that does not export .mjs modules or have the "type": "module" attribute in the package.json.

I add those references since you mentioned you are trying to figure out a way to make the compat issue for v5. Just as a reference, not as something that needs to change in XState specifically.

Sorry that you felt it was something I wanted to be fixed here just to make it work with my current tooling, that was not the intent!! Actually I would love some hints and other links I can read to understand this compat issue better, but this is also out of scope of this issue ;)

Looking forward to see how v5 looks like!! 💪🏼

[Andarist]

The last time that I've checked this, XState was actually compatible with ESM loader in node. That is mostly thanks to us not using default exports. It is true that XState was not compatible with that right from the start. However, that was back when the ESM support was still marked as experimental in node. The improved support for loading CJS modules from ESM modules has been introduced in this PR and ESM modules have only been marked as stable after that.

So I think it's fair to close this issue. If you find any problems with any tooling - I could take a look at your repro cases to assess what's wrong and where. I don't think though that such issues would be XState issues.