ts-node: Fail to load module from node_moules

Since I’m using Typescript for my project, I also prefer Typescript modules as the dependencies. So I think it’s necessary to compile dependencies or project won’t work using ts-node.

Node.js v13 adds new complications. For anyone stumbling here, this is what you may face if you’re using Node.js v13+, but I have not found a solution yet.

It seems the ignore trick no longer works if both using module: 'commonjs' and having dependencies that are published as ES Modules.

So although my ignore options is set correctly like I had it before Node 13 while I have module set to commonjs, I get an error like the following error when Node tries to import an ES Module from the CommonJS module output of ts-node:

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/trusktr/src/infamous+infamous/node_modules/lowclass/index.js
require() of ES modules is not supported.
require() of /Users/trusktr/src/infamous+infamous/node_modules/lowclass/index.js from /Users/trusktr/src/infamous+infamous/src/core/Observable.ts is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/trusktr/src/infamous+infamous/node_modules/lowclass/package.json.

Now you may think “just change module to esnext”. Well, it ain’t that easy!

Here’s the problem: the entry point that loads ts-node is still a CommonJS module, so as soon as ts-node supplies ES Module output, then you’ll get an error when you import your first .ts file, like:

import * as fs from 'fs';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at wrapSafe (internal/modules/cjs/loader.js:1055:16)
    at Module._compile (internal/modules/cjs/loader.js:1103:27)
    at Module.m._compile (/Users/trusktr/src/infamous+infamous/node_modules/ts-node/src/index.ts:536:23)
    at Module._extensions..js (internal/modules/cjs/loader.js:1159:10)
    at Object.require.extensions.<computed> [as .ts] (/Users/trusktr/src/infamous+infamous/node_modules/ts-node/src/index.ts:539:12)
    at Module.load (internal/modules/cjs/loader.js:988:32)
    at Function.Module._load (internal/modules/cjs/loader.js:896:14)
    at Module.require (internal/modules/cjs/loader.js:1028:19)
    at require (internal/modules/cjs/helpers.js:72:18)

Next, you might think “well, now just convert the .js file that loads ts-node into an ES Module instead of CommonJS and give it the .mjs extension or use --input-type=module, and now import your .ts file instead of requireing it”. For example:

import tsNode from 'ts-node'

tsNode.register({
    typeCheck: false,
    transpileOnly: true,
    files: true,
    project: './tsconfig.json',
    ignore: false,

    compilerOptions: {
        baseUrl: './',
        module: 'esnext', // out with the old, in with the new!
    },
})

// require('./readem.ts') // instead of this,
import './readem.ts'   // now try this.

Unfortunately, now this won’t work, because .ts extensions are not something that ES Modules understand by default, so now errors like the following happen:

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /Users/trusktr/src/infamous+infamous/readem.ts imported from /Users/trusktr/src/infamous+infamous/readem.mjs

TLDR: ts-node is not yet in good shape to deal with the new ES Modules (unless I missed something).

Node.js v13 adds new complications. For anyone stumbling here, this is what you may face if you’re using Node.js v13+, but I have not found a solution yet.

It seems the ignore trick no longer works if both using module: 'commonjs' and having dependencies that are published as ES Modules.

So although my ignore options is set correctly like I had it before Node 13 while I have module set to commonjs, I get an error like the following error when Node tries to import an ES Module from the CommonJS module output of ts-node:

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/trusktr/src/infamous+infamous/node_modules/lowclass/index.js
require() of ES modules is not supported.
require() of /Users/trusktr/src/infamous+infamous/node_modules/lowclass/index.js from /Users/trusktr/src/infamous+infamous/src/core/Observable.ts is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/trusktr/src/infamous+infamous/node_modules/lowclass/package.json.

Now you may think “just change module to esnext”. Well, it ain’t that easy!

Here’s the problem: the entry point that loads ts-node is still a CommonJS module, so as soon as ts-node supplies ES Module output, then you’ll get an error when you import your first .ts file, like:

import * as fs from 'fs';
^^^^^^

SyntaxError: Cannot use import statement outside a module
    at wrapSafe (internal/modules/cjs/loader.js:1055:16)
    at Module._compile (internal/modules/cjs/loader.js:1103:27)
    at Module.m._compile (/Users/trusktr/src/infamous+infamous/node_modules/ts-node/src/index.ts:536:23)
    at Module._extensions..js (internal/modules/cjs/loader.js:1159:10)
    at Object.require.extensions.<computed> [as .ts] (/Users/trusktr/src/infamous+infamous/node_modules/ts-node/src/index.ts:539:12)
    at Module.load (internal/modules/cjs/loader.js:988:32)
    at Function.Module._load (internal/modules/cjs/loader.js:896:14)
    at Module.require (internal/modules/cjs/loader.js:1028:19)
    at require (internal/modules/cjs/helpers.js:72:18)

Next, you might think “well, now just convert the .js file that loads ts-node into an ES Module instead of CommonJS and give it the .mjs extension or use --input-type=module, and now import your .ts file instead of requireing it”. For example:

import tsNode from 'ts-node'

tsNode.register({
    typeCheck: false,
    transpileOnly: true,
    files: true,
    project: './tsconfig.json',
    ignore: false,

    compilerOptions: {
        baseUrl: './',
        module: 'esnext', // out with the old, in with the new!
    },
})

// require('./readem.ts') // instead of this,
import './readem.ts'   // now try this.

Unfortunately, now this won’t work, because .ts extensions are not something that ES Modules understand by default, so now errors like the following happen:

TypeError [ERR_UNKNOWN_FILE_EXTENSION]: Unknown file extension ".ts" for /Users/trusktr/src/infamous+infamous/readem.ts imported from /Users/trusktr/src/infamous+infamous/readem.mjs

TLDR: ts-node is not yet in good shape to deal with the new ES Modules (unless I missed something).

Same situation

The hard coded node_modules is actually a problem for repositories relying on how require works to load sub-projects in the same monorepo (like Pouchdb, Cerebraljs). If you store your projects like this:

node_modules // <--- root node_modules for installed deps
packages
  + node_modules
    + @myorg
      + moduleA // <--- now this one can import latest '@myorg/moduleB' without linking and such
      + moduleB

This setup works fine until testing cannot happen because ts-node does not compile in node_modules (which is in the test path).

Please give us a way to fix this.

To allow monorepo packages to be transpiled, something like this works:

// tsconfig.json - transpile all packages in the @monorepo namespace
{
   "ts-node": {
      "ignore": [ "/node_modules/(?!@monorepo)" ]
   }
}

In case it helps anyone, here’s how to do it using the API (instead of CLI) with regex(es):

require('ts-node').register({
  typeCheck: false,
  transpileOnly: true,
  files: true,

  // HERE, you can use RegExp literals here.
  ignore: [
    /node_modules\/(?!@scoped\/package)/,
    /node_modules\/(?!unscoped-package)/
  ],

  compilerOptions: require('./tsconfig.json').compilerOptions,
})

Since ignore wants a pattern. We can use a regex to whitelist package.

Here is an example for my_package => --ignore='node_modules\/(?!my_package)'

@blakeembrey This is really a confused discussion. I will never publish a typescript module and this is not the issue.

The issue is that people work inside a repository that happens to be packages/node_modules. Look at PouchDB source or Cerebral.js.

The issue is that we cannot use ts-node to test this working directory. It has nothing to do with publishing.

For example, these are not installed modules, but the actual packages developed by Cerebral.js: https://github.com/cerebral/cerebral/tree/master/packages/node_modules

@blakeembrey Can you please explain in what sense using a public API is bad practice ?

Using node_modules to load modules is totally part of node.js API. There is no hack whatsoever here. Reasons to use such a folder architecture are absolutely valid. Here is a discussion related to PouchDB: alle monorepo, we don’t have a public summary of our discussion at Cerebral.js but the reasons are similar and I have also good reasons with Lucidity to move away from lerna.

I think there are simple ways to avoid compiling vendor modules and still allow complex open source projects to use ts-node for testing. One such way would be to be able to specify, like .gitignore does for example:

"ignore": [
  "node_modules",
  "!/packages/node_modules"
]

@cleavera No, they don’t. You should probably do some research into the topic yourself to get familiar. TypeScript libraries should be distributed in JavaScript with declarations enabled. Please don’t publish raw TypeScript libraries and fragment the ecosystem for no reason. Internally, you can do whatever you want of course.

Realistically, the setup you’re building is kind of awkward. You’re relying on a global mutation that’ll be done by a dependent. Why not just compile the files to JavaScript? What about having both projects register with ts-node? What if one project uses different compilation settings to the other? Or they have different dependencies on .d.ts files that happen to conflict when you put them together?

No problem 😄 Let me know if there’s any way to improve the docs to avoid issues in the future.

It’s exactly the same and nothing is confused here. If you check the README you can see how to get around it (--ignore=false should work, see https://github.com/TypeStrong/ts-node#configuration-options). I won’t be changing it by default, which is what I already said. Unless you have some magic solution that detects just node modules of your package vs node modules installed, I don’t see a better solution.

I am really sorry. I don’t know how I missed this --ignore flag. I never intended for this to be the default.

I had a similar problem to what @zixia had (sharing internal code between Typescript projects) which led me to this issue. Since it appears to be a design decision not to support .ts files in the node_modules folder, the solution I went with was to use a postinstall script in the library’s package.json.

Example:

  "scripts": {
    "postinstall": "tsc -d -p ."
  },

I also added a tsconfig.json file to the root folder of the library with all the compiler settings.

Thanks @romainPrignon. It seems that the same type of thing is necessary when dealing with esnext tsc-produced libraries as well. We were getting SyntaxError: Unexpected identifier on our esnext or es2015 modules, but the following solved it.

I whitelisted our npm org and things started working:

NODE_ENV=production ts-node \
    -r tsconfig-paths/register \
    --files \
    --project tsconfig.run.json \
    --ignore 'node_modules\/(?!@alienfast)' \
    ./src/index.ts 

As my perspective, I agree with @gaspard before, but I have to say, today I’ll agree with @blakeembrey.

Last year, I want to publish pure TypeScript module to NPM, and actually, I did. I also have a workaround about this, you could have a look at this thread at https://github.com/TypeStrong/ts-node/issues/155#issuecomment-235243048 .

The reason I want to do this is that at first:

1. I fall in love with TypeScript so I want to use it anywhere(ts only)
2. I’m not familiar with public TypeDefination file with my module
3. I’m not familiar with ES6 Module with Rollup tools

Today, I learned from Angular how to publish NPM in UMD format with TypeDefination, it’s very easy to do this with an npm script, just as @blakeembrey said, really is a two seconds work. So I have no reason to keep my module in ts only.

At last, if there’s no dark side of using pure TypeScript NPM module, I believe it is possible to support it by ts-node. However, the dark side I could image is: when a javascript developer found your module from NPM and npm -i it, but it could not be able to run.. This will do hurt the npm eco-system.

So today I vote NO for publishing pure TypeScript module to NPM. Hope this could help others like me to make the right decision today. 😃

I’m also running into similar issues. I have a dependency that distributes itself compiled to ES2015 I believe, with ES modules. I’m having a difficult time messing with all of the possible Node.js and ts-node settings to try and get it to work. Fortunately for me, I control the dependency so I might just go add a commonjs compilation to it.

Until this point, ts-node has generally worked wonderfully for me. The dream for me would be for ts-node to simply handle any files or dependencies that you point at it, be they JavaScript, TypeScript, CommonJS, or ES Modules. IMO ts-node should simply take an initial file as input and just make it work

I believe I’ve hit this issue with a project using Node.js v14. Basically, I’m trying to use a dependency with ES Modules and "type": "module" in its package.json, and getting the error below:

ts-node script.ts

// script.ts
import { BoxBufferGeometry, BufferGeometry, Group, Matrix4, Mesh, Vector3 } from 'three';
import { Geometry } from 'three/examples/jsm/deprecated/Geometry';

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: $PROJECTS/three.js/examples/jsm/deprecated/Geometry.js
require() of ES modules is not supported.
require() of $PROJECTS/three.js/examples/jsm/deprecated/Geometry.js from $PROJECTS/three-to-cannon/test/index.test.ts is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename Geometry.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from $PROJECTS/three.js/examples/jsm/package.json.

It seems to be the same issue described here (or have I got that wrong?) so I’m a bit surprised to see it marked as a “won’t fix”. Are ES Modules simply not supported, and there’s no interest in supporting them? Support for ES Modules in Node.js itself has been improving in recent versions, and I’d be glad to see ts-node take advantage of that. 😃

@gaspard

How does your final --ignore configuration look like? I also have a mono repo setup and find that simply specifying --ignore=false does not work for me, as ts-node would try to compile various files from the vendor’s node_module directory which leads to errors.

On the other hand, I’ve not found a good regex to specify a robust ignore pattern:

• (?<!\/packages)\/node_modules\/ does not work due to JavaScript’s lack of look behind
• \/project_name\/node_modules\/ works, but it hardcodes the root folder name of the repo and would fail if some developer renames the directory
• nesting everything in another subdirectory looks most robust to me, but is extremely ugly considering the fact that the mono repo structure already requires two directory levels.

Any suggestions?

PS: @blakeembrey: Please fix the typo in the ticket’s title (“node_moules”) – it’s impossible to find when searching for ‘node_modules’.

@blakeembrey thanks for point me from #158 to here, sorry for not notice this issue before I post.

I do suggest ts-node to compiling dependencies. because is a good way to modulize pure typescript code if we do not want to compile .ts file any more. (it’s why ts-node here, right?)

@zengfenfei I ran into the same issue as you today. I have to make this work by soft link this:

$ mv node_modules/my_mod .
$ ln ../my_mod node_modules

# then ts-node will compile it...

more detail you can find in my dup issue #158