rollup-plugin-typescript2: rpt2 does not resolve pnpm symlinked dependencies

Fixed the issue!

So I believe I’ve found a fix for this long-standing issue! See #332

Notes on root cause analysis / debugging

Red herring to look at resolveId hook etc

I’ve been looking at the code and I’m mystified. Leaving more info here for myself and other passing-by folks:

It was helpful to see this previous research, but this seemed to be a bit of a red herring… I’ll explain shortly.

This issue doesn’t list “plugin output with verbosity: 3” as requested in the issue template, but #330 did, and I confirmed the fix in its reproduction. I also took the reproduction here and added verbosity: 3 and confirmed it had very similar logs. I also confirmed (by locally editing rpt2 in node_modules) that the fix works for this reproduction as well.

Notably, the logs don’t output anything about “resolving”, so the bug here isn’t in the resolveId hook at all. Which is why looking at resolve etc was a red herring. That portion of the code also resolves directly with TS and with tsModule.sys as its host. As I found out later, ts.sys has a default implementation of realpath, which turns out to be the important missing piece here.

Error is in transform hook actually

It actually errors right after the “transpiling” log statement, which means the error is actually in the transform hook, specifically in getEmitOutput, which is implemented by TS and doesn’t have host-specific code for it… But apparently it seems to rely host.realpath under-the-hood.

Implementing host.realpath fixes this

So I tried implementing host.realpath (as #332 shows), and then everything worked and the type-error disappeared! The typings were generated correctly as well now (#330 had broken typings when using check: false).

TS docs on Compiler APIs leave a lot to be desired… 😕 😞

The TS docs are incredibly sparse when it comes to Compiler integrations such as this plugin. The Compiler API basically has two small docs/examples in the TS Wiki… and that’s it. Notably, there is no mention of realpath what-so-ever in either of those.

Honestly the root cause of a lot of TS integration issues can come down to poor documentation of the Compiler API, which has been noted in lots of places. As a result, this requires even more significant domain knowledge to figure out issues like this, as the docs aren’t even helpful – I had to literally read through the Compiler source code… and have had to do that a handful of times in the past too. That’s obviously not ideal, though at least we can do something like that in open-source.

Basically you need to be a TS Compiler subject matter expert to figure out some of these things, and I’m not sure I would even call myself that despite being a regular contributor to the TS tooling ecosystem (compilers are huge after all!).

Misc

So you are saying rpt2 should pass { preserveSymlinks: false } option to resolve?

false is actually the default of resolve, Rollup itself, TS itself, which all reflect how Node itself treats symlinks (i.e. require.resolve). And that was a red herring anyway

I also think we may not need resolve as a dep at all since we can use built-in Node methods (resolve was originally created for browserify after all).

Released in 0.32.0 🎉

I’ve been looking at the code and I’m mystified. Leaving more info here for myself and other passing-by folks:

• In theory resolve already supports symlinks (browserify/resolve#196 was fixed in browserify/resolve#197)

• This is also supported in @rollup/plugin-resolve via the rollup option preserveSymlinks. Though it seems to use a different resolve algorithm. https://github.com/rollup/plugins/blob/9ad41a0760b4eeed9fd26d2da9b5122aa8a351e5/packages/node-resolve/src/index.js#L148-L156

• This plugin does use resolve.sync if rollupCommonJSResolveHack is set to true. But it does not pass an options object.

https://github.com/ezolenko/rollup-plugin-typescript2/blob/428140bd2260e3ff29b1255029331e2c5b436fa2/src/index.ts#L160-L162

Based on the above, I cannot see any combination of options and/or plugins that will correctly resolve pnpm symlinks. Maybe I could try to implement it? It looks simple enough, which probably means it is more complicated than I can see 😅

rollup-plugin-ts works too and also exports compiled d.ts typings without much of a hassle.

I’m starting to suspect this is not an issue with rpt2, despite what the error says, but rather a problem that’s propagating upstream.

Closing until I have more information.

[… shortened for readability …] So you are saying rpt2 should pass { preserveSymlinks: false } option to resolve?

I got the same problem in my project with pnpm. I debugged line by line and found the problem is getAllReferences function returned a path the symlink, not the path of the real file or directory. https://github.com/ezolenko/rollup-plugin-typescript2/blob/428140bd2260e3ff29b1255029331e2c5b436fa2/src/tscache.ts#L63-L75

for examples, consider this following directory

node_modules
├─.pnpm
|  ├─vue@3.2.2
|  |  ├─node_modules
|  |  |   ├─@vue
|  |  |   ├─vue
packages
|  ├─demo
|  |   ├─node_modules
|  |   │  └─vue(symlink, real path should be `project_root/node_modules/.pnpm/vue@3.2.2/node_modules/vue`)
|  |   ├─public
|  |   └─src

As you see, the real path of vue in demo should be project_root/node_modules/.pnpm/vue@3.2.2/node_modules/vue But after resolving by tsModule.nodeModuleNameResolver in function getAllReferences , it becomes project_root/packages/demo/node_modules/vue

Vue@3.2.2 depends on @vue, which is not exist in project_root/packages/demo/node_modules, so it throw an error while resolving.

I wonder if there is any way to solve this problem. Otherwise i can only use pnpm --shamefully-hoist to hoist all packages to root node_modules, which is similar to yarn/npm.😅

@davelsan

This plugin does use resolve.sync if rollupCommonJSResolveHack is set to true. But it does not pass an options object. […] Based on the above, I cannot see any combination of options and/or plugins that will correctly resolve pnpm symlinks. Maybe I could try to implement it? It looks simple enough, which probably means it is more complicated than I can see 😅

So you are saying rpt2 should pass { preserveSymlinks: false } option to resolve?

Hi @beetaa. Thanks for your interest in this issue. I have updated the original repro and I can confirm the issue is still there. I’m re-opening the discussion as per your request.

Thing is, it works fine with the original ts plugin before I had debug failure errors, and running tsc by itself works fine as well.

Update

According to this comment in the pnpm repo, there was a similar problem caused by the rollup-plugin-commonjs plugin (note that I’m not using it in the repro).

However, none of the provided solutions will work in this case.

// include the actual pnpm registry path
commonjs({ include: '../../node_modules/.pnpm/registry.npmjs.org/**' })

// or set the following option in rollup.config.js
preserveSymlinks: true