dts-bundle-generator: Bug With Symlinked Packages in Monorepo

it’s how dts-bundle-generator setups it internally. If you try to run tsc --build tsconfig.json in the packages/tooling-tests/todo-list-cli - it works without errors. The only difference is that composite: false for the generator …

Because it switches the compiler to another way of working, which is a build mode or know as composite projects (or referenced projects) that aren’t supported right now (see #93).

Ok I was able to run this on wsl, here is my observations:

• by default (if you don’t change the config file) you get bunch errors like error TS2307: Cannot find module or its corresponding type declarations., by digging into the issue I found out that this is the behaviour of the compiler - it treats files in node_modules as something that shouldn’t have an emit so it doesn’t generate any output for them (that’s what we’ve discussed above, I agree with the compiler and having a code that handles this correctly is kind of strange and I wouldn’t like to do this). Even though I don’t quite understand why these files are treated as files from node_modules, I’d expect them to be treated as local ones, because after resolve they don’t have node_modules in their paths. But I think this is the questions to the compiler team:

• if you add followSymlinks: false to the config, then it warns with… the same issue actually but different errors - now the tool complains about files that are in the compilation but they are not d.ts ones. But later the building ended with an error Error: Cannot find symbol for node: Plugin. After debugging this problem I figured out, that for some reason the compiler decides to load types for jest package and it causes this error (it comes directly from its types). If you add "types": ["node"], to tsconfig for the tool the problem goes away. But then of course the compilation fails because in the declaration bundle it finds something that shouldn’t be there, for example a function’s body, and it comes directly from the warning.

So I’d say probably the best solution here is to use paths. Alternatively you can ask the compiler team why the compiler doesn’t generate an output for files that were resolved from node_modules to a local folder (outside of node_modules; so basically the case is the following: node_modules/@internal-package/src/index.ts -> ../internal-package/src/index.ts, but the compiler treats this file as “comes from node_modules” and doesn’t generate an output for it).

Another solution could be set up the right order of building packages and compile them first and then bundle the declarations.

@zaripych sorry for late reply.

After reading this thread again I think that I’m not going to implement compiling files from the node_modules folder, this is not how the compiler works, this is not how the other tools are supposed to work and by adding this “feature” we might allow and “support” to use an incorrect and a kind of bad practice of developing and maintaining packages (think about this in wide, not your particular example that you’d like to solve). Once it is added, it won’t be possible to remove it. This is just not right.

But the good news, it looks like the issue might be related to the way how the linking of packages via pnpm works and how pnpm links dependencies inside the node_modules folder and I think that this is something that the tool can add and maintain (and probably should, if there is a problem).

Is it possible for you to create a repro for your issue (with pnpm usage and getting Error: Cannot find symbol for node: Plugin error) so I can take a look what’s is going on there? I think it would be much helpful to understand why exactly the current solution doesn’t work for pnpm’s workspaces.

---

Below some replies to your messages and questions.

I wonder if these somewhat naive code changes I’ve made (that basically make dts-bundle-generator tsc-compile anything it references on demand when needed) could be a start of a solution to support composite projects? Basically all we need is to call getDeclarationFiles for every referenced project, right?

For supporting composite projects there is an issue, see #93. But IIRC there were some issues with supporting this, because the compiler’s API for composite projects is different it it might not be possible to get the output for everything. Also, there might be a question how to treat your composite parts - as an “externals” or “internals” (but it is possible that I’m messing it up with other tool https://github.com/timocov/ts-transformer-properties-rename).

dts-bundle-generator already slow because it has on demand compilation, because now it compiles more projects - it might be slower - but I think it all comes down to how large the projects are

If you run it against d.ts files, it won’t compile any ts files and will use these files as it. It is not perfect and blazing fast, but at least something and if you already compiled your project, you don’t need to compile it once again.

We could start supporting incremental builds and instead of generating in memory .d.ts allow TS to cache results

If you’re talking about incremental compilation that the compiler supports this might be tricky. From what I’ve seen, the compiler doesn’t store anything about the compilation besides the deps tree, last modified dates and some other meta information that helps the compiler to say “nothing has changed from the last compilation so nothing to do”. But once you change a file, the compiler will compile the project anyway. So from my observations the compiler doesn’t store or cache any results in that way.

The way how the compilation of composite projects works is that the compiler for a dependency doesn’t use ts file but use .d.ts compiled file (that’s why declaration: true is required for dependencies) (it is looking similar to what we were talking about above, isn’t it? 😄).

On the other side, the tool requires the files to be compiler and type checked because the tool relies on that and this cannot be omitted.

@timocov the config didn’t help, and it prevented from other npm packages mounted by pnpm from working as well. To clarify, with following config:

{
  "compilationOptions": {
    "preferredConfigPath": "./tsconfig.json",
    "followSymlinks": false
  },
  "entries": [
    { "filePath": "src/index.ts", "outFile": "dist/dist/main.es.d.ts" },
    { "filePath": "lint.ts", "outFile": "dist/dist/lint.es.d.ts" }
  ]
}

dts-bundle-generator gives me error:

Error: Cannot find symbol for node: Plugin

But with this it works 👍

{
  "compilationOptions": {
    "preferredConfigPath": "./tsconfig.json",
    "followSymlinks": true
  },
  "entries": [
    { "filePath": "src/index.ts", "outFile": "dist/dist/main.es.d.ts" },
    { "filePath": "lint.ts", "outFile": "dist/dist/lint.es.d.ts" }
  ]
}

The Plugin in the error message probably refers to the rollup’s Plugin type that I meant to expose in the interface of my package.

Basically the dts-bundle-generator doesn’t work with pnpm unless it follows symlinks and doesn’t work with my monorepo setup, unless I pre-bundle the entire dependency tree manually first. If you insist that this is an ok behaviour for this product - it’s unfortunate for me.