webpack: Unexpected behavior with linked NPM modules

Hey, just a quick update on our side. We’ve managed to solve our problems using npm linked dependencies by:

Using Webpack 2 and setting

  resolve: { 
      symlinks: false,
      modules: [
        path.resolve(__dirname, '..', 'node_modules'),
        'node_modules'
      ],
  }

So the local/root node_modules is always preferred, and symlinks are not resolved, i.e. same as the node --preserve-symlinks options, this was implemented in: https://github.com/webpack/webpack/issues/2937

This is working as expected for us, if that helps anyone stumbling upon this issue.

I’ve written a resolve plugin which takes care of this issue.

What is does is tries to resolve any module calls to the closest one to the provided rootDir, always preferring local ones, as long as the version of the closer dependency is within the requested package’s SemVer range. Useful especially when using linked modules, but even when not using linked modules it it’s pretty useful, as you don’t end up with duplicate copies of the same package if another copy is within a nested node_modules.

It’s called RootMostResolvePlugin and it’s a part of the webpack-dependency-suite (sorry, no docs yet).

Configuration is pretty simple (example).

Make sure to put the plugin in the resolve.plugins array of your configuration (not plugins).

When instantiating, the first parameter should be the root directory of your project (or the one in which context you want to resolve) and the second, optional parameter is whether you want to enforce it for all paths (e.g. including linked modules, use true), or only for nested node_modules (false).

just to provide a bit more insight of what is working, i was able to get this working in webpack 2 with a similar resolve config as above. thanks @jure!

resolve: { 
  symlinks: false,
  modules: [path.resolve('node_modules')],
}

it seems like it would pretty safe to default to the local node_modules/ without needing to configure resolve.modules, maybe based on context?

I don’t really see why people would want the current behaviour (same package with same version being bundled twice), but it could be an opt-in feature, if you think it’s gonna cause issues for some.

Anyone trying to deal with these issues and spending hours on it not getting anywhere isn’t going to care that the Node.js resolution algorithm is broken, getting dependencies to be properly bundled without duplicates is more important. And of course - by opting in you’d understand what you’re doing and the consequences.

One possible solution is not using npm link at all. If you are not using windows, here is a nice alternative https://github.com/wix/wml

I stumbled upon this issue in a monorepo where one package is a dependency of another via "other-package": "file:../other-package".

But this is no webpack issue, the root cause seems to be that npm installs the deps of the sub project in the sub projects node_modules folder instead of the parent projects one (as it is normally the case).

There seems to be a few issues open in npm regrading this [1], [2], [3]. The last one points out, that this behavior is per spec (see: [4]).

But I found another solution for this: using yarn instead of npm will not create the unnecessary node_modules folder in the sub project and install all deps in the parent project.

1: https://npm.community/t/npm-does-not-flatten-packages-during-install/3989/3 2: https://npm.community/t/npm-doesnt-hoist-linked-module-dependencies-to-project-level-when-new-dependencies-are-added-to-linked-module/5717/3 3: https://npm.community/t/npm-doesnt-install-local-package-dependencies/805 4: https://github.com/npm/cli/blob/latest/doc/spec/file-specifiers.md#installation

@niieani Thanks. Your plugin fixes this nicely npm@4.3.0, webpack@2.2.1

So for this particular problem, there are now two useful solutions (but quite tricky ones):

1.) npm linking the linked module’s dependencies to the project’s dependency, i.e. https://github.com/webpack/webpack/issues/966#issuecomment-95491120

2.) sharing a node_modules folder between linked module and project, i.e. https://github.com/webpack/webpack/issues/554#issuecomment-203128656

I don’t like either of those due to their manual/repetitive/unstable nature. I’d prefer this solution: https://github.com/aurelia/webpack-plugin/issues/44#issuecomment-238028171, a webpack ResolverPlugin that replaces the linked modules node_modules requires with the project’s. That’s what we’re attempting. Any other solutions out there?

Update: (thanks @ganmor https://github.com/webpack/webpack/issues/985#issuecomment-256300077)

3.) not using npm link at all for development, but instead using some kind of file syncing tool, i.e. https://github.com/wix/wml or http://www.freefilesync.org/

Preferring the application node_modules with resolveLoader.root worked well enough with npm 2 - but with npm 3 it means that you will randomly pull in the wrong version of a package when there are duplicate versions within the app.

For instance - say my app uses moduleA which depends on lodash@3 and moduleB which depends on lodash@4. The npm3 install ends up looking like this:

• lodash@3
• moduleA
• moduleB
  • lodash@4

If I set resolveLoader.root - moduleB ends up pulling in lodash 3 rather than 4 (since lodash@3 just happened to get installed at root).

the workaround explained using npm link to get one version of react only (on both webpack and node) helped me https://github.com/webpack/webpack/issues/966#issuecomment-95491120

Just an idea, but why not just compare if the same version of the same package is already bundled in? Or make that an opt-in feature? Like for example, check first three.js package.json to resolve version, and then when three.js is about to be bundled in from another location, check the package.json again and stop doing so if it’s the same.

The underlying issue is that after linking in modules, npm doesn’t deduplicate the node_modules tree so you can end up having /node_modules/three and ../symlinked-package/node_modules/three both available. And then when three is imported from symlinked-package it will use its own local three version, but when three is imported by the host application built by webpack it’ll use /node_modules/three, resulting in 2 three.js versions bundled in at the same time.

But this seemingly npm-specific issue is definitely not going away, considering that symlinking packages into node_modules is a very common workflow. So I think it makes sense for the improvement to be introduced on the webpack side, instead.