webpack: [Performance] Webpack 2.4.0+ slower than 2.3.x

OK, I have a potential fix in enhanced-resolve webpack/enhanced-resolve#92. Essentially, there will be a new resolve option called cacheWithContext, which is on by default, which includes request.context when considering whether a resolved module is cached or not. For users who don’t need to consider context, they can set it to false, and that’ll give back the perf we lost.

On top of that, in Webpack proper #5178, I have a fix to check if the user has provided any resolve or resolveLoader plugins, and if they haven’t (which is the majority use case), set cacheWithContext to false.

And then in Webpack 4, we can make resolve or resolveLoader plugins provide the needsContext property, and if none do, set cacheWithContext to false.

Feel free to test #5178 out folks and see if it speeds up your builds again.

I’m going to look into this today.

v2.3.2: 39 seconds v2.5.1: 64 seconds v3.0.0-rc.1: 62 seconds

So, still a problem in v3

(Cross post) webpack 3 - 65 seconds webpack 3 with @mikesherov PR - 34.4 seconds webpack 2.3.3 - 34.0 seconds

😃

@sokra I think I’m not alone in wanting a resolution to this, like many have already mentioned, it’s keeping me stuck on an older version for one of my (larger) projects. It seems the reason for this pref regression got identified as a result of tighter caches thanks to da29d21 could we make that behavior opt-in or even if you could just show us how we could change our configs to avoid sending that extra context information, as you mentioned, then we’d be able to enjoy all the benefits of webpack@3

Because no internal resolvers need the context information, and introducing an option to provide it would be a breaking change, my plan is as follows:

1. Stop passing context information like da29d21 did unless a custom resolve plugin is provided. This will give performance back to the overwhelming majority of folks who don’t use resolvePlugins.
2. For Webpack 4.0, require resolve plugins to provide a property called needsContext (similar to cacheable for loader plugins). Then, when plugins are registered, Webpack can determine if any custom plugins actually need context, and only provide context in the case at least one needs it.

@sokra thoughts?

We have stopped at v.2.3.3 at my company until this this issue is resolved. +~50% build time is a lot for large bundles.

da29d21 will cause that the resolve cache can be used less often.

It passes issuer to the resolver. This allows to resolve depending on issuer. Maybe we can add an option to specify the context information passed to the resolver, so you get higher performance if you don’t need it.

I went ahead and manually changed the code in that commit back in my local dependencies while using 2.5.1, and it got the performance boost back. So I can confirm that the performance regression was introduced in this commit: https://github.com/webpack/webpack/commit/983904cfa354eb00256f44b973a539090a19c7c6

The problem with this is, that commit is from this pull request: https://github.com/webpack/webpack/pull/4622

so, reverting that commit looks like it would break using context.issuer in custom resolvers.

I’m curious what @sokra was going for with the resolveContextInfo in this commit: https://github.com/webpack/webpack/commit/da29d21ae44056ccbd525282300df7184e922d97#diff-1fd8fe07e83988b2301f9bad408cf5c2

Is it possible to keep the context feature, but also speed up performance back to what it was before that commit?

Another angle to investigate is: is this actually webpack? It doesn’t seem like this change should create such a performance hit?

We managed to sneak even more perf into #5178 thanks to a speedup @gdborton found, so the master branch should be even faster than 2.3.3 !