angular-cli: NgccProcessor slowing down subsequent builds with AngularCompilerPlugin
I’m using the AngularCompilerPlugin
for a medium-sized project and I’ve observed a slowdown in build times after enabling Ivy which appears to come from ngcc
. When Ivy is enabled the AngularCompilerPlugin
runs the NgccProcessor
which is supposed to compile the node_modules
for Ivy on the first run, however in my case any subsequent builds are slowed down as well.
I have a hacky way to disable the NgccProcessor
which I’ve used to measure the differences in build times. Here’s what it looks like:
class MyAngularCompilerPlugin extends AngularCompilerPlugin {
public apply(compiler: Compiler): void {
super.apply(compiler);
compiler.hooks.environment.tap('angular-compiler', () => {
(this as any)._compilerHost.ngccProcessor = undefined;
});
}
}
Here are the timings for 5 production builds on my own machine. Note that these times are after ngcc
has been run once over the node_modules
in a postinstall
script.
// Without ngcc (80737ms on average)
84520ms
72547ms
81733ms
85043ms
79844ms
------------------------
// With ngcc (94208ms on average)
93741ms
93714ms
92922ms
97220ms
93444ms
The 15s slower build time for a production build isn’t a big deal locally, however the difference becomes even larger when running it on our CI server. For our production builds we need to run about 30 Webpack builds (10 at a time in parallel), about 20 which go through the AngularCompilerPlugin
. Disabling the NgccProcessor
on our CI server reduced our build time from around 10 minutes to about 6 minutes. Furthermore keeping the NgccProcessor
led to some flakiness in our builds, presumably because 10 different ngcc
processes were trying to access file at the same time. Our build script looks something like this:
- Run
npm ci
. - Run
ngcc
programmatically as follows:
const ngcc = require('@angular/compiler-cli/ngcc');
ngcc.process({
basePath: path.join(projectRoot, 'node_modules'),
compileAllFormats: false,
propertiesToConsider: ['browser', 'module', 'main'],
createNewEntryPointFormats: true
});
- Run
webpack --mode=production
30 times by splitting them into chunks of 10 that are run concurrently. We have some logic that tries to make sure that we always run the maximum number of builds, but it stays the same no matter whetherngcc
is enabled so it shouldn’t have an effect on the times.
I can see a couple of ways to get around this:
- Short term: add an option to the
AngularCompilerPlugin
config that allows users that know what they’re doing to disable theNgccProcessor
. - Long term: rework either
ngcc
orNgccProcessor
so that it doesn’t have to spend time analyzing thenode_modules
after it has been run already.
cc @filipesilva
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 19 (15 by maintainers)
It is worth taking a look at the package.json after the original ngcc run and then after the CLI compilation?
We should note that multiple properties in
package.json
map to the same code format. E.g.module
andfesm5
andesm5
are the same “format”.So although you have built the
esm5
format in the first run of ngcc (via themodule
property), there might be a case where the CLI attempts to build an additional entry point, which happens to map to a format that has already been built. In that case you would not see a “Compiling…” message but ngcc might still write to the package.json to say that an entry-point has been processed.Actually I think that we don’t need the flag… if the node_modules have already been compiled by ngcc then the CLI+ngcc should not need to write any files, so there should be no flakes…
@crisbeto - thanks for the report. I did a big performance refactoring of ngcc in https://github.com/angular/angular/pull/30525 but I believe that only landed in 8.2.0-next.1.
This fixed a problem where ngcc was parsing the entire node_modules tree every time it was being run from the
AngularCompilerPlugin
.I see that https://github.com/angular/angular/pull/31509 has not appeared in any release yet. But it should be released with 8.2.0-next.2. Can you take another look at the performance once that is released. If it is still a problem then we can prioritize work on it.