angular-cli: NgccProcessor slowing down subsequent builds with AngularCompilerPlugin

I’m using the AngularCompilerPlugin for a medium-sized project and I’ve observed a slowdown in build times after enabling Ivy which appears to come from ngcc. When Ivy is enabled the AngularCompilerPlugin runs the NgccProcessor which is supposed to compile the node_modules for Ivy on the first run, however in my case any subsequent builds are slowed down as well.

I have a hacky way to disable the NgccProcessor which I’ve used to measure the differences in build times. Here’s what it looks like:

class MyAngularCompilerPlugin extends AngularCompilerPlugin {
    public apply(compiler: Compiler): void {
        super.apply(compiler);
        
        compiler.hooks.environment.tap('angular-compiler', () => {
            (this as any)._compilerHost.ngccProcessor = undefined;
        });
    }
}

Here are the timings for 5 production builds on my own machine. Note that these times are after ngcc has been run once over the node_modules in a postinstall script.

// Without ngcc (80737ms on average)
84520ms
72547ms
81733ms
85043ms
79844ms
------------------------
// With ngcc (94208ms on average)
93741ms
93714ms
92922ms
97220ms
93444ms

The 15s slower build time for a production build isn’t a big deal locally, however the difference becomes even larger when running it on our CI server. For our production builds we need to run about 30 Webpack builds (10 at a time in parallel), about 20 which go through the AngularCompilerPlugin. Disabling the NgccProcessor on our CI server reduced our build time from around 10 minutes to about 6 minutes. Furthermore keeping the NgccProcessor led to some flakiness in our builds, presumably because 10 different ngcc processes were trying to access file at the same time. Our build script looks something like this:

  1. Run npm ci.
  2. Run ngcc programmatically as follows:
const ngcc = require('@angular/compiler-cli/ngcc');
ngcc.process({
    basePath: path.join(projectRoot, 'node_modules'),
    compileAllFormats: false,
    propertiesToConsider: ['browser', 'module', 'main'],
    createNewEntryPointFormats: true
});
  1. Run webpack --mode=production 30 times by splitting them into chunks of 10 that are run concurrently. We have some logic that tries to make sure that we always run the maximum number of builds, but it stays the same no matter whether ngcc is enabled so it shouldn’t have an effect on the times.

I can see a couple of ways to get around this:

  1. Short term: add an option to the AngularCompilerPlugin config that allows users that know what they’re doing to disable the NgccProcessor.
  2. Long term: rework either ngcc or NgccProcessor so that it doesn’t have to spend time analyzing the node_modules after it has been run already.

cc @filipesilva

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 19 (15 by maintainers)

Most upvoted comments

It is worth taking a look at the package.json after the original ngcc run and then after the CLI compilation?

We should note that multiple properties in package.json map to the same code format. E.g. module and fesm5 and esm5 are the same “format”.

So although you have built the esm5 format in the first run of ngcc (via the module property), there might be a case where the CLI attempts to build an additional entry point, which happens to map to a format that has already been built. In that case you would not see a “Compiling…” message but ngcc might still write to the package.json to say that an entry-point has been processed.

Actually I think that we don’t need the flag… if the node_modules have already been compiled by ngcc then the CLI+ngcc should not need to write any files, so there should be no flakes…

@crisbeto - thanks for the report. I did a big performance refactoring of ngcc in https://github.com/angular/angular/pull/30525 but I believe that only landed in 8.2.0-next.1.

This fixed a problem where ngcc was parsing the entire node_modules tree every time it was being run from the AngularCompilerPlugin.

I see that https://github.com/angular/angular/pull/31509 has not appeared in any release yet. But it should be released with 8.2.0-next.2. Can you take another look at the performance once that is released. If it is still a problem then we can prioritize work on it.