jest-preset-angular: Slow tests with 9.0.0-next.6

💥 Performance Regression Report

I’ve spent a lot more time than I’d like to admit upgrading to 9.0.0-next* and getting things working, and trying all different configurations to determine if it’s ready to use. This past week was the 3rd time I’ve attempted to upgrade, but my conclusion is that the performance hit from upgrading to 9.0.0-next is still too big. I’m detailing a bunch of things that I tried here in hopes of both helping the performance effort for this project, and helping others with perf troubleshooting.

In a nutshell, I’ve found the release version (jest-preset-angular@8.3.1) to yield acceptably fast tests - “I’d like faster, but Jest is the best option for most tests”. And I’ve found 9.0.0-next.6, and earlier 9.0.0-next versions, with isolatedModules: false, unacceptably slow - 7x slower than @8.3.1 for the first run.

Note that 9.0.0-next.6 with isolatedModules: true is FAST - it’s what I’d hoped for using Ivy in Jest. Unfortunately we don’t want the limitations of typescript compiling with isolatedModules, as there’s no other benefit to using isolatedModules that we care about. If isolatedModules resulted in faster Angular build times, or some other similar benefit, it would probably be worth giving up const enum.

Primary request: Is there any way we can use the same jest compilation path that is being used for isolatedModules: true, but without isolatedModules: true in tsc?

Secondary request: Explanation as to why isolatedModules: true is soooo much faster than isolatedModules: false in 9.0.0-next.

Last working version

Worked up to version: 8.3.1

Stopped working in version: 9.0.0-next.3, 9.0.0-next.6

Motivation

The main reason I’ve been putting so much effort into upgrading to 9.0.0-next.6 is Ivy support: There are a number of benefits (type safety + build perf), but the main reason is that we have code that depends on Ivy behavior, than doesn’t exist in ViewEngine (right now we have to run those tests in Karma). Also, it makes sense to test with the same code used in production.

Since it’s available, I also decided to try Jest + node ESM support. Since we use ESM (primarily) in prod, and many libraries ship separate code for ESM, let’s use the ESM code in tests. And, I was hopeful it would help perf - partly b/c I figured we could use pre-built library code within Jest, reducing Jest’s compilation workload.

Upgrade steps

To clarify the changes between these different test runs:

yarn add -D jest-preset-angular@next jest@next

jest.config.js changes :

require('jest-preset-angular/ngcc-jest-processor');

module.exports = {
  preset: 'jest-preset-angular/presets/defaults', // or 'jest-preset-angular/presets/defaults-esm',

setup-jest.ts changes : Replace import 'jest-preset-angular'; with import 'jest-preset-angular/setup-jest';

Upgrading to ESM

In jest.config.js, use ‘jest-preset-angular/presets/defaults-esm’.

Run tests using node --experimental-vm-modules ./node_modules/jest/bin/jest.js .

Performance Tactics

A few of the things I tried to improve Jest performance:

• Switching to use ESM. It turns out this doesn’t help noticeably, probably b/c UMD modules are loaded by Jest’s default resolver - for now, at least. It also doesn’t achieve my other goal, of using the same library code as we use in production - for now everything run in Jest is UMD (or CommonJS) and ES5.

Note that I could load internal ESM files during Jest compilation by using this in my Jest.config (and equivalent in tsconfig.spec.json):

moduleNameMapper: ‘^@this/([a-z\-]+)/([a-z\-]+)$’: ‘<rootDir>/dist/$1/fesm2015/this-$1-$2.js’

But all the external dependencies (angular, zone, rxjs) are still loaded as UMD - as noted in #751, this requires a resolver that is ESM aware.

• Using pre-built library build output instead of source references, to reduce Jest compile workload.

In other words, changing tsconfig.spec.json from:

    "paths": {
      "@this/*": [
        "libs/*/src/public-api.ts"
      ],
    }

to

    "paths": {
      "@this/*": [
        "dist/*"
      ]
    }

and jest.config.js from :

  moduleNameMapper: {
    '^@this/(.*)$': '<rootDir>/libs/$1/src/public-api.ts'
  }

to

  moduleNameMapper: {
    '^@this/(.*)$': '<rootDir>/dist/$1', <-- Linking doesn't work
    'tslib': '<rootDir>/node_modules/tslib/tslib.es6.js'
  }

This didn’t work for me because we use some language features that break in ES5 (base class properties), and this linking path combined with jest’s default resolver pulls in UMD/ES5 library code. You can work around that for internal libraries using:

  moduleNameMapper: {
	'^@this/([a-z\\-]+)/([a-z\\-]+)$': '<rootDir>/dist/$1/fesm2015/this-$1-$2.js',
    '^@this/([a-z\\-]+)$': '<rootDir>/dist/$1/fesm2015/this-$1.js',
    'tslib': '<rootDir>/node_modules/tslib/tslib.es6.js'
  }

But it will still use UMD/ES5 for external libraries.

Timing data

This is non-scientific, run on an 8 core laptop with SSD.

jest-preset-angular@8.3.1 with jest 26, isolatedModules: false (default) : for 392 tests, 78 suites First run (no cache): 62s 2nd run (uses cache): 27s

jest-preset-angular@8.3.1 with jest 26, isolatedModules: true : for 381 tests, 71 suites (8 suites don’t compile) First run (no cache): 42s 2nd run (uses cache): 32s

jest-preset-angular@9.0.0-next.6 with jest jest@27.0.0-next.2, isolatedModules: false, ‘jest-preset-angular/presets/defaults’, +ngcc-jest-processor for 392 tests, 78 suites First run (no cache): 446s NOTE: The first test doesn’t complete for 115s 2nd run (uses cache): 33s

jest-preset-angular@9.0.0-next.6 with jest jest@27.0.0-next.2, isolatedModules: true, ‘jest-preset-angular/presets/defaults’ for 381 tests, 71 suites (8 suites don’t compile) First run (no cache): 43s 2nd run (uses cache): 32s

jest-preset-angular@9.0.0-next.6 with jest jest@27.0.0-next.2, isolatedModules: false, ‘jest-preset-angular/presets/defaults-esm’ for 392 tests, 78 suites First run (no cache): 435s NOTE: The first test doesn’t complete for 117s 2nd run (uses cache): 41s

jest-preset-angular@9.0.0-next.6 with jest jest@27.0.0-next.2, isolatedModules: true, ‘jest-preset-angular/presets/defaults-esm’ for 282 tests, 52 suites (27 suites don’t compile) First run (no cache): 41s 2nd run (uses cache): 32s

jest-preset-angular@9.0.0-next.6 with jest jest@27.0.0-next.2, isolatedModules: false, ‘jest-preset-angular/presets/defaults-esm’, internal library ESM references for 392 tests, 78 suites First run (no cache): 258s NOTE: The first test doesn’t complete for 40s 2nd run (uses cache): 40s

jest-preset-angular@9.0.0-next.6 with jest jest@27.0.0-next.2, isolatedModules: true, ‘jest-preset-angular/presets/defaults-esm’, internal library ESM references for 287 tests, 55 suites (24 suites don’t compile) First run (no cache): 38s 2nd run (uses cache): 27s

Tests per second for different configurations

Configuration	1st run	2nd run	1st run tests/sec	2nd run tests/sec
8.3.1	62s	27s	6.3	14.5
8.3.1 + isolatedModules	42s	32s	9.1	11.9
9.0	446s	33s	0.9	11.9
9.0 + isolatedModules	43s	32s	8.9	11.9
9.0 + ESM	435s	41s	0.9	9.6
9.0 + ESM + isolatedModules	41s	32s	6.9	8.8
9.0 + ESM + internal library ref	258s	40s	1.5	9.8
9.0 + ESM + isolatedModules + internal library ref	38s	27s	7.6	10.6

Link to repo

I can’t expose all our tests, but the setup is very similar to this repo + branch:

https://github.com/johncrim/repro-ivy-jest-preset-angular/tree/bug/inline-directive

envinfo

System:
    OS: Windows

Npm packages:
    jest: 27.0.0-next.2
    jest-preset-angular: 9.0.0-next.6
    typescript: 4.0.5