eslint-webpack-plugin: Threads massively slows down linting / re-linting

Operating System: Windows 10
Node Version: v14.15.1
YARN Version: 1.22.4
webpack Version: 4.44.2
eslint-webpack-plugin Version: 2.4.0

This is a problem that was experienced in react-scripts / create-react-app v4.0.1 and v4.0.2

Compiling and Re-compiling changes seems extremely slow compared to pre v4 version. After digging a bit, it seems the culprit is threads: true pointed out by @radoi90 here - https://github.com/webpack-contrib/eslint-webpack-plugin/issues/50#issuecomment-732376285

Expected Behavior

Fast compiling

Actual Behavior

Actual very slow compiling / recompiling.

I dug a bit deeper to see if there’s a small issue with the threads and it might be 2 issues, don’t know if it’s actually the same.

On the first compile:

Linting a particular file seems fast but it takes a while to move to a new file - anywhere between 20ms to a few seconds.

I’ve added before this line: https://github.com/webpack-contrib/eslint-webpack-plugin/blob/46fc07a456bb95899ac9957bc836dadf9e970099/src/index.js#L91

  console.timeEnd('start lint next');
  console.time('linted in');

  // Queue file for linting.
  lint(file);

  console.timeEnd('linted in');
  console.time('start lint next');

and my results look like this:

start lint next: 19.038ms
files to lint: E:\project\src\components\layout\aFile.tsx
linted in: 0.702ms

start lint next: 133.277ms
files to lint: E:\project\src\components\settings\bFile.tsx
linted in: 1.106ms

start lint next: 577.165ms
files to lint: E:\project\src\stores\cFile.ts
linted in: 1.451ms

start lint next: 98.303ms
files to lint: E:\project\src\stores\dFile.ts
linted in: 1.143ms

start lint next: 148.661ms
files to lint: E:\project\src\stores\eFile.ts
linted in: 0.999ms

start lint next: 901.431ms
files to lint: E:\project\src\stores\fFile.ts
linted in: 1.096ms

start lint next: 13.968ms
files to lint: E:\project\src\functions\gFile.ts
linted in: 0.657ms

start lint next: 86.591ms
files to lint: E:\project\src\stores\thumb-gen\hFile.ts
linted in: 1.701ms

(this is a small excerpt of 300+ files that need to be linted at first run)

I’ve also measured: https://github.com/webpack-contrib/eslint-webpack-plugin/blob/46fc07a456bb95899ac9957bc836dadf9e970099/src/index.js#L101 which took awaitReport: 29.909ms

and https://github.com/webpack-contrib/eslint-webpack-plugin/blob/46fc07a456bb95899ac9957bc836dadf9e970099/src/index.js#L114 generateReportAsset: 0.177ms

then I’ve edited just a single file to see what happens there and it looks something like this:

files to lint: E:\project\src\components\layout\Header.tsx
linted in: 3.163ms
awaitReport: 16.754s // 16 seconds!!


files to lint: E:\project\src\components\layout\Header.tsx
linted in: 1.515ms
awaitReport: 11.264s // 11 seconds!!

Code

https://github.com/facebook/create-react-app/blob/9b08e3c9b365ac790546a3d5027d24f264b42613/packages/react-scripts/config/webpack.config.js#L753-L771

setting threads: false as here https://github.com/facebook/create-react-app/issues/10154#issuecomment-734757965 solves the slow lint / relint

// additional code, HEY YO remove this block if you don't need it

How Do We Reproduce?

I guess try it on a newly created React project with create-react-app - https://github.com/facebook/create-react-app/issues/10145#issuecomment-734674629

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 19 (11 by maintainers)

Most upvoted comments

@eek I will try to reproduce as well in the morning.

Great.

I’ve also tested it now and what @imsandez reported here - https://github.com/facebook/create-react-app/issues/10145#issuecomment-734674629 is true.

if you create a new react app:

npx create-react-app test-app --template typescript

(I’ve created a TS one to be closer to what I use)

I takes between 3-6s to compile a change in App.tsx (closer to 3s rather than 6s) but once you downgrate the eslint-webpack-plugin by adding resolutions at the end of package.json:

  "resolutions": {
    "react-scripts/eslint-webpack-plugin": "2.3.0"
  }

and yarn again. (don’t know if npm supports resolutions)

Compile will take maximum 0.5s.

With my custom eslintConfig:

  "eslintConfig": {
    "parser": "@typescript-eslint/parser",
    "extends": [
      "airbnb-typescript",
      "plugin:@typescript-eslint/eslint-recommended"
    ],
    "parserOptions": {
      "project": "./tsconfig.json",
      "ecmaVersion": 8,
      "ecmaFeatures": {
        "experimentalObjectRestSpread": true
      }
    },
    "plugins": [
      "@typescript-eslint",
      "import",
      "react"
    ],
    "env": {
      "browser": true
    },
    "globals": {
      "gtag": true,
      "isNaN": true
    },
    "settings": {
      "import/resolver": {
        "node": {
          "extensions": [
            ".js",
            ".jsx",
            ".ts",
            ".tsx"
          ],
          "moduleDirectory": [
            "node_modules",
            "src/"
          ]
        }
      }
    }
  },

on the blank project it always takes 6s. Once I downgrade to the previous eslint-webpack-plugin it takes 0.5s tops.

eek on Nov 30, 2020

Using the defaults provided by npx create-react-app test-app --template typescript I’m getting 1-2ms recompile time editing App.tsx. Adding your lint config makes it substantially slower and exposes what the slow down is.

The jest-worker doesn’t auto-warm up the pool. So until you hit max-pool size each lint request is paying the eslint init cost. After each compile, we cleanup the pool. So, each recompile rebuilds the pool… and since incremental edits often are one file… that takes a pool init + module init + ESLint setup AND run… So, my first stab at this is to disable the pool after the first build. All rebuilds would be on the main process and not respin the pool.

I can go further to preserve the pool (and warmup) such that re-compiles don’t start over. Dispatching seems to eat ~150-200ms. So recompiles will always be slower by that much minimum. Would some lint configs warrant always run off thread? or would just running lint on the main thread after the first compile be sufficient? @ricardogobbosouza @eek @alexander-akait

jsg2021 on Nov 30, 2020

@jsg2021 In my view the problem is here, I think https://github.com/webpack-contrib/eslint-webpack-plugin/blob/master/src/getESLint.js#L58 Is carrying the eslint twice

ricardogobbosouza on Nov 30, 2020