lint-staged: Running multiple instances of lint-staged in a monorepo leads to automatic backup errors

Description

Following the (dated) instructions for how to use lint-staged with husky in a monorepo, i tried to run lint-staged using yarn workspaces foreach run --parallel lint-staged in a husky script but that leads to the git stashing behavior to often report that there was a problem in restoring the automated backup.

so then i also tried running lint-staged with --no-stash and trying to do the stashing myself with git stash --keep-index and git stash pop, but that also leads to some strange behavior when lint-staged also does lint fixing - the stash pop producing errant files, and one of my coworkers reported lint-staged just removing every file in the entire repo on his machine somehow lol.

if i don’t parallelize it, running lint-staged just takes a long time since my project is large, so i’d really like for it to be able to run in parallel. Is there a better pattern for this for handling monorepos?

Steps to reproduce

  • make a yarn workspaces monorepo
  • add lint-staged as a dep to each of the workspaces
  • define a lint-staged file for each workspace, that does eslint --fix
  • install husky to the root, make a precommit hook that calles yarn workspaces foreach run --parallel lint-staged
  • make various edits, see that things happen confusingly

Debug Logs

expand to view 
➤ YN0000: [@every.org/partner-api-cloudflare-worker]: Running lint-staged with the following config:
➤ YN0000: [@every.org/partner-api-cloudflare-worker]: {
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   '*': async (files) => {
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   const prettierFiles = micromatch(
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:     files,
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:     prettierSupportedExtensions.map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   );
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   const eslintFiles = await filterEslintIgnoredFiles(
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:     micromatch(
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:       files,
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:       ["js", "ts", "jsx", "tsx"].map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:     )
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   );
➤ YN0000: [@every.org/partner-api-cloudflare-worker]: 
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   return [
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:     ...(eslintFiles.length > 0
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:       ? [`yarn eslint --fix ${eslintFiles.join(" ")}`]
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:       : []),
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:     ...(prettierFiles.length > 0
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:       ? [`prettier --write ${prettierFiles.map(addQuotes).join(" ")}`]
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:       : []),
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   ];
➤ YN0000: [@every.org/partner-api-cloudflare-worker]: }
➤ YN0000: [@every.org/partner-api-cloudflare-worker]: }
➤ YN0000: [@every.org/partner-api-cloudflare-worker]: ℹ No staged files match any configured task.
➤ YN0000: [@every.org/website-next]: Running lint-staged with the following config:
➤ YN0000: [@every.org/website-next]: {
➤ YN0000: [@every.org/website-next]:   '*': async (files) => {
➤ YN0000: [@every.org/website-next]:   const prettierFiles = micromatch(
➤ YN0000: [@every.org/website-next]:     files,
➤ YN0000: [@every.org/website-next]:     prettierSupportedExtensions.map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/website-next]:   );
➤ YN0000: [@every.org/website-next]:   const eslintFiles = await filterEslintIgnoredFiles(
➤ YN0000: [@every.org/website-next]:     micromatch(
➤ YN0000: [@every.org/website-next]:       files,
➤ YN0000: [@every.org/website-next]:       ["js", "ts", "jsx", "tsx"].map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/website-next]:     )
➤ YN0000: [@every.org/website-next]:   );
➤ YN0000: [@every.org/website-next]: 
➤ YN0000: [@every.org/website-next]:   return [
➤ YN0000: [@every.org/website-next]:     ...(eslintFiles.length > 0
➤ YN0000: [@every.org/website-next]:       ? [`yarn eslint --fix ${eslintFiles.join(" ")}`]
➤ YN0000: [@every.org/website-next]:       : []),
➤ YN0000: [@every.org/website-next]:     ...(prettierFiles.length > 0
➤ YN0000: [@every.org/website-next]:       ? [`prettier --write ${prettierFiles.map(addQuotes).join(" ")}`]
➤ YN0000: [@every.org/website-next]:       : []),
➤ YN0000: [@every.org/website-next]:   ];
➤ YN0000: [@every.org/website-next]: }
➤ YN0000: [@every.org/website-next]: }
➤ YN0000: [@every.org/website-next]: ℹ No staged files match any configured task.
➤ YN0000: [@every.org/common]: Running lint-staged with the following config:
➤ YN0000: [@every.org/common]: {
➤ YN0000: [@every.org/common]:   '*': async (files) => {
➤ YN0000: [@every.org/common]:   const prettierFiles = micromatch(
➤ YN0000: [@every.org/common]:     files,
➤ YN0000: [@every.org/common]:     prettierSupportedExtensions.map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/common]:   );
➤ YN0000: [@every.org/common]:   const eslintFiles = await filterEslintIgnoredFiles(
➤ YN0000: [@every.org/common]:     micromatch(
➤ YN0000: [@every.org/common]:       files,
➤ YN0000: [@every.org/common]:       ["js", "ts", "jsx", "tsx"].map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/common]:     )
➤ YN0000: [@every.org/common]:   );
➤ YN0000: [@every.org/common]: 
➤ YN0000: [@every.org/common]:   return [
➤ YN0000: [@every.org/common]:     ...(eslintFiles.length > 0
➤ YN0000: [@every.org/common]:       ? [`yarn eslint --fix ${eslintFiles.join(" ")}`]
➤ YN0000: [@every.org/common]:       : []),
➤ YN0000: [@every.org/common]:     ...(prettierFiles.length > 0
➤ YN0000: [@every.org/common]:       ? [`prettier --write ${prettierFiles.map(addQuotes).join(" ")}`]
➤ YN0000: [@every.org/common]:       : []),
➤ YN0000: [@every.org/common]:   ];
➤ YN0000: [@every.org/common]: }
➤ YN0000: [@every.org/common]: }
➤ YN0000: [@every.org/common]: [STARTED] Preparing...
➤ YN0000: [@every.org/common]: [SUCCESS] Preparing...
➤ YN0000: [@every.org/common]: [STARTED] Running tasks...
➤ YN0000: [@every.org/common]: [STARTED] Running tasks for *
➤ YN0000: [@every.org/common]: [STARTED] yarn eslint --fix /Users/omar/code/every.org/every.org/packages/commo…
➤ YN0000: [@every.org/common]: [SUCCESS] yarn eslint --fix /Users/omar/code/every.org/every.org/packages/commo…
➤ YN0000: [@every.org/common]: [STARTED] prettier --write "/Users/omar/code/every.org/every.org/packages/commo…
➤ YN0000: [@every.org/common]: [SUCCESS] prettier --write "/Users/omar/code/every.org/every.org/packages/commo…
➤ YN0000: [@every.org/common]: [SUCCESS] Running tasks for *
➤ YN0000: [@every.org/common]: [SUCCESS] Running tasks...
➤ YN0000: [@every.org/common]: [STARTED] Applying modifications...
➤ YN0000: [@every.org/common]: [SUCCESS] Applying modifications...
➤ YN0000: [@every.org/common]: [STARTED] Cleaning up...
➤ YN0000: [@every.org/common]: [SUCCESS] Cleaning up...
➤ YN0000: [@every.org/website]: Running lint-staged with the following config:
➤ YN0000: [@every.org/website]: {
➤ YN0000: [@every.org/website]:   '*': async (files) => {
➤ YN0000: [@every.org/website]:   const prettierFiles = micromatch(
➤ YN0000: [@every.org/website]:     files,
➤ YN0000: [@every.org/website]:     prettierSupportedExtensions.map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/website]:   );
➤ YN0000: [@every.org/website]:   const eslintFiles = await filterEslintIgnoredFiles(
➤ YN0000: [@every.org/website]:     micromatch(
➤ YN0000: [@every.org/website]:       files,
➤ YN0000: [@every.org/website]:       ["js", "ts", "jsx", "tsx"].map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/website]:     )
➤ YN0000: [@every.org/website]:   );
➤ YN0000: [@every.org/website]: 
➤ YN0000: [@every.org/website]:   return [
➤ YN0000: [@every.org/website]:     ...(eslintFiles.length > 0
➤ YN0000: [@every.org/website]:       ? [`yarn eslint --fix ${eslintFiles.join(" ")}`]
➤ YN0000: [@every.org/website]:       : []),
➤ YN0000: [@every.org/website]:     ...(prettierFiles.length > 0
➤ YN0000: [@every.org/website]:       ? [`prettier --write ${prettierFiles.map(addQuotes).join(" ")}`]
➤ YN0000: [@every.org/website]:       : []),
➤ YN0000: [@every.org/website]:   ];
➤ YN0000: [@every.org/website]: }
➤ YN0000: [@every.org/website]: }
➤ YN0000: [@every.org/website]: [STARTED] Preparing...
➤ YN0000: [@every.org/website]: [SUCCESS] Preparing...
➤ YN0000: [@every.org/website]: [STARTED] Running tasks...
➤ YN0000: [@every.org/website]: [STARTED] Running tasks for *
➤ YN0000: [@every.org/website]: [STARTED] yarn eslint --fix /Users/omar/code/every.org/every.org/packages/websi…
➤ YN0000: [@every.org/website]: [SUCCESS] yarn eslint --fix /Users/omar/code/every.org/every.org/packages/websi…
➤ YN0000: [@every.org/website]: [STARTED] prettier --write "/Users/omar/code/every.org/every.org/packages/websi…
➤ YN0000: [@every.org/website]: [SUCCESS] prettier --write "/Users/omar/code/every.org/every.org/packages/websi…
➤ YN0000: [@every.org/website]: [SUCCESS] Running tasks for *
➤ YN0000: [@every.org/website]: [SUCCESS] Running tasks...
➤ YN0000: [@every.org/website]: [STARTED] Applying modifications...
➤ YN0000: [@every.org/website]: [SUCCESS] Applying modifications...
➤ YN0000: [@every.org/website]: [STARTED] Cleaning up...
➤ YN0000: [@every.org/website]: [SUCCESS] Cleaning up...
➤ YN0000: [@every.org/api]: Running lint-staged with the following config:
➤ YN0000: [@every.org/api]: {
➤ YN0000: [@every.org/api]:   '*': async (files) => {
➤ YN0000: [@every.org/api]:   const prettierFiles = micromatch(
➤ YN0000: [@every.org/api]:     files,
➤ YN0000: [@every.org/api]:     prettierSupportedExtensions.map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/api]:   );
➤ YN0000: [@every.org/api]:   const eslintFiles = await filterEslintIgnoredFiles(
➤ YN0000: [@every.org/api]:     micromatch(
➤ YN0000: [@every.org/api]:       files,
➤ YN0000: [@every.org/api]:       ["js", "ts", "jsx", "tsx"].map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/api]:     )
➤ YN0000: [@every.org/api]:   );
➤ YN0000: [@every.org/api]: 
➤ YN0000: [@every.org/api]:   return [
➤ YN0000: [@every.org/api]:     ...(eslintFiles.length > 0
➤ YN0000: [@every.org/api]:       ? [`yarn eslint --fix ${eslintFiles.join(" ")}`]
➤ YN0000: [@every.org/api]:       : []),
➤ YN0000: [@every.org/api]:     ...(prettierFiles.length > 0
➤ YN0000: [@every.org/api]:       ? [`prettier --write ${prettierFiles.map(addQuotes).join(" ")}`]
➤ YN0000: [@every.org/api]:       : []),
➤ YN0000: [@every.org/api]:   ];
➤ YN0000: [@every.org/api]: }
➤ YN0000: [@every.org/api]: }
➤ YN0000: [@every.org/api]: [STARTED] Preparing...
➤ YN0000: [@every.org/api]: [SUCCESS] Preparing...
➤ YN0000: [@every.org/api]: [STARTED] Running tasks...
➤ YN0000: [@every.org/api]: [STARTED] Running tasks for *
➤ YN0000: [@every.org/api]: [STARTED] yarn eslint --fix /Users/omar/code/every.org/every.org/packages/api/s…
➤ YN0000: [@every.org/api]: [SUCCESS] yarn eslint --fix /Users/omar/code/every.org/every.org/packages/api/s…
➤ YN0000: [@every.org/api]: [STARTED] prettier --write "/Users/omar/code/every.org/every.org/packages/api/s…
➤ YN0000: [@every.org/api]: [SUCCESS] prettier --write "/Users/omar/code/every.org/every.org/packages/api/s…
➤ YN0000: [@every.org/api]: [SUCCESS] Running tasks for *
➤ YN0000: [@every.org/api]: [SUCCESS] Running tasks...
➤ YN0000: [@every.org/api]: [STARTED] Applying modifications...
➤ YN0000: [@every.org/api]: [SUCCESS] Applying modifications...
➤ YN0000: [@every.org/api]: [STARTED] Cleaning up...
➤ YN0000: [@every.org/api]: [FAILED] lint-staged automatic backup is missing!
➤ YN0000: Done in 17s 857ms
🚨 Lint fix command failed! Please fix the lint errors by hand.
If you want to ignore this and commit anyway, run HUSKY=0 git commit
husky - pre-commit hook exited with code 1 (error)```

</details>

### Environment

<!-- Tell us about your development environment -->
- **OS:** Mac Big Sur
- **Node.js:* 14.16.1
- **`lint-staged`:** 11.0.0

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 20
  • Comments: 21 (4 by maintainers)

Most upvoted comments

We are working on a better monorepo support. You’re supposed to run lint-staged only once from the root, and then use multiple configurations. This has some other issues at the moment, but it’s the way forward… Maybe you can try to change your setup towards that?

+6
iiroj on Feb 15, 2022

You should not be installing it in every package. If it’s still required for it to work then we need a separate issue with as many details as possible but you should be running one instance from the root with multiple configs in every package. Right @iiroj ?

+4
okonet on Jun 7, 2022

@iiroj

Im getting some pid issues using latest version and in a monorepo setup:

◼ Applying modifications from tasks...
◼ Cleaning up temporary files...
/XXX/node_modules/pidtree/lib/pidtree.js:61
      callback(new Error('No matching pid found'));
               ^

Error: No matching pid found
    at /XXX/node_modules/pidtree/lib/pidtree.js:61:16
    at /XXX/node_modules/pidtree/lib/ps.js:40:7
    at ChildProcess.<anonymous> (/XXX/node_modules/pidtree/lib/bin.js:45:5)

mind you, this was the commit to stage multiple lint-stagedrc files, as soon as they were added everything looks to be in order.

+4
grumpyoldman-io on Mar 21, 2022

You should not be installing it in every package. If it’s still required for it to work then we need a separate issue with as many details as possible but you should be running one instance from the root with multiple configs in every package. Right @iiroj ?

Thanks, you’re right. Tested again with lint-staged install at root and only config files for each package is working for me.

+1
waldronmatt on Jun 8, 2022

It wasn’t clear to me at first from this thread how to actually set up lint-staged with a monorepo. Here is what worked for me following @iiroj 's comment via https://github.com/okonet/lint-staged/issues/988#issuecomment-1039948428

At first I had a lint-staged.config.js at the root of the monorepo. Do NOT do this. This caused the error in the comment above ^ and wiped all my staged changes.

Here’s what worked for me:

  1. Install lint-staged at the monorepo root ONLY.
  2. Install lint-staged in package-a AND include your lint-staged.config.js
  3. Repeat the process for package-b and so on

The repo I have this configured on: https://github.com/waldronmatt/module-federation-template . Using Lerna, Yarn Workspaces, Turbo, Husky, Lint-Staged, Commitizen, Commitlint

+1
waldronmatt on Jun 7, 2022

@wbern currently parent globs like ../*.js are broken, and then there’s this: https://github.com/okonet/lint-staged/pull/1106

+1
iiroj on Feb 15, 2022
Read more comments on GitHub
← lint-staged: Running in a subdirectory of a git repo incorrectly runs in other directories
lint-staged: The command line is too long(windows) →