Running multiple instances of lint-staged in a monorepo leads to automatic backup errors

Description

Following the (dated) instructions for how to use lint-staged with husky in a monorepo, i tried to run lint-staged using yarn workspaces foreach run --parallel lint-staged in a husky script but that leads to the git stashing behavior to often report that there was a problem in restoring the automated backup.

so then i also tried running lint-staged with --no-stash and trying to do the stashing myself with git stash --keep-index and git stash pop, but that also leads to some strange behavior when lint-staged also does lint fixing - the stash pop producing errant files, and one of my coworkers reported lint-staged just removing every file in the entire repo on his machine somehow lol.

if i don’t parallelize it, running lint-staged just takes a long time since my project is large, so i’d really like for it to be able to run in parallel. Is there a better pattern for this for handling monorepos?

Steps to reproduce

make a yarn workspaces monorepo
add lint-staged as a dep to each of the workspaces
define a lint-staged file for each workspace, that does eslint --fix
install husky to the root, make a precommit hook that calles yarn workspaces foreach run --parallel lint-staged
make various edits, see that things happen confusingly

Debug Logs

expand to view

➤ YN0000: [@every.org/partner-api-cloudflare-worker]: Running lint-staged with the following config:
➤ YN0000: [@every.org/partner-api-cloudflare-worker]: {
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   '*': async (files) => {
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   const prettierFiles = micromatch(
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:     files,
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:     prettierSupportedExtensions.map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   );
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   const eslintFiles = await filterEslintIgnoredFiles(
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:     micromatch(
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:       files,
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:       ["js", "ts", "jsx", "tsx"].map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:     )
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   );
➤ YN0000: [@every.org/partner-api-cloudflare-worker]: 
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   return [
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:     ...(eslintFiles.length > 0
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:       ? [`yarn eslint --fix ${eslintFiles.join(" ")}`]
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:       : []),
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:     ...(prettierFiles.length > 0
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:       ? [`prettier --write ${prettierFiles.map(addQuotes).join(" ")}`]
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:       : []),
➤ YN0000: [@every.org/partner-api-cloudflare-worker]:   ];
➤ YN0000: [@every.org/partner-api-cloudflare-worker]: }
➤ YN0000: [@every.org/partner-api-cloudflare-worker]: }
➤ YN0000: [@every.org/partner-api-cloudflare-worker]: ℹ No staged files match any configured task.
➤ YN0000: [@every.org/website-next]: Running lint-staged with the following config:
➤ YN0000: [@every.org/website-next]: {
➤ YN0000: [@every.org/website-next]:   '*': async (files) => {
➤ YN0000: [@every.org/website-next]:   const prettierFiles = micromatch(
➤ YN0000: [@every.org/website-next]:     files,
➤ YN0000: [@every.org/website-next]:     prettierSupportedExtensions.map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/website-next]:   );
➤ YN0000: [@every.org/website-next]:   const eslintFiles = await filterEslintIgnoredFiles(
➤ YN0000: [@every.org/website-next]:     micromatch(
➤ YN0000: [@every.org/website-next]:       files,
➤ YN0000: [@every.org/website-next]:       ["js", "ts", "jsx", "tsx"].map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/website-next]:     )
➤ YN0000: [@every.org/website-next]:   );
➤ YN0000: [@every.org/website-next]: 
➤ YN0000: [@every.org/website-next]:   return [
➤ YN0000: [@every.org/website-next]:     ...(eslintFiles.length > 0
➤ YN0000: [@every.org/website-next]:       ? [`yarn eslint --fix ${eslintFiles.join(" ")}`]
➤ YN0000: [@every.org/website-next]:       : []),
➤ YN0000: [@every.org/website-next]:     ...(prettierFiles.length > 0
➤ YN0000: [@every.org/website-next]:       ? [`prettier --write ${prettierFiles.map(addQuotes).join(" ")}`]
➤ YN0000: [@every.org/website-next]:       : []),
➤ YN0000: [@every.org/website-next]:   ];
➤ YN0000: [@every.org/website-next]: }
➤ YN0000: [@every.org/website-next]: }
➤ YN0000: [@every.org/website-next]: ℹ No staged files match any configured task.
➤ YN0000: [@every.org/common]: Running lint-staged with the following config:
➤ YN0000: [@every.org/common]: {
➤ YN0000: [@every.org/common]:   '*': async (files) => {
➤ YN0000: [@every.org/common]:   const prettierFiles = micromatch(
➤ YN0000: [@every.org/common]:     files,
➤ YN0000: [@every.org/common]:     prettierSupportedExtensions.map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/common]:   );
➤ YN0000: [@every.org/common]:   const eslintFiles = await filterEslintIgnoredFiles(
➤ YN0000: [@every.org/common]:     micromatch(
➤ YN0000: [@every.org/common]:       files,
➤ YN0000: [@every.org/common]:       ["js", "ts", "jsx", "tsx"].map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/common]:     )
➤ YN0000: [@every.org/common]:   );
➤ YN0000: [@every.org/common]: 
➤ YN0000: [@every.org/common]:   return [
➤ YN0000: [@every.org/common]:     ...(eslintFiles.length > 0
➤ YN0000: [@every.org/common]:       ? [`yarn eslint --fix ${eslintFiles.join(" ")}`]
➤ YN0000: [@every.org/common]:       : []),
➤ YN0000: [@every.org/common]:     ...(prettierFiles.length > 0
➤ YN0000: [@every.org/common]:       ? [`prettier --write ${prettierFiles.map(addQuotes).join(" ")}`]
➤ YN0000: [@every.org/common]:       : []),
➤ YN0000: [@every.org/common]:   ];
➤ YN0000: [@every.org/common]: }
➤ YN0000: [@every.org/common]: }
➤ YN0000: [@every.org/common]: [STARTED] Preparing...
➤ YN0000: [@every.org/common]: [SUCCESS] Preparing...
➤ YN0000: [@every.org/common]: [STARTED] Running tasks...
➤ YN0000: [@every.org/common]: [STARTED] Running tasks for *
➤ YN0000: [@every.org/common]: [STARTED] yarn eslint --fix /Users/omar/code/every.org/every.org/packages/commo…
➤ YN0000: [@every.org/common]: [SUCCESS] yarn eslint --fix /Users/omar/code/every.org/every.org/packages/commo…
➤ YN0000: [@every.org/common]: [STARTED] prettier --write "/Users/omar/code/every.org/every.org/packages/commo…
➤ YN0000: [@every.org/common]: [SUCCESS] prettier --write "/Users/omar/code/every.org/every.org/packages/commo…
➤ YN0000: [@every.org/common]: [SUCCESS] Running tasks for *
➤ YN0000: [@every.org/common]: [SUCCESS] Running tasks...
➤ YN0000: [@every.org/common]: [STARTED] Applying modifications...
➤ YN0000: [@every.org/common]: [SUCCESS] Applying modifications...
➤ YN0000: [@every.org/common]: [STARTED] Cleaning up...
➤ YN0000: [@every.org/common]: [SUCCESS] Cleaning up...
➤ YN0000: [@every.org/website]: Running lint-staged with the following config:
➤ YN0000: [@every.org/website]: {
➤ YN0000: [@every.org/website]:   '*': async (files) => {
➤ YN0000: [@every.org/website]:   const prettierFiles = micromatch(
➤ YN0000: [@every.org/website]:     files,
➤ YN0000: [@every.org/website]:     prettierSupportedExtensions.map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/website]:   );
➤ YN0000: [@every.org/website]:   const eslintFiles = await filterEslintIgnoredFiles(
➤ YN0000: [@every.org/website]:     micromatch(
➤ YN0000: [@every.org/website]:       files,
➤ YN0000: [@every.org/website]:       ["js", "ts", "jsx", "tsx"].map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/website]:     )
➤ YN0000: [@every.org/website]:   );
➤ YN0000: [@every.org/website]: 
➤ YN0000: [@every.org/website]:   return [
➤ YN0000: [@every.org/website]:     ...(eslintFiles.length > 0
➤ YN0000: [@every.org/website]:       ? [`yarn eslint --fix ${eslintFiles.join(" ")}`]
➤ YN0000: [@every.org/website]:       : []),
➤ YN0000: [@every.org/website]:     ...(prettierFiles.length > 0
➤ YN0000: [@every.org/website]:       ? [`prettier --write ${prettierFiles.map(addQuotes).join(" ")}`]
➤ YN0000: [@every.org/website]:       : []),
➤ YN0000: [@every.org/website]:   ];
➤ YN0000: [@every.org/website]: }
➤ YN0000: [@every.org/website]: }
➤ YN0000: [@every.org/website]: [STARTED] Preparing...
➤ YN0000: [@every.org/website]: [SUCCESS] Preparing...
➤ YN0000: [@every.org/website]: [STARTED] Running tasks...
➤ YN0000: [@every.org/website]: [STARTED] Running tasks for *
➤ YN0000: [@every.org/website]: [STARTED] yarn eslint --fix /Users/omar/code/every.org/every.org/packages/websi…
➤ YN0000: [@every.org/website]: [SUCCESS] yarn eslint --fix /Users/omar/code/every.org/every.org/packages/websi…
➤ YN0000: [@every.org/website]: [STARTED] prettier --write "/Users/omar/code/every.org/every.org/packages/websi…
➤ YN0000: [@every.org/website]: [SUCCESS] prettier --write "/Users/omar/code/every.org/every.org/packages/websi…
➤ YN0000: [@every.org/website]: [SUCCESS] Running tasks for *
➤ YN0000: [@every.org/website]: [SUCCESS] Running tasks...
➤ YN0000: [@every.org/website]: [STARTED] Applying modifications...
➤ YN0000: [@every.org/website]: [SUCCESS] Applying modifications...
➤ YN0000: [@every.org/website]: [STARTED] Cleaning up...
➤ YN0000: [@every.org/website]: [SUCCESS] Cleaning up...
➤ YN0000: [@every.org/api]: Running lint-staged with the following config:
➤ YN0000: [@every.org/api]: {
➤ YN0000: [@every.org/api]:   '*': async (files) => {
➤ YN0000: [@every.org/api]:   const prettierFiles = micromatch(
➤ YN0000: [@every.org/api]:     files,
➤ YN0000: [@every.org/api]:     prettierSupportedExtensions.map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/api]:   );
➤ YN0000: [@every.org/api]:   const eslintFiles = await filterEslintIgnoredFiles(
➤ YN0000: [@every.org/api]:     micromatch(
➤ YN0000: [@every.org/api]:       files,
➤ YN0000: [@every.org/api]:       ["js", "ts", "jsx", "tsx"].map((extension) => `**/*${extension}`)
➤ YN0000: [@every.org/api]:     )
➤ YN0000: [@every.org/api]:   );
➤ YN0000: [@every.org/api]: 
➤ YN0000: [@every.org/api]:   return [
➤ YN0000: [@every.org/api]:     ...(eslintFiles.length > 0
➤ YN0000: [@every.org/api]:       ? [`yarn eslint --fix ${eslintFiles.join(" ")}`]
➤ YN0000: [@every.org/api]:       : []),
➤ YN0000: [@every.org/api]:     ...(prettierFiles.length > 0
➤ YN0000: [@every.org/api]:       ? [`prettier --write ${prettierFiles.map(addQuotes).join(" ")}`]
➤ YN0000: [@every.org/api]:       : []),
➤ YN0000: [@every.org/api]:   ];
➤ YN0000: [@every.org/api]: }
➤ YN0000: [@every.org/api]: }
➤ YN0000: [@every.org/api]: [STARTED] Preparing...
➤ YN0000: [@every.org/api]: [SUCCESS] Preparing...
➤ YN0000: [@every.org/api]: [STARTED] Running tasks...
➤ YN0000: [@every.org/api]: [STARTED] Running tasks for *
➤ YN0000: [@every.org/api]: [STARTED] yarn eslint --fix /Users/omar/code/every.org/every.org/packages/api/s…
➤ YN0000: [@every.org/api]: [SUCCESS] yarn eslint --fix /Users/omar/code/every.org/every.org/packages/api/s…
➤ YN0000: [@every.org/api]: [STARTED] prettier --write "/Users/omar/code/every.org/every.org/packages/api/s…
➤ YN0000: [@every.org/api]: [SUCCESS] prettier --write "/Users/omar/code/every.org/every.org/packages/api/s…
➤ YN0000: [@every.org/api]: [SUCCESS] Running tasks for *
➤ YN0000: [@every.org/api]: [SUCCESS] Running tasks...
➤ YN0000: [@every.org/api]: [STARTED] Applying modifications...
➤ YN0000: [@every.org/api]: [SUCCESS] Applying modifications...
➤ YN0000: [@every.org/api]: [STARTED] Cleaning up...
➤ YN0000: [@every.org/api]: [FAILED] lint-staged automatic backup is missing!
➤ YN0000: Done in 17s 857ms
🚨 Lint fix command failed! Please fix the lint errors by hand.
If you want to ignore this and commit anyway, run HUSKY=0 git commit
husky - pre-commit hook exited with code 1 (error)```

</details>

### Environment

<!-- Tell us about your development environment -->
- **OS:** Mac Big Sur
- **Node.js:* 14.16.1
- **`lint-staged`:** 11.0.0

Issue Analytics

State:
Created 2 years ago
Reactions:20
Comments:20 (1 by maintainers)

Top GitHub Comments

5reactions

iirojcommented, Feb 15, 2022

We are working on a better monorepo support. You’re supposed to run lint-staged only once from the root, and then use multiple configurations. This has some other issues at the moment, but it’s the way forward… Maybe you can try to change your setup towards that?

3reactions

grumpyoldman-iocommented, Mar 21, 2022

@iiroj

Im getting some pid issues using latest version and in a monorepo setup:

◼ Applying modifications from tasks...
◼ Cleaning up temporary files...
/XXX/node_modules/pidtree/lib/pidtree.js:61
      callback(new Error('No matching pid found'));
               ^

Error: No matching pid found
    at /XXX/node_modules/pidtree/lib/pidtree.js:61:16
    at /XXX/node_modules/pidtree/lib/ps.js:40:7
    at ChildProcess.<anonymous> (/XXX/node_modules/pidtree/lib/bin.js:45:5)

mind you, this was the commit to stage multiple lint-stagedrc files, as soon as they were added everything looks to be in order.

Top Results From Across the Web

lint-staged - npm

Ignoring files. The concept of lint-staged is to run configured linter tasks (or other tasks) on files that are staged in git.

Setup lint-staged on a Monorepo | Horacio Herrera

lint -staged will run the root's lint script, and Yarn workspaces will run each package lint script using the yarn workspaces run lint...

Using lint-staged, husky, and pre-commit hooks to fail fast and ...

Here we have a two scripts that can lint files, and a paired script that will lint them and make any changes it...

Troubleshooting CI/CD - GitLab Docs

GitLab provides several tools to help make troubleshooting your pipelines ... running them in a test environment with a backup of the instance...

Setup a Monorepo with PNPM workspaces and speed it up ...

In this article we're going to have a deep dive into setting up a new monorepo using PNPM workspaces that hosts a Remix...

Troubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.

Start Free

Top Related Reddit Thread

No results found

Top Related Tweet

No results found

Top Related Dev.to Post

No results found

Running multiple instances of lint-staged in a monorepo leads to automatic backup errors

Description

Steps to reproduce

Debug Logs

Issue Analytics

Top GitHub Comments

Top Results From Across the Web

Top Related Medium Post

Top Related StackOverflow Question

Troubleshoot Live Code

Top Related Reddit Thread

Top Related Hackernoon Post

Top Related Tweet

Top Related Dev.to Post

Top Related Hashnode Post