Spurious error: "A worker process has failed to exit gracefully and has been force exited."

🐛 Bug Report

Our CI builds have been failing sporadically with this error from Jest:

“A worker process has failed to exit gracefully and has been force exited. This is likely caused by tests leaking due to improper teardown. Try running with --runInBand --detectOpenHandles to find leaks.”

--detectOpenHandles does not detect any problems. The error is reported randomly by various projects that build independently in a large monorepo, which suggests that this is not actually caused by a bad test.

It occurs more frequently on Windows machines, and on machines with heavier load. But it has also been reported on individual developer MacBooks.

To Reproduce

You can trivially force the warning to be reported by reducing this constant to 0:

https://github.com/facebook/jest/blob/64d5983d20a628d68644a3a4cd0f510dc304805a/packages/jest-worker/src/base/BaseWorkerPool.ts#L20

Investigation

The FORCE_EXIT_DELAY constant is used by this code:

https://github.com/facebook/jest/blob/64d5983d20a628d68644a3a4cd0f510dc304805a/packages/jest-worker/src/base/BaseWorkerPool.ts#L104-L116

500 ms seems way too small for a machine that is under heavy load. My theory is that if we increase the timeout, then the IPC message will eventually be received, and the warnings will go away. I will report back after I have tested that.

jest-haste-map may be the root cause

While debugging this, I noticed that the jest-haste-map library has an obvious flaw where it creates a potentially unlimited number of promises while crawling the disk:

https://github.com/facebook/jest/blob/4fa17e455662dc7918eecdb857dd946e9ce2dc53/packages/jest-haste-map/src/index.ts#L659-L678

For example, I observed that Promise.all() may create 50+ promises that are all crawling the filesystem in parallel. If the build orchestrator is building 8 projects in parallel, and each projects is potentially creating 50 Jest promises, that could obviously thrash the disk to the point where the FORCE_EXIT_DELAY limit is exceeded.

This is just a speculation – I did not have time to try fixing this to see whether it resolves the problem or not.

Either way, it would be a good idea to apply some throttling to _buildHasteMap() to limit the parallelism. @cpojer @SimenB

envinfo

  System:
    OS: Windows 10 10.0.19041
    CPU: (8) x64 AMD Ryzen 7 3700X 8-Core Processor
  Binaries:
    Node: 12.20.1 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.10 - C:\Program Files\nodejs\yarn.CMD
    npm: 6.14.10 - C:\Program Files\nodejs\npm.CMD