jest: Updating to Jest 22 causes coverage to run out of memory

Since updating to Jest 22 in https://github.com/facebook/react/pull/11956, all React test runs with coverage fail:

screen shot 2018-01-05 at 6 56 58 pm

A failure looks like this:

 PASS  packages/react-dom/src/events/__tests__/SelectEventPlugin-test.js
 PASS  packages/react-dom/src/__tests__/ReactDOMOption-test.js (8.546s)
 PASS  packages/react-dom/src/__tests__/ReactMountDestruction-test.js (8.088s)

<--- Last few GCs --->

[199:0x283aea0]   742209 ms: Mark-sweep 1412.2 (1467.6) -> 1412.2 (1466.6) MB, 2408.5 / 0.0 ms  (+ 0.0 ms in 0 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 2409 ms) last resort GC in old space requested
[199:0x283aea0]   744462 ms: Mark-sweep 1412.2 (1466.6) -> 1412.2 (1466.6) MB, 2252.7 / 0.0 ms  last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x2ed934225ee1 <JSObject>
    3: completeUnitOfWork(aka completeUnitOfWork) [/home/circleci/project/packages/react-reconciler/src/ReactFiberScheduler.js:552] [bytecode=0x1e5a3fb9bb11 offset=1017](this=0xa6f6c182311 <undefined>,workInProgress=0x19c3ddb03201 <FiberNode map = 0x327cfdde93a9>)
    5: performUnitOfWork(aka performUnitOfWork) [/home/circleci/project/packages/react-reconciler/src/ReactFiberScheduler.js:646] [by...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]
 2: 0x121a2cc [node]
 3: v8::Utils::ReportOOMFailure(char const*, bool) [node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [node]
 5: v8::internal::Factory::NewFixedArray(int, v8::internal::PretenureFlag) [node]
 6: v8::internal::FeedbackVector::New(v8::internal::Isolate*, v8::internal::Handle<v8::internal::SharedFunctionInfo>) [node]
 7: v8::internal::JSFunction::EnsureLiterals(v8::internal::Handle<v8::internal::JSFunction>) [node]
 8: v8::internal::Compiler::Compile(v8::internal::Handle<v8::internal::JSFunction>, v8::internal::Compiler::ClearExceptionFlag) [node]
 9: v8::internal::Runtime_CompileLazy(int, v8::internal::Object**, v8::internal::Isolate*) [node]
10: 0x2c01a818463d
Done in 745.02s.
cat: ./coverage/lcov.info: No such file or directory
[error] "2018-01-05T18:50:27.351Z"  'error from lcovParse: ' 'Failed to parse string'
[error] "2018-01-05T18:50:27.353Z"  'input: ' ''
[error] "2018-01-05T18:50:27.353Z"  'error from convertLcovToCoveralls'

Not sure if this is a Jest issue but figured it’s worth asking.

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 9
  • Comments: 31 (23 by maintainers)

Commits related to this issue

Most upvoted comments

Is this the same as removing --runInBand? Without --runInBand it used to always OOM. As I mentioned in the past it only made things worse. But I can try!

@gaearon there’s a bug in Circle builds where running jest without --runInBand attempts to use the number of CPUs of the machine rather than the VM (which would make things worse). The --runInBand flag “fixes” this by forcing it to one worker, but you should be able to use -w for up to the number of CPUs given to the Circle VM

So for React, you should be able to use -w=2 since the circle jobs running React have 2 CPUs:

Ok I think I figured this out (see above PR)

Some notes:

  • It is a Jest leak (not jsdom)
  • It is present in all Jest 22 releases
  • It is not related to coverage
  • The new --detectLeak feature correctly indicated that there is a leak in all tests using Jest 22

Setting maxWorkers=2 seems to have fixed the build in the above PR, but there’s definitely still a leak

I’ve also been experiencing issues with jest@22.4.2 in gitlab pipelines – tests ran very slow and finally crashed with a similar error as the one in the initial comment. Downgrading to @^21 seems to help.

The latest discussion is here: #5989. tl;dr - it’s not possible to accurately detect how many cpus are available on the different CIs, with CircleCI being the hardest

The changes related to fixing memory leaks with graceful-fs and other modules that modify native references were rollbacked, mostly due to bugs during the node initialization process 😞. The only thing that remained is an improvement for cloning process.

Running without --runInBand should improve things, since the memory limit is then spread through many processes, increasing the overall available memory (as long as there is enough system memory). Running with a tuned Node (adding max-old-space-size + stack-size) should improve things as well.

I’m not sure at all on what could be causing the leak, but we use the --coverage flag internally and we haven’t seen any of this issues, so I think it could be something specific to React tests. It’d be really nice if we had some sort of automated mechanism to run bisections; that would definitely improve detecting this kind of things.