ts-jest: Module caching memory leak

Cross posting from https://github.com/facebook/jest/issues/11956#issuecomment-994914988

We’ve been recently hit by this bug (or combination of them) in several services of our stack. It’s been an inconvenience for use since once the heap size exceeded the 2GB available by GitHub Actions machines, our CI/CD process was failing 100% of the time.

I believe the “modules accumulating in the heap” bug has been present in Jest for some time, but that in combination with the changes in Node from v16.10 to v16.11 has made it much more severe. We’re certain that bug exists in Jest, as we’ve managed to replicate it with a “dummy repo”, as several other people have stated.

The fact that the ticket created in Node JS was closed as a WONTFIX is a bit worrying. I currently lack the knowledge to judge the discussions which happened there, but if that wasn’t a bug at the end probably the solution falls on the Jest side.

After trying to extract as much information from the different GitHub issues that have been created in Jest, TS-Jest and Node, and trying several approaches, the only path for us was to downgrade back to Node v16.10.

These are some of the statistics we gathered in the process:

Some explanations about the columns:

• Node: version of NodeJS
• @jest/core: version of the Jest core
• isolatedModules: the setting defined via jest.config.json
• runInBand: specified by the --runInBand Jest flag
• Comp. cache: specified by the NodeJS --no-compilation-cache flag
• Initial H: heap size after running the first test
• Final H: heap size after running the last test
• H ratio: heap size increase after each test
• Time: time the whole suite needed to run

The most relevant tickets related to this matter:

• https://github.com/facebook/jest/issues/10550
• https://github.com/facebook/jest/issues/11956
• https://github.com/facebook/jest/issues/12142
• https://github.com/facebook/jest/issues/7311
• https://github.com/facebook/jest/issues/7874
• https://github.com/kulshekhar/ts-jest/issues/1967
• https://github.com/nodejs/node/issues/40014
• https://bugs.chromium.org/p/v8/issues/detail?id=12198

We hope this issue is given the importance it deserves, as keeping the Node version pinned to v16.10 or relying on bigger CI machines to compensate this is a poor strategy for the future.

I had similar issue, I switched to @swc/jest:

-  preset: "ts-jest",
+  transform: {
+    "^.+\\.(t|j)sx?$": "@swc/jest",
+  },

See results:

ts-jest:

./node_modules/.bin/jest --runInBand --logHeapUsage
 -c ./jest.config.js
 PASS packages/regexes/test/index.spec.ts (5.824 s, 727 MB heap size)
 PASS packages/lib/test/index.spec.ts (1395 MB heap size)
 PASS packages/sensitive/test/index.spec.ts (11.137 s, 2021 MB heap size)
 RUNS packages/decoder/test/draw.spec.ts

<--- Last few GCs --->

[13648:0x150008000]    28711 ms: Mark-sweep (reduce) 2045.3 (2085.2) -> 2044.2 (2085.7) MB, 1246.6 / 0.0 ms  (+ 450.4 ms in 89 steps since start of marking, biggest step 11.4 ms, walltime since start of marking -2193503732 ms) (average mu = 0.282, current[13648:0x150008000]    30599 ms: Mark-sweep (reduce) 2045.4 (2085.7) -> 2043.2 (2085.5) MB, 1886.3 / 0.0 ms  (average mu = 0.154, current mu = 0.001) allocation failure scavenge might not succeed

<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory

@swc/jest:

$ ./node_modules/.bin/jest --runInBand --logHeapUsage
-c ./jest.config.js
PASS packages/regexes/test/index.spec.ts (55 MB heap size)
PASS packages/lib/test/index.spec.ts (85 MB heap size)
PASS packages/sensitive/test/index.spec.ts (51 MB heap size)
PASS packages/decoder/test/draw.spec.ts (56 MB heap size)
PASS packages/codegen/test/imports.spec.ts (61 MB heap size)

Test Suites: 5 passed, 5 total
Tests:       24 passed, 24 total
Snapshots:   0 total
Time:        2.36 s, estimated 3 s
Ran all test suites in 5 projects.

This means ts-jest was leaking memory like crazy (or typescript). Also tests are faster now

Thank you. To be honest I’m a little bit surprised that this issue haven’t been resolved/looked at over the last 2 years (That’s when Jest 24 was released). It makes ts-jest 24 or newer unusable for bigger projects. @ahnpnl Is there any plan to resolve it?

I managed to fix the leaks using this combo: https://github.com/reside-eng/jest-runtime (why is needed and why the leak happens is linked in the readme)

https://github.com/magic-akari/jest_workaround why is needed here: https://github.com/swc-project/swc/discussions/5151#discussioncomment-3149154

i had to use the workaround because all my jest.spyOn started failing and using swc removed the need of using ts-jest

I ran into this one today. looking at the heap shows me lots of compiled modules being retained.

@amiedes thank you for sharing. I tested your solution on our project and realized that it’s enough to remove the --runInBand Jest flag and change jest.config.js

  globals: {
    'ts-jest': {
      isolatedModules: true,
    },
  },

and this already greatly reduces the memory problem by 3-4 times + speed, no matter which version of node (I’ve tested on 16.10 and 16.13, on 16.10 it’s better, but not so critical, at least for us)

P.S. try to add --expose-gc

"test": "node --expose-gc --no-compilation-cache ./node_modules/.bin/jest --logHeapUsage",

Adding this to jest.config.js worked for me.

  globals: {
    'ts-jest': {
      isolatedModules: true
    }
  }

I bet my cents on jest 😅

Can confirm that https://github.com/reside-eng/jest-runtime significantly alleviates the memory leaking problems!

I have a medium-sized TypeScript project transformed with @swc/jest containing around 500 test cases that used to take up a massive 5.2 GB of RAM to run from start to finish, but now takes “only” 2.6 GB, effectively halving the RAM usage of Jest.

@derN3rd I think you need to be using at least Jest v29 for it to work.

@willsoto I don’t think this is an issue with jest the more I look into it. I think it is ts-jest that is the issue. The reason being is if I switch my transformers from ‘ts-jest’ to ‘babel-jest’ my tests pass everytime. The memory never goes past 300mb on the suite.

To me this sounds like ts-jest is either responsible for the memory leak or the something in the transformer is crawling and transforming modules in a destructive manner

For what it’s worth, same worsening here on Node v16, we had to abandon jest (temporarily?) to avoid OOM errors in our CI.

Actually @marcoreni , seems this is due to Node.JS V8 engine memory leak in 16.11+

See: https://github.com/facebook/jest/issues/11956 & https://github.com/nodejs/node/issues/40014

Sure I think that is a good idea 👍

Attempts to use @swc/jest, swc-node/jest and esbuild-jest for my NestJS project were ussuccessful. It’s always appearing unexpected errors. If anybody can share an appropriate configuration we will be grateful.

Used https://github.com/reside-eng/jest-runtime in my NestJS application without swc and it helped a lot, hope this fixes upstream someday

I tried bumping the runtime to 18.12.1 instead of 16 now that’s the latest LTS but I see the leak there is worse, ~120MB instead of ~<100MB per suite.

Same tests run in 70s, but there’s a total heap of 1460MB by the end vs 1208MB.

Yet another instance of struggling with this. I can see several compiled JS files leaking and taking up 70% of the heap space, but can’t figure out where they’re coming from.

Setting isolatedModules seems to improve things a little but it still grows.

@Simon-Hayden-iteratec it is not just a node16 issue, just that Node v16.11+ there is a memory leak in V8 engine which “further contributes” and makes it seem “worse than” node14, which is not the fault of ts-jest/jest

However even in node14 and node v16.0<v16.10, the module caching problem remains

@arekzaluski my solution was to switch to Mocha/Chai.

The workaround is following the doc https://kulshekhar.github.io/ts-jest/user/config/isolatedModules

If using isolatedModules: true, you should combine with tsc for type checking.

Has anyone found a solution to this? I know it isnt a ts-jest issue, but it seems like the jest repo has just closed any issue about this. Even with setting “isolatedModules”: true, memory usage gets high and over 1000 MB in my project. I am using --runInBand because my test suites are not able to run in parallel and on Node v14.21.1.

With Jest v29 you can use an option to kill the workers when memory usage reaches x MB. See here: https://jestjs.io/docs/configuration#workeridlememorylimit-numberstring

Can confirm this is an issue for me as well in a mid-size codebase (Node v19, Jest v29.3.1). Recently swapped ts-jest for SWC, but it did not resolve the memory leak. I have taken multiple heap snapshots and am seeing the same things documented here - vast majority of the memory being allocated to (string) as what appear to be cached modules. This appears to be a legitimate issue with Jest and perhaps not ts-jest.

As a workaround, I adapted some code I found in another related issue to batch jest execution like this:

import path from 'path'
import glob from 'glob'
import _ from 'lodash'
import spawn from 'cross-spawn'

const BATCH_SIZE = 10

const args = process.argv.slice(2)

const exec = (...args) =>
  new Promise<void>((resolve, reject) => {
    const cmd = spawn(...args, { stdio: 'inherit' })

    cmd.on('close', code => {
      if (code !== 0) reject(code)
      resolve()
    })

    cmd.on('exit', code => {
      if (code !== 0) reject(code)
      resolve()
    })
  })

const runJest = files => {
  return exec('jest', ['--runInBand', '--logHeapUsage', ...files])
}

const run = async () => {
  const startTime = Date.now()

  const errors = []

  if (args.length) {
    try {
      await runJest(args)
    } catch (e) {
      errors.push(e)
    }
  } else {
    const files = await glob.sync(`${path.join(__dirname, '../../../src')}/**/*.spec.ts`)

    const batches = _.chunk(_.shuffle(files), BATCH_SIZE).map(batch => {
      return async () => {
        try {
          await runJest(batch)
        } catch (e) {
          errors.push(e)
        }
      }
    })

    console.log(`== Running test suite in batches (${batches.length} total) ==`)

    const runBatches = async (index = 0) => {
      if (index > batches.length - 1) return

      const batch = batches[index]

      console.log(`== Running batch ${index + 1} ==`)

      try {
        await batch()

        return await runBatches(index + 1)
      } catch (e) {
        console.log('Runner failed: ', e)
      }
    }

    await runBatches()
  }

  const endTime = Date.now()

  const totalTime = endTime - startTime

  const minutes = totalTime / 60000

  console.log(`== Test suite completed in ${minutes} minutes ==`)

  return errors
}

run()
  .then(errors => {
    if (errors.length) process.exit(1)
    else process.exit(0)
  })
  .catch(err => {
    console.error(err)
    process.exit(1)
  })

Obviously, this is far from ideal (and hopefully a temporary workaround) but at least allows our team to execute the entirety of the suite without running out of memory. Maybe it can be helpful to someone else facing the same issue.

I removed node-cache, and sure enough, nothing much seemed to change in terms of memory use, with the various module strings now being retained by the exit hooks in prisma engine-core.

I’m not sure if I’m chasing my tail here looking at shallow size. If I instead look at retained size, I see the top culprit is “closures” which include some pretty suspicious looking recursive retaining objects between app in ServerResponse and response in app(), coming from http_server and express respectively.

The closures have a total shallow size of 13,470,048 bytes, but a retained size of 235,546,160 bytes.

Looks like jest-runner installs source-map-support in runTests.ts, and install does indeed patch the implementation of Error.prepareStackTrace So I think this could well be an issue with jest-runner not cleaning up properly after each test

@amiedes thank you for sharing. I tested your solution on our project and realized that it’s enough to remove the --runInBand Jest flag and change jest.config.js

  globals: {
    'ts-jest': {
      isolatedModules: true,
    },
  },

and this already greatly reduces the memory problem by 3-4 times + speed, no matter which version of node (I’ve tested on 16.10 and 16.13, on 16.10 it’s better, but not so critical, at least for us)

P.S. try to add --expose-gc

"test": "node --expose-gc --no-compilation-cache ./node_modules/.bin/jest --logHeapUsage",

This works for me but I’m getting my successfull test cases failed due to isolatedModules: true,

Can anyone please help, it will make my day.

+1 from our side. Spent 2 days already trying to make Jest fit into our memory again.

Running ~300 tests used to take 20 seconds on my machine, now it takes >600 seconds. While running it now swaps as 500Gb/second due to 16Gb of RAM being not enough.

Adding my two cents, we are running on Node 14 and see pretty much the exact same issue @broekema41 describes. So its not only a Node 16 issue then?

We are only at the beginning of a larger project and are running only 25 tests right now and already experience around 700MB RAM being used.

At least some of the RAM is also being kept by @jest/transform

Edit: Not sure if it helps, but here are some errors our CI pipeline currently throws:

  ● Test suite failed to run

    ENOMEM: not enough memory, read

      at Runtime.readFile (../../node_modules/jest-runtime/build/index.js:2404:21)
      at Object.<anonymous> (../../node_modules/rxjs/src/internal/util/UnsubscriptionError.ts:1:1)

  ● Test suite failed to run

    ENOMEM: not enough memory, write

      at NgJestTransformer.TsJestTransformer.getCacheKey (../../node_modules/ts-jest/dist/ts-jest-transformer.js:198:22)
      at ScriptTransformer._getCacheKey (../../node_modules/@jest/transform/build/ScriptTransformer.js:280:41)
      at ScriptTransformer._getFileCachePath (../../node_modules/@jest/transform/build/ScriptTransformer.js:351:27)
      at ScriptTransformer.transformSource (../../node_modules/@jest/transform/build/ScriptTransformer.js:588:32)
      at ScriptTransformer._transformAndBuildScript (../../node_modules/@jest/transform/build/ScriptTransformer.js:758:40)
      at ScriptTransformer.transform (../../node_modules/@jest/transform/build/ScriptTransformer.js:815:19)

  ● Test suite failed to run

    jest: failed to cache transform results in: /tmp/jest_2ne/jest-transform-cache-dc78a43d3cfa428010ccba88e31c3c72-5bcdb64e20894c4318a9b7ff46ffc22d/60/testsetup_606c8b9ec8a8bfd206ea84678bbe5fc2.map
    Failure message: ENOMEM: not enough memory, write

      at writeFileSync (../../node_modules/write-file-atomic/index.js:215:10)
      at writeCacheFile (../../node_modules/@jest/transform/build/ScriptTransformer.js:998:33)
      at ScriptTransformer._buildTransformResult (../../node_modules/@jest/transform/build/ScriptTransformer.js:570:7)
      at ScriptTransformer.transformSource (../../node_modules/@jest/transform/build/ScriptTransformer.js:621:17)
      at ScriptTransformer._transformAndBuildScript (../../node_modules/@jest/transform/build/ScriptTransformer.js:758:40)
      at ScriptTransformer.transform (../../node_modules/@jest/transform/build/ScriptTransformer.js:815:19)

It seems this issue is worse in Node v16. Switched project today from Node v14 (Heap usage at end -> ~1600MB), Node v16 hits 2GB before the end…

@willsoto Have you been able to find a solution for this bug? It’s impossible to update jest an ts-jest above version 23 as from 24+ the memory heap goes crazy. For my project even to 12 GB+

If transpileModule still produces memory leak, I also think the problem is more in Jest or TypeScript side. I think the issue might be more on Jest side.

Out of curiosity, have you tried with jest CLI flag runInBand @willsoto ? If yes, does it produce the similar leak behavior ?

@ahnpnl That does not resolve the problem, it simply reduces growth but does not mitigate it entirely.

After looking into this all day yesterday, we managed to reduce the issue to jest + typescript. You can see a minimal reproduction here where we introduced a minimal transformer that just compiles the TS via transpileModule. The memory leak still exists.

Our working assumption right now is that there is some negative interaction between Jest and TypeScript. Given Jest’s history of memory leaks, we assume the fault is on Jest’s side but are still working on a minimal reproduction.

Give this information, do you have any other thoughts on this @ahnpnl ?

An interesting observation as I was exploring this was whether or not I was using a custom tsConfig. I’m not sure if this has a meaningful impact on smaller apps, but for large apps the difference is substantial.

When I point ts-jest at a custom tsconfig with an include: [] (an idea I got from this repo), base memory usage goes down significantly but still climbs consistent with a memory leak. When I use whatever the default is, base memory usage increases significantly.

Apart from using jest-runime, did anyone have any other alternative solutions?

I have successfully used vitest, @swc/jest, esbuild jest, or bun’s built in testing in other projects now

With Jest v29 you can use an option to kill the workers when memory usage reaches x MB. See here: https://jestjs.io/docs/configuration#workeridlememorylimit-numberstring

I don’t think this works when running in band. My tests are not able to be run in parallel, so there is only one worker at a time. appears that doing batch execution can temporarily solve the problem in my case at least

@rcbevans it looks like we’re currently walking down similar paths! I’ve currently investigating a similar leak. We’ve just switched a couple of packages to @swc/jest and it’s equally affected so that does indeed suggest this is Jest. I’m seeing exactly the same in the heap snapshots with strings (imports) and compiled code (dependencies) growing.

What I did notice though was if I compare a before run

 PASS  Tooltip.test.tsx (33.605 s, 424 MB heap size)
 PASS  utils.test.ts (413 MB heap size)
 PASS  HeatmapTooltip.test.tsx (472 MB heap size)
 PASS  buildingHasNoSpaces.test.ts (451 MB heap size)
 PASS  test.ts (216.615 s, 531 MB heap size)
 PASS  test.ts (539 MB heap size)
 PASS  MeetingRoomPersonCountTable.test.tsx (627 MB heap size)
 PASS  utils.test.ts (6.908 s, 631 MB heap size)
 PASS  UsageOverTimeTable.test.tsx (679 MB heap size)
 FAIL  Data.test.tsx (136.964 s, 646 MB heap size)
 PASS  utils.test.ts (636 MB heap size)
 PASS  hooks.test.tsx (681 MB heap size)
 PASS  Chart.test.tsx (753 MB heap size)
 PASS  Legend.test.tsx (801 MB heap size)
 PASS  Table.test.tsx (836 MB heap size)
 PASS  OverviewSection.test.tsx (879 MB heap size)
 PASS  utils.test.ts (883 MB heap size)
 PASS  transformData.test.ts (918 MB heap size)
 PASS  Analysis.test.tsx (5.603 s, 1017 MB heap size)
 PASS  MostOccupiedTable.test.tsx (1052 MB heap size)
 PASS  Tooltip.test.tsx (1065 MB heap size)
 PASS  TooltipRow.test.tsx (1102 MB heap size)
 PASS  MostOccupiedChart.test.tsx (1156 MB heap size)
 PASS  OccupancyHeatmap.test.tsx (1203 MB heap size)
 PASS  transformSelectionToBuildings.test.ts (1226 MB heap size)
 PASS  LiveData.test.tsx (1309 MB heap size)
 PASS  utils.test.ts (1299 MB heap size)

and then look at the same run, but this time taking 4 heap snapshots:

 PASS  Data.test.tsx (36.719 s, 206 MB heap size)
 PASS  OccupancyHeatmap.test.tsx (5.713 s, 285 MB heap size)
 PASS  Analysis.test.tsx (6.123 s, 315 MB heap size)
 PASS  Table.test.tsx (5.727 s, 369 MB heap size)
 PASS  MostOccupiedChart.test.tsx (60.097 s, 377 MB heap size)
 PASS  Chart.test.tsx (5.704 s, 396 MB heap size)
 PASS  MostOccupiedTable.test.tsx (5.075 s, 434 MB heap size)
 PASS  LiveData.test.tsx (406 MB heap size)
 PASS  OverviewSection.test.tsx (5.159 s, 366 MB heap size)
 PASS  UsageOverTimeTable.test.tsx (364 MB heap size)
 PASS  MeetingRoomPersonCountTable.test.tsx (362 MB heap size)
 PASS  Tooltip.test.tsx (360 MB heap size)
 PASS  Legend.test.tsx (378 MB heap size)
 PASS  TooltipRow.test.tsx (379 MB heap size)
 PASS  test.ts (360 MB heap size)
 PASS  utils.test.ts (361 MB heap size)
 PASS  transformData.test.ts (348 MB heap size)
 PASS  utils.test.ts (340 MB heap size)
 PASS  utils.test.ts (340 MB heap size)
 PASS  test.ts (351 MB heap size)
 PASS  hooks.test.tsx (358 MB heap size)
 PASS  transformSelectionToBuildings.test.ts (63.389 s, 350 MB heap size)
 PASS  utils.test.ts (349 MB heap size)
 PASS  Tooltip.test.tsx (312 MB heap size)
 PASS  HeatmapTooltip.test.tsx (314 MB heap size)
 PASS  utils.test.ts (306 MB heap size)
 PASS  buildingHasNoSpaces.test.ts (305 MB heap size)

You can see that the memory stops growing when I take a snapshot, and doesn’t seem to suffer with the same problem anymore (or at least not to the same magnitude). I’ve tried 2 runs and got similar results with fewer heap snapshots. Make me wonder if for some reason there’s no garbage collection running normally?

I hacked my local runTest.js to comment the usage of source-map-support and whilst it didn’t prevent the leak, the inspector was showing strings as retained by applicationinsights diagnostic channel instead. Removing all usage of applicationinsights, they then show as being retained by timer callbacks within node-cache which is another dependency I have.

tearDownEnv calls runtime.teardown() which internally calls _internalModuleRegistry.clear() so perhaps clearing this map is intended to release the handles and prevent leaks.

At this points I’m not sure if I’m on the right track and there are a series of modules which aren’t being cleaned up appropriately causing the leak, or perhaps I’m barking up the wrong tree and misunderstanding what the inspector is telling me.

At the core, it feels like the _cacheFS within jest-runtime isn’t being gc’d properly after each test and I’m not sure if this is something that’s squarely user error, or if it’s something v8/jest runtime should be able to do ensure the that the memory can be appropriately reclaimed even if I don’t.

This is very interesting. A bit late here but I’ll have a look tomorrow morning because this issue has been impacting us quite a lot. I need to dive into the code you mentioned and see if patching it locally can change something.

I wonder if jest-runner is actually monkey-patching Error.prepareStackTrace with an implementation using source-map-support and causing the issue… The only dependency I see in yarn.lock with a dependency on source-map-support is jest-runner.

I’ve spent some time capturing heap profiles and the results seem pretty revealing.

In the timeline I can see ~100+MB allocation spikes that coincide with when I see jest finish one test suite and move to the next.

Looking at what’s in these spikes, the majority of the allocations are (as expected) strings that look like source maps.

Looking at the retainers, I see that these strings stem from prepareStackTrace, in Error(), within V8, which seems to be retaining a closure over retrieveMapHanders in source-map-support which is retrieving the source maps from the module registry.

Looking at the implementation of source-map-support I see that there is a reset function exposed, so I tried importing and calling that in afterAll hoping it might flush out those references but alas, it didn’t seem to help any.

I’m using node 16.18.0 and seeing memory climb from ~300MB for the first suite to run by ~100MB each test suite.

Running a set of 12 typescript test suites with node --expose-gc ./node_modules/.bin/jest --runInBand --logHeapUsage --detectOpenHandles I get output like:

 PASS  src/routes/InvitesRoutes.test.ts (28.953 s, 305 MB heap size)
 PASS  src/routes/ApiRoutes.test.ts (16.683 s, 453 MB heap size)
 PASS  src/routes/OrganizationRoutes.test.ts (565 MB heap size)
 PASS  src/routes/Organization/DomainRules.test.ts (674 MB heap size)
 PASS  src/routes/InviteRoutes.test.ts (755 MB heap size)
 PASS  src/middleware/CallerMiddleware.test.ts (843 MB heap size)
 PASS  src/routes/Organization/InvitesRoutes.test.ts (939 MB heap size)
 PASS  src/routes/OrganizationsRoutes.test.ts (1052 MB heap size)
 PASS  src/routes/UserRouters.test.ts (1165 MB heap size)
 PASS  src/middleware/EarlyAccessMiddleware.test.ts (1276 MB heap size)
 PASS  src/helpers/ObjectHelpers.test.ts (1204 MB heap size)
 PASS  src/helpers/CognitiveServiceHelpers.test.ts (1208 MB heap size)

Test Suites: 2 skipped, 12 passed, 12 of 14 total
Tests:       2 skipped, 383 passed, 385 total
Snapshots:   1 passed, 1 total
Time:        70.863 s, estimated 75 s

my jest.config is

/** @type {import('ts-jest/dist/types').InitialOptionsTsJest} */
module.exports = {
  transform: {
    '^.+\\.(t|j)sx?$': ['ts-jest', { isolatedModules: true }],
  },
  testEnvironment: 'node',
  coveragePathIgnorePatterns: ['node_modules/'],
  globalSetup: '<rootDir>/src/test/globalSetup.jest.ts',
  setupFilesAfterEnv: ['<rootDir>/src/test/setup.jest.ts'],
  testPathIgnorePatterns: ['dist/', 'node_modules/'],
  transformIgnorePatterns: ['node_modules/'],
  maxWorkers: 1,
};

Without specifying isolatedModules: true the tests were consuming ~2x the heap.

Capturing a heap snapshot during the test I see that a little over half of the heap is string allocations. Many are duplicate strings of the same node_modules. From the count, it looks like each test suite might be allocating a new string of the node_module dependencies?

Has anyone had any success resolving this, or at least reducing the impact? I didn’t try going back to 16.10 yet, I could try that.

Based on https://github.com/facebook/jest/issues/11956 it seems like there could be multiple factors at play.

In case other’s find this issue via Google, like I did, there seems to be a resolution on the horizon in jest:

I raised my memory leak problems on the jest issue : https://github.com/facebook/jest/issues/11956#issuecomment-1225474868

And now it seems like there’s a PR which solves our leak problems, as we are forced to use run in band due to the test suite: https://github.com/facebook/jest/pull/13168

https://github.com/facebook/jest/issues/11956#issuecomment-1148857787

This is a custom runtime that can be used by most while this issue is still open

Adding this to jest.config.js worked for me.

  globals: {
    'ts-jest': {
      isolatedModules: true
    }
  }

this works for me!

In my case this is the solution. The only clarification is that I was using an Angular project with several modules and projects, and because of that, I needed to go thrown all the jes.config.js files and add the isolatedModules: true.

...
   'ts-jest': {
      tsconfig: '<rootDir>/tsconfig.spec.json',
      stringifyContentPathRegex: '\\.(html|svg)$',
      isolatedModules: true
    }
....

The problem was, that because I was defining some attributes in the ‘ts-jest’ section, this was removing the isolatedModules that came from the main jest.config.js

And just in case, I also add to the tsconfig.json file, this configuration:

"compilerOptions": {
    ...
    "isolatedModules": true,
    ...
}

For what it’s worth, same worsening here on Node v16, we had to abandon jest (temporarily?) to avoid OOM errors in our CI.

Adding this to jest.config.js worked for me.

  globals: {
    'ts-jest': {
      isolatedModules: true
    }
  }

this works for me!

Currently no unfortunately 😃 for isolatedModules: true nothing much to do for it. For isolatedModules: false, there might be a way.

Any progress on this issue?

Please check https://github.com/facebook/jest/issues/10550 because it relies on that. Besides, you can also look at the current codebase of ts-jest to see any suspicious points, here is the part using TypeScript Compiler API https://github.com/kulshekhar/ts-jest/blob/master/src/compiler/ts-compiler.ts#L72

You can look into this one https://github.com/kulshekhar/ts-jest/blob/master/src/compiler/language-service.ts#L175 . This is the place where I/O overhead appears.

@ahnpnl I am more than willing to work on this issue since this is a big problem once there are hundreds of source files. We’ve seen heap usage climb to 1.5gb in some cases. If you can point me to possible culprits that would be great.