berry: [Bug]: yarn install silently stops if memory is set too low in a CI environment

Self-service

  • I’d be willing to implement a fix

Describe the bug

i migrated a project to yarn 2 with nodeLinker: node-modules

running yarn install on gitlab-ci silently stops:

Bildschirmfoto 2022-01-19 um 22 03 16

the log just ends there (followed by the next command)

I had a memory limit of 500Mi, increasing it to 700Mi fixes the problem.

Problem is though that there is no indication of a lack of memory, yarn just silently crashes.

To reproduce

I used node:14 as a base image in gitlab-ci

I set these env vars to limit the memory of the container:

  KUBERNETES_MEMORY_REQUEST: "350Mi"
  KUBERNETES_MEMORY_LIMIT: "500Mi"

running yarn install --immutable in such a container silently stops.

You have to lower the limit probably depending on the size of the project

Environment

System:
    OS: macOS 12.0.1
    CPU: (16) x64 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
  Binaries:
    Node: 14.18.3 - /private/var/folders/h2/6_301c1x0qdbh_3gsyyjyvrr0000gn/T/xfs-d36c185a/node
    Yarn: 3.1.1 - /private/var/folders/h2/6_301c1x0qdbh_3gsyyjyvrr0000gn/T/xfs-d36c185a/yarn
    npm: 6.14.15 - ~/.nvm/versions/node/v14.18.3/bin/npm
  npmPackages:
    jest: ^27.4.5 => 27.4.5

Additional context

No response

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Reactions: 14
  • Comments: 16 (7 by maintainers)

Most upvoted comments

This needs more attention. It’s definitely happening on the latest version of Yarn 4.2.2 for us on CircleCI - on a Medium machine w/ 4GB of memory.

The unzip process is performed in wasm-land (for portability), so the memory grows but never shrinks.

I experimented in https://github.com/yarnpkg/berry/pull/5738 to move this part in JS-land (initially for performances, but I imagine it should decrease the memory usage as well); I’m a little short in time at the moment and had to pause it, but if you or someone else want to play with it that’d be an interesting project!

so it seems like the issue is here: https://github.com/yarnpkg/yarn/blob/3119382885ea373d3c13d6a846de743eca8c914b/src/cli/index.js#L639 exitCode is null if the child was killed with a SIGKILL, which happens in an OOM condition, so this error just gets swallowed

If only there was a way to limit concurrency in yarn. I tried

  • networkConcurrency: 1
  • taskPoolConcurrency: 1
  • taskPoolMode: async
  • NODE_OPTIONS='--max-old-space-size=1000'

but yarn always seems to spawn 8 concurrent processes (can be seen in top) and OOMs during the Fetch step. (2GB RAM)

Hi!

We recently ran into this error when attempting to upgrade from Yarn 1 -> Yarn Berry

First image is a (silently failing) yarn install using berry which quickly consumed 2GB of memory

image

This second image is a successfully passing yarn install using v1 which consumed less than 1GB of memory. (Sorry that the Y axis is different - we were in the process of right-sizing our CI resources)

image

Our project has 1,147 total dependencies counted using yarn list --depth=0 | wc -l. So not a lot (by NPM standards at least…)


So while the incorrect exit code=0 bug definitely should get fixed, I’m wondering if an alternative angle would be to investigate why memory usage in the new version is so much worse. ~2x memory increase seems to be pretty severe

Yarn can’t do much, it’s your kernel that kills the process with no opportunity to react (if it was Node, you’d have a page fault error with memory addresses etc).