berry: [Bug]: yarn install silently stops if memory is set too low in a CI environment
Self-service
- I’d be willing to implement a fix
Describe the bug
i migrated a project to yarn 2 with nodeLinker: node-modules
running yarn install on gitlab-ci silently stops:
the log just ends there (followed by the next command)
I had a memory limit of 500Mi, increasing it to 700Mi fixes the problem.
Problem is though that there is no indication of a lack of memory, yarn just silently crashes.
To reproduce
I used node:14 as a base image in gitlab-ci
I set these env vars to limit the memory of the container:
KUBERNETES_MEMORY_REQUEST: "350Mi"
KUBERNETES_MEMORY_LIMIT: "500Mi"
running yarn install --immutable in such a container silently stops.
You have to lower the limit probably depending on the size of the project
Environment
System:
OS: macOS 12.0.1
CPU: (16) x64 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
Binaries:
Node: 14.18.3 - /private/var/folders/h2/6_301c1x0qdbh_3gsyyjyvrr0000gn/T/xfs-d36c185a/node
Yarn: 3.1.1 - /private/var/folders/h2/6_301c1x0qdbh_3gsyyjyvrr0000gn/T/xfs-d36c185a/yarn
npm: 6.14.15 - ~/.nvm/versions/node/v14.18.3/bin/npm
npmPackages:
jest: ^27.4.5 => 27.4.5
Additional context
No response
About this issue
- Original URL
- State: open
- Created 5 years ago
- Reactions: 14
- Comments: 16 (7 by maintainers)
This needs more attention. It’s definitely happening on the latest version of Yarn 4.2.2 for us on CircleCI - on a Medium machine w/ 4GB of memory.
The unzip process is performed in wasm-land (for portability), so the memory grows but never shrinks.
I experimented in https://github.com/yarnpkg/berry/pull/5738 to move this part in JS-land (initially for performances, but I imagine it should decrease the memory usage as well); I’m a little short in time at the moment and had to pause it, but if you or someone else want to play with it that’d be an interesting project!
so it seems like the issue is here: https://github.com/yarnpkg/yarn/blob/3119382885ea373d3c13d6a846de743eca8c914b/src/cli/index.js#L639 exitCode is null if the child was killed with a SIGKILL, which happens in an OOM condition, so this error just gets swallowed
If only there was a way to limit concurrency in yarn. I tried
networkConcurrency: 1taskPoolConcurrency: 1taskPoolMode: asyncNODE_OPTIONS='--max-old-space-size=1000'but yarn always seems to spawn 8 concurrent processes (can be seen in
top) and OOMs during theFetch step. (2GB RAM)Hi!
We recently ran into this error when attempting to upgrade from Yarn 1 -> Yarn Berry
First image is a (silently failing)
yarn installusingberrywhich quickly consumed 2GB of memoryThis second image is a successfully passing
yarn installusingv1which consumed less than 1GB of memory. (Sorry that the Y axis is different - we were in the process of right-sizing our CI resources)Our project has 1,147 total dependencies counted using
yarn list --depth=0 | wc -l. So not a lot (by NPM standards at least…)So while the incorrect
exit code=0bug definitely should get fixed, I’m wondering if an alternative angle would be to investigate why memory usage in the new version is so much worse. ~2x memory increase seems to be pretty severeYarn can’t do much, it’s your kernel that kills the process with no opportunity to react (if it was Node, you’d have a page fault error with memory addresses etc).