berry: [Bug]: Stack Overflow on Recursive Dependencies
Self-service
- I’d be willing to implement a fix
Describe the bug
When attempting to convert a large monorepo that works fine in Yarn 1.x, I ran into a stack overflow exception. This appears to be caused by cyclic dependencies where blindly following dependency/devDependency gets you into an infinite loop.
For example:
- A dependsOn B
- B dependsOn C
- C devDependsOn A (for the purposes of documentation)
These can’t simply be converted to Peer Dependencies, as they’re only required for documentation. They shouldn’t be installed when consuming the package via NPM.
To reproduce
I’m not even slightly sure this actually triggers the problem, but it’s the closest I’ve gotten to making it happen. Sherlock crashes with an out of memory.
const { writeFile } = require("fs").promises;
await packageJson({
name: "@atlaskit/fabric",
private: true,
workspaces: ["packages/*"],
dependencies: {
"@forge/ui": "0.7.0",
"@atlaskit/adf-schema": "workspace:*",
"@atlaskit/editor-json-transformer": "workspace:*",
"@atlaskit/adf-utils": "workspace:*",
"@atlaskit/docs": "workspace:*",
"@atlaskit/button": "workspace:*",
"@atlaskit/webdriver-runner": "workspace:*",
"@atlaskit/editor-mobile-bridge": "workspace:*",
"@atlaskit/analytics-gas-types": "workspace:*",
"@atlaskit/analytics-next": "workspace:*",
"@atlaskit/button": "workspace:*",
"@atlaskit/checkbox": "workspace:*",
},
resolutions: {
"@types/react": "16.9.2",
"@types/react-dom": "16.9.2",
"http-proxy": "^1.18.1",
"html-minifier": "^3.5.21",
"is-email": "1.0.1",
"object-path": "^0.11.5",
react: "16.8.6",
"react-dom": "16.8.6",
"js-yaml": "3.13.1",
"dot-prop": "^5.2.0",
"component-cookie": "^1.1.3",
minimist: "^1.2.2",
"node-fetch": "^2.6.1",
prismjs: "^1.21.0",
sharp: "^0.24.0",
xmldom: "^0.2.1",
elliptic: "^6.5.4",
trim: "^0.0.3",
"react-dev-utils": "^11.0.4",
},
engines: {
node: "^12.21.0",
},
});
await packageJson(
{
name: "@atlaskit/adf-schema",
version: "1.0.0",
devDependencies: {
"@atlaskit/editor-json-transformer": "workspace:*",
},
},
{ cwd: "packages/adf-schema" }
);
await packageJson(
{
name: "@atlaskit/editor-json-transformer",
version: "1.0.0",
dependencies: { "@atlaskit/adf-utils": "workspace:*" },
},
{ cwd: "packages/editor-json-transformer" }
);
await packageJson(
{
name: "@atlaskit/adf-utils",
version: "1.0.0",
devDependencies: {
"@atlaskit/docs": "workspace:*",
},
},
{ cwd: "packages/adf-utils" }
);
await packageJson(
{
name: "@atlaskit/docs",
version: "1.0.0",
dependencies: {
"@atlaskit/button": "workspace:*",
},
},
{ cwd: "packages/docs" }
);
await packageJson(
{
name: "@atlaskit/button",
version: "1.0.0",
devDependencies: {
"@atlaskit/webdriver-runner": "workspace:*",
"@atlaskit/checkbox": "workspace:*",
},
},
{ cwd: "packages/button" }
);
await packageJson(
{
name: "@atlaskit/webdriver-runner",
version: "1.0.0",
devDependencies: {
"@atlaskit/editor-mobile-bridge": "workspace:*",
},
},
{ cwd: "packages/webdriver-runner" }
);
await packageJson(
{
name: "@atlaskit/editor-mobile-bridge",
version: "1.0.0",
dependencies: {
"@atlaskit/analytics-gas-types": "workspace:*",
},
},
{ cwd: "packages/editor-mobile-bridge" }
);
await packageJson(
{
name: "@atlaskit/analytics-gas-types",
version: "1.0.0",
dependencies: {
"@atlaskit/analytics-next": "workspace:*",
},
},
{ cwd: "packages/analytics-gas-types" }
);
await packageJson(
{
name: "@atlaskit/analytics-next",
version: "1.0.0",
devDependencies: {
"@atlaskit/button": "workspace:*",
"@atlaskit/docs": "workspace:*",
},
},
{ cwd: "packages/analytics-next" }
);
await packageJson(
{
name: "@atlaskit/checkbox",
version: "1.0.0",
devDependencies: {
"@atlaskit/docs": "workspace:*",
},
},
{ cwd: "packages/checkbox" }
);
await writeFile(".yarnrc.yml", `nodeLinker: node-modules`);
let result = await yarn("install");
Environment
System:
OS: Linux 5.11 Ubuntu 21.04 (Hirsute Hippo)
CPU: (12) x64 Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
Binaries:
Node: 12.22.1 - /tmp/lstanden/xfs-5067c1f9/node
Yarn: 3.1.0-rc.4 - /tmp/lstanden/xfs-5067c1f9/yarn
npm: 7.19.1 - ~/.nvm/versions/node/v12.22.1/bin/npm
npmPackages:
jest: ^26.4.2 => 26.4.2
Additional context
No response
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 2
- Comments: 30 (12 by maintainers)
Commits related to this issue
- fix: Remove recursive dependency check that was causing a stack overflow in certain respositories. https://github.com/yarnpkg/berry/issues/3434 — committed to stevesouth/berry by stevesouth 3 years ago
@larixer finally managed a simple reproduction.
https://github.com/stevesouth/yarn-stack-overflow-repro
@larixer here you go, first PR so not sure if i’ve done it all correctly.
https://github.com/yarnpkg/berry/pull/3561
@stevesouth I can confirm, that removing
stackDepth
check does not trigger the problem in https://github.com/yarnpkg/berry/issues/776 anymore. Let us just remove it. Please open pull request when you have time.CC: @arcanis
I have played a bit with the repro on GitHub. Looks like it is the assertion in Yarn’s code on this line, which triggers stack overflow on the repro project at https://github.com/atlassian-forks/yarn-workspace-repro: https://github.com/yarnpkg/berry/blob/b292d2b9b35070fe5ecde784326c905ac7a2cfdc/packages/yarnpkg-core/sources/Project.ts#L1952
It’s interesting that the stack overflow will be triggered for
stackDepth
from2
to7
, but if I change the condition tostackDepth >= 8
, the resolution succeeds and Yarn continues to install the project.This check was introduced in PR: https://github.com/yarnpkg/berry/pull/783 as far as I understand, to deal with the issue: https://github.com/yarnpkg/berry/issues/776