berry: [Bug]: Stack Overflow on Recursive Dependencies

Self-service

  • I’d be willing to implement a fix

Describe the bug

When attempting to convert a large monorepo that works fine in Yarn 1.x, I ran into a stack overflow exception. This appears to be caused by cyclic dependencies where blindly following dependency/devDependency gets you into an infinite loop.

For example:

  • A dependsOn B
  • B dependsOn C
  • C devDependsOn A (for the purposes of documentation)

These can’t simply be converted to Peer Dependencies, as they’re only required for documentation. They shouldn’t be installed when consuming the package via NPM.

To reproduce

I’m not even slightly sure this actually triggers the problem, but it’s the closest I’ve gotten to making it happen. Sherlock crashes with an out of memory.

const { writeFile } = require("fs").promises;

await packageJson({
  name: "@atlaskit/fabric",
  private: true,
  workspaces: ["packages/*"],
  dependencies: {
    "@forge/ui": "0.7.0",
    "@atlaskit/adf-schema": "workspace:*",
    "@atlaskit/editor-json-transformer": "workspace:*",
    "@atlaskit/adf-utils": "workspace:*",
    "@atlaskit/docs": "workspace:*",
    "@atlaskit/button": "workspace:*",
    "@atlaskit/webdriver-runner": "workspace:*",
    "@atlaskit/editor-mobile-bridge": "workspace:*",
    "@atlaskit/analytics-gas-types": "workspace:*",
    "@atlaskit/analytics-next": "workspace:*",
    "@atlaskit/button": "workspace:*",
    "@atlaskit/checkbox": "workspace:*",
  },
  resolutions: {
    "@types/react": "16.9.2",
    "@types/react-dom": "16.9.2",
    "http-proxy": "^1.18.1",
    "html-minifier": "^3.5.21",
    "is-email": "1.0.1",
    "object-path": "^0.11.5",
    react: "16.8.6",
    "react-dom": "16.8.6",
    "js-yaml": "3.13.1",
    "dot-prop": "^5.2.0",
    "component-cookie": "^1.1.3",
    minimist: "^1.2.2",
    "node-fetch": "^2.6.1",
    prismjs: "^1.21.0",
    sharp: "^0.24.0",
    xmldom: "^0.2.1",
    elliptic: "^6.5.4",
    trim: "^0.0.3",
    "react-dev-utils": "^11.0.4",
  },
  engines: {
    node: "^12.21.0",
  },
});

await packageJson(
  {
    name: "@atlaskit/adf-schema",
    version: "1.0.0",
    devDependencies: {
      "@atlaskit/editor-json-transformer": "workspace:*",
    },
  },
  { cwd: "packages/adf-schema" }
);

await packageJson(
  {
    name: "@atlaskit/editor-json-transformer",
    version: "1.0.0",
    dependencies: { "@atlaskit/adf-utils": "workspace:*" },
  },
  { cwd: "packages/editor-json-transformer" }
);

await packageJson(
  {
    name: "@atlaskit/adf-utils",
    version: "1.0.0",
    devDependencies: {
      "@atlaskit/docs": "workspace:*",
    },
  },
  { cwd: "packages/adf-utils" }
);

await packageJson(
  {
    name: "@atlaskit/docs",
    version: "1.0.0",
    dependencies: {
      "@atlaskit/button": "workspace:*",
    },
  },
  { cwd: "packages/docs" }
);

await packageJson(
  {
    name: "@atlaskit/button",
    version: "1.0.0",
    devDependencies: {
      "@atlaskit/webdriver-runner": "workspace:*",
      "@atlaskit/checkbox": "workspace:*",
    },
  },
  { cwd: "packages/button" }
);

await packageJson(
  {
    name: "@atlaskit/webdriver-runner",
    version: "1.0.0",
    devDependencies: {
      "@atlaskit/editor-mobile-bridge": "workspace:*",
    },
  },
  { cwd: "packages/webdriver-runner" }
);

await packageJson(
  {
    name: "@atlaskit/editor-mobile-bridge",
    version: "1.0.0",
    dependencies: {
      "@atlaskit/analytics-gas-types": "workspace:*",
    },
  },
  { cwd: "packages/editor-mobile-bridge" }
);

await packageJson(
  {
    name: "@atlaskit/analytics-gas-types",
    version: "1.0.0",
    dependencies: {
      "@atlaskit/analytics-next": "workspace:*",
    },
  },
  { cwd: "packages/analytics-gas-types" }
);

await packageJson(
  {
    name: "@atlaskit/analytics-next",
    version: "1.0.0",
    devDependencies: {
      "@atlaskit/button": "workspace:*",
      "@atlaskit/docs": "workspace:*",
    },
  },
  { cwd: "packages/analytics-next" }
);

await packageJson(
  {
    name: "@atlaskit/checkbox",
    version: "1.0.0",
    devDependencies: {
      "@atlaskit/docs": "workspace:*",
    },
  },
  { cwd: "packages/checkbox" }
);

await writeFile(".yarnrc.yml", `nodeLinker: node-modules`);
let result = await yarn("install");

Environment

System:
    OS: Linux 5.11 Ubuntu 21.04 (Hirsute Hippo)
    CPU: (12) x64 Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz
  Binaries:
    Node: 12.22.1 - /tmp/lstanden/xfs-5067c1f9/node
    Yarn: 3.1.0-rc.4 - /tmp/lstanden/xfs-5067c1f9/yarn
    npm: 7.19.1 - ~/.nvm/versions/node/v12.22.1/bin/npm
  npmPackages:
    jest: ^26.4.2 => 26.4.2

Additional context

No response

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 2
  • Comments: 30 (12 by maintainers)

Commits related to this issue

Most upvoted comments

@larixer here you go, first PR so not sure if i’ve done it all correctly.

https://github.com/yarnpkg/berry/pull/3561

@stevesouth I can confirm, that removing stackDepth check does not trigger the problem in https://github.com/yarnpkg/berry/issues/776 anymore. Let us just remove it. Please open pull request when you have time.

CC: @arcanis

I have played a bit with the repro on GitHub. Looks like it is the assertion in Yarn’s code on this line, which triggers stack overflow on the repro project at https://github.com/atlassian-forks/yarn-workspace-repro: https://github.com/yarnpkg/berry/blob/b292d2b9b35070fe5ecde784326c905ac7a2cfdc/packages/yarnpkg-core/sources/Project.ts#L1952

It’s interesting that the stack overflow will be triggered for stackDepth from 2 to 7, but if I change the condition to stackDepth >= 8, the resolution succeeds and Yarn continues to install the project.

This check was introduced in PR: https://github.com/yarnpkg/berry/pull/783 as far as I understand, to deal with the issue: https://github.com/yarnpkg/berry/issues/776