berry: [Bug] Workspaces foreach always exits with zero even on error

  • I’d be willing to implement a fix

Describe the bug

When a build or test fails using workspaces foreach run test yarn should exit with a non-zero status.

To Reproduce

Checkout https://github.com/marcneander/yarn-foreach-error.git

From root directory run yarn run test && echo "done" or yarn workspaces foreach run -pvai test && echo "done"

Expectation: in the output below, done should not output when using foreach

/Volumes/HDD2/GitHub/yarn-foreach-error on master*
$ yarn set version berry
➤ YN0000: Downloading https://github.com/yarnpkg/berry/raw/master/packages/yarnpkg-cli/bin/yarn.js
➤ YN0000: Saving the new release in .yarn/releases/yarn-2.4.0.cjs
➤ YN0000: Done in 2s 116ms
/Volumes/HDD2/GitHub/yarn-foreach-error on master*
$ yarn run test && echo "done"
done
/Volumes/HDD2/GitHub/yarn-foreach-error on master*
$ yarn workspace failure run test && echo "done"
/Volumes/HDD2/GitHub/yarn-foreach-error on master*
$

Note that if I run the test on the workspace directly by name, then the exit code is correct.

Screenshots

Environment if relevant (please complete the following information):

  • OS: [OSX, Linux]
  • Node version [15]
  • Yarn version [2.4.0]

Additional context

In a non-sample project, I am trying to use yarn to run cucumber.js tests in workspaces. The tests are failing, but the build is passing.

yarn workspaces foreach -pviA  run test && echo "done"
➤ YN0000: [workspace A]: Seeing profile=XYZ
➤ YN0000: [common-test]: Seeing profile=XYZ
➤ YN0000: [workspace A]: ..F-
➤ YN0000: [workspace A]: 
➤ YN0000: [workspace A]: Failures:
➤ YN0000: [workspace A]: 
➤ YN0000: [workspace A]: 1) Scenario: Can reach API Proxy # test/features/apigee.feature:9
➤ YN0000: [workspace A]:    ✔ Given I have a REST client # ../../common-test/test/features/step-definitions/apickli.ts:28
➤ YN0000: [workspace A]:    ✔ Given I set "User-Agent" header to 'XXXXXXXX' # ../../common-test/test/features/step-definitions/apickli.ts:34
➤ YN0000: [workspace A]:    ✖ When I GET "/ping" # ../../common-test/test/features/step-definitions/apickli.ts:88
➤ YN0000: [workspace A]:        Error: Error: socket hang up
➤ YN0000: [workspace A]:            at /Volumes/HDD2/apigee-config/src/common-test/test/features/step-definitions/apickli.ts:91:22
➤ YN0000: [workspace A]:            at Request._callback (/Volumes/HDD2/apigee-config/.yarn/cache/apickli-npm-2.3.3-e973675e47-469a7ec0f4.zip/node_modules/apickli/apickli.js:481:14)
➤ YN0000: [workspace A]:            at self.callback (/Volumes/HDD2/apigee-config/.yarn/cache/request-npm-2.88.2-f4a57c72c4-7a74841f30.zip/node_modules/request/request.js:185:22)
➤ YN0000: [workspace A]:            at Request.emit (node:events:329:20)
➤ YN0000: [workspace A]:            at Request.EventEmitter.emit (node:domain:467:12)
➤ YN0000: [workspace A]:            at Request.onRequestError (/Volumes/HDD2/apigee-config/.yarn/cache/request-npm-2.88.2-f4a57c72c4-7a74841f30.zip/node_modules/request/request.js:877:8)
➤ YN0000: [workspace A]:            at ClientRequest.emit (node:events:329:20)
➤ YN0000: [workspace A]:            at ClientRequest.EventEmitter.emit (node:domain:467:12)
➤ YN0000: [workspace A]:            at TLSSocket.socketOnEnd (node:_http_client:502:9)
➤ YN0000: [workspace A]:            at TLSSocket.emit (node:events:341:22)
➤ YN0000: [workspace A]:            at TLSSocket.EventEmitter.emit (node:domain:467:12)
➤ YN0000: [workspace A]:            at endReadableNT (node:internal/streams/readable:1294:12)
➤ YN0000: [workspace A]:            at processTicksAndRejections (node:internal/process/task_queues:80:21)
➤ YN0000: [workspace A]:    - Then response code should be 200 # ../../common-test/test/features/step-definitions/apickli.ts:164
➤ YN0000: [workspace A]: 
➤ YN0000: [workspace A]: 1 scenario (1 failed)
➤ YN0000: [workspace A]: 4 steps (1 failed, 1 skipped, 2 passed)
➤ YN0000: [workspace A]: 0m00.590s (executing steps: 0m00.578s)
➤ YN0000: [common-test]: ..F-
➤ YN0000: [common-test]: 
➤ YN0000: [common-test]: Failures:
➤ YN0000: [common-test]: 
➤ YN0000: [common-test]: 1) Scenario: Can reach API Proxy # test/features/apigee.feature:9
➤ YN0000: [common-test]:    ✔ Given I have a REST client # test/features/step-definitions/apickli.ts:28
➤ YN0000: [common-test]:    ✔ Given I set "User-Agent" header to 'XXXXX' # test/features/step-definitions/apickli.ts:34
➤ YN0000: [common-test]:    ✖ When I GET "/ping" # test/features/step-definitions/apickli.ts:88
➤ YN0000: [common-test]:        Error: Error: socket hang up
➤ YN0000: [common-test]:            at /Volumes/HDD2/apigee-config/src/common-test/test/features/step-definitions/apickli.ts:91:22
➤ YN0000: [common-test]:            at Request._callback (/Volumes/HDD2/apigee-config/.yarn/cache/apickli-npm-2.3.3-e973675e47-469a7ec0f4.zip/node_modules/apickli/apickli.js:481:14)
➤ YN0000: [common-test]:            at self.callback (/Volumes/HDD2/apigee-config/.yarn/cache/request-npm-2.88.2-f4a57c72c4-7a74841f30.zip/node_modules/request/request.js:185:22)
➤ YN0000: [common-test]:            at Request.emit (node:events:329:20)
➤ YN0000: [common-test]:            at Request.EventEmitter.emit (node:domain:467:12)
➤ YN0000: [common-test]:            at Request.onRequestError (/Volumes/HDD2/apigee-config/.yarn/cache/request-npm-2.88.2-f4a57c72c4-7a74841f30.zip/node_modules/request/request.js:877:8)
➤ YN0000: [common-test]:            at ClientRequest.emit (node:events:329:20)
➤ YN0000: [common-test]:            at ClientRequest.EventEmitter.emit (node:domain:467:12)
➤ YN0000: [common-test]:            at TLSSocket.socketOnEnd (node:_http_client:502:9)
➤ YN0000: [common-test]:            at TLSSocket.emit (node:events:341:22)
➤ YN0000: [common-test]:            at TLSSocket.EventEmitter.emit (node:domain:467:12)
➤ YN0000: [common-test]:            at endReadableNT (node:internal/streams/readable:1294:12)
➤ YN0000: [common-test]:            at processTicksAndRejections (node:internal/process/task_queues:80:21)
➤ YN0000: [common-test]:    - Then response code should be 200 # test/features/step-definitions/apickli.ts:164
➤ YN0000: [common-test]: 
➤ YN0000: [common-test]: 1 scenario (1 failed)
➤ YN0000: [common-test]: 4 steps (1 failed, 1 skipped, 2 passed)
➤ YN0000: [common-test]: 0m00.476s (executing steps: 0m00.464s)
done

I expect a non-zero exit code and “done” to be missing from the output.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 15 (5 by maintainers)

Most upvoted comments

@merceyz Somehow, I still can reproduce this bug.

This is my repro: https://github.com/akphi/issue-repo/pull/1. I’m using Yarn@3.0.1 and Node@v14.17.1 on Mac

I setup a test monorepo with 2 workspaces pkg1 and pkg2. Each workspace has a script called doSomething. pkg1 will exit with code 1 since the command error is not known.

At the root directory, if we run the following command, we got some unexpected result:

yarn workspaces foreach run doSomething                                          # no error
yarn workspaces foreach --all run doSomething                                    # no error
yarn workspaces foreach --parallel run doSomething                               # no error
yarn workspaces foreach --all --parallel --topological-dev run doSomething       # error

Let me know if you want me to file this as a different issue.

+2
akphi on Aug 19, 2021

Any reason a fix can’t be backported to Yarn V2?

This is causing our CI to return a false positive when one of our workspaces build or test fails (I.E. Not reporting the error). Moving our project to a Release Candidate version of V3, which also includes breaking changes, doesn’t seem like a reasonable solution here.

Edit: Downgrading to Node 14 seems to be a temporary fix until Yarn V3 is stable/launched.

+2
diesal11 on Jun 29, 2021

yeah, having the same problem with yarn workspaces foreach, for both jest and eslint. With the following workspace:

{
  "name": "@my-scope/package",
  "version": "1.0.0",
  "private": true,
  "main": "src/index.js",
  "devDependencies": {
    "eslint": "^7.20.0",
    "jest": "^26.6.3"
  },
  "scripts": {
    "lint": "eslint",
    "unit-test": "jest --coverage",
    "test": "yarn lint && yarn unit-test",
  }
}

Running either yarn workspaces foreach lint, yarn workspaces foreach unit-test or yarn workspaces foreach test will produce a successful outcome, even if any/all of these are failing. It does log that, Process exited without output (exit code 0), but still exits with the same 0 exit code.

+2
DanielOrtel on Mar 2, 2021

You need to import your plugins from sources as well https://yarnpkg.com/cli/plugin/import/from/sources

+1
merceyz on Mar 17, 2021

I don’t understand - done is not printed when using foreach, as shown in the output you shared (your prompt makes it fairly difficult to parse btw):

You have misread the commandline output. Steps in output

  1. check out repo (not shown)
  2. Set version berry
  3. Run workspace test directly (fails as expected)
  4. Run workspace test via foreach (unexpectedly passes)

Cleaned up some output

/Volumes/HDD2/GitHub/yarn-foreach-error on master*
$ yarn set version berry
➤ YN0000: Downloading https://github.com/yarnpkg/berry/raw/master/packages/yarnpkg-cli/bin/yarn.js
➤ YN0000: Saving the new release in .yarn/releases/yarn-2.4.0.cjs
➤ YN0000: Done in 2s 116ms

/Volumes/HDD2/GitHub/yarn-foreach-error on master*
$ yarn workspace failure run test && echo "done"

/Volumes/HDD2/GitHub/yarn-foreach-error on master*
$ yarn run test && echo "done"
done

Running test using the workspace the test fails correctly (no done)

$ yarn workspace failure run test && echo "done"
$

Even running the test explicitly with foreach does NOT fail correctly

$ yarn workspaces foreach run test -A && echo "done"
done
$

Also note that we purposefully have code setting the exit code to 1 if anything fails, so I suspect your test framework doesn’t actually exits with a non-zero exit code:

🤷🏾‍♂️ That is nice and well, but this issue is easily reproducible for me. The test fails when running without foreach so unlikely to be a test framework issue. My test framework is Cucumber.js which is pretty battle tested and also NOT part of the reproducible example 😉

+1
MarsianMan on Feb 17, 2021

Also note that we purposefully have code setting the exit code to 1 if anything fails, so I suspect your test framework doesn’t actually exits with a non-zero exit code:

https://github.com/yarnpkg/berry/blob/909a1259a515611b500b55bf6bc32d812eb26e18/packages/plugin-workspace-tools/sources/commands/foreach.ts#L323-L328

+1
arcanis on Feb 17, 2021
Read more comments on GitHub
← berry: [Bug] webpack-dev-server can't recompile: Can't open file: No file descriptors available
berry: [Bug]: Yarn 2 has broken scoped authentication functionality since 2.2.0 version →