mocha: Mocha exits with 0 exit code with failing tests
Prerequisites
- [X ] Checked that your issue hasn’t already been filed by cross-referencing issues with the
faq
label Applicable issue #2713 is closed - [X ] Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn’t just a feature that actually isn’t supported in the environment in question or a bug in your code.
- [ X] ‘Smoke tested’ the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
- [X ] Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with:
node node_modules/.bin/mocha --version
(Local) andmocha --version
(Global). We recommend that you not install Mocha globally. not installed globally
Description
Mocha exits with exit code 0 despite having failing tests Visually confirmable by truncated output. Does not print all test lines before printing Done in x. Last test line printed varies run to run.
Steps to Reproduce
Working to determine outside of our test suite. Running sub-suites alone seems to work fine, but running the entire suite does not. Possibly a timing or issue with high test counts?
Expected behavior: Mocha reports a non-zero [success] exit code
Actual behavior: Mocha reports a 0 exit code and does not complete console output. Test Output
$ yarn mocha './mochaWhy/*.test.js'
yarn run v1.9.4
$ C:\projectDir\node_modules\.bin\mocha ./mochaWhy/*.test.js
1ListBuilder
1) fails a test before compliance check
2) creates the links
3) fails a test after compliance check
dataBuilder - Empty
√ It does not create uneccessary data
Compliance
√ Files with bare bones data are compliant (1196ms)
√ Files with simple data are compliant (46ms)
√ Files with bare bones data are compliant (152ms)
√ Files with simple data are compliant (95ms)
orders
retrieveFile
4) handles arrays
√ handles an empty array of uuids
5) throws validation error
retrieveById
6) Finds info, and pulls the associated info
7) Handles b
√ Handles a
save
√ handles empty arrays gracefully
8) calls entity validation
9) doesn't call db-update upon validation failure
Dispatch Sequence
10) Queues b
11) Handles z
12) Doesn't allow x
13) Doesn't allow y
14) Doesn't create g
15) Throws an error if w
16) Throws an error if t
17) Allows partial success when p
structureData
It makes sure 8
√ Handles no
√ Zero-Index input IDs
√ One-Index input IDs
√ Null Ids
√ Skipped Id
√ Mixed Values
Interactor
parseAndSaveJson
- handles a
- handles b
- handles c
- handles d
- not archived if origin is API
- Can determine set
dataInteractor
unzipConvert //not the last test in the suite, but is the last printed.
Done in 4.45s.//[this should be failing count & error details]
$ echo $?
0
Reproduces how often: Currently, every time I run the full suite. This has been a recurring problem for us, sometimes upgrading node has helped, but have not found a solution this time, most likely was a bandaid for the problem.
Versions
Mocha: 3.5.3 and 6.1.4 Node versions 8.10, 8.11, 8.12, 8.16, 10.0, 10.16 (In the past upgrading node 8.9 to node 8.10 seemed to fix, but no longer is a solution)
Gitlab runner: docker:latest node:8 Ubuntu 16.4 64bit Windows 10 64bit - Git bash MINGW64 & Windows powershell
##Related Issues https://github.com/mochajs/mocha/issues/2713 https://github.com/mochajs/mocha/issues/188 https://github.com/mochajs/mocha/issues/187 https://github.com/mochajs/mocha/issues/2438#issuecomment-247223269 in discussion, not main post
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Reactions: 4
- Comments: 18 (11 by maintainers)
This happens with cypress.
Reproduction steps.
npm i
to install packages.packages/reporter
and write a failing test in ‘header/header.spec.jsx’ file like: 'expect(false).to.be(true)`. (This step is added because I fixed the bugs and they might not exist when you cloned the repo.)npm test
.I found this bug because it happened in cypress CircleCI. Check cypress-io/cypress#5770
Trawling node versions doesn’t change anything its a band-aid over the problem. The previously “resolved” ticket was on node 7.2, and our group has seen the issue from node 8 up through node 12
Node 10 branch: https://gitlab.com/nicwest/mochaexitcode_3893/tree/node10 Node 10 output: https://gitlab.com/nicwest/mochaexitcode_3893/-/jobs/221718456 Node 12 branch: https://gitlab.com/nicwest/mochaexitcode_3893/tree/node12 Node 12 output: https://gitlab.com/nicwest/mochaexitcode_3893/-/jobs/221761744
Only additional change from the node 8 branch, repeated tests to extend the run. I’d created master on node 8 to reduce the code needed to reproduce the issue.