jest: failing beforeAll() causes even passing tests in the scope to fail

🐛 Bug Report

I’m looking for the best practice for aborting a describe block if the test setup fails. For example, when testing an API, if authenticating fails, it’s pointless to run any other tests.

As suggested in the issue template, I searched StackOverflow first, where I found this incomplete answer, advising to place the initialization code in a beforeAll block. Hence, raising the issue here because

I haven’t seen anything documenting the practice of returning failure from beforeAll
If a beforeAll block fails, tests in that describe block are still run, and they fail, even if otherwise they would pass!

To Reproduce

describe('test that a 3rd party API remains consistent', () => {
  beforeAll(() => expect('login').toBe('successful'));  // this will fail
  test('API function 1', () => expect(1).toBe(1));  // each...
  test('API function 2', () => expect(2).toBe(2));  // ...of these...
  test('API function 3', () => expect(3).toBe(3));  // ...will fail too
});

Expected behavior

Jest should report that the beforeAll() failed, and bail the describe scope without executing further tests in it. If it’s somehow intended behavior that all tests should still be executed, and still marked as failed (which I find odd), this should be documented under beforeAll and in the Setup and Teardown guide. Other tests in the file, outside of the failing block, should still be executed.

Link to repl or repo (highly encouraged)

https://repl.it/@DanDascalescu/beforeAll-failure-should-bail-the-test

Run `npx envinfo --preset jest`

  System:
    OS: Linux 4.15 Ubuntu 16.04.4 LTS (Xenial Xerus)
    CPU: x64 Intel(R) Core(TM) i7-7500U CPU @ 2.70GHz
  Binaries:
    Node: 10.5.0 - /usr/local/bin/node
    Yarn: 1.7.0 - /usr/bin/yarn
    npm: 6.2.0 - /usr/local/bin/npm
  npmPackages:
    jest: ^23.4.1 => 23.4.1

About this issue

Original URL
State: open
Created 6 years ago
Reactions: 30
Comments: 25 (14 by maintainers)

Commits related to this issue

add jest-circus this solves the issue of seeing two failures for each test in test output when beforeAll fails https://github.com/facebook/jest/issues/6695 — committed to makerdao/dai.js by levity 5 years ago
add jest-circus this solves the issue of seeing two failures for each test in test output when beforeAll fails https://github.com/facebook/jest/issues/6695 — committed to makerdao/dai.js by levity 5 years ago
Use jest-circus runner in main tests. Fixes https://github.com/facebook/jest/issues/6695 — committed to InboxSDK/InboxSDK by Macil 5 years ago

Most upvoted comments

To my eyes this just looks like an issue with the reporting. beforeAll is only run once and none of the other tests actually execute, but they all fail with the same reason, and they all print the same error.

We should probably avoid printing the individual tests if beforeAll fails. Right now the implementation just checks if we’ve had an error and if yes, print it and fail the test without actually executing it

SimenB on May 6, 2019

As a present for this issue’s 5th birthday, I would like to ask a maintainer to reopen #2713, because throwing from beforeAll should really stop executing any further tests. @SimenB?

It’s sad that 3rd party packages (now with tens of thousands of weekly downloads) had to be developed to work around this problem:

dandv on Sep 20, 2023

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

github-actions[bot] on May 26, 2023

Definitely not stale, still see this issue in latest jest.

JaneJeon on Mar 18, 2022

jest-circus seems to not fix this. All 3 tests still run and fail.

Here’s the command I ran:

$ yarn run jest --testRunner=jest-circus/runner --bail

Am I missing a configuration value somewhere?

chrismwendt on Apr 9, 2019

Closing this as the behavior is fixed in jest-circus

It doesn’t seem to be fixed in latest Jest (v23.6.0). So it’s fixed for what users exactly? Facebook internal?

medikoo on Oct 18, 2018

Unstale

thernstig on May 26, 2023

You can follow #6295 for state of making it the default.

SimenB on Oct 18, 2018

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

github-actions[bot] on Mar 17, 2022

Still not working as expected… Not via throwing an error or calling done.fail(err)

Bec-k on Mar 17, 2021

I’ve seen a similar issue with reporting. It’s hard to disentangle from proprietary code, but what happens is that a test.todo('description', callback); reports the MongoDB error thrown by the previously executed test (a bulk write failure) instead of the correct error that only a description should be specified for .todo tests. I tried throwing an error directly in that failing test but that doesn’t reproduce the behavior.

Happy to see if the .todo issue reproduces once this is fixed.

dandv on May 9, 2019

This is still as broken as ~1yr ago, even with circus… Please consider reopening.

dandv on May 5, 2019