jest: Ability to skip rest test cases if one fails

šŸš€ Feature Proposal

Provide ability to skip rest test cases in a file or describe block if one test fails.

Motivation

For E2E testing, the test cases are usually written to be run sequentially, and a failed test should stop the rest from running.

Example

test('should login', async () => {});

test('should edit profile', async () => {});

test('should render profile', async () => {});

Or with describes:

describe('#1', () => {
  test('should login', async () => {});

  describe('#1.1', () => {
    test('should edit profile', async () => {});

    test('should render profile', async () => {});
  });
});

About this issue

  • Original URL
  • State: open
  • Created 5 years ago
  • Reactions: 47
  • Comments: 17 (3 by maintainers)

Most upvoted comments

Here is my temporary solution.

describe("test", () => {
  let skipOther = false
  test("check something", async () => {
    try {
      await checkSomething()
    } catch(e) {
      skipOther = true
      throw new Error("error")
    }
  })

  test("might be skipped", async () => {
    if(skipOther) throw new Error("error occured in previous tests.")
    console.log("display this, if there is no error before this.")
  })
})

#9944 But I doubt it will ever be merged, I havenā€™t heard back a single word from any contributor.

I created a PR that implements asynchronously canceling tests as described in #8604. With that in place, one could implement this functionality through easy monkey patching in user land like this:

let prevFailed;
const _describe = global.describe;
global.describe = (...args) => {
  prevFailed = false;
  return _describe(...args);
};
global.test.step = (name, fn, ...args) => {
  test(name, async function(...fnArgs) {
    if (prevFailed) {
      return this.skip();
    }
    try {
      return await fn(...fnArgs);
    } catch (e) {
      prevFailed = true;
      throw e;
    }
  }, ...args);
};

Playwright already provides a test.step https://playwright.dev/docs/api/class-test/#test-step And this would also make sense for jest specially for e2e tests.

No itā€™s not a duplicate. We are requesting mocha-steps not jasmine-fail-fast.

TL;DR: skip (and auto fail) the following steps after the first failing step within the current describe (not exit on first failure)

@jeysal please reopen.

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

Smoke tests in front of more substantial tests save people time. Without the ability to short circuit subsequent test, the time saving is mitigated.

Where does this functionality need to be added? Would this live in a test runner or is this a jest issue? (Many people donā€™t even realize the distinction between jest and test runnersā€¦)

Personally I think this makes sense, and would love to see it in the jest core under step, as it lets you break tests into smaller parts (allowing you to describe them better, since they each have an it/step message), and increase performance for when youā€™re testing something that is dependent on prior tests.

Take for example this test:

describe('.root', () => {
  it('contains the root package.json', () => {
    // eslint-disable-next-line @typescript-eslint/no-require-imports
    const requirePackageJson = () => require(path.join(pathTo.root, 'package.json'));

    expect(() => requirePackageJson()).not.toThrow();
    expect(requirePackageJson()).toStrictEqual(packageJson);
  });
});

Itā€™s checking that pathTo.root points to a directory with a package.json, and that the package.json is the actual root package.json (by comparing the object returned by the require to one imported via the normal module loader).

Since require throws an error if it canā€™t find the file (the first expect), it makes no sense to test contents of the file (the second expect) if this happens.

But, since its are independent of each other, I either will have two tests failing (which while technically correct means Iā€™ve got pointless noise that adds nothing to the situation), or one ā€œbigā€ it that combines two tests.

Meanwhile, if we had step:

describe('.root', () => {
  // eslint-disable-next-line @typescript-eslint/no-require-imports
  const requirePackageJson = () => require(path.join(pathTo.root, 'package.json'));

  step('has the root package.json', () => {
    expect(() => requirePackageJson()).not.toThrow();
  });
  step('contains the root package.json', () => {
    expect(requirePackageJson()).toStrictEqual(packageJson);
  });
});

This maybe be a result of me being relatively new to the testing game, but it seems to me that step would empower jest further, as it enables you to do feature-like testing letting you nicely test apps as a whole.

I know that step is pretty self explanatory in its advantages, I felt like it could be valuable to put one to paper - Iā€™ve got a lot of tests that are similar to that.

I donā€™t mind step not being in the jest core, just so long itā€™s as easy to use as it, describe, #each, etc are today.

Iā€™m just really excited at the idea of having this feature in jest, as Iā€™ve come across this problem a lot! šŸ˜„

I have created the Jest Environment for this usecase

https://www.npmjs.com/package/jest-environment-steps

Okay then, we can keep this open to track fail-fast within a describe block (although the OP does not explicitly request this). I think itā€™s unlikely that this goes into Jest core though, while #6527 might as itā€™s a logical extension of bail.

My solution to skip rest tests after first failure https://github.com/facebook/jest/issues/6527#issuecomment-760092817 image