mocha: "Warn" ability, instead of fail

Thanks @jbnicolai, and everyone for chiming in. Sorry I’ve been unresponsive here.

#1773 doesn’t really address our need, but I won’t argue — if no one else seems to find value in this idea, alas. =) Thanks for the consideration anyway!

If you’re interested in why / more details, tl;dr:

• As we move to more microservices, some service dependencies can be temporarily down; retrying right away won’t help in those cases.
• We do know about Travis’s allow_failures, as well as .skip (we actually use --grep '#skipci' --invert instead, for flexibility). Those are good hammers, but not everything is a nail.
• In particular, we’re interested in this feature for individual tests (rather than entire suites). And running just those tests separately isn’t straightforward; nor is repeating the entire suites efficient.
• Yes, we do view test output even if it passes — when we run tests locally.

I guess this extended beyond a tl;dr. =)

Our failure rate these days is low enough that we’re okay living with this right now, so I don’t anticipate myself getting to a documentation+tests PR, but thanks for letting me know that’s an option!

In particular, we’re interested in this feature for individual tests (rather than entire suites). And running just those tests separately isn’t straightforward; nor is repeating the entire suites efficient.

@aseemk Gotcha. That’s where both retries as well as my suggestion fall apart, sorry 😦 However, I think we’ve got another PR that might help: https://github.com/mochajs/mocha/pull/1618 With dynamic skip, the resulting specs would look like:

it('spec that might fail', function(done) {
  someAsyncTaskThatMightFail(function(err, res) {
    if (err) return this.skip();

    // otherwise make assertions
  }.bind(this));
});

It doesn’t have the benefit of a separate section in the reporter output (e.g. passing, failing, pending, warnings), but it seems to achieve the desired behaviour?

Here is a simplified workaround

it.allowFail = (title, callback) => {
  it(title, function() {
    return Promise.resolve().then(() => {
      return callback.apply(this, arguments);
    }).catch(() => {
      this.skip();
    });
  });
};

it.allowFail('skip on error', function() {
  assert.ok(false);
});

or use it here if you want.

This could be useful for the HTTP tests.

@kking937 you can do this by not passing a callback to it():

it('should be written later');

@aseemk Would wrapping the finicky tests each in a try/catch work? Dump something to STDOUT…