ember.js: Ember 2.15 - Integration tests with rejected promise

OK, I have finally run this issue to ground. It’s a doozy…

The underlying principle in tests is that rejected promises which are not handled within the same run loop trigger the unhandled rejection handler (which by default calls assert.ok(false, ....) and fails the current test). Generally speaking, this is very desireable ergonomics wise.

Given this test (which passes on Ember <= 2.14 and fails on 2.15+):

test('error', function(assert) {
  assert.expect(1);

  this.set('promise', RSVP.reject('bar'));

  this.render(hbs`{{my-component promise=promise}}`);

  return wait().then(() => {
    assert.equal(this.$().text(), 'error: bar');
  });
});

We can see that the following things are happening:

1. create a rejected promise (RSVP.reject('bar')) which queues up the unhandled rejection assertion
2. invoke this.set (which is essentially Ember.run(Ember.set, this, rejectedPromise))
3. invoke this.render (which is ultimately going to instantiate the component with the provided promise, and call promise.then(handleSuccess, handleFailure))

The primary difference between the Ember versions is how the unhandled rejection assertion is delivered.

In Ember <= 2.14 the assertion is queued up to be ran after a flush of the run loop on the next tick (e.g. setTimeout(flushAndAssert, 0)). Since both 2) and 3) above are synchronous, this meant that under Ember <= 2.14, the promise rejection was handled by the time the unhandled rejection assertion is ran, and therefore no assertion is made.

In Ember 2.15+ (versions which include https://github.com/BackburnerJS/backburner.js/pull/203), the assertion is queued up to be ran after the flush of the next run loop. This means that when the run loop from 2) is ran, the assertion is triggered and therefore the test fails.

Unfortunately, I believe the change in behavior is desirable and should remain.

---

There are a few different ways to successfully test this interaction. I’ve made separate PR’s to the demo repo to show each of them:

1. Pass in a pending promise, then either resolve or reject as needed: https://github.com/amk221/-ember-rsvp-test-failure/pull/1
2. Chain off of the rejected promise to reset its “unhandled rejection” state and avoid the assertion: https://github.com/amk221/-ember-rsvp-test-failure/pull/2
3. Monkey patch Ember.Test.adapter.exception to swallow the unhandled rejection assertion: https://github.com/amk221/-ember-rsvp-test-failure/pull/3

I missed that there was a demo repo in your intro (sorry about that), I’ll try to poke at this later today…

We too experienced a version of this in our tests. We use sinon.js for stubbing functions calls.

So a test like below, which was passing before 2.15,

test('it shows correct error message', function(assert) {
   const sandbox = sinon.sandbox.create();
  // stubs some async call which is made in component
   const stub = sandbox.stub(Model, 'function').returns(RSVP.reject('some error')); // Line 2
   this.render(hbs`{{my-component}}`);
  return wait().then(() => {
    assert.equal('do some assertion');
  });
});

It fails in latest ember (tried in 2.18), where the RSVP.reject propagates up the chain and breaks the test.

Apart from @rwjblue suggestion above, changing the line 2 to use sinon rejects helper also works. Sinon rejects/resolves methods return native JS promise and not RSVP.

const stub = sandbox.stub((Model, 'function').rejects(); // Line 2

I believe this will be addressed by https://github.com/BackburnerJS/backburner.js/pull/277 but need to test the reproduction to confirm.