ember.js: Allowing rejected promises in tests

I’m not here to debate the “right or wrong” of testing exceptions but I did want to offer a workaround for those like me who value testing 500 errors in acceptance tests (whether you are using ember-data or something else that wraps RSVP). Full credit for this goes to my good friend @williamsbdev who found the least hacky/but most functional way to get this working 😃

var application, originalLoggerError, originalTestAdapterException;

module('Acceptance to tdd error handling in your ember app', {
    beforeEach() {
        application = startApp();
        originalLoggerError = Ember.Logger.error;
        originalTestAdapterException = Ember.Test.adapter.exception;
        Ember.Logger.error = function() {};
        Ember.Test.adapter.exception = function() {};
    },
    afterEach() {
        Ember.Logger.error = originalLoggerError;
        Ember.Test.adapter.exception = originalTestAdapterException;
        Ember.run(application, 'destroy');
    }
});

test('some test that blows up when I try to tdd and a 500 makes RSVP reject', (assert) => {
    server.post('/api/foo/1', (db, request) => {
        //return a 500 with ember-cli-mirage/or whatever ajax stub library you choose
    });
    visit('/foo');
    fillIn('#my-input', 999);
    click('#my-btn');
    andThen(() => {
        //assert your modal shows up/or whatever biz thing happened
    });
});

Here’s what I figured out:

1. QUnit is designed to fail tests on all errors, including unhandled rejected promises. Mocha won’t do that unless you use an assertion against a rejected promise.

2. QUnit does not have an API to squelch those errors, making a test pass. There is assert.throws, but:

   • It can’t be used with Cucumber-style tests, which execute a linear list of steps, and steps can’t be nested.
   • It seems to be synchronous. If I get it right, you can’t use it with async code and instead have to catch the failing promise.
   • But there is no access to Ember Data promises from an acceptance test.

3. Ember.onerror is an old and quite poorly documented feature of Ember:

   • It is mentioned once in the guides as a simple hook for Sentry/Bugsnag.
   • It’s never mentioned in the API docs.
   • It does not have an export.
   • It does not offer pub/sub, meaning two addons can’t use it.
   • ember-mocha does not seem to use it.
   • ember-qunit does use it (see below), but does not document it, keeping us clueless.

4. I had an impression that Ember.onerror is a candidate for deprecation and should be avoided. @amk221 kindly helped me understand that it has special behavior in ember-qunit: catching an error in Ember.onerror would prevent QUnit from failing a test!

5. At the same time, QUnit would ensure Ember.onerror is not catching all errors. In case it does, you should see a failing test case. To avoid this, you need to re-throw errors after catching them in Ember.onerror.

6. @amk221 pointed out that Ember.onerror only catches errors that are thrown within the Ember runloop. For some reason, this throw:

   async someMethod() {
     try {
       await store.query(/*...*/);
     } catch (e) {
       throw e;
     }
   }
   

   …would throw the error outside of the runloop, and Ember.onerror will ignore it!

   The solution is to wrap it with run from '@ember/runloop'. This is the first time I’ve had to use run in app code.

Now I have control over errors in tests again, which I lost after Ember.Test.adapter.exception was removed.

Big thanks to @amk221 for taking time and effort to help me figure this out.

@toranb - Thank you for calling attention to this.

I would like to give some background. One of the applications that we were working on had great validations before the request was made so it would not get a 400. Instead of handling all the errors with each ajax request, we wanted to catch all the errors across the application. So we setup something like this:

Ember.RSVP.on("error", function() {
    // handle 401
    // handle 403
    // handle 5XX errors
    // handle other errors
});

The problem then came when we wanted to test. This is why the monkey patching (done above in the comment by @toranb) on Ember.Logger.error and Ember.Test.adapter.exception were done. I was not terribly fond of what I did but it gave me some confidence that the application would respond in the desired manner.

I completely understand what @stefanpenner was saying about handling the error by the end of the run loop. When running the tests, you don’t know if the error has been handled and the test framework is going to let the user know that something is broken. This is great. We just took advantage of the Ember.RSVP.on("error" which allowed it to handle the error and had to trick the test into allowing it.

It does though?

I do this:

  debugger;

  Ember.onerror = (error) => {
    debugger;
  };

I can see the first debugger statement firing, but the second one never fires, and tests are red.

Anyway, I believe, Ember.onerror is (was?) a hook that lets you log your errors to services like Sentry/Bugsnag. It’s not a way to intercept errors.

---

this works for me, is because my AJAX requests are routed through RSVP (because I use ember-fetch), if yours just use fetch, then the squelching won’t work.

We use Ember Data, currently at version 3.13.1. No jQuery.

---

I’m a bit at a loss here. The previous solution of using Ember.Test.adapter.exception has been removed from Ember, and there is no replacement.

I think this post from EmberMap could be very helpful for people that end up in this conversation:

https://embermap.com/notes/86-testing-mirage-errors-in-your-routes

And that is also the current behavior.

Then why is the following code breaking?

function failToDoX() {
  return new Ember.RSVP.Promise(function(resolve, reject) {
    Ember.run(null, reject, { an: 'error' });
  });
}

test('it catches', function(assert) {
  var caught = false;

  failToDoX()
  .catch(function(e) { caught = e; });

  assert.ok(caught != null);
});

I get to the catch, but the test still fails. Is it because failToDoX does its own run-loop wrapping?

All except for the unhandled rejection that rejects with the error

This I can get on board with 😃

@stefanpenner

seems dubious to allow rejections during tests

Seems perfectly normal to me. How else would I test that I show an appropriate error message when I get a 422 response from some API?

Also: this behavior is not just bad, it’s also inconsistent.

This doesn’t break the test:

new Ember.RSVP.Promise(function(resolve, reject) {
  reject();
});

But this does:

new Ember.RSVP.Promise(function(resolve, reject) {
  reject("anything");
});

because in the first case Test.adapter.exception(error); is Test.adapter.exception(undefined); and the test adapter ignores it.