ember-orbit: Acceptance tests do not wait for all requests during a store merge

The Problem

I’m chasing a bug in an existing project where a number of acceptance tests started failing because some (all?) of the async test helpers from ember-test-helpers lost the ability to know the app is settled. For example a await click line in a test will continue forward while async stuff (ember-concurrency task and ajax calls) are still running.

Backstory

I say ‘lost’ above because this occurred in an existing project where we’ve been using orbit for a few months. The tests have been rock solid until… I added ember-auto-import as a top level dependency. Super weird right? I couldn’t make heads of tails of that one, so I started a new project to reproduce the issue in an app with fewer dependencies.

Reproduction App

I’ve been able to reproduce the failure in this public app https://github.com/forge512/async-test-bug.

The app has component which runs an ember-concurrency task to create 4 posts component. And it has a test which verifies it works.

Experiment 1 (ember-data parallel save)

I used ember-data to fire requests to create 4 posts with parallel requests. See the use of ember-concurrency all here.

Tests pass.

Experiment 2 (ember-data parallel save)

In this branch, I adapted the component to fire the requests in serial.

Tests pass.

Experiment 3 (orbit with store forking)

In this branch, I switch to using orbit with a blocking remote store. The component forks the store, adds 4 posts, then merges the store.

Tests fail.

Summary

I’m not sure why one app (the private repo) worked until adding ember-auto-import. I’m also not sure why the example app failed in experiment 3. I had expected experiment 3 to pass, but then mysteriously fail when I added ember-auto-import.

About this issue

Original URL
State: closed
Created 5 years ago
Comments: 16 (13 by maintainers)

Most upvoted comments

Thanks for the detailed report and the app to repro the test issue.

The following changes to your test seem to fix the problem:

import { module, test } from "qunit";
import { click, visit, settled } from "@ember/test-helpers";
import { setupApplicationTest } from "ember-qunit";
import setupMirage from "ember-cli-mirage/test-support/setup-mirage";

module("Acceptance | create many posts", function(hooks) {
  setupApplicationTest(hooks);
  setupMirage(hooks);

  test("it makes the posts", async function(assert) {
    server.timing = 400;
    await visit("/");

    await click("button");

    // await request processing
    await this.owner.lookup('service:store').source.requestQueue.process();

    // await app settled state
    await settled();

    assert.dom(".post").exists({ count: 4 });
  });
});

Waiting for the store’s requestQueue to finish processing is necessary since the action triggers multiple fetch requests. I’ll try to merge something like #164 soon to handle this kind of waiting. Obviously, the waiting for settledness above is just too verbose.

Note that settled in ember-test-helpers still seems to be only xhr-aware. That’s why await settled() does not work on its own - it needs to be preceded by awaiting the requestQueue to process. It’s also why you didn’t experience issues with ember-data. We should really update ember-data and the settled helper to be fetch-aware.

Beyond this, I’m not sure exactly what’s changed and why in your app, especially regarding ember-auto-import. However, if you can reproduce more issues in your demo app, please let me know and I’ll investigate.

dgeb on Jan 28, 2019