c8: async functions show return lines as not being covered from a branch perspective

  • 5.0.1:
  • Windows 10 / Node 12.6.0 with --experimental-modules / tap 14.6.1:

We are finding ourselves needing to always inject /* c8 ignore next */ in our async functions in order to get passing coverage results. I have included sample code below.

Typical command line usage would be c8 tap (oftentimes with --check-coverage flags as well).

It should be noted that these cases where we find this behavior are all in Node --experimental-modules environments where we are using the ES modules format. (This is actually what led us to use c8 / tap in the first place as it they play nicely with native ESModules without need for ESM loader or similar)

There seem to be two different scenarios where branch covereage reporting is problematic.

case 1 - standard function return

async function () {
    // some code
   return; // must ignore this line to get branch coverage
};

case 2 - then() function

const value = await someThenable()
  .then( (data) => {
     // do something;
    return data;
  }); // must ignore this line to get branch coverage

** Sample code under test**

I can get 100% branch coverage on this file only with each async function return ignored from coverage reports as shown.

import mongo from 'mongodb';
const { ObjectID } = mongo;

const getRecord = () => {
  return {
    _id: ObjectID(),
    created: new Date()
  };
};

class MockCollection {
  constructor(opts) {
    this._failKeys = new Set(opts.failKeys);
    this._notFoundKeys = new Set(opts.notFoundKeys);
    this._keyAccessCounts = {};
  }

  _action(key) {
    if (!this._keyAccessCounts.hasOwnProperty(key)) {
      this._keyAccessCounts[key] = 0;
    }
    this._keyAccessCounts[key]++;

    if (this._failKeys.has(key) ) {
      throw new Error(`mock error on key '${key}'`);
    }

    return ( this._notFoundKeys.has(key) ) ? false : true;
  }

  _getKeyAccessCounts(key) {
    return (key) ? this._keyAccessCounts[key] : this._keyAccessCounts;
  }

  async createIndex(keys, opts) {
    this._action('createIndex');

    const parts = [
      'mock',
      Object.keys(keys)
        .reduce( (str, key) => `${str}_${key}_${keys[key]}`, '' )
        // trim leading '_'
        .slice(1)
    ];

    if (opts) {
      parts.push( Object.keys(opts).join('_') );
    }

    /* c8 ignore next */
    return parts.join('_');
  }

  async find() {
    const recordCount = 100;
    const records = [];

    if (this._action('find')) {
      for (let i = 0; i < recordCount; i++) records.push(getRecord());
    }

    /* c8 ignore next */
    return new MockCursor(records);
  }

  async findOne() {
    /* c8 ignore next */
    return (this._action('findOne')) ? getRecord() : null;
  }

  async insertOne() {
    this._action('insertOne');
    /* c8 ignore next */
    return getRecord();
  }

  async updateOne() {
    let result = {
      modifiedCount: 1,
      matchedCount: 1
    };
    const succeeded = this._action('updateOne');
    if (!succeeded) {
      result = {
        modifiedCount: 0,
        matchedCount: 0
      };
    }
    /* c8 ignore next */
    return result;
  }
}

class MockCursor {
  constructor(records) {
    this._records = records;
    this._offset = 0;
    this._limit = 0;
  }

  limit(limit) {
    this._limit = limit;
    return this;
  }

  sort() {
    // not implemented
    return this;
  }

  skip(offset) {
    this._offset = offset;
    return this;
  }

  async toArray() {
    const offset = this._offset;
    const limit = this._limit;
    const end = (limit > 0) ? offset + limit : undefined;
    /* c8 ignore next */
    return this._records.slice(offset, end);
  }
}

class MockDb {
  constructor(opts) {
    this._opts = opts;
    this._collections = {};
  }

  collection(name) {
    if (!this._collections.hasOwnProperty(name)) {
      const opts = {
        failKeys: this._opts.failConfig[name] || [],
        notFoundKeys: this._opts.notFoundConfig[name] || [],
      };
      this._collections[name] = new MockCollection(opts);
    }
    return this._collections[name];
  }
}


/*
The config passed to MockMongoClient allows us to configure failure (thrown errors)
and/or not found behavior from methods called on given collection. The object may look like the following...

{
  failConfig: {
    clientAccounts: ['findOnce']
  },
  notFoundConfig: {
    clientAccounts: ['updateOne']
  }
}

... where the keys for each config objecg specify a mongo collection name and the arrays at each keys represent ths
*/

const defaultOpts = {
  failConfig: {},
  notFoundConfig: {}
};

class MockMongoClient {
  constructor(opts) {
    opts = opts || {};
    this._opts = Object.assign({}, defaultOpts, opts);
    this._db = null;
  }

  db() {
    if (!this._db) {
      this._db = new MockDb(this._opts);
    }
    return this._db;
  }
}

const getMockMongoClient = (opts) => new MockMongoClient(opts);

export {
  getMockMongoClient as default,
  MockCollection,
  MockCursor,
  MockDb,
  MockMongoClient,
  ObjectID
};

About this issue

  • Original URL
  • State: closed
  • Created 5 years ago
  • Comments: 21 (7 by maintainers)

Most upvoted comments

This is fixed in the latest Node.js v14 but not in the latest v12.

Just wanted to add to the conversation.

I’m not seeing this particular issue with all async functions. Instead, in my code, it’s always the closing brace of a try/catch/finally block within an async function that gets reported as a covered line but an uncovered branch:

async function abc() {
  try {
    return 'abc';
  } finally {
    console.log('in finally');
  } // this is a covered line but uncovered branch
}
async function badMath() {
  try {
    return 5 / 0;
  } catch (e) {
    console.error(e);
  } // this is a covered line but uncovered branch
}

Is V8 maybe adding some kind of hidden conditional at the end of a try block inside an async function that gets misreported as an uncovered branch?

@mikeal @dbarnespaychex the way I debug one of these issues, such that I can make a useful upstream report to V8 is as follows:

I create a file with example code, like this:

foo.js

async function abc() {
  try {
    return 'abc';
  } finally {
    console.log('in finally');
  } // this is a covered line but uncovered branch
}

abc()

I run the file with c8, ./bin/c8.js foo.js.

I look at the output in coverage/tmp/coverage-xyz.json, for the function abc. This is the raw output from V8.

In the code above, I see this output:

       {
          "functionName": "abc",
          "ranges": [
            {
              "startOffset": 0,
              "endOffset": 146,
              "count": 1
            },
            {
              "startOffset": 97,
              "endOffset": 145,
              "count": 0
            }
          ],
          "isBlockCoverage": true
        }
}

☝️ what this tells me is that V8 is treating characters 97 - 145 as a distinct block that is not executed… the problem is that this is the following line:

'// this is a covered line but uncovered branch\n

☝️ this would appear to definitely be a V8 bug, and since c8 doesn’t understand JavaScript (it has no AST parser), the cleanest fix has often been to patch the upstream issue in V8 and downstream it to Node.js.


@victorgomes I don’t suppose you could help us on the V8 side? as @mikeal points out, v8’s coverage has become pretty important to the Node ecosystem since ESM became un-flagged.

@maxwell-k 👍 thanks for the reproduction. I’d be surprised if this didn’t turn out to be an issue in V8, I’ll open an upstream tracking issue.