cypress-parallel: Random test cases were failing when we tried to generate integration code coverage report with cypress-parallel having multiple threads

I am able to generate report when there is no cypress-parallel but when I have used cypress-parallel command then code-coverage report is generating but random test cases were failing due to below errors:-

CypressError: cy.task('coverageReport') failed with the following error:

Unexpected end of JSON input

https://on.cypress.io/api/task

Because this error occurred during a after all hook we are skipping all of the remaining tests. at <unknown> (http://localhost:4210/__cypress/runner/cypress_runner.js:145679:78) at tryCatcher (http://localhost:4210/__cypress/runner/cypress_runner.js:11318:23) at Promise._settlePromiseFromHandler (http://localhost:4210/__cypress/runner/cypress_runner.js:9253:31) at Promise._settlePromise (http://localhost:4210/__cypress/runner/cypress_runner.js:9310:18) at Promise._settlePromise0 (http://localhost:4210/__cypress/runner/cypress_runner.js:9355:10) at Promise._settlePromises (http://localhost:4210/__cypress/runner/cypress_runner.js:9431:18) at _drainQueueStep (http://localhost:4210/__cypress/runner/cypress_runner.js:6025:12) at _drainQueue (http://localhost:4210/__cypress/runner/cypress_runner.js:6018:9) at …/…/node_modules/bluebird/js/release/async.js.Async._drainQueues (http://localhost:4210/__cypress/runner/cypress_runner.js:6034:5) at Async.drainQueues (http://localhost:4210/__cypress/runner/cypress_runner.js:5904:14) From Your Spec Code: at Context.generateReport (webpack:///./node_modules/@cypress/code-coverage/support.js:200:0)

From Node.js Internals: SyntaxError: Unexpected end of JSON input at JSON.parse (<anonymous>) at includeAllFiles (/var/jenkins/workspace/sample_project/node_modules/@cypress/code-coverage/task-utils.js:348:28) at coverageReport (/var/jenkins/workspace/sample_project/node_modules/@cypress/code-coverage/task.js:204:7) at invoke (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/child/run_plugins.js:234:16) at <unknown> (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/util.js:59:14) at tryCatcher (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/node_modules/bluebird/js/release/util.js:16:23) at Function.Promise.attempt.Promise.try (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/node_modules/bluebird/js/release/method.js:39:29) at Object.wrapChildPromise (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/util.js:58:23) at RunPlugins.taskExecute (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/child/run_plugins.js:240:10) at RunPlugins.execute (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/child/run_plugins.js:160:21) at EventEmitter.<anonymous> (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/child/run_plugins.js:257:12) at EventEmitter.emit (events.js:314:20) at EventEmitter.emit (domain.js:483:12) at process.<anonymous> (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@packages/server/lib/plugins/util.js:33:22) at process.emit (events.js:314:20) at process.EventEmitter.emit (domain.js:483:12) at process.emit.sharedData.processEmitHook.installedValue [as emit] (/root/.cache/Cypress/10.8.0/Cypress/resources/app/node_modules/@cspotcode/source-map-support/source-map-support.js:745:40) at emit (internal/child_process.js:877:12) at processTicksAndRejections (internal/process/task_queues.js:85:21)

Or Sometime getting this error

CypressError: cy.task('coverageReport') failed with the following error:

Unexpected end of JSON input

https://on.cypress.io/api/task

Because this error occurred during a after all hook we are skipping all of the remaining tests.

FYI , I have followed below link in order to generate cypress integration code coverage report and we need parallelism in order to speed up test execution so I have used below command in order to run test cases in parallel:-

“cypress-parallel -s cypress:run -t 8 -d cypress/e2e/testcases --verbose --reporter cypress-mochawesome-reporter -a ‘"–config baseUrl=http://localhost:4210"’ --strictMode false”,

https://lukas-klement.medium.com/implementing-code-coverage-with-angular-and-cypress-6ed08ed7e617

Cypress version : 10.8.0 cypress-parallel : 0.9.1

Please note we have one CI machine so we have to use cypress-parallel with multi-threading

Total test cases around 50. Update: Seems to be related to parallel Cypress runs. Fairly consistently fails for me when running with parallelization, but succeeds when using a single (non-parallel) job.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Reactions: 1
  • Comments: 15

Most upvoted comments

I hit this same issue and the reason is because cypress-coverage plugin merges the coverage data in a way that is not thread or multiprocessing-safe. This is fine for typical cypress runs but clashes with the way cypress-parallel works. The core problem is in the way the .nyc_output/out.json file is maintained. Under cypress-parallel multiple processes hit this code at the same time and create race conditions where data loss or corruption can occur due to concurrent read and writing to the same file.

My workaround is to add a lock around the above file access as follows:

// cypress.config.js or as described in 
// https://docs.cypress.io/guides/tooling/code-coverage#Install-the-plugin
const lockfile = require('proper-lockfile');
module.exports = defineConfig({
  e2e: {
    setupNodeEvents(on, config) {
      require('@cypress/code-coverage/task')((ignored, tasks) => {
        // we use our own locking here to prevent a race condition with cypress-coverage and
        // cypress-parallel
        const myTasks = {
          ...tasks,
          combineCoverage: async (sentCoverage) => {
            const release = await lockfile.lock('/tmp/cypressCombineCoverage.lock', {
              realpath: false, // allows following symlinks and creating the file
              retries: {
                retries: 10,
                factor: 2,
                minTimeout: 100,
                maxTimeout: 1000,
                randomize: true,
              },
            });
            const ret = await tasks.combineCoverage(sentCoverage);
            await release();
            return ret;
          },
          coverageReport: async () => {
            const release = await lockfile.lock('/tmp/cypressCoverageReport.lock', {
              realpath: false, // allows following symlinks and creating the file
              retries: {
                retries: 10,
                factor: 2,
                minTimeout: 100,
                maxTimeout: 1000,
                randomize: true,
              },
            });
            const ret = await tasks.coverageReport();
            await release();
            return ret;
          },
        };
        on('task', myTasks);
      }, config);

edit: also serialized coverageReport task

+3
isstabb on Oct 11, 2022

I accidentally left part out when adapting it to post here. I updated my comment above. Namely this was missing:

        const myTasks = {
          ...tasks,
+1
isstabb on Oct 11, 2022
Read more comments on GitHub
← cypress-parallel: Question about: Some test suites likely terminated without passing on results
go-mc: getting: auth fail and recv packet for Login fail errors →