cypress: Component tests failing intermittently with uncaught Vite "failed to fetch dynamically imported module" error in CI

Hi all - we are looking to work on this in the next two weeks with the goal of completely solving it, eliminating the need for work arounds.

I am facing same issue

Any update on this issue? It’s unfortunately made Cypress component testing pretty much unusable for my projects since it’s extremely difficult to get full test suites to pass.

@adamscybot optimizeDeps doesn’t work for us. Still getting random errors on CI.

In my case it started happening more and more often after upgrading from 12.7 to 13.1. For some reason CI stopped logging the “[vite] ✨ new dependencies optimized:”, but I could reproduce that easily locally.

After going through Vite docs, I read this bit:

By default, Vite will crawl all your .html files to detect dependencies that need to be pre-bundled (ignoring node_modules, build.outDir, __tests__ and coverage). If build.rollupOptions.input is specified, Vite will crawl those entry points instead.

The bit about “__tests__” folder being ignored from pre-bundling was the problem in my case. Only tests that were nested inside “__tests__” folder were randomly failing. In my case, solution was to add all my test files to entries section of Vite config, like so (all my tests have this test-cy bit added in name):

optimizeDeps: {
  entries: ['src/**/*.test-cy.ts', 'src/**/*.test-cy.tsx', 'index.html']
}

Now for each test, Vite always optimizes all needed dependencies before running the test, and since that the issue has not been happening anymore.

For anyone interested, here’s our “workaround” solution. We have a script called cypress-ct-runner.mjs (under bin/vite-migration:

#!/usr/bin/env node
import { Chalk } from 'chalk';
import cypress from 'cypress';
import fs from 'fs';
import path, { dirname } from 'path';
import { fileURLToPath } from 'url';

const chalk = new Chalk({ level: 1 });
const groupId = process.argv[2];

if (groupId === undefined) {
  console.error('Please provide a group id parameter');
  process.exit(1);
}

const __dirname = dirname(fileURLToPath(import.meta.url));
const groups = JSON.parse(fs.readFileSync(path.join(__dirname, '..', '..', 'component-test-groups.json')));
const group = groups[groupId - 1];
const maxRetries = 5;
let specs = group.join(',');
let shouldFail = false;
let shouldRetry = true;
let retries = 0;

while (shouldRetry) {
  if (retries > 0) {
    console.log(chalk.bgRed(`Retry number: ${retries}`));
    console.log(chalk.bgRed('Retrying failed tests:'));
    console.log(
      chalk.red(
        specs
          .split(',')
          .map((spec) => `- ${spec}`)
          .join('\n')
      )
    );
    console.log();
  }

  const result = await cypress.run({
    spec: specs,
    browser: 'chrome',
    headless: true,
    testingType: 'component',
  });

  const failedTests = result.runs.reduce((failed, { tests, spec }) => {
    return [...failed, ...tests.filter(({ state }) => state === 'failed').map((test) => ({ ...test, spec }))];
  }, []);

  specs = failedTests
    .map(({ spec }) => spec.relative)
    .reduce((result, spec) => {
      if (!result.includes(spec)) result.push(spec);
      return result;
    }, [])
    .join(',');

  if (failedTests.length === 0) {
    shouldRetry = false;
  } else if (retries >= maxRetries) {
    shouldRetry = false;
    shouldFail = true;
  } else {
    retries++;
  }
}

if (shouldFail) {
  console.log(chalk.bgRed("Couldn't recover these failing tests:"));
  console.log(
    chalk.red(
      specs
        .split(',')
        .map((spec) => `- ${spec}`)
        .join('\n')
    )
  );
  process.exit(1);
} else {
  console.log(chalk[retries === 0 ? 'bgBlue' : 'bgRed'](`Finished running tests with ${retries} retries`));
}

Under the same path we also have another script which collects paths of all our component test specs and “groups” them randomly, it looks like this:

#!/usr/bin/env node
const fs = require('fs');
const path = require('path');
const glob = require('glob');

const groupCount = process.argv[2];

if (groupCount === undefined) {
  console.error('Please provide a group count parameter');
  process.exit(1);
}

glob('src/**/*.cytest.tsx', (err, files) => {
  if (err) {
    console.error('An error occurred:', err);
    process.exit(1);
  }

  console.log('Found', files.length, 'files');

  // sort files randomly
  files.sort(() => Math.random() - 0.5);

  // take 1/3 of the files and put them in the first group to warm up Vite's cache
  const firstGroupAmount = Math.ceil(files.length / groupCount / 3);
  const groups = [files.splice(0, firstGroupAmount)];

  // ignore the first group in this loop
  for (let i = 0; i < groupCount - 1; i++) {
    groups.push(files.slice((i * files.length) / (groupCount - 1), ((i + 1) * files.length) / (groupCount - 1)));
  }

  console.log(
    'Created',
    groups.length,
    'groups with following distribution:',
    groups.map((group) => group.length)
  );

  fs.writeFileSync(path.join(__dirname, '..', '..', 'component-test-groups.json'), JSON.stringify(groups, null, 2));
});

On CI we have a following setup of steps:

• Distribute component tests randomly
  • ./bin/vite-migration/distribute-component-tests.js 5
• Component tests [1]
  • ./bin/vite-migration/cypress-ct-runner.mjs 1
• Component tests [2-5]
  • ./bin/vite-migration/cypress-ct-runner.mjs 2-5

Our whole CI flow looks like this:

I hope that someone will find this useful until the root cause gets fixed.

We are having the exact same issue with component tests ( I was just coming here to open a new issue ) . We just migrated from webpack to vite. The error we get is:

The following error originated from your test code, not from Cypress. > Failed to fetch dynamically imported module: http://localhost:3000/__cypress/src/cypress/support/component.ts When Cypress detects uncaught errors originating from your test code it will automatically fail the current test. Cypress could not associate this error to any specific test. We dynamically generated a new test to display this failure.

We are getting this despite the fact that the file does exist. And this occurs randomly.

Vite: 4.1.1 Cypress: 12.5.1 Node: 19.3.0

One related issue: our cypress directory is on the same level as our src directory. When running locally, Cypress correctly expects component.ts to be where we have it in the cypress directory. But when we run the tests in docker, Cypress expects component.ts to be in a cypress directory under the src directory (see above error). Even if we use the ‘supportFolder’ config setting, Cypress still looks for it in the src directory (src is the vite root folder). So I just copied component.ts to the location cypress expects it to be (but still fails randomly despite this).

So locally this is not reproducable for us (so far). This only occurs in a docker container.

We are also running the tests in parallel.

We will look at trying vitejs/vite#12080 in our next sprint of work to see if we can get more fine grained error logging. This is very hard problem to solve.

Chiming in here in the hopes of saving others some time.

We had had intermittent issues with component tests failing in CI. The errors always came near a [vite] ✨ new dependencies optimized line in the job log.

I was able to reproduce the issue locally by clearing node_modules/.vite/deps before running the tests, and adding:

optimizeDeps: {
  entries: ['cypress/**/*'],
},

to our vite.config.js solved those issues. Our test files (.cy.js) are all located within that directory.

That was fine for a while, but I just now had to address another similar issue, this time with a new dependency (vue-router) that was not previously imported directly by our tested components. I eventually figured out that the issue was due to the fact that we cache the node_modules directory in between jobs (we use GitLab CI/CD). I don’t have a full understanding of the Vite internals, but I think it saw that (now stale) .vite/deps data and assumed it didn’t need to optimize (pre-optimize?) the Cypress test files again. I added a simple rm -rf node_modules/.vite/deps to the Cypress test job, and that fixed the issue.

@YossiSaadi we’ve tried upgrade to cypress 12.9 & Vite 4 and the errors still occur.

@lmiller1990 have you found any interesting insights?

@lmiller1990 I created a repro for this issue showing it happening in a NON-CI scenario. Hopefully, it helps get to the root of the problem.

git clone https://github.com/crobinson-expl/vue-scaffold-cypress-e2e-unit-march-15.git
cd vue-scaffold-cypress-e2e-unit-march-15
npm i
npm run test:unit:dev
// Choose Electron
// Click on HelloWorld.cy.ts

https://github.com/vitejs/vite/issues/12080 was released in vite 4.4.0

I experienced this too - for me it was because I was switching between branches with different dependencies, and one of them somehow hadn’t been reinstalled inbetween changing branches (in my case @pinia/testing). So while Cypress was reporting that it couldn’t import components.js, it wasn’t because that file was missing, it was because that file was erroring because of the missing import of @pinia/testing within it. Hope this helps people who end up here in the future (or me, when this happens again and see my own comment).

Still a problem. In my case, all test run fine locally but crashing randomly in pipeline, with the error mentioned. You can run the same job twice and where in the first run the Test A crashes and Test B works, in the second run both can crash or B crash and A works…

Hi. I’m adam-thomas-privitar on my personal account 😃. So I wouldn’t say that my confidence is 100% due to the odd nature of this issue. But I can say so far, the problem has not reoccurred in my branch since adding the test files to the optimizeDeps.entries after kicking it off a fair few times. I was also able to re-enable a certain suite which in particular was causing us especially big problems (but not exclusively) in regards to this issue, which is really promising. Previously we were retrying that test at the CLI level 3 times and even that eventually was not sufficient. So the fact it’s working without those retries strongly suggests there may be a path here.

I’m trying not to get my hopes up too much though since the real test will come when this gets merged to main next week and the build volume increases. So ultimately we still need to proceed with caution.

I will report back, but I feel there’s sufficient data to say that anyone experiencing this should try it so we can add more data points. If it does fix it, @RadomirNowak is a legend 😄, as this problem has truly haunted us at scale. If it suddenly fails, I might go and have a little cry somewhere and hang up my keyboard forever.

It’s perhaps of note that we have our own substantial layer above cypress (test support stuff) that is imported only into tests, in its own package, to abstract common things, and so the proposition that this plays a part is very plausible.

I believe nearly all of the discourse about solutions that seem to work then don’t is likely due to varying load on the agent that’s executing the tests. It’s a race condition in its current state, and so any fluctuation in speed affects it, and I’ve observed this myself. Also, the problem gets worse the more deps you add. If you are running tests in parallel that definitely adds to the randomness also, since that’s also causing varying load between runs. Another reason is if you have many agents and some are more highly specced than others.

If it does emerge that this is the problem then I guess a PR to the Cypress vite integration package is needed to automatically merge the test files into optimizeDeps.entries .

the thing with __tests__ was something that allowed me to find the root issue, which is: if you import packages inside your tests that are not imported in your app (so they are not prebundled by vite) then they will be dynamically optimized during test run which will result in the problem described in this thread. @floroz @adam-thomas-privitar can you try to include your test files in the entries section like I mentioned before? Also another thing could be to include there a glob for all your files (as a test) e.g. **/*.ts

We are using the __tests__ convention as well so the theory is certainly interesting.

I still need to confirm this long term, but I think setting a constant port value for dev server with Cypress tests seems to fix the issue. Please see whether it will work for you.

@adam-thomas-privitar @nive-copia @rszalski I updated the links to the pre-release binary in Mark’s comment. You should be able to install this one. Please comment if you are still having issues installing and thank you for your patience

@nive-copia this should be working, I’ll look into it. I was able to download the binary but not perform npm install.

Even if this solves this issue, I dont think it is the cause auf the problem. It is quite strange that the “manual” optimizeDeps has no effect.

Agreed. Still it would be useful to unblock people. I’m going to create a pre-release build of Cypress that people in this thread can drop into their CI pipelines to see if it has an impact on this error. I’ll update this thread with instructions when it is ready.

I am not sure how vite does detect “Relevant fields in your vite.config.js”. (https://github.com/cypress-io/cypress/issues/25913#issuecomment-1663993439). Maybe find-up should be replaced by loadConfigFromFile (the official vite API).

@OrbisK I played around with find-up vs loadConfigFromFile but I didn’t immediately see the opportunity to replace one with the other as you suggested. Though if I follow what you are suggesting, maybe we could use loadConfigFromFile for the file from the path determined with find-up, before we use vite.mergeConfig here. Is your thinking that, under the hood, vite may then recognize the need to rerun without force? Based on “Relevant fields in your vite.config.js” being mentioned here.

optimizeDeps simply does not work as advertised, at least for us - we regularly see “vite is optimizing…” for entries we’ve already got there, which is in all likelyhood the source of a good chunk of these issues.

I hope adding some more error hooks will help diagnose this.

Is it possible to start the component dev server with the --force cli flag? Maybe vite cache is the problem (related to the “vite is optimizing” problem with included deps)?

Docs:

The pre-bundling step will only need to be re-run when one of the following has changed:

• Package manager lockfile content, e.g. package-lock.json, yarn.lock, pnpm-lock.yaml or bun.lockb.
• Patches folder modification time.
• Relevant fields in your vite.config.js, if present.
• NODE_ENV value.

EDIT: optimizeDeps.force will ignore previously cached optimized dependencies.

FYI: I am not facing this issue (anymore (?) - and i dont know why). I can’t test it myself.

@floroz no sorry, we got caught up in other issues. We will try and get to this this week or next. To clarify, we are going to (or a third party, if anyone is up for it):

In https://github.com/vitejs/vite/pull/12084/files, a new event was added: vite:preloadError. We should listen to this on the server and log any events using DEBUG, like this https://github.com/cypress-io/cypress/blob/14a7416e126f6b296a2da037ae257c27e7915d5b/npm/vite-dev-server/src/plugins/cypress.ts#L10

Then when users run Cyperss with DEBUG=cypress:vite* they will (hopefully) see a more useful error on the server. If we can manifest this in the Cypress UI, that would be even better. The hope is we can find a more specific error that we can use to actually fix this bug.

FYI, this happens in other repos not using Cypress, https://github.com/vitejs/vite/issues/11804, so it’s possible the fix is actually a PR in Vite, eventually. I think the issue is we use Vite in a fashion that makes this error more regularly occurring here than just using Vite as a dev server independently.

@mx2s thanks! Can you share your./vite.config?

May be that helps anyone, Here’s my config for component testing:

import { defineConfig } from 'cypress';
import createBundler from '@bahmutov/cypress-esbuild-preprocessor';
import { devServer } from '@cypress/vite-dev-server';
import viteConfig from './vite.config';

module.exports = defineConfig({
  component: {
    setupNodeEvents(on) {
      on('file:preprocessor', createBundler());
    },
    devServer(devServerConfig) {
      return devServer({
        ...devServerConfig,
        framework: 'react',
        viteConfig: async () => {
          return { ...viteConfig('production') };
        },
      });
    },
  },
});

Tests started to pass for me once I changed viteConfig('production') from development to production (basically NODE_ENV)

Update… this is not forgotten, but we are trying to figure out how to fix this. It’s clear from the investigation so far there’s likely a few bugs that all manifest in the same fashion. In addition, the actual problem looks like it spans both Vite and whatever it’s integrated with (Cypress, PW, or whatever). So far, several developers (here and external) spend some time (days) trying to fix this, but no luck.

I think we need to budget significant (weeks) of time for someone to go a lot deeper into this, and possibly in conjunction with someone from the Vite team. Based on https://github.com/vitejs/vite/issues/11804, it is impacting many users outside of Cypress, too.

If you need another possible test case you can try the https://github.com/patrickcate/vue-uswds/tree/feature/606/cypress-12-upgrade branch of my Vue 3 project. Unlike other’s I’m seeing this error when running the tests locally, both in headed/open and headless/run mode.

The components it fails on appears to be random, but in headed/open mode, refreshing the browser often allows the tests to run fine. In headless/run mode this isn’t possible so I’ve never been able to get through a successful run with the 100 or so total number of components.

I am not using the vite-plugin-warmup plugin or running tests in parallel.

Other details:

• Cypress 12.9.0
• Vite 4.2.1 (also tried Vite 4.3 beta 2)

Local computer:

• MacBook Pro
• MacOS 11.7.4
• 2.6 GHz Quad-Core Intel Core i7
• 16 GB 1600 MHz DDR3

If anyone does this, please post here and let us know if you are still seeing import failures in CI

Before this change, 90% of my Cypress runs in the CI of my little hobby project were failing due to dynamic import issues. Since using your commit, none of them failed due to that reason. (It’s a tiny project, so probably not very representative, but wanted to let you know regardless)

@waldemarennsaed no I don’t think so, since there are different problems described in this issue that don’t relate to #26139. I would like to update the title of the issue to make it more specific though, since like Lachlan pointed out, the “failed to fetch” error can basically mean that any error occurred. I’ll be looking more into this issue today

Nice try @crobinson-expl but unfortunately your error is related to https://github.com/cypress-io/cypress/issues/26138, we don’t support Vite 4.2 yet. That will be fixed in https://github.com/cypress-io/cypress/pull/26139.

The “failed to fetch” is so general, it can basically mean any kind of error occurred. I think this issue you are encountering is unrelated to the one reported here.

Just a check in that over 2 weeks now the problem remains gone!

So its been a couple of days now and the problem has not come back in CI. As others have said, I think this is about capturing everything, including any test-support stuff as well as the test files themselves. Makes sense support file should be added here as well because that is imported separately by cypress and not from the tests.

the thing with __tests__ was something that allowed me to find the root issue, which is: if you import packages inside your tests that are not imported in your app (so they are not prebundled by vite) then they will be dynamically optimized during test run which will result in the problem described in this thread. @floroz @adam-thomas-privitar can you try to include your test files in the entries section like I mentioned before? Also another thing could be to include there a glob for all your files (as a test) e.g. **/*.ts

Just to let you know I am trying your workaround in CI currently, and I should be able to get back to you once its run a sufficient amount of times.

We’ve been stuck on this for about 3 weeks, would really love to hear from anyone with a workaround.

@astone123 @marktnoonan I was able to locally install the drawin arm64 beta version from Mark’s comment and compare with a stable version that we currently run.

Our component tests have 44 specs and I tested both the beta version and a stable v12.17.3 of cypress by running the entire suite 10 times for each version.

In each case the only failures were the loading of dynamically imported module:

As I mentioned above we are not only seeing errors like the above, that fail the spec, but also see the entire test run freeze randomly.

The very interesting thing is that when doing the tests on the beta version the spec never froze. We also saw much more of these dynamically loaded module errors (higher repro rate). The beta version definitely impacts this, but there might be now a completely different failure mode showing up, but manifested in the same cryptic message. Is there a way to get better logs from the Vite’s error hooks? Or maybe from Cypress’ debug logs?

Testing

• v12.17.3

  • runs froze (did not finish) 8 out of 10 times,
  • runs failed with the error above 6 out of 10 times (but the remaining 4 were inconclusive since the full run froze and I suspect it would have failed if ran till the end).

• beta

  • runs never froze (!)
  • runs always failed with the above error. In a single run from 50% to a 100% of the spec files failed with this error. Much more than in v12.17.3.

• I then tried running individual spec files, that I saw fail when run as part of the entire suite. Running a single spec file never failed on beta. Tried this like 50x times.

• I then tried running 2 spec files (both of which had failed in the full run) in this case, always the second spec failed, never the first one. Tried this also dozens of times.

• I tried running the above with DEBUG=cypress:* but that was an overwhelming amount of logs and nothing stood out to me. I’m happy to do some more debugging but I’d need pointers on how to narrow the logs down to what is relevant.

Is the experimental build available to public to test? I see the following error when I try to install.

` Run npm install https://cdn.cypress.io/beta/npm/12.17.5/linux-x64/marktnoonan/25913-vite-logging-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.tgz npm WARN deprecated stable@0.1.8: Modern JS already guarantees Array#sort() is a stable sort, so this library is deprecated. See the compatibility table on MDN: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#browser_compatibility npm WARN deprecated consolidate@0.15.1: Please upgrade to consolidate v1.0.0+ as it has been modernized with several long-awaited fixes implemented. Maintenance is supported by Forward Email at https://forwardemail.net ; follow/watch https://github.com/ladjs/consolidate for updates and release changelog npm WARN deprecated @stylelint/postcss-markdown@0.36.2: Use the original unforked package instead: postcss-markdown npm WARN deprecated babel-eslint@10.1.0: babel-eslint is now @babel/eslint-parser. This package will no longer receive updates. npm WARN deprecated @stylelint/postcss-css-in-js@0.37.2: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info. npm WARN deprecated sortablejs@1.12.0: This version has deprecated in favor of a rollback to version 1.10.2 npm WARN deprecated core-js@3.21.1: core-js@❤️.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js. npm ERR! code 1 npm ERR! path /home/tempuser/work/temp-fork/temp-fork/node_modules/cypress npm ERR! command failed npm ERR! command sh -c node index.js --exec install npm ERR! Note: Overriding Cypress cache directory to: /home/tempuser/.cache/Cypress npm ERR! npm ERR! Previous installs of Cypress may not be found. npm ERR! npm ERR! ⚠ Warning: You are installing a pre-release build of Cypress. npm ERR! npm ERR! Bugs may be present which do not exist in production builds. npm ERR! npm ERR! This build was created from: npm ERR! * Commit SHA: 73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72 npm ERR! * Commit Branch: main npm ERR! * Commit Timestamp: 2023-08-21T22:49:16.000Z npm ERR! npm ERR! Installing Cypress (version: https://cdn.cypress.io/beta/binary/12.17.5/linux-x64/main-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.zip) npm ERR! npm ERR! [STARTED] Task without title. npm ERR! The Cypress App could not be downloaded. npm ERR! npm ERR! Does your workplace require a proxy to be used to access the Internet? If so, you must configure the HTTP_PROXY environment variable before downloading Cypress. Read more: https://on.cypress.io/proxy-configuration npm ERR! npm ERR! Otherwise, please check network connectivity and try again: npm ERR! npm ERR! ---------- npm ERR! npm ERR! URL: https://cdn.cypress.io/beta/binary/12.17.5/linux-x64/main-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.zip npm ERR! Error: Failed downloading the Cypress binary. npm ERR! Response code: 404 npm ERR! Response message: Not Found npm ERR! npm ERR! ---------- npm ERR! npm ERR! Platform: linux-x64 (Ubuntu - 22.04) npm ERR! Cypress Version: 12.17.5 npm ERR! [FAILED] The Cypress App could not be downloaded. npm ERR! [FAILED] npm ERR! [FAILED] Does your workplace require a proxy to be used to access the Internet? If so, you must configure the HTTP_PROXY environment variable before downloading Cypress. Read more: https://on.cypress.io/proxy-configuration npm ERR! [FAILED] npm ERR! [FAILED] Otherwise, please check network connectivity and try again: npm ERR! [FAILED] npm ERR! [FAILED] ---------- npm ERR! [FAILED] npm ERR! [FAILED] URL: https://cdn.cypress.io/beta/binary/12.17.5/linux-x64/main-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.zip npm ERR! [FAILED] Error: Failed downloading the Cypress binary. npm ERR! [FAILED] Response code: 404 npm ERR! [FAILED] Response message: Not Found npm ERR! [FAILED] npm ERR! [FAILED] ---------- npm ERR! [FAILED] npm ERR! [FAILED] Platform: linux-x64 (Ubuntu - 22.04) npm ERR! [FAILED] Cypress Version: 12.17.5

npm ERR! A complete log of this run can be found in: npm ERR! /home/tempuser/.npm/_logs/2023-08-25T00_10_00_723Z-debug.log Error: Process completed with exit code 1. `

Here’s a pre-release build to test in your CI processes. This build hard codes force: true for optimizeDeps. If you try this out, please comment with the result.

npm install https://cdn.cypress.io/beta/npm/12.17.5/linux-x64/marktnoonan/25913-vite-logging-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.tgz

This should actually install the correct binary for your system, but if it doesn’t, here are the platform specific URLs you can point to:

• Windows x64 https://cdn.cypress.io/beta/npm/12.17.5/win32-x64/marktnoonan/25913-vite-logging-5621393b0b31c13e0cb5423560f8abfa5b01d0db/cypress.tgz
• Dawrin arm64 https://cdn.cypress.io/beta/npm/12.17.5/darwin-arm64/marktnoonan/25913-vite-logging-5621393b0b31c13e0cb5423560f8abfa5b01d0db/cypress.tgz
• Darwin x64 https://cdn.cypress.io/beta/npm/12.17.5/darwin-x64/marktnoonan/25913-vite-logging-5621393b0b31c13e0cb5423560f8abfa5b01d0db/cypress.tgz
• Linux arm64 https://cdn.cypress.io/beta/npm/12.17.5/linux-arm64/marktnoonan/25913-vite-logging-5621393b0b31c13e0cb5423560f8abfa5b01d0db/cypress.tgz
• Linux x64 https://cdn.cypress.io/beta/npm/12.17.5/linux-x64/marktnoonan/25913-vite-logging-5621393b0b31c13e0cb5423560f8abfa5b01d0db/cypress.tgz

We have also switched to running Cypress against a pre-built version of the library but did not help solve the issue.

@floroz did you switch to pre-build version for Cypress Component Tests or E2E tests? Asking because I was wondering if this would have been easy for Component Tests (we already did that for E2E as noted above). I’m not sure what changes would be needed since as I understand component tests in Vite (if using the vite bundler) rely on the Vite dev server bundling behavior 🤔

@rszalski we use Cypress for integration testing and visual regression with Percy. The recurring failures were damaging our baseline of screenshots.

We migrated part of our tests involving visual regressions from Component Testing to E2E Testing (not Component) and created “fake pages” to render the components in isolation and perform our snapshots.

We are still keeping the “integration” part in Component testing, but since half of the workflow now has been moved away, we’ve experienced a reduction in the number of times our tests are failing.

Again, whether is a coincidence or the more components/runs the more the chance of this failure, is hard to say.

We are also facing a separate issue where the CT run intermittently freezes (no error, no timeout, just gets stuck on various tests). Not sure if this is at all related and so I’m focusing on the Failed to fetch dynamically imported module error.

Worst case scenario we’ll revert back to running Component tests w/ Webpack until this is resolved.

We’ve also experienced these random freezes, also locally. Again, almost impossible to reproduce the issue reliably to try debug the error (besides the lack of logging).

We have also switched to running Cypress against a pre-built version of the library but did not help solve the issue.

We’ve migrated to E2E tests with “fake pages” to continue simulating the same environment as CT.

Let’s try using force. This issue is happening to us in the Cypress repo, too https://app.circleci.com/pipelines/github/cypress-io/cypress/55051/workflows/57524acc-7e72-42ca-9aee-d5cd707aacc3/jobs/2276758?invite=true#step-112-11815_61

@moritzkalwa

I feel like there should be an easy way to retry tests that fail in this manner, not just better logging.

I tried re-running those tests, we had a PR that did this, it would fail for the same reason many times, it was like Vite would refuse to refresh / serve the dep.

For us this issue started appearing when upgrading from node 16 to 18. We had decided to put this upgrade on hold until a solution is found, however with node 16 going EOL next month we might have to upgrade without a fix and completely break our CI pipeline. Would be really awesome if a solution is found until then. I feel like there should be an easy way to retry tests that fail in this manner, not just better logging.

Hey @lmiller1990 - do you have updates on this topic? We are experiencing pretty much the same as the rest in this thread. We are having one single component test which sometimes (I would guess 30%) fails on our GitHub actions pipeline with the “failed to fetch” error. After one manual retry we have a successful run.

I am surprised that makes any difference. tsc is usually smart enough to coerce a type only import from import to import type.

@barbados-clemens I think you are correct about the CI load as the underlying issue, which is why no-one can reproduce this locally, and why it seems to be GH Actions specific (seems these machines suffer load issues more than other providers).

'@typescript-eslint/consistent-type-imports': 'error',

Add this to eslint rules. After about 10k lines of changes in our monorepo, some of the cryptic Vite errors have subsided.

I think we solved it on our side, we had an import that was no longer valid inside component.ts. Didn’t happen again for a few days already, both on CI and locally for a few of us.

It does not explain, though, why when we upgraded to vite4 it stopped happening for a few hours, so I hope it was the reason and that it won’t reproduce…

@lmiller1990 Progress!!! Took a quick look at this and it appears we have a network loop issue - something is generating a network request against /__cypress/src/ (default devServerPublicPathRoute) which is looping through the Node proxy agent and eventually killing the process thru a “too many file handles” error. I added the “ESCAPE HATCH” block below in agent.ts and was able to run the full test suite provided by @patrickcate without issue using my local cypress dev env, remove that block and it blows up.

// packages/network/lib/agent.ts

  if (!options.href) {
    debug('Defaulting href from path "%s"', options.path)
    // options.path can contain query parameters, which url.format will not-so-kindly urlencode for us...
    // so just append it to the resultant URL string
    options.href = url.format({
      protocol: isHttps ? 'https:' : 'http:',
      slashes: true,
      hostname: options.host,
      port: options.port,
    }) + options.path
  }

  // ESCAPE HATCH
  if (options.href === 'http://localhost:3030/__cypress/src/') {
    return
  }

  if (!options.uri) {
    options.uri = url.parse(options.href)
  }

Since this seems CT-specific I’m assuming we have some sort of bad interaction between our configs in routes-ct.ts, server-ct.ts, and perhaps the base Vite dev server configs - will dig in a bit more on Monday

Argh, the dream was short lived. It’s hard to say if this made a difference or if I just got lucky with 4 consecutive passes. 😢

Hmm you are running 4 containers - I’ll try running more than 1, maybe that’s related.

Sample size of 1 but seems to work so far: https://github.com/vuetifyjs/vuetify/actions/runs/4574802124 Looks like evan is interested in having it be part of vite itself: https://twitter.com/youyuxi/status/1641402565631025152

Aaand still busted: https://github.com/vuetifyjs/vuetify/actions/runs/4576029601/jobs/8079626308 unless process.env.CYPRESS isn’t set somehow but it seems to be locally

Just collecting research for myself to debug this, if it is a slow start caching issue, maybe we can adapt https://github.com/bluwy/vite-plugin-warmup

Edit @KaelWD interesting, I got a failure on first try for Vuetify. I added the plugin above, and it passed four times consecutively. https://github.com/lmiller1990/vuetify/actions. WDYT? Can you try adding that plugin to your CI like I did and see if it helps?

Edit: I removed the plugin and it failed. Either this was dumb luck, or we isolated the issue to a race condition, solved by having a warm cache.

@stnokott can you try the plugin too? And @chojnicki? Like this: https://github.com/lmiller1990/vuetify/commit/ff65846fcea3461092e908aa60cc94d07aa83102#diff-ee0cbc5388a0d00ac00c818b7cf76db70ff9c4e7945808ea2340d4eb4423d4cfR58-R63 include a pattern to match your spec files, and your cypress/component file (the one with mount).

@stnokott thanks for this - the TS issue is, in fact, a bug in TypeScript. Follow this issue: https://github.com/cypress-io/cypress/issues/26148#issuecomment-1487574243. It looks like there is a open PR in TS fixing the issue. 🤞 Looking at your CI, it seems the same test, Settings.cy.tsx keeps failing, and the Vite optimize step always precedds it? https://github.com/stnokott/r6-dissect-influx/actions/runs/4467180776/jobs/7846310109#step:5:271. I wonder if you add those deps to optimizeDeps in your vite.config, will it prebundle them and avoid the flake 🤔 ?

@KaelWD thanks, I will spend some more time looking into this today.

Lol, I did not had to wait too long. Failed on first try after update cypress to 12.9.0.

✖ 1 of 191 failed (1%) 00:37 654 653 1 - -

Running:  landing/LandingArticleCard.spec.js                                          (109 of 191)


  (Attempt 1 of 2) An uncaught error was detected outside of a test
  1) An uncaught error was detected outside of a test

  0 passing (419ms)
  1 failing

  1) An uncaught error was detected outside of a test:
     TypeError: The following error originated from your test code, not from Cypress.

  > Failed to fetch dynamically imported module: http://localhost:3000/__cypress/src/cypress/support/index.js

When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.

Cypress could not associate this error to any specific test.

We dynamically generated a new test to display this failure.

Again after around 50% finished tests, again random one, component is just fine (it’s very basic) and it will work after rerun, it happens only on GH actions (works locally). So nothing changed 😦

Like I said before, I tried with “retries” functionality in Cypress, but it ignores it. Like you see, there is (Attempt 1 of 2) but it never tries second time. I guess it only retries when there are assertion errors, but not general errors, which is shame 😦 I would really like to just rerun that single test, instead all queue and all CI/CD process. Any way to do that?

Edit: Also it’s failing on importing cypress/support/index.js which for me does not make any sense.

So I can confirm it was not fixed in my case because based on #26138 it was due to vite 4.2.0, but I’m still on 4.1.1 😕 I will update Cypress anyway with that fix, and wait until it will happen again.

Okay we have a potential solution for this that should at least help prevent failures due to this issue. If anyone is willing, you can try installing a pre-release binary from my commit here.

If anyone does this, please post here and let us know if you are still seeing import failures in CI

@chojnicki thanks a lot, I’ll ping you if we need access, I think our reproduction should be enough. Is your reproduction consistent? CI only?

@lmiller1990 like i described in previous comment - only CI/CD with GH actions. Locally (with exact same docker image, also on linux as host) it does not happening, not even once. But on GH it’s very random.

We have an internal repro 🔥 FINALLY!

It’s Cypress org private project, but dropping the link here so someone at Cypress can internally… https://cloud.cypress.io/projects/d9sdrd/runs/4193/test-results/2f924501-7fb2-4c80-b69f-819010c67c87

Now we’ve got a repro I can dig into it… for us it’s CI only too, so sounds like a resources/race condition.