cypress: Component tests failing intermittently with uncaught Vite "failed to fetch dynamically imported module" error in CI

Current behavior

We started migrating our project from CRA to Vite and we have almost 300 component and 40 E2E Cypress tests in place. Unfortunately, after fixing all the other issues, we are still not able to stabilize our tests since there is always 1 or 2 failing randomly on “Failed to fetch dynamically imported module” errors.

We noticed that it’s somehow related to the load of CI. Under some conditions more tests are failing like this, in other times it succeeds. But it’s random. We checked our tests and we are pretty sure it’s not caused by any logic we have.

We’ve checked some of the existing issues on Cypress & Vite, tried various workaround but no luck with any of them.

What we think is happening is that Cypress is not waiting for Vite to “boot up” properly and retries don’t help with it, only when new spec is run, it works.

Note: it only happens with component tests. For E2E tests we had similar stability issues but we solved them by building production version and then just serving it with vite preview. This made integration tests faster and very stable. Previously they were also timeouting.

Note 2: we have a lot of components and lot of dependencies in our project, we also use MUI as our base. But with CRA we were able to have stable tests, it was just around two times slower. That’s why we want to use Vite now.

Note 3: we are running “Component tests” in parallel, currently in 4 groups.

Desired behavior

No random problems with Cypress + Vite combo.

Test code to reproduce

Unfortunately can’t provide this since our project is private. And I’m afraid that it’s related to project’s complexity and that’s why we can’t easily create another one for reproduction.

Cypress Version

v12.6.0

Node version

v16.18.1

Operating System

Docker Hub image: cypress/included for v12.6.0 version

Debug Logs

No response

Other

No response

About this issue

  • Original URL
  • State: open
  • Created 4 years ago
  • Reactions: 76
  • Comments: 181 (52 by maintainers)

Commits related to this issue

Most upvoted comments

I am facing same issue

We are having the exact same issue with component tests ( I was just coming here to open a new issue ) . We just migrated from webpack to vite. The error we get is:

The following error originated from your test code, not from Cypress. > Failed to fetch dynamically imported module: http://localhost:3000/__cypress/src/cypress/support/component.ts When Cypress detects uncaught errors originating from your test code it will automatically fail the current test. Cypress could not associate this error to any specific test. We dynamically generated a new test to display this failure.

We are getting this despite the fact that the file does exist. And this occurs randomly.

Vite: 4.1.1 Cypress: 12.5.1 Node: 19.3.0

One related issue: our cypress directory is on the same level as our src directory. When running locally, Cypress correctly expects component.ts to be where we have it in the cypress directory. But when we run the tests in docker, Cypress expects component.ts to be in a cypress directory under the src directory (see above error). Even if we use the ‘supportFolder’ config setting, Cypress still looks for it in the src directory (src is the vite root folder). So I just copied component.ts to the location cypress expects it to be (but still fails randomly despite this).

So locally this is not reproducable for us (so far). This only occurs in a docker container.

We are also running the tests in parallel.

Hi all - we are looking to work on this in the next two weeks with the goal of completely solving it, eliminating the need for work arounds.

Any update on this issue? It’s unfortunately made Cypress component testing pretty much unusable for my projects since it’s extremely difficult to get full test suites to pass.

@adamscybot optimizeDeps doesn’t work for us. Still getting random errors on CI.

In my case it started happening more and more often after upgrading from 12.7 to 13.1. For some reason CI stopped logging the “[vite] ✨ new dependencies optimized:”, but I could reproduce that easily locally.

After going through Vite docs, I read this bit:

By default, Vite will crawl all your .html files to detect dependencies that need to be pre-bundled (ignoring node_modules, build.outDir, __tests__ and coverage). If build.rollupOptions.input is specified, Vite will crawl those entry points instead.

The bit about “__tests__” folder being ignored from pre-bundling was the problem in my case. Only tests that were nested inside “__tests__” folder were randomly failing. In my case, solution was to add all my test files to entries section of Vite config, like so (all my tests have this test-cy bit added in name):

optimizeDeps: {
  entries: ['src/**/*.test-cy.ts', 'src/**/*.test-cy.tsx', 'index.html']
}

Now for each test, Vite always optimizes all needed dependencies before running the test, and since that the issue has not been happening anymore.

Chiming in here in the hopes of saving others some time.

We had had intermittent issues with component tests failing in CI. The errors always came near a [vite] ✨ new dependencies optimized line in the job log.

I was able to reproduce the issue locally by clearing node_modules/.vite/deps before running the tests, and adding:

optimizeDeps: {
  entries: ['cypress/**/*'],
},

to our vite.config.js solved those issues. Our test files (.cy.js) are all located within that directory.

That was fine for a while, but I just now had to address another similar issue, this time with a new dependency (vue-router) that was not previously imported directly by our tested components. I eventually figured out that the issue was due to the fact that we cache the node_modules directory in between jobs (we use GitLab CI/CD). I don’t have a full understanding of the Vite internals, but I think it saw that (now stale) .vite/deps data and assumed it didn’t need to optimize (pre-optimize?) the Cypress test files again. I added a simple rm -rf node_modules/.vite/deps to the Cypress test job, and that fixed the issue.

For anyone interested, here’s our “workaround” solution. We have a script called cypress-ct-runner.mjs (under bin/vite-migration:

#!/usr/bin/env node
import { Chalk } from 'chalk';
import cypress from 'cypress';
import fs from 'fs';
import path, { dirname } from 'path';
import { fileURLToPath } from 'url';

const chalk = new Chalk({ level: 1 });
const groupId = process.argv[2];

if (groupId === undefined) {
  console.error('Please provide a group id parameter');
  process.exit(1);
}

const __dirname = dirname(fileURLToPath(import.meta.url));
const groups = JSON.parse(fs.readFileSync(path.join(__dirname, '..', '..', 'component-test-groups.json')));
const group = groups[groupId - 1];
const maxRetries = 5;
let specs = group.join(',');
let shouldFail = false;
let shouldRetry = true;
let retries = 0;

while (shouldRetry) {
  if (retries > 0) {
    console.log(chalk.bgRed(`Retry number: ${retries}`));
    console.log(chalk.bgRed('Retrying failed tests:'));
    console.log(
      chalk.red(
        specs
          .split(',')
          .map((spec) => `- ${spec}`)
          .join('\n')
      )
    );
    console.log();
  }

  const result = await cypress.run({
    spec: specs,
    browser: 'chrome',
    headless: true,
    testingType: 'component',
  });

  const failedTests = result.runs.reduce((failed, { tests, spec }) => {
    return [...failed, ...tests.filter(({ state }) => state === 'failed').map((test) => ({ ...test, spec }))];
  }, []);

  specs = failedTests
    .map(({ spec }) => spec.relative)
    .reduce((result, spec) => {
      if (!result.includes(spec)) result.push(spec);
      return result;
    }, [])
    .join(',');

  if (failedTests.length === 0) {
    shouldRetry = false;
  } else if (retries >= maxRetries) {
    shouldRetry = false;
    shouldFail = true;
  } else {
    retries++;
  }
}

if (shouldFail) {
  console.log(chalk.bgRed("Couldn't recover these failing tests:"));
  console.log(
    chalk.red(
      specs
        .split(',')
        .map((spec) => `- ${spec}`)
        .join('\n')
    )
  );
  process.exit(1);
} else {
  console.log(chalk[retries === 0 ? 'bgBlue' : 'bgRed'](`Finished running tests with ${retries} retries`));
}

Under the same path we also have another script which collects paths of all our component test specs and “groups” them randomly, it looks like this:

#!/usr/bin/env node
const fs = require('fs');
const path = require('path');
const glob = require('glob');

const groupCount = process.argv[2];

if (groupCount === undefined) {
  console.error('Please provide a group count parameter');
  process.exit(1);
}

glob('src/**/*.cytest.tsx', (err, files) => {
  if (err) {
    console.error('An error occurred:', err);
    process.exit(1);
  }

  console.log('Found', files.length, 'files');

  // sort files randomly
  files.sort(() => Math.random() - 0.5);

  // take 1/3 of the files and put them in the first group to warm up Vite's cache
  const firstGroupAmount = Math.ceil(files.length / groupCount / 3);
  const groups = [files.splice(0, firstGroupAmount)];

  // ignore the first group in this loop
  for (let i = 0; i < groupCount - 1; i++) {
    groups.push(files.slice((i * files.length) / (groupCount - 1), ((i + 1) * files.length) / (groupCount - 1)));
  }

  console.log(
    'Created',
    groups.length,
    'groups with following distribution:',
    groups.map((group) => group.length)
  );

  fs.writeFileSync(path.join(__dirname, '..', '..', 'component-test-groups.json'), JSON.stringify(groups, null, 2));
});

On CI we have a following setup of steps:

  • Distribute component tests randomly
    • ./bin/vite-migration/distribute-component-tests.js 5
  • Component tests [1]
    • ./bin/vite-migration/cypress-ct-runner.mjs 1
  • Component tests [2-5]
    • ./bin/vite-migration/cypress-ct-runner.mjs 2-5

Our whole CI flow looks like this: image

I hope that someone will find this useful until the root cause gets fixed.

EDIT: In the end this doesn’t work, it just made the problem rarer.

I’m still having this problem. I’m not really satisfied with the last solution (retrying the run), so I explored the “warmup files” approach suggested earlier by @adamscybot.

The solution provided was

viteConfig: {
  server: {
    warmup: {
      clientFiles: ['**/*'],
    },
  },
},

However, as @apdrsn mentioned, this leads to numerous errors on my end: [vite] Pre-transform error: Failed to resolve import ...

The Vite documentation also advises against warming up too many files, so I was reluctant to use a wildcard to preload everything.

Instead, I refined this solution by warming up only the specific file that was failing to load. This approach seems cleaner and has been working well so far:

viteConfig: {
  server: {
    warmup: {
      clientFiles: ['**/cypress/support/component.ts'],
    },
  },
},

Note that this still requires Vite 5 as the warmup config isn’t available in earlier versions.

We will look at trying vitejs/vite#12080 in our next sprint of work to see if we can get more fine grained error logging. This is very hard problem to solve.

For me setting cypress files to optimizedDeps worked! At first, I updated vite to 5.4.11 and added warmup config but it didn’t work by its own. I’m using cypress 13.15.2.

optimizeDeps: {
      entries: ['cypress/**/*']
    },
   server: {
        warmup: {
            clientFiles: ['**/cypress/support/component.ts']
        }
    },

UPDATE: Not working consistently in CI, it’s still failing 😕

I know this is brute force, but try:

    server: {
              warmup: {
                   clientFiles: ['**/*'],
              }
          }

And yes there will be warnings in console. Ignore them for the sake of testing this. Currently you are assuming the problem is coming from one of your support files, but actually, it can be anything. Including the rest of the support files, third party modules, the tests themselves, and anything they import.

This is just to try to prove a point about if it works or not. There’s a lack of data points at the moment as some people accurately report the problem can return or still exist at a lower level with warmup configured, but up to now we only have that data for folks who have a selective warmup configuration which could be why (or not…).

I am still using **/* and haven’t had it resurface. But that of course, is not proof of anything either. Just need more folks trying and reporting back.

@YossiSaadi we’ve tried upgrade to cypress 12.9 & Vite 4 and the errors still occur.

@lmiller1990 have you found any interesting insights?

@lmiller1990 I created a repro for this issue showing it happening in a NON-CI scenario. Hopefully, it helps get to the root of the problem.

git clone https://github.com/crobinson-expl/vue-scaffold-cypress-e2e-unit-march-15.git
cd vue-scaffold-cypress-e2e-unit-march-15
npm i
npm run test:unit:dev
// Choose Electron
// Click on HelloWorld.cy.ts

image

This finally flared again enough for me to attend to it. The recurrence was somehow related to a Vite 5 upgrade. Vite 5 now has a new option server.warmup which can get the cache ready in advance.

We removed configuration around optimizeDeps and tried a (brutal) wildcard glob in warmup:

 // ...etc
 component: {
    // ...etc
    devServerPublicPathRoute: '',
    devServer: {
      bundler: 'vite',
      framework: 'react',
      viteConfig: {
           // ...etc
          server: {
              warmup: {
                   clientFiles: ['**/*'],
              }
          }
      }
    },

The problem is now gone again. If you are on Vite 5, I recommend trying this. If you are not, I recommend upgrading Vite (you also need at least Cypress 13.10 to be compatible) and then trying it.

Probably this glob pattern can be reduced to just my test file, but just getting this information out there. We also haven’t done enough testing to know if it was actually the removal of optimizeDeps which is the key ingredient.

Either way, we defintely need to be clear what Vite versions we have when reporting back.

Update… this is not forgotten, but we are trying to figure out how to fix this. It’s clear from the investigation so far there’s likely a few bugs that all manifest in the same fashion. In addition, the actual problem looks like it spans both Vite and whatever it’s integrated with (Cypress, PW, or whatever). So far, several developers (here and external) spend some time (days) trying to fix this, but no luck.

I think we need to budget significant (weeks) of time for someone to go a lot deeper into this, and possibly in conjunction with someone from the Vite team. Based on https://github.com/vitejs/vite/issues/11804, it is impacting many users outside of Cypress, too.

We have an internal repro 🔥 FINALLY!

It’s Cypress org private project, but dropping the link here so someone at Cypress can internally… https://cloud.cypress.io/projects/d9sdrd/runs/4193/test-results/2f924501-7fb2-4c80-b69f-819010c67c87

Now we’ve got a repro I can dig into it… for us it’s CI only too, so sounds like a resources/race condition.

Hey @ivanalemunioz, definitely love community engagement and integrations with Cypress. Please remember to keep threads on topic though for the benefit of everyone following this specific issue. 🙏🏻 If you’d like to share your work you can share in our community chat or social media.

Hi, everyone. I’ve been working on a solution to visually debug my browser automation crashes, and I’ve now made it free for the public to use: https://buglesstack.com

I don’t mean to spam or bother you; I just want to share this solution. If you have any questions or suggestions for improvement, feel free to contact me.

Thanks for reading! ❤️

We’re getting this issue after upgrading to Cypress 13.17.0 from 13.14.2, if I revert to 13.14.2 the problem goes away.

One thing to note, we see the issue reliably if we have code coverage turned on (we’re using @bahmutov/cypress-code-coverage@2.7.2), we don’t run coverage on our branches so our PR test runs never suffer from this issue. Its only when we merge to main (after getting a green test run) that the tests then start to fail as this is when we generate coverage reports.

Edit: at the same time as upgrading to 13.7.0 we also upgraded from Vite 5 to Vite 6 and I see someone above mentions that they started seeing this after going to Vite 6. As I say above though, reverting to 13.14.2 fixes the issue for us so it feels more like a Cypress issue. I also tried 13.15.0 but had the same problems with that so this suggests it was something introduced at 13.15.0

Edit 2: Switching to the standard cypress code coverage plugin has resolved this issue for us

Something interesting is happening here. After I claimed victory on this issue, we did indeed continue error-free for many months (thousands of runs), which I think is solid evidence of optimizeDeps having something to do with it, if only in part.

However. Yesterday, due to some other requirement, I upgraded Cypress from v12 to v13, and at the same time, I upgraded Vite from v3 to v5. It is probably tertiary, but it is also worth noting this also involved an upgrade of the Cypress docker image, which would have also resulted in a Chrome upgrade,

It was only once, and I hardcore logged off shortly after in total denial as Friday evening drew to a close – but after doing this, the issue re-emerged for the first time in months on a single CI run. Deep down, I know the curse has returned.

Perhaps an ever-present race condition has suddenly reactivated due to the subtle execution speed differences all these upgrades made, or perhaps there was a relevant behavioural change that affected this issue amongst those changes.

I think it would be useful for everyone to post their exact Vite and Cypress versions and their optimizeDeps settings when posting their analysis. if you’ve already posted, it would be super helpful to just say “Im using Cypress x with Vite x and this optimizeDeps deps setting and it’s happening for me”. Perhaps also note what docker image it runs in CI (or if you don’t do that and it’s on bare metal).

Monday will probably be interesting, but England just got to the semi-finals so I’m pretending not to be affected by the blaze of inevitable red all over CI.

This is a total stretch but I’m also someone who is loading service workers (MSW) as part of the test story. I doubt its related, but would be interested to know in the rare event there’s some consistency between that cause and this affect.

I experienced this too - for me it was because I was switching between branches with different dependencies, and one of them somehow hadn’t been reinstalled inbetween changing branches (in my case @pinia/testing). So while Cypress was reporting that it couldn’t import components.js, it wasn’t because that file was missing, it was because that file was erroring because of the missing import of @pinia/testing within it. Hope this helps people who end up here in the future (or me, when this happens again and see my own comment).

Still a problem. In my case, all test run fine locally but crashing randomly in pipeline, with the error mentioned. You can run the same job twice and where in the first run the Test A crashes and Test B works, in the second run both can crash or B crash and A works…

Hi. I’m adam-thomas-privitar on my personal account 😃. So I wouldn’t say that my confidence is 100% due to the odd nature of this issue. But I can say so far, the problem has not reoccurred in my branch since adding the test files to the optimizeDeps.entries after kicking it off a fair few times. I was also able to re-enable a certain suite which in particular was causing us especially big problems (but not exclusively) in regards to this issue, which is really promising. Previously we were retrying that test at the CLI level 3 times and even that eventually was not sufficient. So the fact it’s working without those retries strongly suggests there may be a path here.

I’m trying not to get my hopes up too much though since the real test will come when this gets merged to main next week and the build volume increases. So ultimately we still need to proceed with caution.

I will report back, but I feel there’s sufficient data to say that anyone experiencing this should try it so we can add more data points. If it does fix it, @RadomirNowak is a legend 😄, as this problem has truly haunted us at scale. If it suddenly fails, I might go and have a little cry somewhere and hang up my keyboard forever.

It’s perhaps of note that we have our own substantial layer above cypress (test support stuff) that is imported only into tests, in its own package, to abstract common things, and so the proposition that this plays a part is very plausible.

I believe nearly all of the discourse about solutions that seem to work then don’t is likely due to varying load on the agent that’s executing the tests. It’s a race condition in its current state, and so any fluctuation in speed affects it, and I’ve observed this myself. Also, the problem gets worse the more deps you add. If you are running tests in parallel that definitely adds to the randomness also, since that’s also causing varying load between runs. Another reason is if you have many agents and some are more highly specced than others.

If it does emerge that this is the problem then I guess a PR to the Cypress vite integration package is needed to automatically merge the test files into optimizeDeps.entries .

the thing with __tests__ was something that allowed me to find the root issue, which is: if you import packages inside your tests that are not imported in your app (so they are not prebundled by vite) then they will be dynamically optimized during test run which will result in the problem described in this thread. @floroz @adam-thomas-privitar can you try to include your test files in the entries section like I mentioned before? Also another thing could be to include there a glob for all your files (as a test) e.g. **/*.ts

We are using the __tests__ convention as well so the theory is certainly interesting.

I still need to confirm this long term, but I think setting a constant port value for dev server with Cypress tests seems to fix the issue. Please see whether it will work for you.

@adam-thomas-privitar @nive-copia @rszalski I updated the links to the pre-release binary in Mark’s comment. You should be able to install this one. Please comment if you are still having issues installing and thank you for your patience

@nive-copia this should be working, I’ll look into it. I was able to download the binary but not perform npm install.

Even if this solves this issue, I dont think it is the cause auf the problem. It is quite strange that the “manual” optimizeDeps has no effect.

Agreed. Still it would be useful to unblock people. I’m going to create a pre-release build of Cypress that people in this thread can drop into their CI pipelines to see if it has an impact on this error. I’ll update this thread with instructions when it is ready.

I am not sure how vite does detect “Relevant fields in your vite.config.js”. (https://github.com/cypress-io/cypress/issues/25913#issuecomment-1663993439). Maybe find-up should be replaced by loadConfigFromFile (the official vite API).

@OrbisK I played around with find-up vs loadConfigFromFile but I didn’t immediately see the opportunity to replace one with the other as you suggested. Though if I follow what you are suggesting, maybe we could use loadConfigFromFile for the file from the path determined with find-up, before we use vite.mergeConfig here. Is your thinking that, under the hood, vite may then recognize the need to rerun without force? Based on “Relevant fields in your vite.config.js” being mentioned here.

optimizeDeps simply does not work as advertised, at least for us - we regularly see “vite is optimizing…” for entries we’ve already got there, which is in all likelyhood the source of a good chunk of these issues.

I hope adding some more error hooks will help diagnose this.

Is it possible to start the component dev server with the --force cli flag? Maybe vite cache is the problem (related to the “vite is optimizing” problem with included deps)?

Docs:

The pre-bundling step will only need to be re-run when one of the following has changed:

  • Package manager lockfile content, e.g. package-lock.json, yarn.lock, pnpm-lock.yaml or bun.lockb.
  • Patches folder modification time.
  • Relevant fields in your vite.config.js, if present.
  • NODE_ENV value.

EDIT: optimizeDeps.force will ignore previously cached optimized dependencies.

FYI: I am not facing this issue (anymore (?) - and i dont know why). I can’t test it myself.

@floroz no sorry, we got caught up in other issues. We will try and get to this this week or next. To clarify, we are going to (or a third party, if anyone is up for it):

In https://github.com/vitejs/vite/pull/12084/files, a new event was added: vite:preloadError. We should listen to this on the server and log any events using DEBUG, like this https://github.com/cypress-io/cypress/blob/14a7416e126f6b296a2da037ae257c27e7915d5b/npm/vite-dev-server/src/plugins/cypress.ts#L10

Then when users run Cyperss with DEBUG=cypress:vite* they will (hopefully) see a more useful error on the server. If we can manifest this in the Cypress UI, that would be even better. The hope is we can find a more specific error that we can use to actually fix this bug.

FYI, this happens in other repos not using Cypress, https://github.com/vitejs/vite/issues/11804, so it’s possible the fix is actually a PR in Vite, eventually. I think the issue is we use Vite in a fashion that makes this error more regularly occurring here than just using Vite as a dev server independently.

@mx2s thanks! Can you share your./vite.config?

May be that helps anyone, Here’s my config for component testing:

import { defineConfig } from 'cypress';
import createBundler from '@bahmutov/cypress-esbuild-preprocessor';
import { devServer } from '@cypress/vite-dev-server';
import viteConfig from './vite.config';

module.exports = defineConfig({
  component: {
    setupNodeEvents(on) {
      on('file:preprocessor', createBundler());
    },
    devServer(devServerConfig) {
      return devServer({
        ...devServerConfig,
        framework: 'react',
        viteConfig: async () => {
          return { ...viteConfig('production') };
        },
      });
    },
  },
});

Tests started to pass for me once I changed viteConfig('production') from development to production (basically NODE_ENV)

If you need another possible test case you can try the https://github.com/patrickcate/vue-uswds/tree/feature/606/cypress-12-upgrade branch of my Vue 3 project. Unlike other’s I’m seeing this error when running the tests locally, both in headed/open and headless/run mode.

The components it fails on appears to be random, but in headed/open mode, refreshing the browser often allows the tests to run fine. In headless/run mode this isn’t possible so I’ve never been able to get through a successful run with the 100 or so total number of components.

I am not using the vite-plugin-warmup plugin or running tests in parallel.

Other details:

  • Cypress 12.9.0
  • Vite 4.2.1 (also tried Vite 4.3 beta 2)

Local computer:

  • MacBook Pro
  • MacOS 11.7.4
  • 2.6 GHz Quad-Core Intel Core i7
  • 16 GB 1600 MHz DDR3

Just collecting research for myself to debug this, if it is a slow start caching issue, maybe we can adapt https://github.com/bluwy/vite-plugin-warmup

Edit @KaelWD interesting, I got a failure on first try for Vuetify. I added the plugin above, and it passed four times consecutively. https://github.com/lmiller1990/vuetify/actions. WDYT? Can you try adding that plugin to your CI like I did and see if it helps?

Edit: I removed the plugin and it failed. Either this was dumb luck, or we isolated the issue to a race condition, solved by having a warm cache.

@stnokott can you try the plugin too? And @chojnicki? Like this: https://github.com/lmiller1990/vuetify/commit/ff65846fcea3461092e908aa60cc94d07aa83102#diff-ee0cbc5388a0d00ac00c818b7cf76db70ff9c4e7945808ea2340d4eb4423d4cfR58-R63 include a pattern to match your spec files, and your cypress/component file (the one with mount).

image

If anyone does this, please post here and let us know if you are still seeing import failures in CI

Before this change, 90% of my Cypress runs in the CI of my little hobby project were failing due to dynamic import issues. Since using your commit, none of them failed due to that reason. (It’s a tiny project, so probably not very representative, but wanted to let you know regardless)

@waldemarennsaed no I don’t think so, since there are different problems described in this issue that don’t relate to #26139. I would like to update the title of the issue to make it more specific though, since like Lachlan pointed out, the “failed to fetch” error can basically mean that any error occurred. I’ll be looking more into this issue today

Nice try @crobinson-expl but unfortunately your error is related to https://github.com/cypress-io/cypress/issues/26138, we don’t support Vite 4.2 yet. That will be fixed in https://github.com/cypress-io/cypress/pull/26139.

The “failed to fetch” is so general, it can basically mean any kind of error occurred. I think this issue you are encountering is unrelated to the one reported here.

Hello! Is this actively being worked on, or has anyone found a good workaround for it ?

For me setting cypress files to optimizedDeps worked! At first, I updated vite to 5.4.11 and added warmup config but it didn’t work by its own. I’m using cypress 13.15.2.

optimizeDeps: {
      entries: ['cypress/**/*']
    },
   server: {
        warmup: {
            clientFiles: ['**/cypress/support/component.ts']
        }
    },

UPDATE: Not working consistently in CI, it’s still failing 😕

The only thing for us that completely killed this issue was when we switched to a non-shared CI runner. Our Infra team kindly gave us a dedicated machine so we don’t share CPU/Memory with other CI jobs and that seems to have completely stopped the issue for us.

Don’t know how much this would help as to me it points to an issue in Vite rather than cypress?

Nice! Not many people have reported back on the warmup trick, but there’s also an absence of negative reports. Its still working for us. I suspect though we are still tweaking around a still-existing race condition.

Glad to see it working for someone.

I report back positively back then as well. Since we use the warmup method, it seems to work for us.

Got the same problem on CI/CD pipeline:

Vite: 5.2.0 Cypress: 13.13.0

Had no entries in optimizeDeps. Will evaluate if server.warmup.clientFiles works

After some time i can confirm that it works for me, but im afraid that this could change in the future with bigger components, etc.

We’re having this problem still, after adding the experimentalJustInTimeCompile flag and optimizeDeps stuff. It’s random and intermittent, which somehow makes it even worse. The warmup stuff requires us to upgrade to Vite 5, which I don’t want to do just yet, so I’m stumped as what to try next.

npm ls vite └── vite@4.5.3

npm ls cypress └── cypress@13.14.0

vite.config.js:
...
optimizeDeps: {
      entries: ['cypress/**/*'],
    },
cypress.config.ts:
...
experimentalJustInTimeCompile: true,

Error:

 Running:  MyTest.cy.tsx                                          (1 of 3)
  Estimated: 0 seconds
Browserslist: caniuse-lite is outdated. Please run:
  npx update-browserslist-db@latest
  Why you should do it regularly: https://github.com/browserslist/update-db#readme
[TypeScript] Found 0 errors. Watching for file changes.
[TypeScript] Found 0 errors. Watching for file changes.
5:16:42 AM [vite] ✨ new dependencies optimized: cypress/react18, @emotion/cache, @mui/material/styles, @mui/material, notistack, @emotion/react, tss-react, react-router-dom, tss-react/mui, react-i18next, react-draggable, underscore, html2canvas, i18next, dayjs, highcharts/highstock, highcharts/modules/draggable-points, webgl-plot, graphql-tag, @segment/analytics-next, @mui/icons-material/Close, @mui/icons-material/CheckCircleOutlined, @mui/icons-material/CancelOutlined, @mui/icons-material/InfoOutlined, @mui/material/Card, @mui/material/Collapse, @mui/material/CardActions, @mui/icons-material/ExpandMore, @mui/icons-material/AccountCircle, @mui/material/IconButton, @mui/icons-material/CloseOutlined, @mui/icons-material/CheckCircleOutline, @mui/icons-material/ArrowBack, dayjs/plugin/utc, dayjs/plugin/timezone, dayjs/plugin/advancedFormat, dayjs/plugin/duration, react-dom, @apollo/client, @apollo/client/utilities, @apollo/client/link/context, @apollo/client/link/retry, @apollo/client/link/subscriptions, graphql-ws, @growthbook/growthbook-react, i18next-http-backend, i18next-browser-languagedetector, oidc-client-ts, dexie, highcharts-react-official, highcharts/modules/xrange, highcharts/modules/pattern-fill, @mui/icons-material/Height, @mui/material/Tooltip
5:16:42 AM [vite] ✨ optimized dependencies changed. reloading
  1) An uncaught error was detected outside of a test
  0 passing (879ms)
  1 failing
  1) An uncaught error was detected outside of a test:
     TypeError: The following error originated from your test code, not from Cypress.
  > Failed to fetch dynamically imported module: http://localhost:3000/__cypress/src/cypress/support/component.ts
When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.
Cypress could not associate this error to any specific test.
We dynamically generated a new test to display this failure.

The other 2 runs did not fail, only the first one.

We recently released the experimentalJustInTimeCompile flag for component testing in 13.14.0. We have not completely evaluated whether that flag fixes this issue, but it does address performance issues in some cases when running tests in CT, so I wanted to mention it as something those in this thread might want to try out.

Ah yes! I had seen this just today also. The warmup trick is still working for me, but I will give this a go as well.

Just a check in that over 2 weeks now the problem remains gone!

So its been a couple of days now and the problem has not come back in CI. As others have said, I think this is about capturing everything, including any test-support stuff as well as the test files themselves. Makes sense support file should be added here as well because that is imported separately by cypress and not from the tests.

@dyc3 I had the same issue - turned out I was not catching all files with all import - in my case it was also cypress/support.ts file which was not caught by the src/**/* glob pattern in entries section.

it could be also a different problem on your side, as I’m using React so maybe there’s something different in Vue 🤔 Anyways I’d recommend investigating the output of CLI when you run the tests for any [vite] ✨ new dependencies optimized: and also check contents of node_modules/.vite/deps and how they change between different test suites - ideally to mitigate the error the contents of this folder should always be the same - if it changes then there’s some dependencies not caught by the optimizeDeps.entries option

the thing with __tests__ was something that allowed me to find the root issue, which is: if you import packages inside your tests that are not imported in your app (so they are not prebundled by vite) then they will be dynamically optimized during test run which will result in the problem described in this thread. @floroz @adam-thomas-privitar can you try to include your test files in the entries section like I mentioned before? Also another thing could be to include there a glob for all your files (as a test) e.g. **/*.ts

Just to let you know I am trying your workaround in CI currently, and I should be able to get back to you once its run a sufficient amount of times.

We’ve been stuck on this for about 3 weeks, would really love to hear from anyone with a workaround.

@astone123 @marktnoonan I was able to locally install the drawin arm64 beta version from Mark’s comment and compare with a stable version that we currently run.

Our component tests have 44 specs and I tested both the beta version and a stable v12.17.3 of cypress by running the entire suite 10 times for each version.

In each case the only failures were the loading of dynamically imported module: Screenshot 2023-08-30 at 14 40 45

As I mentioned above we are not only seeing errors like the above, that fail the spec, but also see the entire test run freeze randomly.

The very interesting thing is that when doing the tests on the beta version the spec never froze. We also saw much more of these dynamically loaded module errors (higher repro rate). The beta version definitely impacts this, but there might be now a completely different failure mode showing up, but manifested in the same cryptic message. Is there a way to get better logs from the Vite’s error hooks? Or maybe from Cypress’ debug logs?

Testing

  • v12.17.3

    • runs froze (did not finish) 8 out of 10 times,
    • runs failed with the error above 6 out of 10 times (but the remaining 4 were inconclusive since the full run froze and I suspect it would have failed if ran till the end).
  • beta

    • runs never froze (!)
    • runs always failed with the above error. In a single run from 50% to a 100% of the spec files failed with this error. Much more than in v12.17.3.
  • I then tried running individual spec files, that I saw fail when run as part of the entire suite. Running a single spec file never failed on beta. Tried this like 50x times.

  • I then tried running 2 spec files (both of which had failed in the full run) in this case, always the second spec failed, never the first one. Tried this also dozens of times.

  • I tried running the above with DEBUG=cypress:* but that was an overwhelming amount of logs and nothing stood out to me. I’m happy to do some more debugging but I’d need pointers on how to narrow the logs down to what is relevant.

Is the experimental build available to public to test? I see the following error when I try to install.

` Run npm install https://cdn.cypress.io/beta/npm/12.17.5/linux-x64/marktnoonan/25913-vite-logging-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.tgz npm WARN deprecated stable@0.1.8: Modern JS already guarantees Array#sort() is a stable sort, so this library is deprecated. See the compatibility table on MDN: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#browser_compatibility npm WARN deprecated consolidate@0.15.1: Please upgrade to consolidate v1.0.0+ as it has been modernized with several long-awaited fixes implemented. Maintenance is supported by Forward Email at https://forwardemail.net ; follow/watch https://github.com/ladjs/consolidate for updates and release changelog npm WARN deprecated @stylelint/postcss-markdown@0.36.2: Use the original unforked package instead: postcss-markdown npm WARN deprecated babel-eslint@10.1.0: babel-eslint is now @babel/eslint-parser. This package will no longer receive updates. npm WARN deprecated @stylelint/postcss-css-in-js@0.37.2: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info. npm WARN deprecated sortablejs@1.12.0: This version has deprecated in favor of a rollback to version 1.10.2 npm WARN deprecated core-js@3.21.1: core-js@❤️.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js. npm ERR! code 1 npm ERR! path /home/tempuser/work/temp-fork/temp-fork/node_modules/cypress npm ERR! command failed npm ERR! command sh -c node index.js --exec install npm ERR! Note: Overriding Cypress cache directory to: /home/tempuser/.cache/Cypress npm ERR! npm ERR! Previous installs of Cypress may not be found. npm ERR! npm ERR! ⚠ Warning: You are installing a pre-release build of Cypress. npm ERR! npm ERR! Bugs may be present which do not exist in production builds. npm ERR! npm ERR! This build was created from: npm ERR! * Commit SHA: 73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72 npm ERR! * Commit Branch: main npm ERR! * Commit Timestamp: 2023-08-21T22:49:16.000Z npm ERR! npm ERR! Installing Cypress (version: https://cdn.cypress.io/beta/binary/12.17.5/linux-x64/main-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.zip) npm ERR! npm ERR! [STARTED] Task without title. npm ERR! The Cypress App could not be downloaded. npm ERR! npm ERR! Does your workplace require a proxy to be used to access the Internet? If so, you must configure the HTTP_PROXY environment variable before downloading Cypress. Read more: https://on.cypress.io/proxy-configuration npm ERR! npm ERR! Otherwise, please check network connectivity and try again: npm ERR! npm ERR! ---------- npm ERR! npm ERR! URL: https://cdn.cypress.io/beta/binary/12.17.5/linux-x64/main-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.zip npm ERR! Error: Failed downloading the Cypress binary. npm ERR! Response code: 404 npm ERR! Response message: Not Found npm ERR! npm ERR! ---------- npm ERR! npm ERR! Platform: linux-x64 (Ubuntu - 22.04) npm ERR! Cypress Version: 12.17.5 npm ERR! [FAILED] The Cypress App could not be downloaded. npm ERR! [FAILED] npm ERR! [FAILED] Does your workplace require a proxy to be used to access the Internet? If so, you must configure the HTTP_PROXY environment variable before downloading Cypress. Read more: https://on.cypress.io/proxy-configuration npm ERR! [FAILED] npm ERR! [FAILED] Otherwise, please check network connectivity and try again: npm ERR! [FAILED] npm ERR! [FAILED] ---------- npm ERR! [FAILED] npm ERR! [FAILED] URL: https://cdn.cypress.io/beta/binary/12.17.5/linux-x64/main-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.zip npm ERR! [FAILED] Error: Failed downloading the Cypress binary. npm ERR! [FAILED] Response code: 404 npm ERR! [FAILED] Response message: Not Found npm ERR! [FAILED] npm ERR! [FAILED] ---------- npm ERR! [FAILED] npm ERR! [FAILED] Platform: linux-x64 (Ubuntu - 22.04) npm ERR! [FAILED] Cypress Version: 12.17.5

npm ERR! A complete log of this run can be found in: npm ERR! /home/tempuser/.npm/_logs/2023-08-25T00_10_00_723Z-debug.log Error: Process completed with exit code 1. `

Here’s a pre-release build to test in your CI processes. This build hard codes force: true for optimizeDeps. If you try this out, please comment with the result.

npm install https://cdn.cypress.io/beta/npm/12.17.5/linux-x64/marktnoonan/25913-vite-logging-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.tgz

This should actually install the correct binary for your system, but if it doesn’t, here are the platform specific URLs you can point to:

We have also switched to running Cypress against a pre-built version of the library but did not help solve the issue.

@floroz did you switch to pre-build version for Cypress Component Tests or E2E tests? Asking because I was wondering if this would have been easy for Component Tests (we already did that for E2E as noted above). I’m not sure what changes would be needed since as I understand component tests in Vite (if using the vite bundler) rely on the Vite dev server bundling behavior 🤔

@rszalski we use Cypress for integration testing and visual regression with Percy. The recurring failures were damaging our baseline of screenshots.

We migrated part of our tests involving visual regressions from Component Testing to E2E Testing (not Component) and created “fake pages” to render the components in isolation and perform our snapshots.

We are still keeping the “integration” part in Component testing, but since half of the workflow now has been moved away, we’ve experienced a reduction in the number of times our tests are failing.

Again, whether is a coincidence or the more components/runs the more the chance of this failure, is hard to say.

We are also facing a separate issue where the CT run intermittently freezes (no error, no timeout, just gets stuck on various tests). Not sure if this is at all related and so I’m focusing on the Failed to fetch dynamically imported module error.

Worst case scenario we’ll revert back to running Component tests w/ Webpack until this is resolved.

We’ve also experienced these random freezes, also locally. Again, almost impossible to reproduce the issue reliably to try debug the error (besides the lack of logging).

We have also switched to running Cypress against a pre-built version of the library but did not help solve the issue.

We’ve migrated to E2E tests with “fake pages” to continue simulating the same environment as CT.

Let’s try using force. This issue is happening to us in the Cypress repo, too https://app.circleci.com/pipelines/github/cypress-io/cypress/55051/workflows/57524acc-7e72-42ca-9aee-d5cd707aacc3/jobs/2276758?invite=true#step-112-11815_61

@moritzkalwa

I feel like there should be an easy way to retry tests that fail in this manner, not just better logging.

I tried re-running those tests, we had a PR that did this, it would fail for the same reason many times, it was like Vite would refuse to refresh / serve the dep.

For us this issue started appearing when upgrading from node 16 to 18. We had decided to put this upgrade on hold until a solution is found, however with node 16 going EOL next month we might have to upgrade without a fix and completely break our CI pipeline. Would be really awesome if a solution is found until then. I feel like there should be an easy way to retry tests that fail in this manner, not just better logging.

Hey @lmiller1990 - do you have updates on this topic? We are experiencing pretty much the same as the rest in this thread. We are having one single component test which sometimes (I would guess 30%) fails on our GitHub actions pipeline with the “failed to fetch” error. After one manual retry we have a successful run.

I am surprised that makes any difference. tsc is usually smart enough to coerce a type only import from import to import type.

@barbados-clemens I think you are correct about the CI load as the underlying issue, which is why no-one can reproduce this locally, and why it seems to be GH Actions specific (seems these machines suffer load issues more than other providers).

'@typescript-eslint/consistent-type-imports': 'error',

Add this to eslint rules. After about 10k lines of changes in our monorepo, some of the cryptic Vite errors have subsided.

I think we solved it on our side, we had an import that was no longer valid inside component.ts. Didn’t happen again for a few days already, both on CI and locally for a few of us.

It does not explain, though, why when we upgraded to vite4 it stopped happening for a few hours, so I hope it was the reason and that it won’t reproduce…

@lmiller1990 Progress!!! Took a quick look at this and it appears we have a network loop issue - something is generating a network request against /__cypress/src/ (default devServerPublicPathRoute) which is looping through the Node proxy agent and eventually killing the process thru a “too many file handles” error. I added the “ESCAPE HATCH” block below in agent.ts and was able to run the full test suite provided by @patrickcate without issue using my local cypress dev env, remove that block and it blows up.

// packages/network/lib/agent.ts

  if (!options.href) {
    debug('Defaulting href from path "%s"', options.path)
    // options.path can contain query parameters, which url.format will not-so-kindly urlencode for us...
    // so just append it to the resultant URL string
    options.href = url.format({
      protocol: isHttps ? 'https:' : 'http:',
      slashes: true,
      hostname: options.host,
      port: options.port,
    }) + options.path
  }

  // ESCAPE HATCH
  if (options.href === 'http://localhost:3030/__cypress/src/') {
    return
  }

  if (!options.uri) {
    options.uri = url.parse(options.href)
  }

Since this seems CT-specific I’m assuming we have some sort of bad interaction between our configs in routes-ct.ts, server-ct.ts, and perhaps the base Vite dev server configs - will dig in a bit more on Monday

Argh, the dream was short lived. It’s hard to say if this made a difference or if I just got lucky with 4 consecutive passes. 😢

Hmm you are running 4 containers - I’ll try running more than 1, maybe that’s related.

Sample size of 1 but seems to work so far: https://github.com/vuetifyjs/vuetify/actions/runs/4574802124 Looks like evan is interested in having it be part of vite itself: https://twitter.com/youyuxi/status/1641402565631025152

Aaand still busted: https://github.com/vuetifyjs/vuetify/actions/runs/4576029601/jobs/8079626308 unless process.env.CYPRESS isn’t set somehow but it seems to be locally

@stnokott thanks for this - the TS issue is, in fact, a bug in TypeScript. Follow this issue: https://github.com/cypress-io/cypress/issues/26148#issuecomment-1487574243. It looks like there is a open PR in TS fixing the issue. 🤞 Looking at your CI, it seems the same test, Settings.cy.tsx keeps failing, and the Vite optimize step always precedds it? https://github.com/stnokott/r6-dissect-influx/actions/runs/4467180776/jobs/7846310109#step:5:271. I wonder if you add those deps to optimizeDeps in your vite.config, will it prebundle them and avoid the flake 🤔 ?

@KaelWD thanks, I will spend some more time looking into this today.

Lol, I did not had to wait too long. Failed on first try after update cypress to 12.9.0.

✖ 1 of 191 failed (1%) 00:37 654 653 1 - -

Running:  landing/LandingArticleCard.spec.js                                          (109 of 191)


  (Attempt 1 of 2) An uncaught error was detected outside of a test
  1) An uncaught error was detected outside of a test

  0 passing (419ms)
  1 failing

  1) An uncaught error was detected outside of a test:
     TypeError: The following error originated from your test code, not from Cypress.

  > Failed to fetch dynamically imported module: http://localhost:3000/__cypress/src/cypress/support/index.js

When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.

Cypress could not associate this error to any specific test.

We dynamically generated a new test to display this failure.

Again after around 50% finished tests, again random one, component is just fine (it’s very basic) and it will work after rerun, it happens only on GH actions (works locally). So nothing changed 😦

Like I said before, I tried with “retries” functionality in Cypress, but it ignores it. Like you see, there is (Attempt 1 of 2) but it never tries second time. I guess it only retries when there are assertion errors, but not general errors, which is shame 😦 I would really like to just rerun that single test, instead all queue and all CI/CD process. Any way to do that?

Edit: Also it’s failing on importing cypress/support/index.js which for me does not make any sense.

So I can confirm it was not fixed in my case because based on #26138 it was due to vite 4.2.0, but I’m still on 4.1.1 😕 I will update Cypress anyway with that fix, and wait until it will happen again.

Okay we have a potential solution for this that should at least help prevent failures due to this issue. If anyone is willing, you can try installing a pre-release binary from my commit here.

If anyone does this, please post here and let us know if you are still seeing import failures in CI

@chojnicki thanks a lot, I’ll ping you if we need access, I think our reproduction should be enough. Is your reproduction consistent? CI only?

@lmiller1990 like i described in previous comment - only CI/CD with GH actions. Locally (with exact same docker image, also on linux as host) it does not happening, not even once. But on GH it’s very random.