cypress: Component tests failing intermittently with uncaught Vite "failed to fetch dynamically imported module" error in CI
Current behavior
We started migrating our project from CRA to Vite and we have almost 300 component and 40 E2E Cypress tests in place. Unfortunately, after fixing all the other issues, we are still not able to stabilize our tests since there is always 1 or 2 failing randomly on “Failed to fetch dynamically imported module” errors.
We noticed that it’s somehow related to the load of CI. Under some conditions more tests are failing like this, in other times it succeeds. But it’s random. We checked our tests and we are pretty sure it’s not caused by any logic we have.
We’ve checked some of the existing issues on Cypress & Vite, tried various workaround but no luck with any of them.
What we think is happening is that Cypress is not waiting for Vite to “boot up” properly and retries don’t help with it, only when new spec is run, it works.
Note: it only happens with component tests. For E2E tests we had similar stability issues but we solved them by building production version and then just serving it with vite preview. This made integration tests faster and very stable. Previously they were also timeouting.
Note 2: we have a lot of components and lot of dependencies in our project, we also use MUI as our base. But with CRA we were able to have stable tests, it was just around two times slower. That’s why we want to use Vite now.
Note 3: we are running “Component tests” in parallel, currently in 4 groups.
Desired behavior
No random problems with Cypress + Vite combo.
Test code to reproduce
Unfortunately can’t provide this since our project is private. And I’m afraid that it’s related to project’s complexity and that’s why we can’t easily create another one for reproduction.
Cypress Version
v12.6.0
Node version
v16.18.1
Operating System
Docker Hub image: cypress/included for v12.6.0 version
Debug Logs
No response
Other
No response
About this issue
- Original URL
- State: open
- Created 4 years ago
- Reactions: 76
- Comments: 181 (52 by maintainers)
Commits related to this issue
- ci: use cypress pre-release build to see if flakey tests are fixed see: https://github.com/cypress-io/cypress/issues/25913#issuecomment-1481802265 — committed to dyc3/opentogethertube by dyc3 4 years ago
- ci: use cypress pre-release build to see if flakey tests are fixed (#844) * ci: use cypress pre-release build to see if flakey tests are fixed see: https://github.com/cypress-io/cypress/issues/259... — committed to dyc3/opentogethertube by dyc3 4 years ago
I am facing same issue
We are having the exact same issue with component tests ( I was just coming here to open a new issue ) . We just migrated from webpack to vite. The error we get is:
The following error originated from your test code, not from Cypress. > Failed to fetch dynamically imported module: http://localhost:3000/__cypress/src/cypress/support/component.ts When Cypress detects uncaught errors originating from your test code it will automatically fail the current test. Cypress could not associate this error to any specific test. We dynamically generated a new test to display this failure.We are getting this despite the fact that the file does exist. And this occurs randomly.
Vite: 4.1.1 Cypress: 12.5.1 Node: 19.3.0
One related issue: our cypress directory is on the same level as our src directory. When running locally, Cypress correctly expects component.ts to be where we have it in the cypress directory. But when we run the tests in docker, Cypress expects component.ts to be in a cypress directory under the src directory (see above error). Even if we use the ‘supportFolder’ config setting, Cypress still looks for it in the src directory (src is the vite root folder). So I just copied component.ts to the location cypress expects it to be (but still fails randomly despite this).
So locally this is not reproducable for us (so far). This only occurs in a docker container.
We are also running the tests in parallel.
Hi all - we are looking to work on this in the next two weeks with the goal of completely solving it, eliminating the need for work arounds.
Any update on this issue? It’s unfortunately made Cypress component testing pretty much unusable for my projects since it’s extremely difficult to get full test suites to pass.
@adamscybot optimizeDeps doesn’t work for us. Still getting random errors on CI.
In my case it started happening more and more often after upgrading from
12.7to13.1. For some reason CI stopped logging the “[vite] ✨ new dependencies optimized:”, but I could reproduce that easily locally.After going through Vite docs, I read this bit:
The bit about “__tests__” folder being ignored from pre-bundling was the problem in my case. Only tests that were nested inside “__tests__” folder were randomly failing. In my case, solution was to add all my test files to
entriessection of Vite config, like so (all my tests have thistest-cybit added in name):Now for each test, Vite always optimizes all needed dependencies before running the test, and since that the issue has not been happening anymore.
Chiming in here in the hopes of saving others some time.
We had had intermittent issues with component tests failing in CI. The errors always came near a
[vite] ✨ new dependencies optimizedline in the job log.I was able to reproduce the issue locally by clearing
node_modules/.vite/depsbefore running the tests, and adding:to our
vite.config.jssolved those issues. Our test files (.cy.js) are all located within that directory.That was fine for a while, but I just now had to address another similar issue, this time with a new dependency (
vue-router) that was not previously imported directly by our tested components. I eventually figured out that the issue was due to the fact that we cache thenode_modulesdirectory in between jobs (we use GitLab CI/CD). I don’t have a full understanding of the Vite internals, but I think it saw that (now stale).vite/depsdata and assumed it didn’t need to optimize (pre-optimize?) the Cypress test files again. I added a simplerm -rf node_modules/.vite/depsto the Cypress test job, and that fixed the issue.For anyone interested, here’s our “workaround” solution. We have a script called
cypress-ct-runner.mjs(underbin/vite-migration:Under the same path we also have another script which collects paths of all our component test specs and “groups” them randomly, it looks like this:
On CI we have a following setup of steps:
./bin/vite-migration/distribute-component-tests.js 5./bin/vite-migration/cypress-ct-runner.mjs 1./bin/vite-migration/cypress-ct-runner.mjs 2-5Our whole CI flow looks like this:
I hope that someone will find this useful until the root cause gets fixed.
EDIT: In the end this doesn’t work, it just made the problem rarer.
I’m still having this problem. I’m not really satisfied with the last solution (retrying the run), so I explored the “warmup files” approach suggested earlier by @adamscybot.
The solution provided was
However, as @apdrsn mentioned, this leads to numerous errors on my end:
[vite] Pre-transform error: Failed to resolve import ...The Vite documentation also advises against warming up too many files, so I was reluctant to use a wildcard to preload everything.
Instead, I refined this solution by warming up only the specific file that was failing to load. This approach seems cleaner and has been working well so far:
Note that this still requires Vite 5 as the
warmupconfig isn’t available in earlier versions.We will look at trying vitejs/vite#12080 in our next sprint of work to see if we can get more fine grained error logging. This is very hard problem to solve.
I know this is brute force, but try:
And yes there will be warnings in console. Ignore them for the sake of testing this. Currently you are assuming the problem is coming from one of your support files, but actually, it can be anything. Including the rest of the support files, third party modules, the tests themselves, and anything they import.
This is just to try to prove a point about if it works or not. There’s a lack of data points at the moment as some people accurately report the problem can return or still exist at a lower level with
warmupconfigured, but up to now we only have that data for folks who have a selectivewarmupconfiguration which could be why (or not…).I am still using
**/*and haven’t had it resurface. But that of course, is not proof of anything either. Just need more folks trying and reporting back.@YossiSaadi we’ve tried upgrade to cypress 12.9 & Vite 4 and the errors still occur.
@lmiller1990 have you found any interesting insights?
@lmiller1990 I created a repro for this issue showing it happening in a NON-CI scenario. Hopefully, it helps get to the root of the problem.
This finally flared again enough for me to attend to it. The recurrence was somehow related to a Vite 5 upgrade. Vite 5 now has a new option
server.warmupwhich can get the cache ready in advance.We removed configuration around
optimizeDepsand tried a (brutal) wildcard glob inwarmup:The problem is now gone again. If you are on Vite 5, I recommend trying this. If you are not, I recommend upgrading Vite (you also need at least Cypress 13.10 to be compatible) and then trying it.
Probably this glob pattern can be reduced to just my test file, but just getting this information out there. We also haven’t done enough testing to know if it was actually the removal of
optimizeDepswhich is the key ingredient.Either way, we defintely need to be clear what Vite versions we have when reporting back.
https://github.com/vitejs/vite/issues/12080 was released in vite 4.4.0
Update… this is not forgotten, but we are trying to figure out how to fix this. It’s clear from the investigation so far there’s likely a few bugs that all manifest in the same fashion. In addition, the actual problem looks like it spans both Vite and whatever it’s integrated with (Cypress, PW, or whatever). So far, several developers (here and external) spend some time (days) trying to fix this, but no luck.
I think we need to budget significant (weeks) of time for someone to go a lot deeper into this, and possibly in conjunction with someone from the Vite team. Based on https://github.com/vitejs/vite/issues/11804, it is impacting many users outside of Cypress, too.
We have an internal repro 🔥 FINALLY!
It’s Cypress org private project, but dropping the link here so someone at Cypress can internally… https://cloud.cypress.io/projects/d9sdrd/runs/4193/test-results/2f924501-7fb2-4c80-b69f-819010c67c87
Now we’ve got a repro I can dig into it… for us it’s CI only too, so sounds like a resources/race condition.
Hey @ivanalemunioz, definitely love community engagement and integrations with Cypress. Please remember to keep threads on topic though for the benefit of everyone following this specific issue. 🙏🏻 If you’d like to share your work you can share in our community chat or social media.
Hi, everyone. I’ve been working on a solution to visually debug my browser automation crashes, and I’ve now made it free for the public to use: https://buglesstack.com
I don’t mean to spam or bother you; I just want to share this solution. If you have any questions or suggestions for improvement, feel free to contact me.
Thanks for reading! ❤️
We’re getting this issue after upgrading to Cypress 13.17.0 from 13.14.2, if I revert to 13.14.2 the problem goes away.
One thing to note, we see the issue reliably if we have code coverage turned on (we’re using @bahmutov/cypress-code-coverage@2.7.2), we don’t run coverage on our branches so our PR test runs never suffer from this issue. Its only when we merge to main (after getting a green test run) that the tests then start to fail as this is when we generate coverage reports.
Edit: at the same time as upgrading to 13.7.0 we also upgraded from Vite 5 to Vite 6 and I see someone above mentions that they started seeing this after going to Vite 6. As I say above though, reverting to 13.14.2 fixes the issue for us so it feels more like a Cypress issue. I also tried 13.15.0 but had the same problems with that so this suggests it was something introduced at 13.15.0
Edit 2: Switching to the standard cypress code coverage plugin has resolved this issue for us
Something interesting is happening here. After I claimed victory on this issue, we did indeed continue error-free for many months (thousands of runs), which I think is solid evidence of
optimizeDepshaving something to do with it, if only in part.However. Yesterday, due to some other requirement, I upgraded Cypress from v12 to v13, and at the same time, I upgraded Vite from v3 to v5. It is probably tertiary, but it is also worth noting this also involved an upgrade of the Cypress docker image, which would have also resulted in a Chrome upgrade,
It was only once, and I hardcore logged off shortly after in total denial as Friday evening drew to a close – but after doing this, the issue re-emerged for the first time in months on a single CI run. Deep down, I know the curse has returned.
Perhaps an ever-present race condition has suddenly reactivated due to the subtle execution speed differences all these upgrades made, or perhaps there was a relevant behavioural change that affected this issue amongst those changes.
I think it would be useful for everyone to post their exact Vite and Cypress versions and their
optimizeDepssettings when posting their analysis. if you’ve already posted, it would be super helpful to just say “Im using Cypress x with Vite x and thisoptimizeDepsdeps setting and it’s happening for me”. Perhaps also note what docker image it runs in CI (or if you don’t do that and it’s on bare metal).Monday will probably be interesting, but England just got to the semi-finals so I’m pretending not to be affected by the blaze of inevitable red all over CI.
This is a total stretch but I’m also someone who is loading service workers (MSW) as part of the test story. I doubt its related, but would be interested to know in the rare event there’s some consistency between that cause and this affect.
I experienced this too - for me it was because I was switching between branches with different dependencies, and one of them somehow hadn’t been reinstalled inbetween changing branches (in my case @pinia/testing). So while Cypress was reporting that it couldn’t import components.js, it wasn’t because that file was missing, it was because that file was erroring because of the missing import of @pinia/testing within it. Hope this helps people who end up here in the future (or me, when this happens again and see my own comment).
Still a problem. In my case, all test run fine locally but crashing randomly in pipeline, with the error mentioned. You can run the same job twice and where in the first run the Test A crashes and Test B works, in the second run both can crash or B crash and A works…
Hi. I’m adam-thomas-privitar on my personal account 😃. So I wouldn’t say that my confidence is 100% due to the odd nature of this issue. But I can say so far, the problem has not reoccurred in my branch since adding the test files to the
optimizeDeps.entriesafter kicking it off a fair few times. I was also able to re-enable a certain suite which in particular was causing us especially big problems (but not exclusively) in regards to this issue, which is really promising. Previously we were retrying that test at the CLI level 3 times and even that eventually was not sufficient. So the fact it’s working without those retries strongly suggests there may be a path here.I’m trying not to get my hopes up too much though since the real test will come when this gets merged to main next week and the build volume increases. So ultimately we still need to proceed with caution.
I will report back, but I feel there’s sufficient data to say that anyone experiencing this should try it so we can add more data points. If it does fix it, @RadomirNowak is a legend 😄, as this problem has truly haunted us at scale. If it suddenly fails, I might go and have a little cry somewhere and hang up my keyboard forever.
It’s perhaps of note that we have our own substantial layer above cypress (test support stuff) that is imported only into tests, in its own package, to abstract common things, and so the proposition that this plays a part is very plausible.
I believe nearly all of the discourse about solutions that seem to work then don’t is likely due to varying load on the agent that’s executing the tests. It’s a race condition in its current state, and so any fluctuation in speed affects it, and I’ve observed this myself. Also, the problem gets worse the more deps you add. If you are running tests in parallel that definitely adds to the randomness also, since that’s also causing varying load between runs. Another reason is if you have many agents and some are more highly specced than others.
If it does emerge that this is the problem then I guess a PR to the Cypress vite integration package is needed to automatically merge the test files into
optimizeDeps.entries.the thing with
__tests__was something that allowed me to find the root issue, which is: if you import packages inside your tests that are not imported in your app (so they are not prebundled by vite) then they will be dynamically optimized during test run which will result in the problem described in this thread. @floroz @adam-thomas-privitar can you try to include your test files in theentriessection like I mentioned before? Also another thing could be to include there a glob for all your files (as a test) e.g.**/*.tsWe are using the
__tests__convention as well so the theory is certainly interesting.I still need to confirm this long term, but I think setting a constant port value for dev server with Cypress tests seems to fix the issue. Please see whether it will work for you.
@adam-thomas-privitar @nive-copia @rszalski I updated the links to the pre-release binary in Mark’s comment. You should be able to install this one. Please comment if you are still having issues installing and thank you for your patience
@nive-copia this should be working, I’ll look into it. I was able to download the binary but not perform npm install.
Agreed. Still it would be useful to unblock people. I’m going to create a pre-release build of Cypress that people in this thread can drop into their CI pipelines to see if it has an impact on this error. I’ll update this thread with instructions when it is ready.
@OrbisK I played around with
find-upvsloadConfigFromFilebut I didn’t immediately see the opportunity to replace one with the other as you suggested. Though if I follow what you are suggesting, maybe we could useloadConfigFromFilefor the file from the path determined withfind-up, before we usevite.mergeConfighere. Is your thinking that, under the hood, vite may then recognize the need to rerun without force? Based on “Relevant fields in your vite.config.js” being mentioned here.Is it possible to start the component dev server with the
--forcecli flag? Maybe vite cache is the problem (related to the “vite is optimizing” problem with included deps)?Docs:
EDIT:
optimizeDeps.forcewill ignore previously cached optimized dependencies.FYI: I am not facing this issue (anymore (?) - and i dont know why). I can’t test it myself.
@floroz no sorry, we got caught up in other issues. We will try and get to this this week or next. To clarify, we are going to (or a third party, if anyone is up for it):
In https://github.com/vitejs/vite/pull/12084/files, a new event was added:
vite:preloadError. We should listen to this on the server and log any events usingDEBUG, like this https://github.com/cypress-io/cypress/blob/14a7416e126f6b296a2da037ae257c27e7915d5b/npm/vite-dev-server/src/plugins/cypress.ts#L10Then when users run Cyperss with
DEBUG=cypress:vite*they will (hopefully) see a more useful error on the server. If we can manifest this in the Cypress UI, that would be even better. The hope is we can find a more specific error that we can use to actually fix this bug.FYI, this happens in other repos not using Cypress, https://github.com/vitejs/vite/issues/11804, so it’s possible the fix is actually a PR in Vite, eventually. I think the issue is we use Vite in a fashion that makes this error more regularly occurring here than just using Vite as a dev server independently.
@mx2s thanks! Can you share your
./vite.config?May be that helps anyone, Here’s my config for component testing:
Tests started to pass for me once I changed
viteConfig('production')fromdevelopmenttoproduction(basically NODE_ENV)If you need another possible test case you can try the https://github.com/patrickcate/vue-uswds/tree/feature/606/cypress-12-upgrade branch of my Vue 3 project. Unlike other’s I’m seeing this error when running the tests locally, both in headed/open and headless/run mode.
The components it fails on appears to be random, but in headed/open mode, refreshing the browser often allows the tests to run fine. In headless/run mode this isn’t possible so I’ve never been able to get through a successful run with the 100 or so total number of components.
I am not using the
vite-plugin-warmupplugin or running tests in parallel.Other details:
Local computer:
Just collecting research for myself to debug this, if it is a slow start caching issue, maybe we can adapt https://github.com/bluwy/vite-plugin-warmup
Edit @KaelWD interesting, I got a failure on first try for Vuetify. I added the plugin above, and it passed four times consecutively. https://github.com/lmiller1990/vuetify/actions. WDYT? Can you try adding that plugin to your CI like I did and see if it helps?
Edit: I removed the plugin and it failed. Either this was dumb luck, or we isolated the issue to a race condition, solved by having a warm cache.
@stnokott can you try the plugin too? And @chojnicki? Like this: https://github.com/lmiller1990/vuetify/commit/ff65846fcea3461092e908aa60cc94d07aa83102#diff-ee0cbc5388a0d00ac00c818b7cf76db70ff9c4e7945808ea2340d4eb4423d4cfR58-R63 include a pattern to match your spec files, and your
cypress/componentfile (the one withmount).Before this change, 90% of my Cypress runs in the CI of my little hobby project were failing due to dynamic import issues. Since using your commit, none of them failed due to that reason. (It’s a tiny project, so probably not very representative, but wanted to let you know regardless)
@waldemarennsaed no I don’t think so, since there are different problems described in this issue that don’t relate to #26139. I would like to update the title of the issue to make it more specific though, since like Lachlan pointed out, the “failed to fetch” error can basically mean that any error occurred. I’ll be looking more into this issue today
Nice try @crobinson-expl but unfortunately your error is related to https://github.com/cypress-io/cypress/issues/26138, we don’t support Vite 4.2 yet. That will be fixed in https://github.com/cypress-io/cypress/pull/26139.
The “failed to fetch” is so general, it can basically mean any kind of error occurred. I think this issue you are encountering is unrelated to the one reported here.
Hello! Is this actively being worked on, or has anyone found a good workaround for it ?
For me setting cypress files to optimizedDeps worked! At first, I updated vite to 5.4.11 and added warmup config but it didn’t work by its own. I’m using cypress 13.15.2.
UPDATE: Not working consistently in CI, it’s still failing 😕
The only thing for us that completely killed this issue was when we switched to a non-shared CI runner. Our Infra team kindly gave us a dedicated machine so we don’t share CPU/Memory with other CI jobs and that seems to have completely stopped the issue for us.
Don’t know how much this would help as to me it points to an issue in Vite rather than cypress?
I report back positively back then as well. Since we use the warmup method, it seems to work for us.
After some time i can confirm that it works for me, but im afraid that this could change in the future with bigger components, etc.
We’re having this problem still, after adding the experimentalJustInTimeCompile flag and optimizeDeps stuff. It’s random and intermittent, which somehow makes it even worse. The warmup stuff requires us to upgrade to Vite 5, which I don’t want to do just yet, so I’m stumped as what to try next.
npm ls vite└── vite@4.5.3npm ls cypress└── cypress@13.14.0Error:
The other 2 runs did not fail, only the first one.
Ah yes! I had seen this just today also. The warmup trick is still working for me, but I will give this a go as well.
Just a check in that over 2 weeks now the problem remains gone!
So its been a couple of days now and the problem has not come back in CI. As others have said, I think this is about capturing everything, including any test-support stuff as well as the test files themselves. Makes sense support file should be added here as well because that is imported separately by cypress and not from the tests.
@dyc3 I had the same issue - turned out I was not catching all files with all import - in my case it was also
cypress/support.tsfile which was not caught by thesrc/**/*glob pattern inentriessection.it could be also a different problem on your side, as I’m using React so maybe there’s something different in Vue 🤔 Anyways I’d recommend investigating the output of CLI when you run the tests for any
[vite] ✨ new dependencies optimized:and also check contents ofnode_modules/.vite/depsand how they change between different test suites - ideally to mitigate the error the contents of this folder should always be the same - if it changes then there’s some dependencies not caught by theoptimizeDeps.entriesoptionJust to let you know I am trying your workaround in CI currently, and I should be able to get back to you once its run a sufficient amount of times.
We’ve been stuck on this for about 3 weeks, would really love to hear from anyone with a workaround.
@astone123 @marktnoonan I was able to locally install the drawin arm64 beta version from Mark’s comment and compare with a stable version that we currently run.
Our component tests have 44 specs and I tested both the
betaversion and a stablev12.17.3ofcypressby running the entire suite 10 times for each version.In each case the only failures were the loading of dynamically imported module:
As I mentioned above we are not only seeing errors like the above, that fail the spec, but also see the entire test run freeze randomly.
The very interesting thing is that when doing the tests on the
betaversion the spec never froze. We also saw much more of these dynamically loaded module errors (higher repro rate). Thebetaversion definitely impacts this, but there might be now a completely different failure mode showing up, but manifested in the same cryptic message. Is there a way to get better logs from the Vite’s error hooks? Or maybe from Cypress’ debug logs?Testing
v12.17.3betav12.17.3.I then tried running individual spec files, that I saw fail when run as part of the entire suite. Running a single spec file never failed on
beta. Tried this like 50x times.I then tried running 2 spec files (both of which had failed in the full run) in this case, always the second spec failed, never the first one. Tried this also dozens of times.
I tried running the above with
DEBUG=cypress:*but that was an overwhelming amount of logs and nothing stood out to me. I’m happy to do some more debugging but I’d need pointers on how to narrow the logs down to what is relevant.Is the experimental build available to public to test? I see the following error when I try to install.
` Run npm install https://cdn.cypress.io/beta/npm/12.17.5/linux-x64/marktnoonan/25913-vite-logging-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.tgz npm WARN deprecated stable@0.1.8: Modern JS already guarantees Array#sort() is a stable sort, so this library is deprecated. See the compatibility table on MDN: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort#browser_compatibility npm WARN deprecated consolidate@0.15.1: Please upgrade to consolidate v1.0.0+ as it has been modernized with several long-awaited fixes implemented. Maintenance is supported by Forward Email at https://forwardemail.net ; follow/watch https://github.com/ladjs/consolidate for updates and release changelog npm WARN deprecated @stylelint/postcss-markdown@0.36.2: Use the original unforked package instead: postcss-markdown npm WARN deprecated babel-eslint@10.1.0: babel-eslint is now @babel/eslint-parser. This package will no longer receive updates. npm WARN deprecated @stylelint/postcss-css-in-js@0.37.2: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info. npm WARN deprecated sortablejs@1.12.0: This version has deprecated in favor of a rollback to version 1.10.2 npm WARN deprecated core-js@3.21.1: core-js@❤️.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js. npm ERR! code 1 npm ERR! path /home/tempuser/work/temp-fork/temp-fork/node_modules/cypress npm ERR! command failed npm ERR! command sh -c node index.js --exec install npm ERR! Note: Overriding Cypress cache directory to: /home/tempuser/.cache/Cypress npm ERR! npm ERR! Previous installs of Cypress may not be found. npm ERR! npm ERR! ⚠ Warning: You are installing a pre-release build of Cypress. npm ERR! npm ERR! Bugs may be present which do not exist in production builds. npm ERR! npm ERR! This build was created from: npm ERR! * Commit SHA: 73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72 npm ERR! * Commit Branch: main npm ERR! * Commit Timestamp: 2023-08-21T22:49:16.000Z npm ERR! npm ERR! Installing Cypress (version: https://cdn.cypress.io/beta/binary/12.17.5/linux-x64/main-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.zip) npm ERR! npm ERR! [STARTED] Task without title. npm ERR! The Cypress App could not be downloaded. npm ERR! npm ERR! Does your workplace require a proxy to be used to access the Internet? If so, you must configure the HTTP_PROXY environment variable before downloading Cypress. Read more: https://on.cypress.io/proxy-configuration npm ERR! npm ERR! Otherwise, please check network connectivity and try again: npm ERR! npm ERR! ---------- npm ERR! npm ERR! URL: https://cdn.cypress.io/beta/binary/12.17.5/linux-x64/main-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.zip npm ERR! Error: Failed downloading the Cypress binary. npm ERR! Response code: 404 npm ERR! Response message: Not Found npm ERR! npm ERR! ---------- npm ERR! npm ERR! Platform: linux-x64 (Ubuntu - 22.04) npm ERR! Cypress Version: 12.17.5 npm ERR! [FAILED] The Cypress App could not be downloaded. npm ERR! [FAILED] npm ERR! [FAILED] Does your workplace require a proxy to be used to access the Internet? If so, you must configure the HTTP_PROXY environment variable before downloading Cypress. Read more: https://on.cypress.io/proxy-configuration npm ERR! [FAILED] npm ERR! [FAILED] Otherwise, please check network connectivity and try again: npm ERR! [FAILED] npm ERR! [FAILED] ---------- npm ERR! [FAILED] npm ERR! [FAILED] URL: https://cdn.cypress.io/beta/binary/12.17.5/linux-x64/main-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.zip npm ERR! [FAILED] Error: Failed downloading the Cypress binary. npm ERR! [FAILED] Response code: 404 npm ERR! [FAILED] Response message: Not Found npm ERR! [FAILED] npm ERR! [FAILED] ---------- npm ERR! [FAILED] npm ERR! [FAILED] Platform: linux-x64 (Ubuntu - 22.04) npm ERR! [FAILED] Cypress Version: 12.17.5
npm ERR! A complete log of this run can be found in: npm ERR! /home/tempuser/.npm/_logs/2023-08-25T00_10_00_723Z-debug.log Error: Process completed with exit code 1. `
Here’s a pre-release build to test in your CI processes. This build hard codes
force: trueforoptimizeDeps. If you try this out, please comment with the result.npm install https://cdn.cypress.io/beta/npm/12.17.5/linux-x64/marktnoonan/25913-vite-logging-73e67122f5feeeb7f9ea316ca1c6e94a33a3ca72/cypress.tgzThis should actually install the correct binary for your system, but if it doesn’t, here are the platform specific URLs you can point to:
@rszalski we use Cypress for integration testing and visual regression with Percy. The recurring failures were damaging our baseline of screenshots.
We migrated part of our tests involving visual regressions from Component Testing to E2E Testing (not Component) and created “fake pages” to render the components in isolation and perform our snapshots.
We are still keeping the “integration” part in Component testing, but since half of the workflow now has been moved away, we’ve experienced a reduction in the number of times our tests are failing.
Again, whether is a coincidence or the more components/runs the more the chance of this failure, is hard to say.
We’ve also experienced these random freezes, also locally. Again, almost impossible to reproduce the issue reliably to try debug the error (besides the lack of logging).
We have also switched to running Cypress against a pre-built version of the library but did not help solve the issue.
We’ve migrated to E2E tests with “fake pages” to continue simulating the same environment as CT.
Let’s try using force. This issue is happening to us in the Cypress repo, too https://app.circleci.com/pipelines/github/cypress-io/cypress/55051/workflows/57524acc-7e72-42ca-9aee-d5cd707aacc3/jobs/2276758?invite=true#step-112-11815_61
@moritzkalwa
I tried re-running those tests, we had a PR that did this, it would fail for the same reason many times, it was like Vite would refuse to refresh / serve the dep.
For us this issue started appearing when upgrading from node 16 to 18. We had decided to put this upgrade on hold until a solution is found, however with node 16 going EOL next month we might have to upgrade without a fix and completely break our CI pipeline. Would be really awesome if a solution is found until then. I feel like there should be an easy way to retry tests that fail in this manner, not just better logging.
Hey @lmiller1990 - do you have updates on this topic? We are experiencing pretty much the same as the rest in this thread. We are having one single component test which sometimes (I would guess 30%) fails on our GitHub actions pipeline with the “failed to fetch” error. After one manual retry we have a successful run.
I am surprised that makes any difference.
tscis usually smart enough to coerce a type only import fromimporttoimport type.@barbados-clemens I think you are correct about the CI load as the underlying issue, which is why no-one can reproduce this locally, and why it seems to be GH Actions specific (seems these machines suffer load issues more than other providers).
'@typescript-eslint/consistent-type-imports': 'error',Add this to eslint rules. After about 10k lines of changes in our monorepo, some of the cryptic Vite errors have subsided.
I think we solved it on our side, we had an import that was no longer valid inside
component.ts. Didn’t happen again for a few days already, both on CI and locally for a few of us.It does not explain, though, why when we upgraded to vite4 it stopped happening for a few hours, so I hope it was the reason and that it won’t reproduce…
@lmiller1990 Progress!!! Took a quick look at this and it appears we have a network loop issue - something is generating a network request against
/__cypress/src/(defaultdevServerPublicPathRoute) which is looping through the Node proxy agent and eventually killing the process thru a “too many file handles” error. I added the “ESCAPE HATCH” block below inagent.tsand was able to run the full test suite provided by @patrickcate without issue using my local cypress dev env, remove that block and it blows up.Since this seems CT-specific I’m assuming we have some sort of bad interaction between our configs in
routes-ct.ts,server-ct.ts, and perhaps the base Vite dev server configs - will dig in a bit more on MondayArgh, the dream was short lived. It’s hard to say if this made a difference or if I just got lucky with 4 consecutive passes. 😢
Hmm you are running 4 containers - I’ll try running more than 1, maybe that’s related.
Sample size of 1
but seems to work so far: https://github.com/vuetifyjs/vuetify/actions/runs/4574802124 Looks like evan is interested in having it be part of vite itself: https://twitter.com/youyuxi/status/1641402565631025152Aaand still busted: https://github.com/vuetifyjs/vuetify/actions/runs/4576029601/jobs/8079626308 unless
process.env.CYPRESSisn’t set somehow but it seems to be locally@stnokott thanks for this - the TS issue is, in fact, a bug in TypeScript. Follow this issue: https://github.com/cypress-io/cypress/issues/26148#issuecomment-1487574243. It looks like there is a open PR in TS fixing the issue. 🤞 Looking at your CI, it seems the same test,
Settings.cy.tsxkeeps failing, and the Vite optimize step always precedds it? https://github.com/stnokott/r6-dissect-influx/actions/runs/4467180776/jobs/7846310109#step:5:271. I wonder if you add those deps tooptimizeDepsin yourvite.config, will it prebundle them and avoid the flake 🤔 ?@KaelWD thanks, I will spend some more time looking into this today.
Lol, I did not had to wait too long. Failed on first try after update cypress to 12.9.0.
✖ 1 of 191 failed (1%) 00:37 654 653 1 - -
Again after around 50% finished tests, again random one, component is just fine (it’s very basic) and it will work after rerun, it happens only on GH actions (works locally). So nothing changed 😦
Like I said before, I tried with “retries” functionality in Cypress, but it ignores it. Like you see, there is
(Attempt 1 of 2)but it never tries second time. I guess it only retries when there are assertion errors, but not general errors, which is shame 😦 I would really like to just rerun that single test, instead all queue and all CI/CD process. Any way to do that?Edit: Also it’s failing on importing
cypress/support/index.jswhich for me does not make any sense.So I can confirm it was not fixed in my case because based on #26138 it was due to vite 4.2.0, but I’m still on 4.1.1 😕 I will update Cypress anyway with that fix, and wait until it will happen again.
Okay we have a potential solution for this that should at least help prevent failures due to this issue. If anyone is willing, you can try installing a pre-release binary from my commit here.
If anyone does this, please post here and let us know if you are still seeing import failures in CI
@lmiller1990 like i described in previous comment - only CI/CD with GH actions. Locally (with exact same docker image, also on linux as host) it does not happening, not even once. But on GH it’s very random.