cypress: ChunkLoadError > Loading chunk cypress-support-file failed
Current behavior
After updating from cypress@12.16.0 to cypress@13.6.2, we are seeing this error occasionally on our CI runs.
ChunkLoadError: The following error originated from your test code, not from Cypress.
> Loading chunk cypress-support-file failed.
[(error: http://localhost:8080/__cypress/src/cypress-support-file.js)]
When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.
Cypress could not associate this error to any specific test.
We dynamically generated a new test to display this failure.
[at __webpack_require__.f.j (http://localhost:8080/__cypress/src/runtime.js:272:29)]
[at <unknown> (http://localhost:8080/__cypress/src/runtime.js:126:40)]
at Array.reduce (<anonymous>)
[at __webpack_require__.e (http://localhost:8080/__cypress/src/runtime.js:125:67)]
[at Object.load (http://localhost:8080/__cypress/src/cypress-entry.js:33:192)]
[at <unknown> (http://localhost:8080/__cypress/runner/cypress_runner.js:110819:21)]
[at tryCatcher (http://localhost:8080/__cypress/runner/cypress_runner.js:1807:23)]
[at Object.gotValue (http://localhost:8080/__cypress/runner/cypress_runner.js:6476:18)]
[at Object.gotAccum (http://localhost:8080/__cypress/runner/cypress_runner.js:6465:25)]
[at Object.tryCatcher (http://localhost:8080/__cypress/runner/cypress_runner.js:1807:23)]
Unfortunately it is not reproducable locally, so I can not give you a repository to check. I know that this might make it hard to investigate but maybe somebody already fell over the same issue or others are experiencing it as well.
The test fails around 30% of the time.
Desired behavior
No response
Test code to reproduce
no reproduction repository
Cypress Version
13.6.2
Node version
18.19.0
Operating System
Ubuntu 22.04.3
Debug Logs
No response
Other
No response
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 37
- Comments: 147 (68 by maintainers)
Any progress on the possible fix (by providing configuration options) according to #29073 ? We still encounter this issue quite often and a (possible) fix would be highly appreciated 🤗
We’re planning to set aside time over the next few months to try to find a resolution to this. Hopefully nothing pulls those plans off track.
We have had good results in turning off
splitChunks.chunks = 'all'and leveragingsplitChunks.chunks = 'initial'to avoid async chunks. However, this isn’t easy to do currently with@cypress/webpack-dev-serversince theoptimizationoption is overwritten . I created a feature request https://github.com/cypress-io/cypress/issues/29073 to allow users to configure this option. There are more details on a possible work around for this issue on the feature request and interested to see who gives it a try!We’re spiking into this to see if we can find what’s causing this.
@mirobo We intend to release this as an experimental feature to enable with a newer Cypress version. Hopefully the next release if all goes well.
Hey all. I have been working on a solution to the chunk load errors, in particular the just-in-time compiling solution proposed in https://github.com/cypress-io/cypress/issues/29244. Effectively, the experimental flag
experimentalJustInTimeCompilewill only compile assets that are currently under test (the current spec) as opposed to all related CT specs in the project. The idea here is less entries should mean smaller dev server which in some cases might mean smaller chunks. We have seen with our own testing that memory usage is much lower (in our case about half the memory used) and we have not run into chuck load errors in testing.However, since this issue is unique in the fact that everyones set up is different, I would like to see if this experiment alleviates the problem for you all and understand possible cases where it may not. I have some pre release binaries build on this commit (note: this is based on 13.13.3).
In particular I am curious about the
cypress-support-filechunk failing, since I don’t believe this chunk really can get much smaller for users since the support file is somewhat constant. As we get closer to releasing the feature I will post updates here, but the prerelease is available if anyone wants to give it a try! You can turn it on like this:Sorry for the delay in an update. We are continuing to experiment and try a few things on our end. @anilanar’s comment of
I think is likely the correct speculation. We are also wondering if the async chunks are being rebuilt on the fly in the dev-server cache and that is also causing more problems.
Either way, on our end we are experimenting with turning async chunking off and opting for
initialchunking. The issue is @cypress/webpack-dev-server overrides this configuration for users (likely when it shouldn’t), so we needed to patch the package in order to try this.We are gathering more data with this change. It has caused a spike in some of our resource usage, but I think we have that ironed out. Once we have more information on whether the configuration changes help alleviate the symptoms I will report back!
Hey all. Sorry for the delay on an update here. I’ve been investigating the issue heads down for about a week now. I’m able to reproduce the error on my end but only at the start of the run and sparingly in the middle of the run. I think there are two issues possibly going on here
At the start of the run
At the end of
webpackcompilation, thedev-server:compile:successevent is emitted by the cypress webpack dev server, but nothing is actively listening to this event. The only exception iswatchForChangesbeing enabled inside the SocketCT handler inside the Cypress server, which is unset in run mode and isn’t applicable here.In other words, we can get ourselves into a race condition where
webpacktakes a long time to build all assets, but the cypress run has begun and is able to resolve theindex.htmlchunk, but not other chunks in the test because they are not present in the output directory. That is what is happening in this example:In the middle of a run
Adding to what @benpatterson mentioned and a bit more complicated to reproduce, but does happen, is
webpack-dev-serverrecompiling in the middle of a test. We can see something like this here with some logging I added in this diagnostic binary.Test starts. Something triggers
webpack-dev-serverto recompile. test fails with chunk errorJust in time recompile (just by chance) while the browser is relaunching for the second test to run. second test passes
For this, it seems like the dev-server is watching assets and recompiling. What I need to further investigate is
webpack.configfrom version 4 to 5.Waiting for the dev server to compile is a bit tricky as it can trigger a compilation at any time, but if we can answer the above question, it puts us in a better position to solve the problem correctly.
Some questions that might be helpful for us in fixing the issue
CYPRESS_INTERNAL_BROWSER_CONNECT_TIMEOUT=120000 DEBUG='cypress:server:open_project,cypress:webpack-dev-server:CypressCTWebpackPlugin npx cypress run --component ...which might give us an idea what webpack is doing under your test.I will give more updates as soon as I have them. I appreciate everyones patience with the issue resolution.
It’s a big chunk of work, where we do understand the problem well now. We’re prioritizing some of the support for new component testing frameworks atm since that’s quicker work to get out and hoping to get to this work after.
Just an update after 2 weeks: we don’t have ChunkLoadErrors anymore with a custom “dev server” that serves prebundled static files.
Found this in our own tests: https://cloud.cypress.io/projects/ypt4pf/runs/53776/overview/c7135bc3-5fcb-4555-8497-c7b93658a628?roarHideRunsWithDiffGroupsAndTags=1
Closing as this will be considered done with Cypress 14 upcoming release
This makes it difficult to depend on Cypress component tests for validation in CI. These chunk failures are making CI flakey. If we could get some official focus on this, that’d be appreciated. It’s a real issue right now for us early adopters of Cypress component tests.
We are getting the same error also intermittently. I don’t think we had it as far back as 12.17.4 but I can’t say that for certain.
It only seems to happen toward the end of our Component Test runs. We have 1500+ component tests that we are running.
I’m confused as to why this was closed? The flag “experimentalJustInTimeCompile” made things worse for us and I wonder what solution is going to come with Cypress 14?
@jennifer-shehane you’re prioritizing adding support for new component testing frameworks? That sucks. This is a pretty big bug in an existing component testing framework. It’s impactful for companies doubling-down on Cypress component tests (like us). Can’t supporting existing users be a higher priority?
Any updates? This is blocking our upgrade to Cypress v13.
We’re seeing this on 13.2
We ended up reverting and pinning to
13.5.1to avoid the crashes. It’s not a solid perm fix since we eventually will need to upgrade and pull in other critical bug fixes, but helps us keep our tests running.Adding these event handlers to the support / component file got our CI passing reliably
Yes, we do run the tests in parallel and component tests run takes significantly longer, especially on CI. Given that webpack bundle doesn’t change during the test execution, I don’t see a reason of involving webpack bundle for each spec, as that’s a major performance drawback, especially when testing complex enterprise apps. I recon that a simple wait (even static if there’s no milestone on which you could rely) for webpack to finish or a one-time retry in case of ChunkLoadError would make the error gone for most of the users.
No more errors 🤞👏 Stability is more important than speed. I’ll stick to that setting for now. I definitely think performance should be improved in the future as i dont want to move to playwright for now
seeing a decrease in performance after applying this flag experimentalJustInTimeCompile AFTER:4m 40s BEFORE:3m 5s
This looks very promising. 4 different smoke tests in CI are working correctly. We will continue to monitor the behavior. Thanks for the release so far
The experimentalJustInTimeCompile feature released today with Cypress
13.14.0. Please give this a try and give us your feedback here or in this github discussion on whether the experiment improves the chunk load error outcomes.Any news on this, we are still pinned on v12 until this can be fixed?
@AtofStryker, great investigation and summary. I can confirm that the error happens only at the beginning of the run for me; I haven’t noticed it in the middle. I also haven’t noticed any webpack re-compilation in the middle of the run.
The compilation times on CI are around 19 seconds for the first run (which is allowed to fail) and 14 seconds for the second run.
We made an update from webpack 4 to webpack 5 in 12.17.4, I wonder if it has anything to do with that upgrade. https://docs.cypress.io/guides/references/changelog#12-17-4
Hi, I’m currently seeing this issue on my own project, but instead with a random chunk (not the support file) like #30521 and other posts above.
I don’t have much to contribute to the conversation, but I know my specific issue isn’t rooted in chrome since we don’t use it for running. We do use webpack
I appreciate the work looking into this and look forward to when this issue can be reopened. Thanks.
we are still experiencing this issue quite a lot and it causes PR pipelines to fail - even multiple times in a row. @jennifer-shehane - as multiple other people above still run into the same issue, can you please re-open it and give us a status?
we tried updating to 14.x using default settings
mitigations we tried
I get that webpack is theoretically not a hard requirement (mentioned that in a comment earlier) except that it is because without it the mocking/stubing functionality is not working with Vite due to the nature of esm. At least that was a problem a few months ago when I last tried to migrate away from using the webpack dev server. That’s actually the only reason why we still have webpack in our project - to support Cypress component tests. The app itself is built with Vite. If anything changed in that regard, I’m happy to try migrating again.
I would not rule out a memory leak somewhere – the project consists of almost 2000 Vue components and a similar number of ts and js files. But investigating that is not an easy task as you might imagine.
Don’t get me wrong, I’ve been a proponent of Cypress pretty much since its inception. But the amount of frustration it has been generating for our team over the last years is beginning to be hard to ignore. Of course, a lot of that might come from the pains of a rapidly growing codebase. So please do take my rant with a grain of salt. 😅
edit: we are also using the code coverage plugin for component tests which is contributing to the memory consumption, though I haven’t measured exactly by how much.
@AtofStryker That’s something that CI providers often provide out of the box (load balancing tests based on timings). It might help with the overall time, but it does nothing to help with the budget side of things. Throwing more machines at the problem should not be the only solution to a performance problem like this. Even if it is the simplest way to go around it, compute is ultimately not free. Unless of course, Cypress starts offering alternative hosted CI runners where clients can pay on a per test basis rather than per compute minutes.
Bill and I found that when intending to upgrade to v13.15.0, I must have typed in 13.5.0, that is whay it seemed like the ChunkLoadError has come back. We used 13.15.0 and the error went away with the flag
experimentalJustInTimeCompileHowever, we uncovered a new issue where time to time, the last component test does not crash, but hangs there indefinately.
What kind of logs or debugging can I give to aid with diagnosing this one?
@AtofStryker
regarding
experimentalJustInTimeCompileWe do see it in console.log, but we do not actually see it in the runner, there is no entry for it anymore, perhaps since a recent version.I sent you a link of the zipped repo on Discord, if you can log in within 24 hours.
I am sending the zipped repo to support right away.
Worst case scenario, is there a way we can flag the ChunLoadError a false positive, and not have things fail but perhaps just skip when we see it?
https://files.slack.com/files-pri/T047EH9G6G2-F07RK079PV0/image.png
@jennifer-shehane @AtofStryker
I have reported previously that
experimentalJustInTimeCompilemade things better for the chunkLoad error.However, as the component suite grows, we still sporadically see it, with a failure rate of 16% (the Cypress Cloud link should be accessible by Cypress employees).
We are somewhat desperate for a solution, as we are vested into component testing with over 500 files/1600 it blocks.
Unless there is a solution, the only plausible way forward for us could be to execute tests based on source code changes. We would be missing a lot of issues, but we would not be alerted falsely (and cost 1600+ runs from our credit) at a 16% rate.
Any updates? This problem makes components testing completely unusable.
I came across this issue today using Cypress 13.11.0. Not sure if this will help others, but for my particular test this was caused by me trying to set a env variable in the test to a dummy value in the beforeEach().
The error went away when I commented out the variable in my .env.local file. Therefore, I updated the test to check if the variable is already set before trying to set it in the beforeEach().
I think memory problems are caused by either source maps (
webpackConfig.devtools) or webpack cache (webpackConfig.cache) or both.I have a feeling chunk load errors happen due to swallowed OOM errors though. Did people try using high
NODE_OPTIONS=--max-old-space-size? I noticed we started getting chunk load errors more often when I reduced CI machine size to a lower RAM class.Nevertheless, I’ve given up on running a “dev server” on CI because that doesn’t make sense to me as I previously expressed. Why run a long running build process and be prone to potential memory errors and harder to debug problems? Webpack dev server stores all generated assets in memory with a lot of overhead due to internal stats. Running webpack build and then serving static files from a simple static http server is much simpler, or it would’ve been simpler if cypress didn’t pass spec name with a query parameter/header. Even then, serving them with an express server isn’t too much trouble. Still, I wish cypress team took this path instead of potentially fighting an uphill battle.
So my implementation of a “custom dev server” still uses high memory during build but significantly reduced memory usage during test run, so I can even use different machine sizes for building and for running and save $$$.
Example custom dev server:
This requires running webpack beforehand with a config like:
Good luck everyone!
We noticed that service workers are leaking memory unless cypress is ran with
experimentalSingleTabRunMode: trueconfig.We noticed that we get
ChunkLoadErrors whenever CI machines run out of memory.We don’t know what’s causing cypress to leak memory and whether if that’s on our side or not. We use
msw, therefore there’s a chance that the service worker is leaking memory. Nevertheless, memory usage keeps increasing with no boundary and eventually hits 100% and crashes regardless of amount of memory, unless tests finish before being able to hit 100%.We don’t use React nor “monaco” (we use Angular) and we have the same issue with component tests with the ChunkLoadError… 🤔
We have been running into this for a while now in our component tests. Specifically with the monaco editor loading web workers.
Our fix is to pin to Cypress version 13.5.1.
Cypress 13.5.1 works consistently. Upgrading to new versions after that, up to version 13.6.6 give us random failures in our component tests.
Looking forward to a fix.
Update: Still seeing the issue sometimes on version 13.5.1
Try this workaround, as I previously suggested:
If other async chunks are causing middle-of-the-run builds, then try using https://webpack.js.org/plugins/limit-chunk-count-plugin/ and limit your chunk count to 1. Or try turning off chunk splitting, see https://webpack.js.org/plugins/split-chunks-plugin/#optimizationsplitchunks .
Again, I doubt this is a misconfiguration in webpack dev server. Async chunks are normal things that optimize initial bundle size by lazy loading things later. They are usually caused by async imports, but if you don’t have an async import on your side or if you don’t know what that even means, then it must be used by one of the libraries you use.
This is a problem with async chunks triggering builds and those builds not finishing before test times out.
The solution is one of:
Edit: typo
Hey all. Just a few updates.
Mid recompilation with
webpack-dev-serverhas stopped with the update fromv3tov4, which is great! However, we are still hittingChunkLoadErrors 🙁 .This issue has been extremely challenging to reproduce, but we have a few theories. My guess is that there are chunk loading issues associated with the main bundle mainly due to us bundling every component test, regardless of what is used under test. The only exception to this is if the
--specoption is leveraged to pass in specs. For example:cypress openis used to to develop a single component test and there are 39 other component tests, ALL 40 component tests are compiled when the app is launched.cypress runis used to test all specs, ALL component tests are compiled before the tests are initialized.cypress runis used to test multiple specs via the--parallelflag in CI, ALL component tests are compiled before the tests are initialized, regardless of what tests are run.cypress runis used to test a single spec via the spec argument, ONE single spec is compiled before the test is run.cypress runis used to test multiple specs via the spec argument, ONLY THE SPECS PASSED IN are compiled before the test is run.The larger
Nis in this case (our example above was 40), the worse the dev server scales as far as building is concerned. Each spec is considered its own entry point in webpack. We don’t have proof that reducing the bundle size would fix the chunk errors, but I think this is in large part of the problem as this seems to happen mostly in projects where there are dozens of CT tests.In most cases, even if the build takes minutes, cypress waits for the dev server to return a response before executing tests. @anilanar In other words, we are pre-bundling everything needed under test already but the assets are not loaded into the browser all at once, which might explain the service worker issue you are running into and I’m glad you were able to find a solution! Something is causing the chunk to fail to load meaning it either doesn’t exist, or it timed out loading. Since the assets are held in memory, it is difficult to figure out which has occurred. Either way, I am not convinced flaky tests are due to builds taking too long.
What we are trying internally with one of our projects where this error happens frequently, is to leverage the
--specoption to group our tests under execution and use circleci to split the tests per container (@cwillsea-mtg this might be something you can try depending on your setup and would be great feedback for us if it lessens or resolve the issue for you). This will only bundle the assets needed under test, reducing the chunk and bundle sizes under test and likely the stress to load them into memory. This could give us direction on if we need to change some aspects of the dev server depending on whetherChunkLoadErrors continue to surface or not and our ability to tune them. I will report back when I gather more data/information.My team is using the default dev server that comes with cypress:
The errors seem to always happen at the start of a test file, but pop up in random places in our test suite. For example, here is a recent failure in the 73th test file out of a suite of 90 where the other 89 runs all passed (edited to remove component/file names):
The frequency with which these errors occur has been increasing as we add more test files. If there’s any debug/diagnostic config we can set to get more info happy to try as this issue is happening frequently in our CI
I’m leaving behind some notes after digging into this further.
We are experiencing this error on ~9% of our component test builds. (I can’t remember offhand how many tests that entails.)
Indeed error messages above are consistent with what we see, here’s an excerpt (with a slight edit on chunk name tbh):
It includes the localhost path as well on the next line (which I left out here).
Noteworthy…going up a few lines in the build, we are seeing something like this (I changed more file names out of caution):
It appears to me that the
ChunkLoadErroroccurs because the next test is running before webpack compilation is completed on the devserver. I think this includes related CSS compilations, but honestly I don’t know if that’s all the time or a subset of examples.Ultimately I’m convinced the next test needs to wait until compilation completes and the devserver has updated the cache. I attempted to set devserver settings around hot reload, etc, but those didn’t work – likely because the test is still trying to serve the page and is not waiting for the devserver to signal it’s up to date.
When it comes to waiting for compilation to complete, we do have logic around this in
CypressCTWebpackPluginin this function. As-written, it’s awaiting specs, and this is why I’m wondering if we need something to wait for related assets, too. Regardless, I think we need a way of ensuring all compilation completes, including any styling, before proceeding to the next test.There had been a previous PR that leveraged the webpack devserver’s index HTML file, which I believe would yield the result I’m thinking of (change here). That PR was intended to fix a situation specific to MacOS version, but I’m optimistic it could resolve the ChunkLoadError. The PR was merged but the code in the file today is older than the PR, so I don’t know if it was rolled back at a later time.
We have been running into this sporadically when Component testing after migrating from CY 12.x to 13.6.2. Similarly to cases above, it happens towards the end of running a batch of suites (files) together.
Resorted to running cypress with
DEBUG='cypress:*'. This is an instance of a crash:There is a lot in the logs that makes it hard to share but please do let us know if we should be looking for or can share anything useful to help unraveling this.