cypress: ChunkLoadError > Loading chunk cypress-support-file failed

We have had good results in turning off splitChunks.chunks = 'all' and leveraging splitChunks.chunks = 'initial' to avoid async chunks. However, this isn’t easy to do currently with @cypress/webpack-dev-server since the optimization option is overwritten . I created a feature request https://github.com/cypress-io/cypress/issues/29073 to allow users to configure this option. There are more details on a possible work around for this issue on the feature request and interested to see who gives it a try!

Sorry for the delay in an update. We are continuing to experiment and try a few things on our end. @anilanar’s comment of

This is a problem with async chunks triggering builds and those builds not finishing before test times out.

I think is likely the correct speculation. We are also wondering if the async chunks are being rebuilt on the fly in the dev-server cache and that is also causing more problems.

Either way, on our end we are experimenting with turning async chunking off and opting for initial chunking. The issue is @cypress/webpack-dev-server overrides this configuration for users (likely when it shouldn’t), so we needed to patch the package in order to try this.

We are gathering more data with this change. It has caused a spike in some of our resource usage, but I think we have that ironed out. Once we have more information on whether the configuration changes help alleviate the symptoms I will report back!

We’re spiking into this to see if we can find what’s causing this.

Hey all. Sorry for the delay on an update here. I’ve been investigating the issue heads down for about a week now. I’m able to reproduce the error on my end but only at the start of the run and sparingly in the middle of the run. I think there are two issues possibly going on here

At the start of the run

At the end of webpack compilation, the dev-server:compile:success event is emitted by the cypress webpack dev server, but nothing is actively listening to this event. The only exception is watchForChanges being enabled inside the SocketCT handler inside the Cypress server, which is unset in run mode and isn’t applicable here.

In other words, we can get ourselves into a race condition where webpack takes a long time to build all assets, but the cypress run has begun and is able to resolve the index.html chunk, but not other chunks in the test because they are not present in the output directory. That is what is happening in this example:

In the middle of a run

Adding to what @benpatterson mentioned and a bit more complicated to reproduce, but does happen, is webpack-dev-server recompiling in the middle of a test. We can see something like this here with some logging I added in this diagnostic binary.

Test starts. Something triggers webpack-dev-server to recompile. test fails with chunk error

Just in time recompile (just by chance) while the browser is relaunching for the second test to run. second test passes

For this, it seems like the dev-server is watching assets and recompiling. What I need to further investigate is

• what is being compiled here and does it need to be compiled? Maybe something configuration-wise changed in the webpack.config from version 4 to 5.

Waiting for the dev server to compile is a bit tricky as it can trigger a compilation at any time, but if we can answer the above question, it puts us in a better position to solve the problem correctly.

Some questions that might be helpful for us in fixing the issue

• Does the error happen at the beggining of the cypress run or in the middle? It sounds like the first case for @TomaszG where compilation fails in the beginning of the test and the second case for @benpatterson where the test fails in the middle. I am curious with others running into this issue where it is happening for you.
• What are the stats on your webpack build? Any idea how long it is taking for webpack to recompile?
• if willing, we have a diagnostic binary build here (this is not a fix) that you can install and run with the following flags CYPRESS_INTERNAL_BROWSER_CONNECT_TIMEOUT=120000 DEBUG='cypress:server:open_project,cypress:webpack-dev-server:CypressCTWebpackPlugin npx cypress run --component ... which might give us an idea what webpack is doing under your test.

I will give more updates as soon as I have them. I appreciate everyones patience with the issue resolution.

Found this in our own tests: https://cloud.cypress.io/projects/ypt4pf/runs/53776/overview/c7135bc3-5fcb-4555-8497-c7b93658a628?roarHideRunsWithDiffGroupsAndTags=1

We are getting the same error also intermittently. I don’t think we had it as far back as 12.17.4 but I can’t say that for certain.

It only seems to happen toward the end of our Component Test runs. We have 1500+ component tests that we are running.

The following error originated from your test code, not from Cypress.

  > Loading chunk cypress-support-file failed.
(error: http://localhost:8080/__cypress/src/cypress-support-file.js)

When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.

Cypress could not associate this error to any specific test.

We dynamically generated a new test to display this failure.
ChunkLoadError: The following error originated from your test code, not from Cypress.
  > Loading chunk cypress-support-file failed.
(error: http://localhost:8080/__cypress/src/cypress-support-file.js
When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.
Cypress could not associate this error to any specific test.
We dynamically generated a new test to display this failure.
    at __webpack_require__.f.j (http://localhost:8080/__cypress/src/main.js:8702:29
    at <unknown> (http://localhost:8080/__cypress/src/main.js:8552:40
    at Array.reduce (<anonymous>)
    at __webpack_require__.e (http://localhost:8080/__cypress/src/main.js:8551:67
    at Object.load (http://localhost:8080/__cypress/src/main.js:1510:582
    at <unknown> (http://localhost:8080/__cypress/runner/cypress_runner.js:110819:21
    at tryCatcher (http://localhost:8080/__cypress/runner/cypress_runner.js:1807:23
    at Object.gotValue (http://localhost:8080/__cypress/runner/cypress_runner.js:6476:18
    at Object.gotAccum (http://localhost:8080/__cypress/runner/cypress_runner.js:6465:25
    at Object.tryCatcher (http://localhost:8080/__cypress/runner/cypress_runner.js:1807:23
[Open in build log](https://teamcity.haivision.com/buildConfiguration/CineNet4x_Projects_CypressComponents/690401?hideProblemsFromDependencies=false&expandBuildTestsSection=true&hideTestsFromDependencies=false&expandBuildChangesSection=true&showLog=690401_2000000247_2000000247&logFilter=debug)

Just an update after 2 weeks: we don’t have ChunkLoadErrors anymore with a custom “dev server” that serves prebundled static files.

We made an update from webpack 4 to webpack 5 in 12.17.4, I wonder if it has anything to do with that upgrade. https://docs.cypress.io/guides/references/changelog#12-17-4

We noticed that service workers are leaking memory unless cypress is ran with experimentalSingleTabRunMode: true config.

We noticed that we get ChunkLoadErrors whenever CI machines run out of memory.

We don’t know what’s causing cypress to leak memory and whether if that’s on our side or not. We use msw, therefore there’s a chance that the service worker is leaking memory. Nevertheless, memory usage keeps increasing with no boundary and eventually hits 100% and crashes regardless of amount of memory, unless tests finish before being able to hit 100%.

We have been running into this for a while now in our component tests. Specifically with the monaco editor loading web workers.

Loading chunk vendors-node_modules_yaml_browser_index_js failed.

Our fix is to pin to Cypress version 13.5.1.

Cypress 13.5.1 works consistently. Upgrading to new versions after that, up to version 13.6.6 give us random failures in our component tests.

Looking forward to a fix.

Update: Still seeing the issue sometimes on version 13.5.1

Hey all. Just a few updates.

Mid recompilation with webpack-dev-server has stopped with the update from v3 to v4, which is great! However, we are still hitting ChunkLoadErrors 🙁 .

This issue has been extremely challenging to reproduce, but we have a few theories. My guess is that there are chunk loading issues associated with the main bundle mainly due to us bundling every component test, regardless of what is used under test. The only exception to this is if the --spec option is leveraged to pass in specs. For example:

• if cypress open is used to to develop a single component test and there are 39 other component tests, ALL 40 component tests are compiled when the app is launched.
• If cypress run is used to test all specs, ALL component tests are compiled before the tests are initialized.
• If cypress run is used to test multiple specs via the --parallel flag in CI, ALL component tests are compiled before the tests are initialized, regardless of what tests are run.
• If cypress run is used to test a single spec via the spec argument, ONE single spec is compiled before the test is run.
• If cypress run is used to test multiple specs via the spec argument, ONLY THE SPECS PASSED IN are compiled before the test is run.

The larger N is in this case (our example above was 40), the worse the dev server scales as far as building is concerned. Each spec is considered its own entry point in webpack. We don’t have proof that reducing the bundle size would fix the chunk errors, but I think this is in large part of the problem as this seems to happen mostly in projects where there are dozens of CT tests.

In most cases, even if the build takes minutes, cypress waits for the dev server to return a response before executing tests. @anilanar In other words, we are pre-bundling everything needed under test already but the assets are not loaded into the browser all at once, which might explain the service worker issue you are running into and I’m glad you were able to find a solution! Something is causing the chunk to fail to load meaning it either doesn’t exist, or it timed out loading. Since the assets are held in memory, it is difficult to figure out which has occurred. Either way, I am not convinced flaky tests are due to builds taking too long.

What we are trying internally with one of our projects where this error happens frequently, is to leverage the --spec option to group our tests under execution and use circleci to split the tests per container (@cwillsea-mtg this might be something you can try depending on your setup and would be great feedback for us if it lessens or resolve the issue for you). This will only bundle the assets needed under test, reducing the chunk and bundle sizes under test and likely the stress to load them into memory. This could give us direction on if we need to change some aspects of the dev server depending on whether ChunkLoadErrors continue to surface or not and our ability to tune them. I will report back when I gather more data/information.

My team is using the default dev server that comes with cypress:

  component: {
    devServer: {
      bundler: 'webpack',
      framework: 'next',
    },
  },

The errors seem to always happen at the start of a test file, but pop up in random places in our test suite. For example, here is a recent failure in the 73th test file out of a suite of 90 where the other 89 runs all passed (edited to remove component/file names):

  Running:  components/COMPONENT_NAME.cy.tsx                          (73 of 90)


  1) An uncaught error was detected outside of a test

  0 passing (665ms)
  1 failing

  1) An uncaught error was detected outside of a test:
     ChunkLoadError: The following error originated from your test code, not from Cypress.

  > Loading chunk vendors-node_modules_SOME_STRING.js failed.
(error: http://localhost:8080/__cypress/src/vendors-SOME_STRING.js)

When Cypress detects uncaught errors originating from your test code it will automatically fail the current test.

Cypress could not associate this error to any specific test.

We dynamically generated a new test to display this failure.
      at __webpack_require__.f.j (http://localhost:8080/__cypress/src/webpack.js:340:29)
      at <unknown> (http://localhost:8080/__cypress/src/webpack.js:137:40)
      at Array.reduce (<anonymous>)
      at __webpack_require__.e (http://localhost:8080/__cypress/src/webpack.js:136:67)
      at Object.load (http://localhost:8080/__cypress/src/main.js:485:577)
      at <unknown> (http://localhost:8080/__cypress/runner/cypress_runner.js:110819:21)
      at tryCatcher (http://localhost:8080/__cypress/runner/cypress_runner.js:1807:23)
      at Object.gotValue (http://localhost:8080/__cypress/runner/cypress_runner.js:6476:18)
      at Object.gotAccum (http://localhost:8080/__cypress/runner/cypress_runner.js:6465:25)
      at Object.tryCatcher (http://localhost:8080/__cypress/runner/cypress_runner.js:1807:23)


  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        1                                                                                │
  │ Passing:      0                                                                                │
  │ Failing:      1                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  1                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     0 seconds                                                                        │
  │ Spec Ran:     components/COMPONENT_NAME.cy.tsx                             │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

The frequency with which these errors occur has been increasing as we add more test files. If there’s any debug/diagnostic config we can set to get more info happy to try as this issue is happening frequently in our CI

@AtofStryker, great investigation and summary. I can confirm that the error happens only at the beginning of the run for me; I haven’t noticed it in the middle. I also haven’t noticed any webpack re-compilation in the middle of the run.

The compilation times on CI are around 19 seconds for the first run (which is allowed to fail) and 14 seconds for the second run.

I’m leaving behind some notes after digging into this further.

We are experiencing this error on ~9% of our component test builds. (I can’t remember offhand how many tests that entails.)

Indeed error messages above are consistent with what we see, here’s an excerpt (with a slight edit on chunk name tbh):

  1) An uncaught error was detected outside of a test:
     ChunkLoadError: The following error originated from your test code, not from Cypress.

  > Loading chunk name1-name2-file~spec-11~spec-14~spec-15~spec-17~spec-22~spec-23~spec-25~spec-27~spec-28~43d59dbf failed.

It includes the localhost path as well on the next line (which I left out here).

Noteworthy…going up a few lines in the build, we are seeing something like this (I changed more file names out of caution):

  Running:  dir/myComponentSpec.spec.tsx                                                  
  Estimated: 1 second


ℹ ｢wdm｣: Compiling...
  (Attempt 1 of 3) An uncaught error was detected outside of a test
ℹ ｢wdm｣: wait until bundle finished: /<root+directory>/related-file~spec-34.js
ℹ ｢wdm｣: wait until bundle finished: /<root+directory>/related-file.js
  (Attempt 2 of 3) An uncaught error was detected outside of a test
  1) An uncaught error was detected outside of a test

It appears to me that the ChunkLoadError occurs because the next test is running before webpack compilation is completed on the devserver. I think this includes related CSS compilations, but honestly I don’t know if that’s all the time or a subset of examples.

Ultimately I’m convinced the next test needs to wait until compilation completes and the devserver has updated the cache. I attempted to set devserver settings around hot reload, etc, but those didn’t work – likely because the test is still trying to serve the page and is not waiting for the devserver to signal it’s up to date.

When it comes to waiting for compilation to complete, we do have logic around this in CypressCTWebpackPlugin in this function. As-written, it’s awaiting specs, and this is why I’m wondering if we need something to wait for related assets, too. Regardless, I think we need a way of ensuring all compilation completes, including any styling, before proceeding to the next test.

There had been a previous PR that leveraged the webpack devserver’s index HTML file, which I believe would yield the result I’m thinking of (change here). That PR was intended to fix a situation specific to MacOS version, but I’m optimistic it could resolve the ChunkLoadError. The PR was merged but the code in the file today is older than the PR, so I don’t know if it was rolled back at a later time.

We have been running into this sporadically when Component testing after migrating from CY 12.x to 13.6.2. Similarly to cases above, it happens towards the end of running a batch of suites (files) together.

Resorted to running cypress with DEBUG='cypress:*'. This is an instance of a crash:

 cypress: server: reporter got mocha event 'test'
with args: [u {
    title: 'An uncaught error was detected outside of a test',
    fn: [Function: o] {
        toString: [Function(anonymous)]
    },
    body: '() => {\n        throw err;\n      }',
    async: undefined,
    sync: undefined,
    _timeout: 2000,
    _slow: 250,
    _enableTimeouts: true,
    timedOut: undefined,
    _retries: -1,
    _currentRetry: 0,
    pending: false,
    type: 'test',
    duration: undefined,
    state: 'skipped',
    parent: p {
        title: '',
        ctx: {},
        suites: [],
        tests: [Array],
        pending: false,
        _beforeEach: [],
        _beforeAll: [],
        _afterEach: [],
        _afterAll: [],
        root: true,
        _timeout: 2000,
        _enableTimeouts: true,
        _slow: 250,
        _bail: false,
        _retries: -1,
        _onlyTests: [],
        _onlySuites: [],
        delayed: false,
        _events: [Object: null prototype],
        _eventsCount: 1,
        file: 'VALID_PATH_TO_FILE_DEDACTED',
        id: 'r1',
        type: 'suite'
    },
    id: 'r2',
    _testConfig: {
        testConfigList: [],
        unverifiedTestConfig: {}
    },
    order: 1,
    invocationDetails: {
        function: 'Object.getInvocationDetails',
        fileUrl: 'http://localhost:8080/__cypress/runner/cypress_runner.js',
        originalFile: 'http://localhost:8080/__cypress/runner/cypress_runner.js',
        line: 97377,
        column: 17,
        whitespace: '    ',
        stack: 'Error\n' + '    at Object.getInvocationDetails 
(http://localhost:8080/__cypress/runner/cypress_runner.js:97377:17)\n' + '    at Suite.addTest 
(http://localhost:8080/__cypress/runner/cypress_runner.js:145737:85)\n' + '    at Object.createRootTest 
(http://localhost:8080/__cypress/runner/cypress_runner.js:145872:21)\n' + '    at Object.normalizeAll 
(http://localhost:8080/__cypress/runner/cypress_runner.js:163115:17)\n' + '    at CDPBrowserSocket.<anonymous> 
(http://localhost:8080/__/assets/index-caf2649c.js:146264:46)\n' + '    at CDPBrowserSocket.on2 
(http://localhost:8080/__/assets/index-caf2649c.js:34575:11)\n' + '    at Emitter2.emit 
(http://localhost:8080/__/assets/index-caf2649c.js:34616:24)\n' + '    at http://localhost:8080/__/assets/index-caf2649c.js:37709:15'

    }
}] + 9 ms

There is a lot in the logs that makes it hard to share but please do let us know if we should be looking for or can share anything useful to help unraveling this.