cypress: Cypress is running really slow after upgrading to v10

Thought I would just share a finding - when running my tests with cypress set to Chrome 113, they ran extremely slow. When I switched to Electron 100, it ran as fast as it used to when Chrome was selected. It’s possible something changed in chrome that is slowing cypress down.

My fix is to use Electron 100 instead of Chrome. But I suspect when Electron gets to 113 it may become slow as well. I would suggest the Cypress team try to find the underlying issue here.

@inorganik I had a similar issue, the problem I was having was chrome was running in emulation mode (I had M1 but was a darwin binary, I think - or something along these lines). Not sure if that’s an issue you might be running into, but sharing in case it helps anyone else.

Thought I would just share a finding - when running my tests with cypress set to Chrome 113, they ran extremely slow. When I switched to Electron 100, it ran as fast as it used to when Chrome was selected. It’s possible something changed in chrome that is slowing cypress down.

I’m having the same problem actually. I’m using v9.7.0 (can’t upgrade easily right now, hence the older version). Chrome 113 and Edge 113 are running horribly. Very laggy, ever since upgrading to 113 (112 was fine). Same issue either in open or run mode. macOS 12.6.5 M1

As you can see from this quick video, the duration clock is lagging quite a lot… https://github.com/cypress-io/cypress/assets/16207039/eba30235-6155-4ebf-9743-f2ee53555465

It’s not just you, I am able to reproduce (finally). It’s not a minimal reproduction, but we are getting there. This data is the same as the videos I posted above, but I ran it on CI, which is plain old Ubuntu. It’s on GitHub Actions.

Notes:

• More apparently on CI (almost twice as slow)
• Reproduced by loading 200 JS modules
• See here
• I am NOT using a dev server - this is the result of yarn vite build.

Numbers

Cypress 9

Done in 13.51s.                                                                                                  

real    0m13.656s
user    0m8.576s
sys     0m1.459s


Cypress 10

Done in 24.01s.                                                                                                  

real    0m24.185s
user    0m9.431s
sys     0m2.132s

I am still finding a minimal reproduction so we can actually FIX this, but it’s pretty clear to me that, at least, loading lots of JS modules (many lots of <files>, I’m not sure yet) is significantly slower between Cypress 9 and Cypress 10.

Please wait a bit more while I debug this, I will fix it.

Seems like updating to Cypress 12.3 from 11.2 (with wiping “node_modules” folder) helped me to restore test perfomance on Windows distro (I compared it with WSL distro on the same machine). Thanks!

Hello all, I’m still debugging this. I am chipping away this, bit by bit. In this reproduction, you can see that after exactly 138 JS module requests, it goes slow. The JS modules appear to be loaded in chunks of 6. It loads 6, waits for a (0.5s?) and then loads some more. This seems like a reproduction in line with what @VictorGosse is encountering.

Based on the below, it looks like it’s not between Cy 9.7 (last version of Cy 9) and Cy 10.0.0 (first version of Cy 10) but a later regression. This also lines up with the OP:

We were on 9.5.4. Then did the upgrade to the latest version of cypress 10.3.1.

So, assuming this is the same bug for everyone involved, and there’s only one bug, it was somewhere between 9.7.0 and 10.3.1.

I had to turn off cache to reproduce this. Makes sense - if the modules do not go through the Cypress network layer, they don’t go through the bottleneck.

I will now try to find the commit that introduced this.

• 10.0.0 - ok
• 10.2.0 - ok
• ??? somewhere in here is not ok
• 10.3.0 - not ok
• 10.4.0 - not ok
• 10.5.0 - not ok

Must be in here:

git log --pretty=oneline v10.2.0...v10.3.0 tags/v10.2.0

Edit: yes, here is the offending commit! 172c36402127acbea0473c93760cc6435ccad471. The PR in question: https://github.com/cypress-io/cypress/pull/22462

@kylemh – I’m having the same issue on M1. You can confirm in the Chrome About page.

@inorganik I had a similar issue, the problem I was having was chrome was running in emulation mode (I had M1 but was a darwin binary, I think - or something along these lines). Not sure if that’s an issue you might be running into, but sharing in case it helps anyone else.

@lmiller1990 how do I check if this is an issue and how do I resolve it? Feels very much like tha problem for me.

Basically seeing the same as others where Firefox and Electron take seconds, but Cypress crawls

After a certain number of requests this starts happening:

https://github.com/cypress-io/cypress/blob/9bc3715c05c8ae021c478454e7cb419218701b28/packages/proxy/lib/http/util/prerequests.ts#L150

[cy:open:dev:576967]: GET /assets/mod-130.4b328c39.js 200 5.620 ms - -
[cy:open:dev:576967]: GET /assets/mod-131.20fe225b.js 200 6.070 ms - -
[cy:open:dev:576967]: GET /assets/mod-132.0411856e.js 200 10.253 ms - -
[cy:open:dev:576967]: GET /assets/mod-133.0ab33992.js 200 7.729 ms - -
[cy:open:dev:576967]: GET /assets/mod-134.37cfe664.js 200 6.695 ms - -
[cy:open:dev:576967]: GET /rune_delta.png 200 1041.509 ms - -
[cy:open:dev:576967]: Never received pre-request for request GET-http://localhost:5566/assets/mod-135.43411b4c.js after waiting 500ms. Continuing without one.
[cy:open:dev:576967]: ->>>>> pending
[cy:open:dev:576967]: GET /assets/mod-135.43411b4c.js 200 507.778 ms - -
[cy:open:dev:576967]: Never received pre-request for request GET-http://localhost:5566/assets/mod-136.99fc4cd4.js after waiting 500ms. Continuing without one.
[cy:open:dev:576967]: ->>>>> pending
[cy:open:dev:576967]: Never received pre-request for request GET-http://localhost:5566/assets/mod-137.4841f7c7.js after waiting 500ms. Continuing without one.

I’ll find out more.

I found the point where the regression was introduced, here it is:

172c36402127acbea0473c93760cc6435ccad471. The PR in question: https://github.com/cypress-io/cypress/pull/22462

I will find out more about what that PR fixed and if there’s another way to handle that fix that doesn’t impact performance.

Ok team, I think I had some luck reproducing this. I will record and get some numbers, will update the post as I go.

Edit: run mode similar on my local machine (linux, beefy 32GB ram, beast CPU) but slower on CI (see here).

Cypress 9 (Run one spec) - 4.71s user 0.95s system 55% cpu 10.268 total

warning package.json: No license field
$ /home/lachlan/code/dump/rhythm/node_modules/.bin/cypress run --config video=false

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        9.7.0                                                                          │
  │ Browser:        Electron 100 (headless)                                                        │
  │ Node Version:   v16.17.0 (/home/lachlan/.nvm/versions/node/v16.17.0/bin/node)                  │
  │ Specs:          1 found (auth.cy.ts)                                                           │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────
                                                                                                    
  Running:  auth.cy.ts                                                                      (1 of 1)


  authentication flow
    ✓ signs up (7033ms)


  1 passing (7s)


  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        1                                                                                │
  │ Passing:      1                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     7 seconds                                                                        │
  │ Spec Ran:     auth.cy.ts                                                                       │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


====================================================================================================

  (Run Finished)


       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  auth.cy.ts                               00:07        1        1        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        00:07        1        1        -        -        -  

Done in 10.18s.
yarn cypress run --config video=false  4.71s user 0.95s system 55% cpu 10.268 total

Cypress 10 (Run one spec) - 4.38s user 1.15s system 46% cpu 11.858 total

yarn run v1.22.11
warning package.json: No license field
$ /home/lachlan/code/dump/rhythm/node_modules/.bin/cypress run --config video=false

====================================================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress:        11.0.1                                                                         │
  │ Browser:        Electron 106 (headless)                                                        │
  │ Node Version:   v16.17.0 (/home/lachlan/.nvm/versions/node/v16.17.0/bin/node)                  │
  │ Specs:          1 found (auth.cy.ts)                                                           │
  │ Searched:       cypress/e2e/**/*.cy.{js,jsx,ts,tsx}                                            │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


────────────────────────────────────────────────────────────────────────────────────────────────────
                                                                                                    
  Running:  auth.cy.ts                                                                      (1 of 1)


  authentication flow
    ✓ signs up (5395ms)


  1 passing (5s)


  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        1                                                                                │
  │ Passing:      1                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        false                                                                            │
  │ Duration:     5 seconds                                                                        │
  │ Spec Ran:     auth.cy.ts                                                                       │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘


====================================================================================================

  (Run Finished)


       Spec                                              Tests  Passing  Failing  Pending  Skipped  
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔  auth.cy.ts                               00:05        1        1        -        -        - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔  All specs passed!                        00:05        1        1        -        -        -  

Done in 8.55s.

Open mode looks suspect - but most of the problems here seem to be around CI. Here’s what I was able to reproduce:

Cypress 9

Pretty quick.

https://user-images.githubusercontent.com/19196536/205818247-59fe8108-6984-4954-8d6f-187c9d9a91ff.mp4

Cypress 10

Look how much longer it takes! I will try reproduce this on CI.

https://user-images.githubusercontent.com/19196536/205818293-8f058f29-0e7b-4436-a908-30578476847e.mp4

I reproduced it by adding lots of tiny JS modules:

import * as mod1 from './mod-1'
import * as mod2 from './mod-2'
import * as mod3 from './mod-3'
import * as mod4 from './mod-4'
...
import * as mod200 from './mod-200'

Each module is just export const foo = 'FOO'. It looks like somewhere between Cypress 9 and 10, something changed that makes loading lots of JS modules (and maybe other files) much slower.

This is a Vite app. I tried a Vanilla app – no dev server – and there’s no difference in Cy 9 vs Cy 10. That Vanilla app is here.

I suspect we might be running into the same issue. We were trying to do an upgrade between v6.6.0 and v11.2.0. I let Cypress automatically handle renaming the integration folder to e2e, converting cypress.json to cypress.config.js, etc. and that all went fine. Then I noticed that the build times for our Cypress jobs in CI went from less than 15 minutes to over 30 minutes.

Once you have a version to fix the issue reported above, I would be interested to test it out to see if it also addresses the performance in our upgrade scenario.

@lmiller1990 it seems not cache to blame, when i click rerun, it’s almost no different times to load. I am just running one test. it seems cypress will proxy and rewrite all js files? may be the problem is here? i noticed the CPU is about 30% when i refresh page in cypress, while in browser is about 8%.

There is some rewriting - I need to look into this part of the code base more, and find out exactly why.

by the way, do you know how to set timeout of mocha testcase? in here it was set to false?

There’s a few ways to change the timeout, how about:

describe('some test', { defaultCommandTimeout: 10000 }, () => {
  // ...
})

You can also set them globally in your cypress.config.js.

Timeout is set to false - a test times out when a command fails to complete. There is a few different timeouts you can control, see here.

I think the problem is our network proxy has a bottleneck when there is many XHR requests. This website’s login page makes over 800 XHR requests, and ~500 CSS resources. Just to clarify, @SadiqRahim, is that intentional? It takes 30 seconds to finish even outside of Cypress - I think what’s happening is Cypress is waiting for all network activity to stop, which takes a very long time.

Prod in browser:

Cypress:

I do not think this is duplicate of #22353, because that is only related to Component Testing, where we changed from 1 tab per run to 1 tab per spec. E2E changed from one browser launch per spec to a single browser launch, opening a new per spec, which is substantially faster.

Also, #22353 points out the entire run time is slower, but the individual spec run time is not - the time is getting lost on rendering the runner UI, parsing JS, etc. This screenshot seems to imply the actual spec execution is slower, which would indicate a different issue.

@SadiqRahim I’m quite curious here… your test takes 20 seconds to execute:

cy.window().then((appWindow) => {
  let spreadHostElement = appWindow.document.querySelector(pageObj.forecastObj.hostElement)
  let spread = appWindow.GC.Spread.Sheets.findControl(spreadHostElement)
  let activeSheet = spread.getActiveSheet()
  activeSheet.setActiveCell(4, 5)
  activeSheet.startEdit(true, '10')
  cy.get('.cc-forecasting.cc-ui-block').type('{enter}')
  cy.wait(300)
})

cy.get(':nth-child(3) > .cc-ui-button').click({force: true})
cy.get('.cc-company-forecasting-save > .cc-ui-field > input').type('New forecast')

cy.intercept('POST', '/graphql', (req) => {
  aliasMutation(req, 'SaveForecast')
})
cy.get('.cc-company-forecasting-save__actions > .cc-ui-button').click({force: true})
cy.wait('@SaveForecast')
cy.get('.is-active > .cc-ui-icon > .iconPlus').click({ force: true })
cy.wait(1500)

Do you have any feeling on which part of this is slow? cy.get and click are basically instant. The only thing that looks like it would be taking time here is waiting for API requests. Are you able to clarify which part(s) of your test seem to be taking longer/account for the additional 5 seconds?