cypress: Tests that iterate over many DOM elements with `.each()` run slower in Electron during `cypress run` in 3.8.0

Current behavior:

We have a storybook cypress smoke test that opens all the stories and checks that they render. The test itself is very simple:

it('can load all components', () => {
    cy.get('[id^=explorer]').each($component => {
      cy.wrap($component)
        .click()
        .get(`[id^=${$component[0].id}--]`)
        .each($story => {
          cy.wrap($story)
            .click()
            .wait(50)
            .componentHasLoaded()
        })
    })
  })

With

Cypress.Commands.add('componentHasLoaded', () => {
  cy.get('#storybook-preview-iframe').then($iframe => {
    const doc = $iframe.contents()
    cy.wrap(doc.find('#root')).should('not.be.empty')
  })
})

Our test duration went from roughly 2 minutes to roughly 8 minutes as seen in the graph: Screenshot 2019-12-17 at 16 15 11 (after some commits we decided to roll back to cypress@3.7.0)

There were absolutely no changes in the app code.

The only change was moving from cypress@3.7.0 to cypress@3.8.0. From what we’ve seen, there are no noticeable changes in the test (no new logs or errors) other than the interaction just being slower. The command to run cypress is exactly the same but we’ve noticed that the Electron version has changed from 73 to 78 as seen in the screenshots: Screenshot 2019-12-17 at 16 20 54 Screenshot 2019-12-17 at 16 21 11

Desired behavior:

We’d like to know if there is something to do on our end to go back to the old speed or if there is a bug in cypress that created this performance regression.

Steps to reproduce: (app code and test code)

Run a storybook test with:

it('can load all components', () => {
    cy.get('[id^=explorer]').each($component => {
      cy.wrap($component)
        .click()
        .get(`[id^=${$component[0].id}--]`)
        .each($story => {
          cy.wrap($story)
            .click()
            .wait(50)
            .componentHasLoaded()
        })
    })
  })

With

Cypress.Commands.add('componentHasLoaded', () => {
  cy.get('#storybook-preview-iframe').then($iframe => {
    const doc = $iframe.contents()
    cy.wrap(doc.find('#root')).should('not.be.empty')
  })
})

Versions

cypress@3.8.0

Performance regression found in

Build-agent version 1.0.22569-ba1c5aa5 (2019-12-16T14:39:46+0000)
Docker Engine Version: 18.09.6
Kernel Version: Linux b9cc89c6eb3c 4.15.0-1052-aws #54-Ubuntu SMP Tue Oct 1 15:43:26 UTC 2019 x86_64 Linux
Starting container cypress/base:12.6.0
  image cache not found on this host, downloading cypress/base:12.6.0

Also locally in MacOS 10.15.2

About this issue

Original URL
State: closed
Created 5 years ago
Reactions: 18
Comments: 28 (15 by maintainers)

Most upvoted comments

I used the Chrome dev tools Performance tab to profile one of my tests running under 3.7.0 vs 3.8.3. There is a definite difference in the code path.

3.7.0:

3.8.3

It seems that 3.8.0 might’ve introduced some change in the isVisible check? The root cause of the extra slowness in this particular timing is layout thrashing due to forced reflow.

Based on what I’m seeing in this trace, I am guessing that the culprit is https://github.com/cypress-io/cypress/pull/6000 or https://github.com/cypress-io/cypress/pull/5916, but I haven’t done a bisect.

fr0 on Feb 3, 2020

I have the same problem. In my case it’s limited to Electron. Version 4 is the same

Running times : 
3.7.0
------ 
Chrome 80
1st run 95.31
2nd run 94.98

Electron 73
1st run 49.4
2nd 60.06
3rd 50.93

3.8.X
------


Chrome 80
1st run 103.15
2nd run 101
3rd run 102.98

Electron 78
1st run 110.90
2nd run 355.23
3nd run doesn't complete

vjau on Feb 29, 2020

Concerning the first example in https://github.com/cypress-io/cypress/issues/5987#issuecomment-583414882 I can recreate the behavior of the specs running slowing in headless Electron in 3.8.0.

Repro

describe('CAN LOOP', () => {
    it('SHOULD LOOP', () => {
        cy.visit('https://en.wikipedia.org/wiki/Star_Trek');
        cy.get('.toc li').each(($el) => {
            cy.wrap($el).find('a').each(($a) => {
                cy.wrap($a).click();
            });
        });

        cy.get('.toc li').each(($el) => {
            cy.wrap($el).find('a').each(($a) => {
                cy.wrap($a).click();
            });
        });
    });
});

Timings

cypress run Electron headless or headed:

3.7.0: 25 secs
3.8.0: 92 secs
5.3.0: 108 secs

cypress open Electron:

3.7.0: 175.25 secs
3.8.0: 115.6 secs
5.3.0: 141.61 secs

cypress open Chrome:

3.7.0: 92.72 secs
3.8.0: 92.21 secs
5.3.0: 111.88 secs

jennifer-shehane on Sep 29, 2020

Hi,

The following test takes about 25 seconds on 3.7.0 and about 1:30 on 3.8.3.

I wonder if the issue is something to do with cy.get(...).each()?

describe('CAN LOOP', () => {
    it('SHOULD LOOP', () => {
        cy.visit('https://en.wikipedia.org/wiki/Star_Trek');
        cy.get('.toc li').each(($el) => {
            cy.wrap($el).find('a').each(($a) => {
                cy.wrap($a).click();
            });
        });

        cy.get('.toc li').each(($el) => {
            cy.wrap($el).find('a').each(($a) => {
                cy.wrap($a).click();
            });
        });

        cy.get('.toc li').each(($el) => {
            cy.wrap($el).find('a').each(($a) => {
                cy.wrap($a).click();
            });
        });

    });
});

If I simplify the loops in the test to

describe('CAN LOOP', () => {
    it('SHOULD LOOP', () => {
        cy.visit('https://en.wikipedia.org/wiki/Star_Trek');
        cy.get('.toc li a').each(($a) => {
            cy.wrap($a).click();
        });

        cy.get('.toc li a').each(($a) => {
            cy.wrap($a).click();
        });

        cy.get('.toc li a').each(($a) => {
            cy.wrap($a).click();
        });
    });

});

Then is takes ~15seconds on 3.7.0 and ~23seconds on 3.8.3

marmite22 on Feb 18, 2020

There is a performance issue introduced in Chrome 79 described here: https://github.com/cypress-io/cypress/issues/6023#issuecomment-574031655

Just wanted to link that issue here in case anyone does find their issue isolated completely to Chrome 79, it may be that issue.

jennifer-shehane on Jan 14, 2020