cypress: Tests randomly fails on timeout to find an element by attribute on v4.5.0

Current behavior:

On running our test suite in Cypress v4.5.0, we get random fails whenever the test suite runs in our CI environment (Drone - custom Node v12 docker image - running electron headless).

Examples of two independent runs failing in different tests: image image

All tests pass when running locally either on chrome and in headless electron. Downgrading to v4.4.1 fixes the issue in our CI.

Desired behavior:

All tests pass independent of version and environment.

Versions

  • Cypress: v4.5.0
  • OS: Linux Docker - Node 12 custom image
  • Browser - Headless Electron

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 9
  • Comments: 27 (8 by maintainers)

Most upvoted comments

This issue is closed as no one has provided a way to reproduce it, so there is no solution avaialable. If you can provide a way to reproduce, we’ll take a look at it.

it’s reproducible in a way, that the tests relying on cy.route() and cy.wait() fails randomly in CI (for me the tests are running successfully locally)

Please re-open the issue, we have the same problem. It fails randomly on CI and local. At least for me, it fails randomly in Chrome and Firefox but works in Edge. 😮 What I’ve also noticed is, that each test alone works, if I run many I get random fails.

@darshna09 @giancarlo88 so what I noticed was that the issue happens always in combination with network requests (simple fetch or axios). We were able to fix some of the issues of this kind by waiting for the network request to resolve, bevore making the assertions. What we noticed is that sometimes our CI-runners just didn’t provide the performance so some requests took a long time to resolve.

So before, we had something like this:

// mocking network response
cy.intercept()

// asserting/finding/getting elements that are being rendered conditionally, depending on the network response
// on our pipeline, this failed sometimes
cy.get()

Now, we are doing:

// mocking network response
// naming the request
cy.intercept().as('getUserInfo')

// WAIT for the response
cy.wait('@getUserInfo')

// then asserting/finding/getting the conditionally rendered elements, based on the network response
// no fail on pipeline anymore
cy.get()

So take a closer look at the time your cy instance needs to run all the tests on the pipeline and compare it to your local time. If the times are very far apart - then you might try out what we did.

We’re also having the same issue. I think the problem is it’s difficult to come up with a reproducible example to share with you as our codebase is internal and we can’t easily spin up something and run it on a Drone CI pipeline (which we’d also have to spin up on our own). I’m going to see if I can try to create an example for you but it might not be entirely possible.

In my case I had to increase the defaultCommandTimeout significantly in order to solve for very poor container performance. I was getting many failures at the default 4 seconds, and consistent success at 10 seconds.

Still facing this with even cypress 10 and it fails randomly so I cannot create a reproduction for this, but it fails on both local and CI

Right now there doesn’t seem to be enough information to reproduce the problem on our end. We’ll have to close this issue until we can reproduce it. This does not mean that your issue is not happening - it just means that we do not have a path to move forward.

Please comment in this issue with a reproducible example and we will consider reopening the issue. Here are some tips for providing a Short, Self Contained, Correct, Example and our own Troubleshooting Cypress guide.