testcafe: Testcafe tests run perfectly fine locally but fail on CI (Gitlab CI)
Are you requesting a feature or reporting a bug?
Reporting a bug
What is the current behavior?
I am trying to run tests on CI. They fail giving a weird script error. They run fine Tried using the stable(alpha) as well as the latest version of testcafe. Using this Docker to run on CI
Locally:
testcafe chrome:headless __tests__/Sequences1_testcafe.js
Using locally installed version of TestCafe.
Running tests in:
- HeadlessChrome 0.0.0 / Linux 0.0.0
Getting Started
✓ Sequences1
1 passed (54s)
In docker, the tests pass too.
testcafe chrome:headless __tests__/Sequences1_testcafe.js
using locally installed version of TestCafe.
Running tests in:
- HeadlessChrome 0.0.0 / Linux 0.0.0
Getting Started
✓ Sequences1
1 passed (58s)
`
On gitlab CI with the same image up there I get this CI error On the CI , it runs on chrome 63.0.3239 and other times it runs on 0.0.0 ( what is this?) what kind of chrome version is this?
What is the expected behavior?
The tests should run on CI given they run locally as well as the docker image I have been using to run on CI
How would you reproduce the current behavior (if this is a bug)?
Cannot reporoduce since this is a company private repo and a private server
Specify your
- operating system: Linux
- testcafe version: 0.18.0-alpha2
- node.js version: 6.0.0
Any help would be greatly appreciated! Thank you
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 20 (4 by maintainers)
I fixed my issue and once again the cause was that change detection wasn’t running automatically. This didn’t happen locally on my Macbook Pro but it happened both in Heroku’s CI environment (~2gb mem) and also on the VMs Crossbrowsertesting.com uses for automated testing.
So there’s definitely an Angular <> Test Cafe performance issue. I’m not certain how to troubleshoot that but the fix is to manually invoke the change detector. I have to do it after every form input right now but only once the user is 90% through the flow.
Like @Lwdthe1 I am also using Heroku and calling testcafe chrome:headless from inside of the dyno and having weird issues. My test runs from local against the remote dyno and works. However, if I run it inside the dyno then most of the time it fails but it has also passed with the same code and config.
Previously, I had an issue running locally where Angular wasn’t updating the view unless I added a call to detectChanges() in my test. So I suspect there might be performance interactions with Test Cafe and large angular applications. Running inside of a dyno likely makes that worse.
Edit: Heroku runs CI tests on a “performance-m” dyno by default which is fairly high-powered so I’m less certain server performance is the issue.
I was able to fix this by adding
maximizeWindow(). My assumption was that the element was visible but off screen in CI which is why I was getting this error in CI but not locally:The element that matches the specified selector is not visible.