playwright: [REGRESSION]: Performance downgrade (almost 3 times slower)
Context:
- GOOD Playwright Version: 1.28
- BAD Playwright Version: 1.31
- Operating System: Ubuntu 20.04, Windows
- Extra: jest runner
Describe the bug
Not sure, if this is proper type of “issue”. However, I experienced downgrade of performance in the test execution. Between the version 1.28 and 1.31, speed of execution has been decreased almost 3 times. Not only for the workers, but also for a single run.
Here is a piece of Jenkins log (same agent in both cases): 1.28
[Pipeline] sh
- npm list playwright Ra.Playwright@1.0.0 /opt/jenkins/workspace/Tests/Ra.Playwright └── playwright@1.28.1
[Pipeline] sh
- npm run support – CI
Ra.Playwright@1.0.0 support jest --verbose --maxWorkers=10 --updateSnapshot --config ./jest.config.js ./tests/support/.*.spec.js CI
PASS browser: chromium tests/support/support.spec.js (17.505 s) Support (menu item) tests ✓ 01: check if links/icons/titles are displayed (1173 ms) ✓ 02: check change log details (4247 ms)
› 2 snapshots written. 📦 report is created on: /opt/jenkins/workspace/Tests/Ra.Playwright/reports/jest_html_reporters.html Snapshot Summary › 2 snapshots written from 1 test suite.
Test Suites: 1 passed, 1 total Tests: 2 passed, 2 total Snapshots: 2 written, 2 total Time: 18.434 s Ran all test suites matching /./tests/support/.*.spec.js|CI/i. [Pipeline] } [Pipeline] // timeout [Pipeline] echo All done!`
1.31
[Pipeline] sh
- npm list playwright Ra.Playwright@1.0.0 /opt/jenkins/workspace/Tests/Ra.Playwright └── playwright@1.31.2
[Pipeline] sh
- npm run support – CI
Ra.Playwright@1.0.0 support jest --verbose --maxWorkers=10 --updateSnapshot --config ./jest.config.js ./tests/support/.*.spec.js CI
PASS browser: chromium tests/support/support.spec.js (75.641 s) Support (menu item) tests ✓ 01: check if links/icons/titles are displayed (33185 ms) ✓ 02: check change log details (27162 ms)
› 2 snapshots written. 📦 report is created on: /opt/jenkins/workspace/Tests/Ra.Playwright/reports/jest_html_reporters.html Snapshot Summary › 2 snapshots written from 1 test suite.
Test Suites: 1 passed, 1 total Tests: 2 passed, 2 total Snapshots: 2 written, 2 total Time: 76.184 s Ran all test suites matching /./tests/support/.*.spec.js|CI/i. [Pipeline] } [Pipeline] // timeout [Pipeline] echo All done!
About this issue
- Original URL
- State: closed
- Created a year ago
- Comments: 15 (8 by maintainers)
Yeah Chai and Playwright’s assertions are significant different, because our assertions retry until the condition is met, what Chai does not. (ours are specifically made for Playwright and end-to-end testing). This ends up in the state that chai or normal expect assertions are flaky and ours are not.
So it should just be:
await expect(page.frameLocator(..).locator(..)).toBeVisible()await expect(page.frameLocator(..).locator(..)).toHaveAttribute('src', '...')await expect(page.frameLocator(..).locator(..)).toHaveText('my-text')Its unfortunately hard to tell from above what the issue is. I would recommend the following: make sure to use best-practise assertions, migrate to Playwright test-runner (you are missing out on html report, horizontal/vertical scaling, better error reporting, vscode extension support etc. and then see if you still experience issues with Playwright.