playwright: [BUG] Webkit Flaky & Slow tests when attempting to click/check/etc elements in a form (unstable elements, change of viewport, etc)

Context:

Playwright Version: 1.19.2 (It has been happening since 1.15)
Operating System: Windows
Node.js version: 16
Browser: WebKit

Code Snippet

Navigate to https://github.com/garciallorente-fer/webkit-unstable-playwright and clone the project locally & run it (or check the report html or trace)
Notice how elements in webkit are unstable at first (using the trace viewer)
Notice how it sometimes fails to check elements (flaky) (not only with checkboxes, this is just an example)
Check the uploaded reports and images proving that

locator.setChecked: Clicking the checkbox did not change its state
=========================== logs ===========================
waiting for selector "input[type=radio][id=CheckboxYes3] ~ span"
  selector resolved to visible <span class="c-radio-group__text">Yes</span>
attempting click action
  waiting for element to be visible, enabled and stable
    element is not stable - waiting...
  element is visible, enabled and stable
  scrolling into view if needed
  done scrolling
  element is outside of the viewport
retrying click action, attempt #1
  waiting for element to be visible, enabled and stable
    element is not stable - waiting...
  element is visible, enabled and stable
  scrolling into view if needed
  done scrolling
  performing click action
  click action done
  waiting for scheduled navigations to finish
  navigations have finished

Describe the bug

Webkit is very flaky and takes long time, has to change constantly the viewport in search for elements before performing the action, and also waits for elements to be stable (only in webkit). This is just a little example of code, in some of the larger tests I run it is much slower and very flaky (it sometimes does not check the elements or it does not click on them) with larger forms and webpages, only when running it with webkit, and has been happening for months (to this specific project/pages)

About this issue

Original URL
State: closed
Created 2 years ago
Reactions: 2
Comments: 17 (11 by maintainers)

Most upvoted comments

@garciallorente-fer I have run the tests on Mac OS and Windows 11. You mean this?

MacOS

Chrome and Firefox shows uniform step times and the same log in the trace: and the log for the 4th checkbox:

waiting for selector "input[type=radio][id=CheckboxYes4] ~ span"
selector resolved to visible <span class="c-radio-group__text">Yes</span>
attempting click action
waiting for element to be visible, enabled and stable
element is visible, enabled and stable
scrolling into view if needed
done scrolling
performing click action
click action done
waiting for scheduled navigations to finish
navigations have finished

But with webkit, we can see that some tests take more time and the logs report several waits to be stable…

  waiting for selector "input[type=radio][id=CheckboxYes4] ~ span"
  selector resolved to visible <span class="c-radio-group__text">Yes</span>
  attempting click action
  waiting for element to be visible, enabled and stable
+ element is not stable - waiting...
+ element is visible, enabled and stable
+ scrolling into view if needed
+ done scrolling
+ element is outside of the viewport
+ retrying click action, attempt #1
+ waiting for element to be visible, enabled and stable
+ element is not stable - waiting...
  element is visible, enabled and stable
  scrolling into view if needed
  done scrolling
  performing click action
  click action done
  waiting for scheduled navigations to finish
  navigations have finished

Windows 11

locator.setChecked: Clicking the checkbox did not change its state
=========================== logs ===========================
waiting for selector "input[type=radio][id=CheckboxYes3] ~ span"
  selector resolved to visible <span class="c-radio-group__text">Yes</span>
attempting click action
  waiting for element to be visible, enabled and stable
    element is not stable - waiting...
  element is visible, enabled and stable
  scrolling into view if needed
  done scrolling
  element is outside of the viewport
retrying click action, attempt #1
  waiting for element to be visible, enabled and stable
    element is not stable - waiting...
  element is visible, enabled and stable
  scrolling into view if needed
  done scrolling
  performing click action
  click action done
  waiting for scheduled navigations to finish
  navigations have finished
============================================================

  13 |   await page.locator('input[type=radio][id=CheckboxYes2] ~ span').setChecked(true);
  14 |
> 15 |   await page.locator('input[type=radio][id=CheckboxYes3] ~ span').setChecked(true);
     |                                                                   ^
  16 |
  17 |   await page.locator('input[type=radio][id=CheckboxYes4] ~ span').setChecked(true);
  18 | });

    at C:\tmp\webkit-unstable-playwright\tests\webkit-unstable.test.ts:15:67

And, in trace viewer the same log as in MacOS

waiting for selector "input[type=radio][id=CheckboxYes3] ~ span"
selector resolved to visible <span class="c-radio-group__text">Yes</span>
attempting click action
waiting for element to be visible, enabled and stable
element is not stable - waiting...
element is visible, enabled and stable
scrolling into view if needed
done scrolling
element is outside of the viewport
retrying click action, attempt #1
waiting for element to be visible, enabled and stable
element is not stable - waiting...
element is visible, enabled and stable
scrolling into view if needed
done scrolling
performing click action
click action done
waiting for scheduled navigations to finish
navigations have finished

Some others test runs pass but take longer time like in MacOS vs chrome

davidglezz on Mar 4, 2022

Would there be a fix in future releases where that line wont be needed anymore?

Yes, it will be fixed in the next release and the workaround won’t be needed.

yury-s on May 4, 2022

await page.addStyleTag({ content: 'html { scroll-behavior: initial; }' });
Can you try the following instead, it should fix the problem in webkit:
await page.addStyleTag({ content: 'html { scroll-behavior: initial !important; }' });
The issue is caused by this CSS style rule in your project, we should probably always disable this smooth scrolling behavior during automation.

Hello @yury-s , thanks a lot for your response, I can confirm that using that line of code makes it work perfectly again, like in Chromium & Firefox. 😃 Would there be a fix in future releases where that line wont be needed anymore?

garciallorente-fer on May 4, 2022

If you read the description above you can see that the author has created a repository with a snippet that reproduces the issue. @pavelfeldman Please reopen

davidglezz on Mar 14, 2022

My triaging shift started and I’m going over the bugs marked as ‘triaging’. Unfortunately, there is too much context for me to be able to grasp and to be helpful. If you could narrow down the problem to something that we could reproduce, it would make this issue actionable on our side. We need a snippet of code reproducing the issue along with the expectations and actual results. That way we can prioritize the fix. Please open a new issue if you manage to create something we can reproduce and link this issue there for context.

pavelfeldman on Mar 14, 2022

@yury-s Could it be a Playwright race condition?, since this test doesn’t contain JS, just plain html+css.

davidglezz on Mar 8, 2022