playwright-python: [BUG] Hanging when browser runs out of memory

Context:

  • Playwright Version: 1.31.1
  • Operating System: Linux+docker
  • Python version: 3.10.6
  • Browser: Chromium

Code Snippet

My code hangs when trying to save a trace to a file every so often

from pathlib import Path
import tempfile
from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    with browser.new_context() as context:
        page = browser.new_page()
        context.tracing.start(screenshots=True, snapshots=True)
        try:
            page.goto("https://some_url.com")
            # Do some stuff that fails
        finally:
            file = tempfile.NamedTemporaryFile(suffix=".zip")
            path = Path(file.name)
            file.close()
            # This hangs forever, using 100% CPU
            context.tracing.stop(path=path)
            # Would return path to some parent function...
    browser.close()

Describe the bug It seems like something goes wrong with the trace saving and then Playwright gets caught in an infinite loop on these two lines:

        while not task.done():
            self._dispatcher_fiber.switch()

The main Python process is pegged at 100% CPU executing these two lines. When I use lsof, no process even has the zipfile I’m trying to save the trace to open. There may have been an exception that inside the try block that Playwright is unable to handle saving traces for. I will add more logging to figure this out and report back, but I feel like Playwright should not get hung here regardless.

hung_trace_capture

About this issue

  • Original URL
  • State: closed
  • Created a year ago
  • Comments: 17 (7 by maintainers)

Most upvoted comments

The zombie processes listed above were totally solved by using dumb-init, so there’s the solution to that issue.

I’ve done some tests where I run a local docker container with artificially constrained memory to observe the behavior. As of yet I have not get it to hang as I see in production, but I suspect it might take the OOM memory handler killing the right process at just the right time to trigger this hang. So next step for me would be to run this test many times and see if I can get it to ever hang. My test command was along these lines: docker run --rm -it --memory=250m --memory-swap=250m --env-file bots/.env -v "$(pwd)/test_mem.py:/test.py" mcr.microsoft.com/playwright/python:v1.32.1-jammy python /test.py

However, there’s a chance dumb-init may have solved the issue and I’ve made some adjustments to memory allocations that mean this hasn’t happened again for a while so I’m going to leave this for now.

I would recommend for now to upgrade to 1.32.1+ since we changed tracing a lot and how it works under the hood to process it, when getting stopped.