playwright: [BUG] Tests do not respect timeout

Context:

• Playwright Version: 1.26.1
• Operating System: Linux
• Node.js version: 14.20.1
• Browser: Chromium

Playwright config publicly available

Code Snippet

// Config file for integration tests
import { PlaywrightTestConfig } from "@playwright/test";

const config: PlaywrightTestConfig = {
  testDir: ".",
  testMatch: "tests.ts",
  expect: {
    timeout: 5000,
  },
  // If nothing is printed for 10m, then CircleCI times us out, which means we don't
  // find out which test hung. So timeout here first so that it tells us what hung.
  globalTimeout: 600000,
  // in ms. 30000 = 30s
  timeout: 30000,
  use: {
    actionTimeout: 5000,
    headless: true,
    trace: "retain-on-failure",
    screenshot: "off",
    video: "on",
  },
  reporter: [
    ["list"],
    ["json", { outputFile: "rundir/test_results/integration_tests.json" }],
    ["junit", { outputFile: "rundir/test_results/integration_tests.xml" }],
  ],
};

export default config;

Describe the bug

Sometimes when I run my playwright tests, some of the tests hang despite a timeout being set.

For example:

https://app.circleci.com/pipelines/github/darklang/dark/4283/workflows/59ac7d0e-36d7-44a9-8ebf-e947dd9ee268/jobs/44444/parallel-runs/0/steps/0-109

In this CI run, the globalTimeout kicks in after 10 minutes, but the individual test timeout did not kick in after 30s.

(Note the overall run time of these tests is typically less than 5m, and before we added the globaltimeout we’d see the same thing, except circleci would timeout due to no output for 10m.)