artillery: Playwright tests not working with GitHub Action? browserType.launch: Executable doesn't exist at /github/home/.cache/ms-playwright/chromium-1076/chrome-linux/chrome

Version info:

v2.0.0-37

Running this command:

run tests/my-test.yml --output ./report.json

I expected to see this happen:

The test to run like it does locally.

Instead, this happened:

worker error, id: 1 browserType.launch: Executable doesn't exist at /github/home/.cache/ms-playwright/chromium-1076/chrome-linux/chrome
╔═════════════════════════════════════════════════════════════════════════╗
║ Looks like Playwright Test or Playwright was just installed or updated. ║
║ Please run the following command to download new browsers:              ║
║                                                                         ║
║     npx playwright install                                              ║
║                                                                         ║
║ <3 Playwright Team                                                      ║
╚═════════════════════════════════════════════════════════════════════════╝
    at Array.scenario (/home/node/artillery/node_modules/artillery-engine-playwright/index.js:56:38)
    at runScenario (/home/node/artillery/node_modules/@artilleryio/int-core/lib/runner.js:331:32)
    at EventEmitter.<anonymous> (/home/node/artillery/node_modules/@artilleryio/int-core/lib/runner.js:203:25)
    at EventEmitter.emit (/home/node/artillery/node_modules/eventemitter3/index.js:181:35)
    at UniformProcess.<anonymous> (/home/node/artillery/node_modules/@artilleryio/int-core/lib/phases.js:220:[12](https://github.com/nansen-ai/nansen-artillery/actions/runs/6467513570/job/17557698774#step:9:13))
    at /home/node/artillery/node_modules/arrivals/lib/index.js:47:10
    at NanoTimer.setInterval (/home/node/artillery/node_modules/nanotimer/lib/nanotimer.js:2[20](https://github.com//actions/runs/6467513570/job/17557698774#step:9:21):5)
    at Immediate._onImmediate (/home/node/artillery/node_modules/nanotimer/lib/nanotimer.js:203:68) ***
  name: 'Error'

Files being used: Workflow:

name: Load Test

on:
    workflow_dispatch:
 
jobs:
  artillery:
    runs-on: ubuntu-latest
 
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Set up Node.js
        uses: actions/setup-node@v3
        with:
          node-version: 18

      - name: Setup bun dependencies
        uses: oven-sh/setup-bun@v1
        with:
          bun-version: latest
        
      - name: Install dependencies
        run: |
          bun install
          npx playwright install

      - name: Write the serviceAccountKey.json file 📝
        run: |
            echo '${{ secrets.SERVICE_ACCOUNT_KEY }}' > serviceAccountKey.json
 
      - name: Execute load tests
        uses: artilleryio/action-cli@v1
        with:
          command: run tests/my-test.yml --output ./report.json
        
      - name: Upload artifact
        uses: actions/upload-artifact@v3
        if: always()
        with:
          name: artillery-report
          # Reference the generated report in the file system.
          path: ./report.json

About this issue

Original URL
State: open
Created 9 months ago
Comments: 17 (4 by maintainers)

Most upvoted comments

Going to add more context here for anyone running into similar issue and trying to debug. (Information is accurate at the time of writing and will probably go out of date soon)

Our Playwright integration is currently pinned to v1.39.0 of Playwright. (We’d like to offer the ability to pick any version of Playwright, but as the Playwright team do a good job of maintaining backwards-compatibility, it’s not a priority right now.)

If you’re running a test on Fargate (with run-fargate) with the most recent version of Artillery, the Docker image that will be used to execute the tests already includes Playwright v1.39.0 and an installation of Chromium. It should all just work without you needing to do anything. If it doesn’t, then it’s a bug we need to fix.

If you’re running tests elsewhere (e.g. locally, or in GitHub Actions), then you have to make sure to install the browsers. The path where the browsers are expected to be found (e.g. /ms-playwright/chromium-1076/chrome-linux/chrome) is set by Playwright itself and not by Artillery, and it changes between version of Playwright, so you have to make sure you’re installing v1.39.0 exactly.

This appears to have changed again. I’m new to Playwright and Artillery so it’s my first encounter with the issue. It seems like many have worked around it. I’ll spend some time trying to do the same.

Never mind, I had a more current version of the browser deps in the cache and hadn’t yet installed the older ones. Indeed it works with the version of Playwright specified in packages.json

lee@suzanne:~/src/artillery_tutorial$ npm install playwright@v1.39.0

added 2 packages in 4s
lee@suzanne:~/src/artillery_tutorial$ artillery run playwright.yml 
Test run id: tth9d_6eebc96bz85p8kfrghmfjk78m8afg_rqyd
Phase started: unnamed (index: 0, duration: 1s) 15:33:03(-0800)

Phase completed: unnamed (index: 0, duration: 1s) 15:33:04(-0800)

worker error, id: 1 browserType.launch: Executable doesn't exist at /home/lee/.cache/ms-playwright/chromium-1084/chrome-linux/chrome
╔═════════════════════════════════════════════════════════════════════════╗
║ Looks like Playwright Test or Playwright was just installed or updated. ║
║ Please run the following command to download new browsers:              ║
║                                                                         ║
║     npx playwright install                                              ║
║                                                                         ║
║ <3 Playwright Team                                                      ║
╚═════════════════════════════════════════════════════════════════════════╝
    at Array.scenario (/usr/lib/node_modules/artillery/node_modules/artillery-engine-playwright/index.js:67:38)
    at runScenario (/usr/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/runner.js:373:32)
    at EventEmitter.<anonymous> (/usr/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/runner.js:203:25)
    at EventEmitter.emit (/usr/lib/node_modules/artillery/node_modules/eventemitter3/index.js:181:35)
    at UniformProcess.<anonymous> (/usr/lib/node_modules/artillery/node_modules/@artilleryio/int-core/lib/phases.js:220:12)
    at /usr/lib/node_modules/artillery/node_modules/arrivals/lib/index.js:47:10
    at NanoTimer.setInterval (/usr/lib/node_modules/artillery/node_modules/nanotimer/lib/nanotimer.js:220:5)
    at Immediate._onImmediate (/usr/lib/node_modules/artillery/node_modules/nanotimer/lib/nanotimer.js:203:68) {
  name: 'Error'
}
lee@suzanne:~/src/artillery_tutorial$ ls -ld /home/lee/.cache/ms-playwright/chromium-*
drwxrwxr-x 3 lee lee 4096 Dec 11 11:42 /home/lee/.cache/ms-playwright/chromium-1091
lee@suzanne:~/src/artillery_tutorial$ npx playwright --version
Version 1.39.0

lazzarello on Dec 16, 2023

That makes more sense - we were using artillery in the package.json and likely having version conflicts in package-lock.json because of it. Now that I am aware that’ll cause non-ideal results, I’ll switch our codebase to an npx artillery approach for Artillery itself. Thanks for the clarification.

freyjadomville on Nov 17, 2023