storybook: [Bug]: Regression in test-storybook after upgrading to 8.5.x: Axe is already running error

Describe the bug

Summary

After upgrading to Storybook 8.5.1, I encountered an issue when running yarn test-storybook in combination with axe-playwright. Specifically, the test runner throws the following error:

page.evaluate: Error: Axe is already running. Use `await axe.run()` to wait for the previous run to finish before starting a new run.

This issue occurs when running tests with --maxWorkers=2 or higher. However, it works fine if I downgrade back to 8.4.7 or if I reduce the number of workers to --maxWorkers=1 in 8.5.1.

Environment Details:

  • Storybook Version: 8.5.x

  • @storybook/test-runner Version: Latest (installed alongside Storybook 8.5.1)

  • Test Runner Command:

yarn test-storybook --url http://127.0.0.1:6006/ --config-dir ./apps/storybook/.storybook --maxWorkers=2
  • Temporary Solution: Changing --maxWorkers to 1 avoids the issue:
yarn test-storybook --url http://127.0.0.1:6006/ --config-dir ./apps/storybook/.storybook --maxWorkers=1

Questions:

  • Was there a change in how @storybook/test-runner or Storybook 8.5.x handles parallel worker processes?

  • Is there a way to ensure compatibility with axe-playwright when running test-storybook with multiple workers?

  • Are there any known workarounds or fixes to address this regression?

Reproduction link

no

Reproduction steps

I can reproduce this with this code

yarn concurrently -k -s first -n "SB,TEST" -c "magenta,blue" "yarn http-server dist/storybook/storybook --port 6006 --silent" "yarn wait-on http://127.0.0.1:6006 && yarn test-storybook --config-dir ./apps/storybook/.storybook --maxWorkers=2
//.storybook/test-runner.ts

import type { TestRunnerConfig } from '@storybook/test-runner';
import { getStoryContext } from '@storybook/test-runner';

import { injectAxe, checkA11y, configureAxe } from 'axe-playwright';

/*
 * See https://storybook.js.org/docs/writing-tests/test-runner#test-hook-api
 * to learn more about the test-runner hooks API.
 */
const config: TestRunnerConfig = {
  async preVisit(page) {
    await injectAxe(page);
  },
  async postVisit(page, context) {
    // Get the entire context of a story, including parameters, args, argTypes, etc.
    const storyContext = await getStoryContext(page, context);

    // Apply story-level a11y rules
    await configureAxe(page, {
      rules: storyContext.parameters?.a11y?.config?.rules,
    });

    const element = storyContext.parameters?.a11y?.element ?? 'body';
    await checkA11y(page, element, {
      detailedReport: true,
      detailedReportOptions: {
        html: true,
      },
    });
  },
};

export default config;

System

Storybook Environment Info:

  System:
    OS: macOS 15.2
    CPU: (8) arm64 Apple M1 Pro
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.15.1 - /private/var/folders/4v/rybl0kls7918cw1lzk1120gw0000gp/T/xfs-f358ab08/node
    Yarn: 3.6.4 - /private/var/folders/4v/rybl0kls7918cw1lzk1120gw0000gp/T/xfs-f358ab08/yarn <----- active
    npm: 10.7.0 - /usr/local/bin/npm
  Browsers:
    Chrome: 131.0.6778.265
    Safari: 18.2
  npmPackages:
    @storybook/addon-a11y: ^8.5.1 => 8.5.1 
    @storybook/addon-designs: ^8.0.4 => 8.0.4 
    @storybook/addon-docs: ^8.5.1 => 8.5.1 
    @storybook/addon-essentials: ^8.5.1 => 8.5.1 
    @storybook/addon-interactions: ^8.5.1 => 8.5.1 
    @storybook/addon-links: ^8.5.1 => 8.5.1 
    @storybook/addon-themes: ^8.5.1 => 8.5.1 
    @storybook/blocks: ^8.5.1 => 8.5.1 
    @storybook/core-server: ^8.5.1 => 8.5.1 
    @storybook/react: ^8.5.1 => 8.5.1 
    @storybook/react-vite: ^8.5.1 => 8.5.1 
    @storybook/test: ^8.5.1 => 8.5.1 
    @storybook/test-runner: ^0.21.0 => 0.21.0 
    chromatic: ^11.24.0 => 11.24.0 
    eslint-plugin-storybook: ^0.11.2 => 0.11.2 
    storybook: ^8.5.1 => 8.5.1 
    storybook-addon-pseudo-states: ^4.0.2 => 4.0.2 

Additional context

  • This issue does not occur when using the Storybook dev server (yarn storybook) instead of http-server. The same test works fine when pointed to http://localhost:4400 (the Storybook dev server) with --maxWorkers=2.

  • I suspect this might be related to how lifecycle events are handled in @storybook/test-runner for static vs. dynamic Storybook builds in the 8.5.x release.

  • Similar issue https://github.com/storybookjs/storybook/issues/29805#issuecomment-2594439211

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 14
  • Comments: 23 (13 by maintainers)

Most upvoted comments

That’s correct. And because Storybook doesn’t run axe automatically anymore it doesn’t conflict with axe which is set up via the test-runner

I’ve resolved the issue “Axe is already running error” by enabling a11y manual in the Storybook .storybook/preview.ts|tsx file.

const preview: Preview = {
 globals: {
    a11y: {
      manual: true
    }
  }
};

The fix should be released in test-runner 0.21.1 https://github.com/storybookjs/test-runner/releases/tag/v0.21.1

Hi @akibarika,

Thank you for reporting the issue!

Can you please try out this canary release of the test runner? 0.21.1--canary.349c0dd.0. It contains a fix for your issue. Unfortunately, I can’t test it because the issue is not reproducible for me.

Related PR: https://github.com/storybookjs/test-runner/pull/536

I am still experiencing this issue with test-runner 0.21.2 and addon-a11y 8.5.8.

Setting manual: true works as a workaround, but is not ideal.

Even though the workaround is working, I still want to identify the root cause. Currently, the test is being run manually in the Storybook UI as well.

@valentinpalkovic the canary release worked for me, thanks! Can’t wait for a patch version to be released!