storybook: [Bug]: Console errors in the Storybook test-runner

Describe the bug

Since upgrading to version 8.5.2 and subsequently 8.5.3 the test runner is outputting errors in the GitHub pipeline console:

PASS browser: chromium ../../../tmp/791c8e78dec52614e7407c37b81a8e7e/components-nptile.test.js (9.434 s)
  ● Console

    console.log
      An error occurred in the following story, most likely because of a navigation: "Components / NpTile/With Long Title". Retrying...

      at Object.log (../../../tmp/791c8e78dec52614e7407c37b81a8e7e/components-nptile.test.js:112:19)

    console.log
      An error occurred in the following story, most likely because of a navigation: "Components / NpTile/Disabled". Retrying...

      at Object.log (../../../tmp/791c8e78dec52614e7407c37b81a8e7e/components-nptile.test.js:172:19)

The tests do appear to be passing however, according to the report:

Test Suites: 52 passed, 52 total
Tests:       227 passed, 227 total
Snapshots:   0 total
Time:        84.877 s
Ran all test suites.
[test-storybook] Cleaning up /tmp/0a06b7ea4f9918e03ffedd164cc92a0c

Reproduction link

Unable to supply as this is a runner issue

Reproduction steps

No response

System

System:
    OS: Linux 5.4 Ubuntu 20.04.6 LTS (Focal Fossa)
    CPU: (4) x64 Intel Xeon Processor (Cascadelake)
    Shell: 5.0.17 - /bin/bash
  Binaries:
    Node: 18.20.6 - /usr/bin/node
    npm: 9.9.4 - /usr/bin/npm <----- active
  npmPackages:
    @storybook/addon-a11y: ^8.5.2 => 8.2.10 
    @storybook/addon-essentials: ^8.5.2 => 8.5.2 
    @storybook/addon-interactions: ^8.5.2 => 8.5.2 
    @storybook/addon-links: ^8.5.2 => 8.5.2 
    @storybook/blocks: ^8.5.2 => 8.5.2 
    @storybook/manager-api: ^8.5.2 => 8.5.2 
    @storybook/test: ^8.5.2 => 8.5.2 
    @storybook/test-runner: ^0.21.0 => 0.21.0 
    @storybook/theming: ^8.5.2 => 8.5.2 
    @storybook/vue3: ^8.5.2 => 8.5.2 
    @storybook/vue3-vite: ^8.5.2 => 8.5.2 
    eslint-plugin-storybook: ^0.11.2 => 0.11.2 
    storybook: ^8.5.2 => 8.5.2

Additional context

No response

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Reactions: 6
  • Comments: 18 (4 by maintainers)

Most upvoted comments

Do you all also use the @storybook/addon-a11y addon? If so, can you remove it for testing purposes and see whether the logs disappear?

As far as I can tell the logs were still there after removing @storybook/addon-a11y from addons.

Had the same issue using interaction testing and it was causing some tests to fail randomly.

I added this as a temporary workaround :

// File: .storybook/test-runner.js
import type { TestRunnerConfig } from "@storybook/test-runner";

const config: TestRunnerConfig = {
  async preVisit(page) {
    await page.waitForTimeout(500); // Avoid weird issue with the test runner that sometimes fails (with message "An error occurred in the following story, most likely because of a navigation")
  },
};
export default config;

I hope a fix will be found because thise workaround also cause the CI step to be longer (as well as the project pre-commit which also run storybook test runner on changed files).

I can confirm I’m seeing the same thing. In my case, it also appears to be related to the @storybook/addon-a11y addon. If I either downgrade or disable that addon the message goes away.

I have the same together with failing smoke tests. It also seems to log randomly and might be the cause of my failing tests. Also the failing smoke-tests seem to be flaky. This worked fine with v8.4.2