lost-pixel: False positives for Storybook shots

Bug description

We are noticing a lot of false positives in our visreg tests for storybook. The height of the story seems to be variable. Sometimes truncating the stories.

Please see attached image:

How to reproduce

using simple config like:

module.exports = {
  storybookShots: {
    storybookUrl: './storybook-static',
  },
  lostPixelProjectId: 'your-id',
  apiKey: process.env.LOST_PIXEL_API_KEY,
};

And storybook config:

const config: StorybookConfig = {
  stories: ['../src/**/*.stories.@(ts|tsx)'],
  addons: [
    '@storybook/addon-links',
    '@storybook/addon-essentials',
    '@storybook/addon-interactions',
  ],
  framework: {
    name: '@storybook/nextjs',
    options: {},
  },
  docs: {
    autodocs: 'tag',
  },
  features: {
    storyStoreV7: true,
  },
};

Build storybook
Use lost-pixel v3.4.0 image to upload diffs
See false negatives in lost-pixel app.

Expected behavior

We would expect consistent image size between builds

Lost Pixel information

No response

lost-pixel logs from CI

No response

About this issue

Original URL
State: closed
Created 4 months ago
Reactions: 2
Comments: 15

Most upvoted comments

Concurrency is definitely at play here. How heavy are the pages?

We recommend 4-5 for oss users if your gh action runners are not super beefy. But as you legitimately noticed the heaviness of the pages also matter.

Your runner just can’t handle rendering all of the stuff concurrently and there is no easy solution to that apart from:

Splitting visual tests into more sensible(smaller) ones
reduce concurrency
bump up machine resources

I am not sure what storybook did across the update but it might in a minor way influence something. Still I can’t imagine anything major

d-ivashchuk on Mar 18, 2024

Yes, I’m in OSS mode running on GitHub Actions

mbaumanndev on Mar 18, 2024

yeah, if we can somehow remedy this on our side - would be happy to do so! reproduction would be very helpful!

@benevbright @mbaumanndev - you are running in OSS mode, right?

d-ivashchuk on Mar 18, 2024

It would be 3.13.0

d-ivashchuk on Feb 21, 2024

hey @e-e-e! could you please try the most up to date version of lost pixel first. If you storybook build is not public and you are on the platform we could look at it together in our shared slack or private discord channel, this could move things way faster!

d-ivashchuk on Feb 21, 2024