storybook: StoryShots/Storyshots-Puppeteer is inconsistent and buggy with docker

Describe the bug We are trying to use Storyshots with puppeteer to take snapshots of our storybook and integrate it to out ci Jenkins pipeline, but are running into many problems when using it with docker, specifically these issues : Screenshots are inconsistent, primarily each time we run the tests with the exact same project and storybook hosted locally, we keep getting these errors related to mismatched resolution and diffs show all pages are slightly misaligned to the right.

We have 107 stories and somehow this is what happens when we run in docker

107 Fails + 107 Passes ?

To Reproduce My base chromium docker image, the rest is copying the project and running yarn test

FROM node:11.7.0-alpine
ENV CHROME_BIN="/usr/bin/chromium-browser"
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD="true"

RUN apk update && apk upgrade && \
    echo @edge http://nl.alpinelinux.org/alpine/edge/community >> /etc/apk/repositories && \
    echo @edge http://nl.alpinelinux.org/alpine/edge/main >> /etc/apk/repositories && \
    apk add --no-cache \
      chromium@edge=~73.0.3683.103 \
      nss@edge \
      freetype@edge \
      freetype-dev@edge \
      harfbuzz@edge \
      ttf-freefont@edge

RUN addgroup -S pptruser && adduser -S -g pptruser pptruser \
    && mkdir -p /home/pptruser/Downloads /app \
    && chown -R pptruser:pptruser /home/pptruser \
    && chown -R pptruser:pptruser /app

The docker image based on it.
Copies the storybook and runs the tests.

FROM node:11.7.0-puppeteer

WORKDIR /src
COPY --from= /src       .
COPY .                      storybook
RUN yarn install:ci:dev
CMD cd storybook && yarn test

imageSnapshots.js

import initStoryshots from '@storybook/addon-storyshots';
import { imageSnapshot } from '@storybook/addon-storyshots-puppeteer';

require('babel-plugin-require-context-hook/register')();

const waitBeforeSnapshotTime = 400;

// Hide all elements with classes containing these phrases
const ignoredElementsSelectors = ['MuiCircularProgress', 'MuiLinearProgress'];

const disableAnimationElements = ignoredElementsSelectors => {
  const animationSelectors = ignoredElementsSelectors.map(animationClass => `[class*='${animationClass}']`);
  const animatedPageElements = document.querySelectorAll(animationSelectors);

  animatedPageElements.forEach(pageElement => {
    pageElement.style.visibility = 'hidden';
  });
};

const storybookUrl = 'http://app-storybook:7071/';

const storiesToTestRegex = '^((?!.*?(Sketch|IFrame)).)*$';

const beforeScreenshot = page =>
  new Promise(resolve =>
    page
      .evaluate(disableAnimationElements, ignoredElementsSelectors)
      .then(() => page.setViewport({ width: 1920, height: 1080 }))
      .then(() => setTimeout(resolve, waitBeforeSnapshotTime)),
  );

const customizePage = page => page.setViewport({ width: 1920, height: 1080 });


initStoryshots({
  framework: 'react',
  suite: 'Image Storyshots',
  storyKindRegex: storiesToTestRegex,
  test: imageSnapshot({
    storybookUrl,
    beforeScreenshot,
    customizePage,
    chromeExecutablePath: process.env.CHROME_BIN,
  }),
});

Expected behavior The screenshots should consistent with the local ones especially since I only install puppeteer with addon-storyshots-puppeteer so there is no version mismatch.

The documentation does not make it clear that there might be differences between os’s and that results might be inconsistent.

Currently, for me, it is broken and I am trying super hard to make it work and to understand most the process required for it to work.

Thank you for reading my issue, I’m really hoping for a response and hope this will be helpful for others. 😃