BackstopJS: BackstopJS hangs in Docker container

I just added a link to this issue in the Docs.

Ok… I solved the issue. It was a docker issue all along. 😓 Linking the reference material that got me to solving the problem (which was ironically because I was doing research into alternatives to chromy and found puppeteer): https://github.com/GoogleChrome/puppeteer/blob/master/docs/troubleshooting.md#tips

essentially adding --shm-size=2gb and --cap-add=SYS_ADMIN to my docker run command let chrome have enough shared memory to finish running the test.

What was probably happening was that the chrome process was running out of shared memory and couldn’t finish loading the page (huge page, lots of assets, etc…) So then when that occurred, it was being treated as a gotoTimeout because it couldn’t load the page with the shared space provided by docker by default (which is 64mb by default).

After increasing the shared memory size to 2gbs the process completed successfully. 😄

@kiran-redhat When I saw this issue, either larger pages or large screencap sizes seemed to require more shared memory – I don’t remember which or if I ever tried to figure out the difference. Obviously more running at once (higher asyncCaptureLimit) also made it happen more often, but I remember I would have the problems with some pages/captures even with asyncCaptureLimite = 1 until I raised the docker shared memory available.

I did not need 2gb for my tests to run at asyncCaptureLimit=10, just 512mb, but if your pages/captures are large enough maybe you need even more, or maybe there’s another problem.

I don’t believe the actual error I got involved “GotoTimeoutError”. I think it was WaitTimeoutError: evaluate(). I could be wrong – I no longer have the error in front of me, but I remember it matching #537 pretty closely.

When debugging, I was able to run df and see the usage of /dev/shm going up and run ps to see chrome processes running and then getting killed (all in the backstop container while backstop was running). I also turned on enough logging (some by tweaking the src of installed js libs – again, in the container) to actually see chrome being killed with a bus error trying to address invalid or out of bounds memory addresses.

Good luck!

Even with a gotoTimeout of 5min+ the process silently fails and eventually receives the exit signal with code 0.

Added to cli/index.js:50:

  process.on('exit', function (code) {
    console.log(`Exit signal recieved: ${code}`);
    process.exit(exitCode);
  });

And then in my docker container’s bash:

Starting Chromy: {"chromeFlags":["--disable-gpu","--force-device-scale-factor=1","--disable-infobars=true","--window-size=1024,768"],"port":9226,"waitTimeout":120000,"visible":false,"gotoTimeout":300000}
Cookie state restored with: []
SCENARIO > en-us WoW Homepage Full Page check
9226 Chrome v61 detected.
9226 ***WARNING! CHROME VERSION 62 OR GREATER IS REQUIRED. PLEASE UPDATE YOUR CHROME APP!***
Starting Chromy: {"chromeFlags":["--disable-gpu","--force-device-scale-factor=1","--disable-infobars=true","--window-size=1600,900"],"port":9227,"waitTimeout":120000,"visible":false,"gotoTimeout":300000}
Cookie state restored with: []
SCENARIO > en-us WoW Homepage Full Page check
9227 Chrome v61 detected.
9227 ***WARNING! CHROME VERSION 62 OR GREATER IS REQUIRED. PLEASE UPDATE YOUR CHROME APP!***
Exit signal recieved: 0

Note that the process doesn’t even enter the report portion.

Edit: Kept tracking all the way down into Chromy itself. I’m not sure why but the process just hangs on the this.client.Page.navigate({url: url}) call. That’s controlled by the chrome-remote-interface which just fails after the timeout and causes chromy to throw GotoTimeoutError. But it still ends the process, which is annoying. 😕