vscode-extension-tester: [🚫 Bug] "WebDriverError: unknown error: Chrome failed to start: exited abnormally" issue

I’m evaluating vscode-extension-tester, and it looks promising, but I’m running into a few issues. Namely, I can run (more or less) once, but subsequent attempts to run I get the error, “WebDriverError: unknown error: Chrome failed to start: exited abnormally”.

I searched for this issue, and I see that it’s been reported in the past, but the issues have been closed. I’ve done a lot of A-B testing to try and troubleshoot this and narrow down the issue, but I’m still stumped. Here’s what I’m experiencing…

1. After running npm install --save-dev vscode-extension-tester and then cloning vscode-extension-tester,

2. I cd into vscode-extension-tester/sample-projects/

3. When I ran ls -ll, only helloworld-sample is listed (this is important later)

4. I then cd into helloworld-sample

5. I then run npm install

6. I then run npm run ui-test

7. At this point, the resources were dowloaded and the tests started to run

8. I’m then presented with:

 WARNING  LICENSE.md, LICENSE.txt or LICENSE not found
Do you want to continue? [y/N]

Issue 1: OK, this is a minor annoyance, but it would be nice if the repro had a LICENSE.md file, so I’m not prompted everytime I run.

I press Y and continue…

1. VS Code opens, and the tests run…

2. The “Get a view control and open its associated view” test passes, but…

3. Issue 2: the “Shows explorer view control (container)” test fails with the following error:

 1) Activity Bar Example Tests
       Shows explorer view control (container):
     StaleElementReferenceError: stale element reference: element is not attached to the page document
  (Session info: chrome=100.0.4896.160)
      at Object.throwDecodedError (node_modules/selenium-webdriver/lib/error.js:522:15)
      at parseHttpResponse (node_modules/selenium-webdriver/lib/http.js:589:13)
      at Executor.execute (node_modules/selenium-webdriver/lib/http.js:514:28)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)
      at async Driver.execute (node_modules/selenium-webdriver/lib/webdriver.js:740:17)

OK, at this point what I see in steps 8 and 11 are minor annoyances, but now, here’s the issue…

1. I then run npm run ui-test again…

Issue 3: This time, VS Code does not launch, and none of the tests run. In the console, I see this:

Launching browser...
  1) "before all" hook in "{root}"
  2) "after all" hook in "{root}"

  0 passing (778ms)
  2 failing

  1) "before all" hook in "{root}":
     WebDriverError: unknown error: Chrome failed to start: exited abnormally.
  (chrome not reachable)
  (The process started from chrome location /var/folders/rn/p50bsb8j3ml9tklgv2b2yx2w0000gp/T/extest-code/Contents/MacOS/Electron is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
      at Object.throwDecodedError (node_modules/selenium-webdriver/lib/error.js:522:15)
      at parseHttpResponse (node_modules/selenium-webdriver/lib/http.js:589:13)
      at Executor.execute (node_modules/selenium-webdriver/lib/http.js:514:28)
      at processTicksAndRejections (node:internal/process/task_queues:96:5)

  2) "after all" hook in "{root}":
     TypeError: Cannot read properties of undefined (reading 'manage')
      at VSBrowser.<anonymous> (node_modules/vscode-extension-tester/out/browser.js:126:48)
      at Generator.next (<anonymous>)
      at /Users/my-user/src/github.com/redhat-developer/vscode-extension-tester/sample-projects/helloworld-sample/node_modules/vscode-extension-tester/out/browser.js:8:71
      at new Promise (<anonymous>)
      at __awaiter (node_modules/vscode-extension-tester/out/browser.js:4:12)
      at VSBrowser.quit (node_modules/vscode-extension-tester/out/browser.js:125:16)
      at Context.<anonymous> (node_modules/vscode-extension-tester/out/suite/runner.js:81:35)
      at Generator.next (<anonymous>)
      at /Users/my-user/src/github.com/redhat-developer/vscode-extension-tester/sample-projects/helloworld-sample/node_modules/vscode-extension-tester/out/suite/runner.js:8:71
      at new Promise (<anonymous>)

I spent a fair amount of time this afternoon trying to figure this out, and attempted things like deleting the files in test-resources, but what I found was this…

After ui-tests successfully runs (more or less) the first time, in vscode-extension-tester/sample-projects, if I run ls -ll, I get:

srwxr-xr-x   1 my-user  staff     0B Aug 17 16:12 helloworld-samp
drwxrwxr-x@ 18 my-user  staff   576B Aug 17 16:13 helloworld-sample

So there’s an extra file, named “helloworld-samp” that’s being created.

I’ve found is that if I…

1. run npm run ui-test (first time)
2. run rm ../helloworld-samp
3. then run npm run ui-test …that it works (well, the “Shows explorer view control (container)” test still fails, but at least I’m able to run and I don’t get the “WebDriverError: unknown error: Chrome failed to start: exited abnormally” error.

…and then I repeat #2 and #3

Curious as to what the “s” file attribute is, I read here (https://askubuntu.com/questions/919952/what-does-the-s-in-srwxr-xr-x-in-the-ls-output-mean) that the “s” stands for socket. So it looks like during the test, something is opening a socket called “helloworld-samp”, saving it in vscode-extension-tester/sample-projects, and then not cleaning up.

If it matters, I’m running this locally on a mac OSX x64 machine.