cypress: 10.3.0 regression - test runner randomly fails when running electron locally

I should also note that I’m seeing this crash super reliably right after the 32nd spec, rather than jumping around to various locations as you described. Could possibly be an OS difference.

Can you share the cypress.config.ts for that project? It looks like something is failing in the child process where the config is executed.

Hi, @lmiller1990 I was trying to create a reproducible example, but unfortunately, I could not. The cypress.config.ts and all the cypress-related files (support, index etc.) are the same for all projects. Then I have solved my problem by deleting the Cypress folder under the Application Support dir. When I track it down, I found that the following directory is the root cause: /Users/<myUsername>/Library/Application Support/Cypress/Partitions/interactive/Service Worker. Unfortunately, I could not track down which file under it caused the issue.

I hope it helps the resolution of the issue in general, or at least for the ones who face similar to my case.

Alrighty! thats awesome intel, thank you @lmiller1990 - ill try and give that a go and see if the error still happens - and report back. 😃

Ok, I looked into this, here’s how you can use old Chromium versions. First, obtain the version you want. I grabbed one here: https://vikyd.github.io/download-chromium-history-version/#/

I downloaded the zip called chrome-linux.zip. I also noticed Cypress has some old Chrome versions, if that’s useful to someone. Those are here.

I got my desired version (say 90) and extracted it to my desktop:

$ ls /home/lachlan/Desktop/chr/chrome-linux

chrome			icudtl.dat	       nacl_irt_x86_64.nexe
chrome_100_percent.pak	libEGL.so	       product_logo_48.png
chrome_200_percent.pak	libGLESv2.so	       resources
chrome_sandbox		locales		       resources.pak
chrome-wrapper		MEIPreload	       swiftshader
ClearKeyCdm		nacl_helper	       v8_context_snapshot.bin
crashpad_handler	nacl_helper_bootstrap  xdg-mime
extensions		nacl_helper_nonsfi     xdg-settings

chrome is the binary. I’m on linux so I can just ./chrome to launch it, if you want to try browsing the web with an old verison of Chromium for whatever reason.

You can customize available browsers in Cypress. See the docs. I did it:

import { defineConfig } from 'cypress'

export default defineConfig({
  'e2e': {
    'setupNodeEvents': (on, config) => {
      config.browsers = config.browsers.concat({
        name: 'Old Chromium',
        channel: 'stable',
        family: 'chromium',
        displayName: 'Old Chromium',
        version: '90',
        path: '/home/lachlan/Desktop/chr/chrome-linux/chrome', // <= this is the important part, pass the path
        majorVersion: '90',
      })
      return config
    },
    'baseUrl': 'http://localhost:3500',
  },
})

Now I can see it:

You could easily queue up many versions like this for testing. I also think you can just do yarn cypress open --browser <path_to_browser> but I like to have lots of them available for testing various things.

Hmm good point, this got me thinking… it’s definitely possible to use a different version of Electron if you have it installed locally. Documentation.

We should be able to isolate this. First, we need to answer the question:

• is this an Electron version issue (eg - would downgrading electron even solve the issue)?
• or, is this a Cypress 10 only issue?
• finally, does this only happen with component testing? That is the assumption here - I didn’t think about it too much, but if this is a CT only issue, that narrows the surface area a lot.

Downgrading the electron version we ship is a little more involved in just changing a dependency, but maybe we debug it like this:

1. Run Cypress 10 and passing different electron versions, like documented: cypress run --component --browser /usr/bin/some-electron-version
2. Try lots of Electrons
3. Try the same thing, but with Cypress 9
4. Isolate issue!

Good input - I feel like we are slowly closing in on this.

I think we’d all love to see this fixed 🥲 along with this other long standing electron issue: https://github.com/cypress-io/cypress/issues/15932

I am really unsure if this is even a bug in Cypress or a bug in electron. If anyone wants to dig into this, happy to assist - tons of good info here, but I don’t think I can actively debug this in the immediate future.

Thanks! Good to know this is isolated to electron… why is electron less reliable, I thought it would be shipping pretty much the same code as chrome, at least the core components.

FYI i worked out that the tests only ran in CI because we use chrome in CI. 🤪 I have updated the title to remove the reference to it being OK in CI

Yeah, it appears to work fine under Chrome, no crashes at all, so that’s a workaround. Added ‘electron’ to the title to help others locate the issue.

@glomotion are you able to invite @BlueWinds to your repo, too? They will be taking a look at this one. Thanks!

@glomotion Right, that’s what the GH repo invite was. (Was out for 1 week).

I will try running this and see if I run into the same problem. Having a minimal repro makes it sooooo much easier to debug and by extension much more likely the problem is solved, thanks for this!

Hi Tim - thanks for the bug report, your issues are always informative.

Normally the issue is the opposite - CI fails, local passes. Disabling code coverage -> success is even more weird. What plugins/libs/versions are you using for code coverage?

Does this happen in the previous version of Cypress? If it really was introduced 10.3.0, that’s a small surface area, it should be easy to fix - can you provide a minimal reproduction, along with answers to the above questions?