cypress: OpenSSL errors when cached SSL certificates are corrupted

Current behavior

There is a race condition in https-proxy that can cause the CA store to become corrupted if multiple Cypress processes are sharing the same appdata directory simultaneously. This causes errors like the following when visiting HTTPS websites:

Error: error:0b000074:X.509 certificate routines:OPENSSL_internal:KEY_VALUES_MISMATCH
Error: error:0900006e:PEM routines:OPENSSL_internal:NO_START_LINE

And other errors relating to corrupt/mismatched private keys and SSL certificates.

Desired behavior

The race condition is avoided, probably with the addition of a lockfile when generating/writing CA certificates.

Note: Cypress is not generally designed to run as multiple processes sharing a home directory.

Workaround (Linux-only)

A workaround is to set a different XDG_CONFIG_HOME environment variable for each Cypress process that is running. This will cause each process to have its own CA store, eliminating the possibility of a race condition.

Example:

# assuming these are somehow run simultaneously
XDG_CONFIG_HOME=/tmp/cyhome1 cypress run...
XDG_CONFIG_HOME=/tmp/cyhome2 cypress run...

This may have side-effects outside of just fixing this issue, but it is the only workaround for now.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Reactions: 5
  • Comments: 32 (10 by maintainers)

Most upvoted comments

@ts-23 I’ve experienced same problem, and after I cleared cypress app data as mentioned in Troubleshooting guide, problem gone 😃 https://docs.cypress.io/guides/references/troubleshooting.html#To-clear-App-Data

Hey guys! do we have a fixed solution for this problem?; here I am still having the same problem dictated by you.

Wow, ok, thanks to @CypressHarry I realize now that the env var for the workaround is NOT XDG_HOME_DIR as I originally posted, it is actually XDG_CONFIG_HOME. So the workaround is actually:

# assuming these are somehow run simultaneously
XDG_CONFIG_HOME=/tmp/cyhome1 cypress run...
XDG_CONFIG_HOME=/tmp/cyhome2 cypress run...

My bad 😅

The code for this is done in cypress-io/cypress#17628, but has yet to be released. We’ll update this issue and reference the changelog when it’s released.

@adrian861 I think you’re right, the workaround is hacky and plus it does not fix this for everyone all the time, so this warrants further investigation. Especially since this issue seems to occur even with 1 Cypress process running sometimes, and even in CI.

The solution would be to add error handling in https-proxy where it calls addContext so it can recover appropriately if the SSL cert cached on disk is corrupted.

We are running Cypress in Google Cloud Build with parallelization through separate steps (which are docker containers that share disk volume mounts). We also ran into this same error: Error: error:0900006e:PEM routines:OPENSSL_internal:NO_START_LINE - ERR_OSSL_PEM_NO_START_LINE.

We set the environment variable suggested XDG_HOME_DIR=/tmp/cyhome1…(2,3,4, etc.); however, it did not resolve the issue for us. Any other suggestions?

I’m not sure why these .pem and .key files were empty for a handful of domains, but deleting them and rerunning a cypress test successfully regenerated them as non-empty and the tests started to pass.

Cypress should probably have better error handling, it sounds like Cypress exited in the middle of a HTTPS visit or something and left these files empty, but there’s no reason it can’t gracefully handle errors here…

Unfortunately, the XDG_CONFIG_HOME workaround doesn’t work if you use the cypress.run() API. Although the API accepts an env option with environment variables, those variables aren’t actually set as OS-level environment variables, so doing something like this has no effect:

cypress.run({
  // Doesn't work
  env: { XDG_CONFIG_HOME: '/tmp/foo' }
});

Downgrading from version 5.6.0 to 5.1.0 fixed it for us. Judging by the other bug reports, this issue must have been introduced between versions 5.1.0 and 5.4.0. Here’s a full backtrace from version 5.6.0, in case it’s helpful:

12:30:57 Error: error:0900006e:PEM routines:OPENSSL_internal:NO_START_LINE
12:30:57     at Object.createSecureContext (_tls_common.js:129:17)
12:30:57     at Server.addContext (_tls_wrap.js:1407:32)
12:30:57     at /root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/lib/server.js:222:23
12:30:57     at tryCatcher (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/util.js:16:23)
12:30:57     at Promise._settlePromiseFromHandler (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:512:31)
12:30:57     at Promise._settlePromise (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:569:18)
12:30:57     at Promise._settlePromise0 (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:614:10)
12:30:57     at Promise._settlePromises (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:694:18)
12:30:57     at Promise._fulfill (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:638:18)
12:30:57     at Promise._settlePromise (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:582:21)
12:30:57     at Promise._settlePromise0 (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:614:10)
12:30:57     at Promise._settlePromises (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:694:18)
12:30:57     at Promise._fulfill (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:638:18)
12:30:57     at Promise._resolveCallback (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:432:57)
12:30:57     at Promise._settlePromiseFromHandler (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:524:17)
12:30:57     at Promise._settlePromise (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:569:18)
12:30:57     at Promise._settlePromise0 (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:614:10)
12:30:57     at Promise._settlePromises (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:694:18)
12:30:57     at Promise._fulfill (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:638:18)
12:30:57     at PromiseArray._resolve (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise_array.js:126:19)
12:30:57     at PromiseArray._promiseFulfilled (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise_array.js:144:14)
12:30:57     at PromiseArray._iterate (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise_array.js:114:31)
12:30:57     at PromiseArray.init [as _init] (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise_array.js:78:10)
12:30:57     at Promise._settlePromise (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:566:21)
12:30:57     at Promise._settlePromise0 (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:614:10)
12:30:57     at Promise._settlePromises (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:694:18)
12:30:57     at Promise._fulfill (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:638:18)
12:30:57     at PromiseArray._resolve (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise_array.js:126:19)
12:30:57     at PromiseArray._promiseFulfilled (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise_array.js:144:14)
12:30:57     at Promise._settlePromise (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:574:26)
12:30:57     at Promise._settlePromise0 (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:614:10)
12:30:57     at Promise._settlePromises (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/promise.js:694:18)
12:30:57     at _drainQueueStep (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/async.js:138:12)
12:30:57     at _drainQueue (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/async.js:131:9)
12:30:57     at Async._drainQueues (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/async.js:147:5)
12:30:57     at Immediate.Async.drainQueues [as _onImmediate] (/root/.cache/Cypress/5.6.0/Cypress/resources/app/packages/https-proxy/node_modules/bluebird/js/release/async.js:17:14)
12:30:57     at processImmediate (internal/timers.js:456:21)
12:30:57  {
12:30:57   library: 'PEM routines',
12:30:57   function: 'OPENSSL_internal',
12:30:57   reason: 'NO_START_LINE',
12:30:57   code: 'ERR_OSSL_PEM_NO_START_LINE'
12:30:57 }

@ts-23 that is the same issue as the OP. Unless you are saying you have no other Cypress processes running?

Yes @flotwig, that is correct there was only one Cypress process running.