setup-gcloud: docker-credential-gcloud + gcloud 298.0.0 + ubuntu-18.04: "OPENSSL_1_1_1 not found" error

Confirming that this issue also exists in Ubuntu 20.04 with gcloud version 301.0.0.

To fix:

sudo apt install google-cloud-sdk=297.0.1-0

What worked for us was to use the env var:

CLOUDSDK_PYTHON: /usr/bin/python

The magical incantation

export LD_LIBRARY_PATH=/usr/local/lib

as described above, worked for me.

I was having this problem using the google/cloud-sdk image on CircleCI.

My workaround was changing the docker-compose install method from the one recommended on the CircleCI docs to:

pip3 install docker-compose --upgrade

export LD_LIBRARY_PATH=/usr/local/lib worked for me too.

@jtrh getting the same error as you while doing docker-compose build. I tried prefixing it with CLOUDSDK_PYTHON=$(which python3) didn’t work either, same error.

System info:

$ gcloud -v
Google Cloud SDK 302.0.0
alpha 2020.07.17
beta 2020.07.17
bq 2.0.58
cloud-build-local
core 2020.07.17
gsutil 4.52
kubectl 1.15.11
pubsub-emulator 0.1.0

$ lsb_release --all
No LSB modules are available.
Distributor ID: LinuxMint
Description:    Linux Mint 19.3 Tricia
Release:        19.3
Codename:       tricia

$ docker-compose version
docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.1.0j  20 Nov 2018

EDIT: this fixed it for me https://github.com/openssl/openssl/issues/5845#issuecomment-378601109

I got the same error on Ubuntu 18.04 and gcloud 304.0.0

export CLOUDSDK_PYTHON=/usr/bin/python worked for me as well.

@bharathkkb The issue I have with this new version is that it breaks gcloud auth configure-docker.

Here is my use case below.

build-deploy:
    needs: bump-version
    name: "Build and Deploy"
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0

      - name: Setup GCP credentials
        uses: google-github-actions/setup-gcloud@master
        with:
          project_id: ${{ secrets.GCLOUD_PROJECT_ID }}
          service_account_key: ${{ secrets.GCP_SA_KEY_JSON }}
          export_default_credentials: true

      - name: Docker pull
        run: |
          gcloud auth configure-docker
          docker-compose pull

The error I get is:

Run gcloud auth configure-docker
  gcloud auth configure-docker
  docker-compose pull
  shell: /bin/bash -e ***0***
  env:
    GKE_PROJECT_ID: ***
    GKE_EMAIL: ***
    GITHUB_SHA: 819aa0a83ff3641e84c2ce6676260186c4344330
    GITHUB_BRANCH: refs/heads/master
    GKE_ZONE: ***
    GKE_CLUSTER: ***
    REGISTRY_HOSTNAME: eu.gcr.io
    DEPLOYMENT_ENV: rc
    DEPLOYMENT_COMMIT_SHA: 819aa0a83ff3641e84c2ce6676260186c4344330
    SEM_VERSION: ***
    VERSION_TIMESTAMP: 1614099296
    BUILDHASH: 819aa0a83ff3641e8412de6676260186c4344330
    BUILDTAG: 0.16.75.0
    CLOUDSDK_METRICS_ENVIRONMENT: github-actions-setup-gcloud
    GCLOUD_PROJECT: ***
    GOOGLE_APPLICATION_CREDENTIALS: /home/runner/work/core/core/c7331fe3-c987-4291-9289-4b0ac681b435
Adding credentials for all GCR repositories.
WARNING: A long list of credential helpers may cause delays running 'docker build'. We recommend passing the registry name to configure only the registry you are using.
After update, the following will be written to your Docker config file
 located at [/home/runner/.docker/config.json]:
 ***
  "credHelpers": ***
    "gcr.io": "gcloud",
    "us.gcr.io": "gcloud",
    "eu.gcr.io": "gcloud",
    "asia.gcr.io": "gcloud",
    "staging-k8s.gcr.io": "gcloud",
    "marketplace.gcr.io": "gcloud"
  ***
***

Do you want to continue (Y/n)?  
Docker configuration file updated.
Pulling xxxxxxxxxx        ... 
Pulling xxxxxxxxxx    ... 
Pulling xxxxxxxxxx ... 
Pulling xxxxxxxxxx ... 
Pulling xxxxxxxxxx-service        ... 
Pulling xxxxxxxxxx-service        ... 
ERROR: gcloud failed to load: /tmp/_MEINIeVty/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by /usr/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so)
    gcloud_main = _import_gcloud_main()
    import googlecloudsdk.gcloud_main
    from googlecloudsdk.calliope import base
    from googlecloudsdk.calliope import arg_parsers
    from googlecloudsdk.core import log
ERROR: for artifacts-service  Credentials store error: StoreError('Credentials store docker-credential-gcloud exited with "".')
Traceback (most recent call last):
  File "docker/credentials/store.py", line 80, in _execute
  File "subprocess.py", line 411, in check_output
  File "subprocess.py", line 512, in run
subprocess.CalledProcessError: Command '['/opt/hostedtoolcache/gcloud/329.0.0/x64/bin/docker-credential-gcloud', 'get']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "docker/auth.py", line 264, in _resolve_authconfig_credstore
  File "docker/credentials/store.py", line 35, in get
  File "docker/credentials/store.py", line 93, in _execute
docker.credentials.errors.StoreError: Credentials store docker-credential-gcloud exited with "".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 80, in main
  File "compose/cli/main.py", line 192, in perform_command
  File "compose/metrics/decorator.py", line 18, in wrapper
  File "compose/cli/main.py", line 828, in pull
  File "compose/project.py", line 763, in pull
  File "compose/project.py", line 813, in parallel_pull
  File "compose/parallel.py", line 106, in parallel_execute
  File "compose/parallel.py", line 204, in producer
  File "compose/project.py", line 795, in pull_service
  File "compose/service.py", line 1236, in _do_pull
  File "docker/api/image.py", line 411, in pull
  File "docker/auth.py", line 48, in get_config_header
  File "docker/auth.py", line 324, in resolve_authconfig
  File "docker/auth.py", line 235, in resolve_authconfig
  File "docker/auth.py", line 281, in _resolve_authconfig_credstore
docker.errors.DockerException: Credentials store error: StoreError('Credentials store docker-credential-gcloud exited with "".')
Error: Process completed with exit code 255.

Ran into this issue today using docker-compose to pull an image from GCR from my Ubuntu 20.10 WSL 2 distro.

I already had Python 3 installed (3.8.6 to be exact), but export CLOUDSDK_PYTHON=/usr/bin/python3 didn’t work for me. The workaround export LD_LIBRARY_PATH=/usr/local/lib worked for me.

Thank you @sorliem. The solution you linked also worked for me (gcloud version 304).

Also remark that adding

LD_LIBRARY_PATH=/usr/local/lib;

in the command line didn’t worked. Variable has to be exported.

I get this error when, for example, I execute docker-compose build, but if I exectue this command with sudo it works.

I was able to reproduce this here. I was also able to fix it with the temporary workaround from https://github.com/openssl/openssl/issues/5845#issuecomment-378601109 suggest by @sorliem.

I will continue to look into a possible long term fix.

The magical incantation

export LD_LIBRARY_PATH=/usr/local/lib

as described above, worked for me.

With newest version of gcloud (326.0.0) this no longer works 😦

297.0.1 for arm64 doesn’t seem to be available via apt-get anymore…

340.0.0-0 works with

export LD_LIBRARY_PATH=/usr/local/lib

export LD_LIBRARY_PATH=/usr/local/lib worked for me too.

I had the exact same problem. The above mentioned export worked for me.

Ubuntu 20.04 Google Cloud SDK 322.0.0

Thanks

I can confirm that @bharathkkb workaround mentioned with the export LD_LIBRARY_PATH=/usr/local/lib works for me !

By the way, to make things a bit harder to debug,I am using https://github.com/asdf-vm/asdf to install my stuff. But the workaround works

$ asdf list

argocd
  1.5.4
docker-compose
  1.26.0
gcloud
  293.0.0
gohugo
  extended_0.72.0
helm
  3.2.0
kubectl
  1.15.11
  1.16.11
kubesec
  0.9.2
kustomize
  3.5.5
  3.8.1
sops
  v3.5.0
stern
  1.11.0
terraform
  0.12.16
  0.12.18

So with

docker-compose
  1.26.0
gcloud
  293.0.0

Debian 10 Buster gcloud 308 export CLOUDSDK_PYTHON=/usr/bin/python Worked for me too!

Why is LD_LIBRARY_PATH set to /tmp/_MEI3WIF2B/? Is this a bug with vendored python dependencies in docker-compose?

None of the solutions mentioned above helped me.

Before finally finding my own workaround, I tried:

• downgrading gcloud to v297
• upgrading gcloud to v340 (it was 330 initially)
• export CLOUDSDK_PYTHON=/usr/bin/python3 and export LD_LIBRARY_PATH=/usr/local/lib (both were already in my .zshrc)
• pip3 install docker-compose --upgrade
• changed my local version of Python using pyenv to 3.7.10 which is the same as CPython used by my docker-compose (inspired by this answer)

For anyone using WSL, and probably future me:

What eventually helped me was realizing that because my Ubuntu runs docker-compose through WSL and the entire Docker runs on Windows, I might have some luck setting up the same container in Windows, and then just getting it back up from my usual WSL console and Ubuntu filesystem.

I cloned my repo to Windows filesystem, installed gcloud for Windows, set up gcloud authentication, ran docker-compose up from cmd, and sure enough Docker pulled the images and the container built just fine. Now that my Docker for Windows has all the dependency images, I don’t have any trouble running the container from inside WSL.

I ran into this problem using docker-compose on Ubuntu 20.10 WSL 2 distro.

This fixed it for me:

export CLOUDSDK_PYTHON=/usr/bin/python2

(That’s a 2 not a 3)

Same here. When running docker-compose with latest google SDK, showing the similiar message when pulling base image from gcr. The workaround by using 297.0.1 got it work.

Just ran into this today and the previously suggested fixes didn’t work for me. I ended up re-installing docker-compose which sorted out this issue (pip install docker-compose)

Config:

GH Action runner 2.287.1
Ubuntu v20.04.3
google-github-actions/setup-gcloud@v0.3.0
Google Cloud SDK 370.0.0
Python 3.8.10

Thanks for the solutions posted in the comments.

In my case, what worked (gcloud sdk version from ~1st of October 2021) with Ubuntu 20.04 that had Python 3.8 installed, was:

1. installing app-engine-python (if I’m correct, installs Python 2.7)
2. export CLOUDSDK_PYTHON=/usr/bin/python2.7

Tried with to python 3 path but was still giving me the OPENSSL_1_1_1 not found error, but pinning/forcing to Python 2 is so far working without issues.

Just happened with github actions ubuntu:latest and command gcloud auth configure-docker -q

Debian 10 here, with gcloud v346 and python 3.7.3 (from debian packages). the export workaround didn’t worked: gcloud still tries to find libraries in /tmp/something

I get this error when, for example, I execute docker-compose build

I also got the OPENSSL_1_1_1 error when executing docker-compose build with google-cloud-sdk Debian package version 302.0.0-0. Reverting to version 301.0.0-0 did not fix the problem, but reverting to 300.0.0-0 did.

System information:

• OS:

  # lsb_release --description
  Description:  Ubuntu 20.04 LTS
  

• Google Cloud SDK installed through apt-get.
• Docker Compose:

  # docker-compose version
  docker-compose version 1.26.2, build eefe0d31
  docker-py version: 4.2.2
  CPython version: 3.7.7
  OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019
  

Everyone here put their lucky number with this sdk. i will start from the most like answer

https://github.com/google-github-actions/setup-gcloud/issues/128#issuecomment-673864153 CLOUDSDK_PYTHON=/usr/bin/python worked for me while export CLOUDSDK_PYTHON=$(which python3) didn’t work.

Same problem here. Is there any known fix for the gihub action as described by the OP ?

I’ve started getting this after updating docker-compose on my machine. It happens if I run docker-compose pull.

I was previously on a much older version of docker-compose, which worked. I can’t remember which version, but it might have been installed as long ago as 2017.

EDIT: LD_LIBRARY_PATH=/usr/lib docker-compose pull my_service works. I suppose I shall use that as a workaround for now. Why gcloud expects to find libssl in /tmp/... is… not something I understand.

Error:

ERROR: gcloud failed to load: /tmp/_MEIMOBjio/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by /usr/lib/python3.6/lib-dynload/_ssl.cpython-36m-x86_64-linux-gnu.so)
    gcloud_main = _import_gcloud_main()
    import googlecloudsdk.gcloud_main
    from googlecloudsdk.api_lib.iamcredentials import util as iamcred_util
    from googlecloudsdk.api_lib.util import apis_internal
    from googlecloudsdk.core import properties
    from googlecloudsdk.core import config
    from oauth2client import client
    from oauth2client import transport
    import httplib2
    from httplib2.python3.httplib2 import *
    import ssl
    import _ssl             # if we can't import it, let the error propagate

This usually indicates corruption in your gcloud installation or problems with your Python interpreter.

Please verify that the following is the path to a working Python 2.7 or 3.5+ executable:
    /usr/bin/python3

The above is truncated. Full error here.

Relevant versions:

/usr/bin/python3 --version Python 3.6.9

I assume the Python executable being used is the one in /usr/bin/python3, since that’s what’s in the error message. However, for completeness:

which python3 /home/thom/.pyenv/shims/python3

python3 --version Python 3.6.9

gcloud version Google Cloud SDK 319.0.0 alpha 2020.11.13 beta 2020.11.13 bq 2.0.62 core 2020.11.13 gsutil 4.55 kubectl 1.16.13

lsb_release -d Description: Ubuntu 18.04.5 LTS

docker-compose version docker-compose version 1.27.4, build 40524192 docker-py version: 4.3.1 CPython version: 3.7.7 OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019

openssl version OpenSSL 1.1.1 11 Sep 2018

which openssl /usr/bin/openssl

locate libssl.so.1.1 /snap/core18/1885/usr/lib/x86_64-linux-gnu/libssl.so.1.1 /snap/core18/1932/usr/lib/x86_64-linux-gnu/libssl.so.1.1 /snap/gnome-3-28-1804/128/usr/lib/x86_64-linux-gnu/libssl.so.1.1 /snap/gnome-3-28-1804/145/usr/lib/x86_64-linux-gnu/libssl.so.1.1 /snap/gnome-3-34-1804/36/usr/lib/x86_64-linux-gnu/libssl.so.1.1 /snap/gnome-3-34-1804/60/usr/lib/x86_64-linux-gnu/libssl.so.1.1 /usr/lib/x86_64-linux-gnu/libssl.so.1.1

@michaelr, that only works if you install python2 or add a symlink from /usr/bin/python2 to /usr/bin/python3 (the latter not being recommended since it can lead to programs expecting Python 2 to crash because they’re actually running under Python 3).

Ideally, this problem should be solved within the current state of modern Linux distros, which do not include Python 2.