kubernetes: Got "repomd.xml signature could not be verified for kubernetes" error when installing Kubernetes from yum repo on Amazon Linux 2

I’m seeing the same. You can get it to work by setting repo_gpgcheck=0 in /etc/yum.repos.d/kubernetes.repo but that is obviously not a real solution.

mv /etc/yum.repos.d/kubernetes.repo /etc/yum.repos.d/kubernetes.repo.bak
cat >> /etc/yum.repos.d/kubernetes.repo <<EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
EOF

retry

Remove these two lines from your kubernetes.repo file-> gpgcheck=1 repo_gpgcheck=1 It worked for me. These verify package signatures after download.

You can re-import the keys: rpm --import https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

Hi, I’m writing here on behalf of the Amazon Linux team.

This bug is an intersection of the version of GnuPG 2 available in AL2, the version of GCC used to build it, and PGP keys that contain signature notations flagged as “critical”.

We’ve applied the change from GnuPG 2.1 and later that fixes this issue, and it will be available in our next repository push. I’ll post here again when the package is published.

It’s still failing while I was trying to install kubectl on ec2 with amazon linux

repo_gpgcheck=0 worked for me as well, needs a better solution

It’s still failing while I was trying to install kubectl on ec2 with amazon linux

Still failing, had to disable gpgcheck as a workaround. Is there any chance for a fix? I ask it as the issue is close to 1 year old.

I think I’m agreeing with @mattsawyer77.

While we’re running a different setup (not on Amazon Linux 2), in CentOS7 installs have started to fail with the https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/repodata/repomd.xml: [Errno -1] repomd.xml signature could not be verified for kubernetes error.

I can reproduce the issue by starting a centos7 container and trying to install anything.

Interesting data points:

• This was working fine as of 6pm EDT on 2021-03-31
• We noticed first failures around 8am EDT on 2021-04-01
• If we compare an existing, recent installation that is still up the keys look different, but still fail even on the “old” node.

Current output when installing:

kubernetes/x86_64/signature                                                                                                                                                        |  844 B  00:00:00
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>"
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Is this ok [y/N]: y
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
kubernetes/x86_64/signature                                                                                                                                                        | 1.4 kB  00:00:11 !!!
https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/repodata/repomd.xml: [Errno -1] repomd.xml signature could not be verified for kubernetes
Trying other mirror.

output when trying yum commands on older, previously functioning node:

kubernetes/signature                                                                                                                                                               |  844 B  00:00:00
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0x307EA071:
Userid     : "Rapture Automatic Signing Key (cloud-rapture-signing-key-2021-03-01-08_01_09.pub)"
Fingerprint: 7f92 e05b 3109 3bef 5a3c 2d38 feea 9169 307e a071
From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Is this ok [y/N]: y
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
kubernetes/signature                                                                                                                                                               | 1.4 kB  00:00:17 !!!
https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/repodata/repomd.xml: [Errno -1] repomd.xml signature could not be verified for kubernetes
Trying other mirror.
updates

To reproduce:

docker run -it centos:centos7 /bin/bash

then:

cat <<EOF | tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF

yum list -y kubectl

Repeating this with docker run -it centos:centos8 /bin/bash works. Note in centos8 gnupg2-2.2.20-2.el8 is installed instead of gnupg2-2.0.22-5.el7_5.

I think this is due to Amazon Linux 2 shipping an old version of GnuPG, and something about the repomd.xml.asc signature requires a newer version.

repomd.xml is signed with key ID 6A030B21BA07F4FB, one of two keys present in https://packages.cloud.google.com/yum/doc/yum-key.gpg . This can be used to verify the signature as follows:

$ gpg2 --version | head -n1
gpg (GnuPG) 2.2.13
$ curl -fsSL https://packages.cloud.google.com/yum/doc/yum-key.gpg | gpg2 --import
gpg: key 3746C208A7317B0F: public key "Google Cloud Packages Automatic Signing Key <gc-team@google.com>" imported
gpg: key 6A030B21BA07F4FB: public key "Google Cloud Packages Automatic Signing Key <gc-team@google.com>" imported
gpg: Total number processed: 2
gpg:               imported: 2
$ curl -fsSL -O https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/repodata/repomd.xml
$ curl -fsSL -O https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/repodata/repomd.xml.asc
$ gpg2  --verify repomd.xml.asc 
gpg: assuming signed data in 'repomd.xml'
gpg: Signature made Tue 28 May 2019 05:57:25 AM PDT
gpg:                using RSA key 6A030B21BA07F4FB
gpg: Good signature from "Google Cloud Packages Automatic Signing Key <gc-team@google.com>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 54A6 47F9 048D 5688 D7DA  2ABE 6A03 0B21 BA07 F4FB

The story is different on Amazon Linux 2, unfortunately:

$ gpg2 --version | head -n1
gpg (GnuPG) 2.0.22
$ curl -fsSL https://packages.cloud.google.com/yum/doc/yum-key.gpg | gpg2 --import
gpg: keyring `/root/.gnupg/secring.gpg' created
gpg: key A7317B0F: public key "Google Cloud Packages Automatic Signing Key <gc-team@google.com>" imported
gpg: key BA07F4FB: public key "Google Cloud Packages Automatic Signing Key <gc-team@google.com>" imported
gpg: Total number processed: 2
gpg:               imported: 2  (RSA: 2)
$ curl -fsSL -O https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/repodata/repomd.xml
$ curl -fsSL -O https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/repodata/repomd.xml.asc
$ gpg2  --verify repomd.xml.asc
gpg: Signature made Sun 02 Jun 2019 07:17:55 PM UTC using RSA key ID BA07F4FB
gpg: assuming bad signature from key BA07F4FB due to an unknown critical bit
gpg: BAD signature from "Google Cloud Packages Automatic Signing Key <gc-team@google.com>"

GnuPG 2.0.22 outright rejects the signature on the repository metadata with assuming bad signature from key BA07F4FB due to an unknown critical bit. I haven’t been able to figure out what critical bit it’s referring to – there don’t appear to be any on the signature or key – but whatever GnuPG 2.0.22 is upset about is most likely the root cause.

This only affects the repomd signature, so there’s zero reason to disable gpgcheck as several others have suggested. Disabling repo_gpgcheck is sufficient and preserves package signature verification (although it’s still not an ideal workaround…)

Hi,

Same problem here running, using repo_gpgcheck=0 fixes it but shouldn’t that be fixed ?

So funny thing… following along @drakedevel’s line of thinking, I installed the CentOS gpg (still 2.0.22), on amzn2 and now kubernetes signatures are being accepted:

sudo yum install -y \
http://mirror.centos.org/centos/7/os/x86_64/Packages/gnupg2-2.0.22-5.el7_5.x86_64.rpm

gpg (GnuPG) 2.0.22
libgcrypt 1.5.3
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Home: ~/.gnupg
Supported algorithms:
Pubkey: RSA, ?, ?, ELG, DSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
        CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: MD5, SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2

just faced the same issue on AWS EC@ instance with Amazon Linux 2 installed

Any thought?

You can re-import the keys: rpm --import https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

This fixed the issue on my side

this did not fix it for me, but:

sudo yum install -y http://mirror.centos.org/centos/7/os/x86_64/Packages/gnupg2-2.0.22-5.el7_5.x86_64.rpm

did

mv /etc/yum.repos.d/kubernetes.repo /etc/yum.repos.d/kubernetes.repo.bak
cat >> /etc/yum.repos.d/kubernetes.repo <<EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
EOF

retry

sudo yum install -y
http://mirror.centos.org/centos/7/os/x86_64/Packages/gnupg2-2.0.22-5.el7_5.x86_64.rpm

it worked for me

Try this way.It worked for me. rpm --import https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg sudo yum install -y http://mirror.centos.org/centos/7/os/x86_64/Packages/gnupg2-2.0.22-5.el7_5.x86_64.rpm

@mattsawyer77 / @luk4z7, see https://github.com/kubernetes/kubernetes/issues/97077#issuecomment-741698560

Solved the issue for me.

---

EDIT: Nevermind, it did not 😦

It appears that this issue, or one very similar, has recurred in the last few days. We’ve found that we can’t create new nodes currently, and we’re getting essentially the same error as the OP.

I can confirm it’s in the package repo now, I did not expect to find this issue solved just after its second birthday, 🔔 ring the bell

Updated:
gnupg2.x86_64 0:2.0.22-5.amzn2.0.4

This can be closed

The same issue. Amazon Linux

load back /etc/yum.repos.d/kubernetes.repo in editor
flip repo_gpgcheck=0 to 0
yum update
yum install kubectl
-= success =-

paste below inside kuberne repo config file then retry.

[kubernetes] name=Kubernetes baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

Might be linked to https://bugzilla.redhat.com/show_bug.cgi?id=1768206 Workaround: this command will import gpg key for yum yum -q makecache -y --disablerepo='*' --enablerepo=kubernetes