images: Yarn GPG key expiry prevents running apt

Create a very simple Dockerfile like this:

ARG VARIANT=3.8
FROM mcr.microsoft.com/devcontainers/python:0-${VARIANT}
RUN apt-get update

If you try to build this with docker build . you will get the error:

user@localhost $ docker build .
[+] Building 1.7s (5/5) FINISHED                                                                                                     
 => [internal] load build definition from Dockerfile                                                                            0.0s
 => => transferring dockerfile: 135B                                                                                            0.0s
 => [internal] load .dockerignore                                                                                               0.0s
 => => transferring context: 2B                                                                                                 0.0s
 => [internal] load metadata for mcr.microsoft.com/devcontainers/python:0-3.8                                                   0.3s
 => CACHED [1/2] FROM mcr.microsoft.com/devcontainers/python:0-3.8@sha256:619df220ed3f7c91e96785e8794c1b7ac7af8c27e50be16c8f08  0.0s
 => ERROR [2/2] RUN apt-get update                                                                                              1.4s
------                                                                                                                               
 > [2/2] RUN apt-get update:                                                                                                         
#5 0.190 Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]                                                              
#5 0.198 Get:2 https://dl.yarnpkg.com/debian stable InRelease [17.1 kB]                                                              
#5 0.227 Get:3 http://deb.debian.org/debian-security bullseye-security InRelease [48.4 kB]                                           
#5 0.242 Get:4 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]                                                     
#5 0.246 Err:2 https://dl.yarnpkg.com/debian stable InRelease
#5 0.246   The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging <yarn@dan.cx>
#5 0.330 Get:5 http://deb.debian.org/debian bullseye/main arm64 Packages [8072 kB]
#5 0.536 Get:6 http://deb.debian.org/debian-security bullseye-security/main arm64 Packages [211 kB]
#5 0.558 Get:7 http://deb.debian.org/debian bullseye-updates/main arm64 Packages [12.0 kB]
#5 1.142 Reading package lists...
#5 1.375 W: GPG error: https://dl.yarnpkg.com/debian stable InRelease: The following signatures were invalid: EXPKEYSIG 23E7166788B63E1E Yarn Packaging <yarn@dan.cx>
#5 1.375 E: The repository 'https://dl.yarnpkg.com/debian stable InRelease' is not signed.
------
executor failed running [/bin/sh -c apt-get update]: exit code: 100

An issue for this error can be seen here: https://github.com/yarnpkg/yarn/issues/7866

Attempts to use any of the solutions such as:

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -

sudo apt-key adv --refresh-keys --keyserver keyserver.ubuntu.com

wget -qO - https://raw.githubusercontent.com/yarnpkg/releases/gh-pages/debian/pubkey.gpg | sudo apt-key add -

prior to the apt-get update do not fix the problem.

You can see here: https://github.com/yarnpkg/releases/blob/gh-pages/debian/pubkey.gpg The pubkey was updated 13 hours ago as of the time of this posting. Even that brand new key does not seem to work.

I consider this issue to be quite urgent as it completely prevents me from building my VSCode Python dev environment. I can’t get any work done unless I come up with a new environment using a different docker base image.

About this issue

Original URL
State: closed
Created a year ago
Reactions: 16
Comments: 16 (7 by maintainers)

Commits related to this issue

Bump python devcontainer Bump devcontainer to 3.11 to fix https://github.com/devcontainers/images/issues/370 Signed-off-by: Jean-Louis Dupond <jean-louis@dupond.be> — committed to dupondje/homeassistant-core by dupondje a year ago

Most upvoted comments

Yarn’s key must have just expired, so the images need to be rev’d to get the latest key. Tactically, you can update the key or remove the yarn source to work around this. e.g., in a Dockerfile

To update the key (now that apt-key is deprecated):

RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | gpg --dearmor > /usr/share/keyrings/yarn-archive-keyring.gpg

Or to remove the source

RUN rm -f /etc/apt/sources.list.d/yarn.list

+11

Chuxel on Jan 24, 2023

Just use curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -, which fixed it here.

Debian from bullseye has deprecated apt-key.

Ref.: https://manpages.debian.org/bullseye/apt/apt-key.8.en.html

jvdavim on Jan 24, 2023

The images release is still in progress, however, new images for “Python” & “Ruby” are published. The issue should be fixed for these two images.

@Apreche / @sreid99 Can you rebuild and try again?

I have confirmed it is now working. Thank you very much.

Apreche on Jan 25, 2023

Thanks again for the quick fix on this, I would just like to discuss what strategy there is for this problem going forward.

Is there some sort of automated system in place to prevent this from happening again? I can imagine something that detects if the keys change, or waits for the expiration date, and then automatically makes a pull request. Sort of like dependabot.

Is there something that can be done in yarn or apt? It was good that they were diligent and put out a new key pretty much as soon as the old one expired. But is there some way they could have had a brief overlapping period with two valid keys to allow a smoother transition?

Any sort of defense mechanism to ensure that images can’t be broken by the actions of a third party would be most welcome.

Once you have a stable image you like, we strongly encourage pre-building your images to avoid these kinds of issues blocking container creation (rather than installing something on the fly). For awareness, the problem seen here also happened for local Linux users, caused by the up-stream source (yarn itself). If you are using Codespaces, doing Codespace pre-builds can help as well.

The problem is that older images will still have the previous key, so there is not a magic bullet fix. But, if everything you need is in an image already, this won’t block you.

@samruddhikhandale we could try to monitor expirations in this case or just switch away from using apt (though that would mean updates for existing containers would not be automated). The next expiration isn’t until 2026 from what I saw.

Chuxel on Jan 25, 2023

All images are published. Closing as completed 🚀

samruddhikhandale on Jan 25, 2023

The images release is still in progress, however, new images for “Python” & “Ruby” are published. The issue should be fixed for these two images.

@Apreche / @sreid99 Can you rebuild and try again?

samruddhikhandale on Jan 25, 2023

Same thing is happening in the Ruby image.

rafaelfranca on Jan 24, 2023