containerd: [regression in runc v1.0.0] Cannot run a container on fedora 34 since containerd 1.4.8 upgrade (`cannot fetch program from id: get program by id: permission denied`)

Description

I am not able to run a container on fedora 34 since containerd 1.4.8 upgrade, neither with docker nor with nerdctl. I always get the following error message:

FATA[0000] OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: process_linux.go:508: setting cgroup config for procHooks process caused: cannot fetch program from id: get program by id: permission denied: unknown

Steps to reproduce the issue:

Start containerd with systemd
Try to run a container with nerdctl: sudo ./nerdctl run hello-world

Describe the results you received:

Following error message:

FATA[0000] OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: process_linux.go:508: setting cgroup config for procHooks process caused: cannot fetch program from id: get program by id: permission denied: unknown

Describe the results you expected:

I expected the container to run

What version of containerd are you using:

$ containerd --version
containerd containerd.io 1.4.8 7eba5930496d9bbe375fdf71603e610ad737d2b2

Any other relevant information (runC version, CRI configuration, OS/Kernel version, etc.):

runc --version

$ runc --version
runc version 1.0.0
commit: v1.0.0-0-g84113ee
spec: 1.0.2-dev
go: go1.15.14
libseccomp: 2.5.0

I noticed that this commit: https://github.com/containerd/containerd/commit/34861f1aa3d6666381e5a9539289598db0b37381 changed the runc version to 1.0.1, while I’m still on 1.0.0. If this is the cause of this issue, please close it, as runc 1.0.1 should be available soon for fedora 34: https://src.fedoraproject.org/rpms/runc

uname -a

$ uname -a
Linux pils 5.12.15-300.fc34.x86_64 #1 SMP Wed Jul 7 19:46:50 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

About this issue

Original URL
State: closed
Created 3 years ago
Reactions: 10
Comments: 16 (6 by maintainers)

Most upvoted comments

As a “workaround”, setting SELinux to “permissive” works (but of course removes protection of SELinux for other things);

setenforce 0
docker run --rm alpine echo hello-world
hello-world

setenforce 1
docker run --rm alpine echo hello-world
docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: process_linux.go:545: container init caused: process_linux.go:508: setting cgroup config for procHooks process caused: cannot fetch program from id: get program by id: permission denied: unknown.
ERRO[0000] error waiting for container: context canceled

thaJeztah on Jul 20, 2021

containerd v1.4.9 and v1.5.5 have been released, which have runc v1.0.1: https://github.com/containerd/containerd/releases

If you’re using docker’s containerd.io package from download.docker.com, new packages of v1.4.9 have been published (download.docker.com does not yet have packages for containerd v1.5.x)

A patch release of Docker (20.10.8) will also be available soon (hopefully start of next week) for people that use the static binaries (which bundles docker, the docker cli, containerd and runc in a single tar) or the official docker-in-docker images from https://hub.docker.com/_/docker/)

Let me go ahead and close this ticket

thaJeztah on Jul 30, 2021

Same issue. See:

max-wittig on Jul 27, 2021

Yes, I should mention I did this on my personal NUC, and not a production instance. I am awaiting the proper fix from upstream.

kevinquillen on Jul 20, 2021

Great! As mentioned; really consider it a “workaround”; as of course, disabling SELinux is not something to “recommend”, makes security people weep, can lead to cute fluffy kittens being tortured, and so on. So “tread carefully” 😅

thaJeztah on Jul 20, 2021

Yes, I asked the maintainers to consider doing v1.4.9 and v1.5.5 releases with runc v1.0.1. I only just asked, so haven’t had a reply yet.

thaJeztah on Jul 20, 2021

This also worked for me on Fedora 34 after the last containerd update:

sudo setenforce 0

Now my projects will start without error.

kevinquillen on Jul 20, 2021