containerd: Kubeadm unknown service runtime.v1alpha2.RuntimeService

Problem Following Kubernetes official installation instruction for containerd and kubeadm init will fail with unknown service runtime.v1alpha2.RuntimeService.

# Commands from https://kubernetes.io/docs/setup/production-environment/container-runtimes/#containerd
apt-get update && apt-get install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
apt-get update && apt-get install -y containerd.io

# Configure containerd
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
systemctl restart containerd

kubeadm init
...
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
	[ERROR CRI]: container runtime is not running: output: time="2020-09-24T11:49:16Z" level=fatal msg="getting status of runtime failed: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1

Solution:

rm /etc/containerd/config.toml
systemctl restart containerd
kubeadm init

Versions:

• Ubuntu 20.04 (focal)
• containerd.io 1.3.7
• kubectl 1.19.2
• kubeadm 1.19.2
• kubelet 1.19.2

Thanks! You help-me with this solution!!!

I followed the official instructions here https://kubernetes.io/docs/setup/production-environment/container-runtimes/

and I was getting similar error

root@green-1:~# kubeadm init  --config=config.yaml 
W1125 12:58:32.733485   26426 configset.go:348] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[init] Using Kubernetes version: v1.19.4
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR CRI]: container runtime is not running: output: time="2020-11-25T12:58:32Z" level=fatal msg="getting status of runtime failed: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher

I checked /etc/containerd/config.toml and saw ‘disabled_plugins = []’

Note the only thing I changed in the config.toml was to use systemd as true -it was different from the way docs has mentioned (maybe this was the problem?)

[plugins."io.containerd.grpc.v1.cri"]
    systemd_cgroup = false --> to true

from docs

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
  ...
  [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
    SystemdCgroup = true

deleting this config.toml as given in the first post and restarting containerd service solved and kubeadm could proceed

Apparently it is my fault this time. My ansible playbook did not override config.toml file as I expected. Am sorry for taking up your time, default installation instructions work great.

In the config.toml file installed by package containerd.io there is the line disabled_plugins = ["cri"] that am guessing creating the issue. That maybe is bad default setting to have in the package containerd.io but that is for another issue/bug.

Closing.

Heads up, this just happened to me on a clean install of Kubernetes v1.24.0 on Ubunutu 20.04.4 LTS. The original fix helped me as well.

Exception:

[init] Using Kubernetes version: v1.24.0
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
	[ERROR CRI]: container runtime is not running: output: time="2022-05-16T23:41:59Z" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
To see the stack trace of this error execute with --v=5 or higher

Corrected:

user@k8s-master:~/$ sudo rm /etc/containerd/config.toml
user@k8s-master:~/$ sudo systemctl restart containerd
user@k8s-master:~/$ sudo kubeadm init
[init] Using Kubernetes version: v1.24.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
..
..

Apparently it is my fault this time. My ansible playbook did not override config.toml file as I expected. Am sorry for taking up your time, default installation instructions work great.

In the config.toml file installed by package containerd.io there is the line disabled_plugins = ["cri"] that am guessing creating the issue. That maybe is bad default setting to have in the package containerd.io but that is for another issue/bug.

Closing.

This comment saved my day 👍 . The default docker configuration removes CRI.

CentOS 7 Linux 5.19.1-1.el7.elrepo.x86_64

Case 1：Kubernetes 1.2x binary installation and reports this error solution:

• containerd: v1.6.4
• kubelet: 1.24.6

containerd config default > /etc/containerd/config.toml
sed -i 's/SystemdCgroup \= false/SystemdCgroup \= true/' /etc/containerd/config.toml
sed -i 's/snapshotter = "overlayfs"/snapshotter = "native"/' /etc/containerd/config.toml

Case 2：kuberspray installs kubernetes 1.25.3 and reports this error solution:

• containerd: v1.6.9
• kubelet: 1.25.3

sed -i 's@# containerd_snapshotter: "native"@containerd_snapshotter: "native"@g' inventory/mycluster/group_vars/all/containerd.yml

Then rerun kubespray

It got fixed for me finally:

fix:

Update contained to the latest and fix toml file with below changes:

disabled_plugins = [“cri”] -->> disabled_plugins = [“”]

Done. Thnak me later 😉

Got this error too

[init] Using Kubernetes version: v1.26.0
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
	[ERROR CRI]: container runtime is not running: output: E0103 00:14:31.026921    5282 remote_runtime.go:948] "Status from runtime service failed" err="rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
time="2023-01-03T00:14:31Z" level=fatal msg="getting status of runtime: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1

The following steps worked for me ->

sudo apt-get update && sudo apt-get install -y containerd
sudo mkdir -p /etc/containerd
sudo containerd config default | sudo tee /etc/containerd/config.toml
# update SystemdCgroup to true in `/etc/containerd/config.toml`
sudo systemctl restart containerd

I struggled with this too. The solution for me was to comment this line out in /etc/containerd/config.toml

disabled_plugins = ["cri"]

I just ran apt-get upgrade and now my control plane and all workers are failing to run containerd and thus also kubelet. The logs for sudo service containerd status show:

Jan 24 23:15:55 kube-master containerd[431]: time="2023-01-24T23:15:55.608380181-06:00" level=info msg="loading plugin \"io.containerd.grpc.v1.cri\"..." type=io.containerd.grpc.v1
Jan 24 23:15:55 kube-master containerd[431]: time="2023-01-24T23:15:55.609315014-06:00" level=warning msg="failed to load plugin io.containerd.grpc.v1.cri" error="invalid plugin config

It seems the apt-get upgrade reverted my changes to /etc/containerd/config.toml and set SystemdCgroup back to false as well as systemd_cgroup. Why does this keep on reverting? Additionally, why are these defaulted to false?

Seems like perhaps there should be some enhanced logic in the generate default config that detects if systemd is in use and set those values to true?

EDIT

Update + solution: in my case, I had to set SystemdCgroup = true and systemd_cgroup = false. Leaving systemd_cgroup = true resulted in an error on containerd startup.

Hi folks, my bash history from the new control plane is below, it works.

   18  apt update
   19  apt upgrade -y
   20  reboot
   21  cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF

   22  sudo modprobe overlay
   23  sudo modprobe br_netfilter
   24  cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF

   25  sudo sysctl --system
   26  sudo apt-get update
   27  wget https://github.com/containerd/containerd/releases/download/v1.7.1/containerd-1.7.1-linux-amd64.tar.gz
   28  tar Cxzvf /usr/local containerd-1.7.1-linux-amd64.tar.gz
   29  systemctl daemon-reload
   30  systemctl enable --now containerd
   31  nano /usr/local/lib/systemd/system/containerd.service
   32  systemctl status sshd
   33  ls -lh /lib/systemd/system/
   34  nano /lib/systemd/system/containerd.service
   35  systemctl daemon-reload
   36  systemctl status containerd.service
   37  nano /lib/systemd/system/containerd.service
   38  ls /usr/local/bin/containerd
   39  systemctl enable --now containerd
   40  wget https://github.com/opencontainers/runc/releases/download/v1.1.7/runc.amd64
   41  install -m 755 runc.amd64 /usr/local/sbin/runc
   42  wget https://github.com/containernetworking/plugins/releases/download/v1.3.0/cni-plugins-linux-amd64-v1.3.0.tgz
   43  tar Cxzvf /opt/cni/bin cni-plugins-linux-amd64-v1.3.0.tgz
   44  systemctl status containerd.service
   45  systemctl restart containerd.service
   46  systemctl status containerd.service
   47  containerd -v
   48  sudo mkdir -p /etc/containerd
   49  sudo containerd config default | sudo tee /etc/containerd/config.tom
   50  sudo containerd config default | sudo tee /etc/containerd/config.toml
   51  ls /etc/containerd/
   52  rm /etc/containerd/config.tom
   53  sudo systemctl restart containerd
   54  sudo systemctl status containerd
   55  sudo swapoff -a
   56  nano /etc/fstab
   57  sudo apt-get update && sudo apt-get install -y apt-transport-https curl
   58  curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
   59  cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

   60  sudo apt-get update
   61  sudo apt-get install -y kubelet=1.26.0-00 kubeadm=1.26.0-00 kubectl=1.26.0-00
   62  sudo apt-mark hold kubelet kubeadm kubectl
   63  sudo kubeadm init --pod-network-cidr 192.168.0.0/16 --kubernetes-version 1.26.0

Got the same issue and fixed it by installing the containerd.io package from the docker repository instead of the one from ubuntu’s repository. see: https://docs.docker.com/engine/install/ubuntu/#set-up-the-repository

I have ubuntu 22.04.2 on VMs and raspberry pies

Also it seems there is presently an issue to retrieve the gpg key from https://packages.cloud.google.com/apt/doc/apt-key.gpg

@hamedsol Thanks for the hint. I was finally able to init the cluster after installing the containerd binaries by following this guide: https://www.itzgeek.com/how-tos/linux/ubuntu-how-tos/install-containerd-on-ubuntu-22-04.html

Problem Following Kubernetes official installation instruction for containerd and kubeadm init will fail with unknown service runtime.v1alpha2.RuntimeService.

# Commands from https://kubernetes.io/docs/setup/production-environment/container-runtimes/#containerd
apt-get update && apt-get install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
apt-get update && apt-get install -y containerd.io

# Configure containerd
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
systemctl restart containerd

kubeadm init
...
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
	[ERROR CRI]: container runtime is not running: output: time="2020-09-24T11:49:16Z" level=fatal msg="getting status of runtime failed: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1

Solution:

rm /etc/containerd/config.toml
systemctl restart containerd
kubeadm init

Versions:

• Ubuntu 20.04 (focal)
• containerd.io 1.3.7
• kubectl 1.19.2
• kubeadm 1.19.2
• kubelet 1.19.2

Thou Saved The Day

Problem Following Kubernetes official installation instruction for containerd and kubeadm init will fail with unknown service runtime.v1alpha2.RuntimeService.

# Commands from https://kubernetes.io/docs/setup/production-environment/container-runtimes/#containerd
apt-get update && apt-get install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
apt-get update && apt-get install -y containerd.io

# Configure containerd
mkdir -p /etc/containerd
containerd config default > /etc/containerd/config.toml
systemctl restart containerd

kubeadm init
...
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
	[ERROR CRI]: container runtime is not running: output: time="2020-09-24T11:49:16Z" level=fatal msg="getting status of runtime failed: rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService"
, error: exit status 1

Solution:

rm /etc/containerd/config.toml
systemctl restart containerd
kubeadm init

Versions:

• Ubuntu 20.04 (focal)
• containerd.io 1.3.7
• kubectl 1.19.2
• kubeadm 1.19.2
• kubelet 1.19.2

Thanks for the reply, this is the only solution that worked for me.

rm /etc/containerd/config.toml systemctl restart containerd kubeadm init

I have a same issue with kubeadm v1.24.0 in CentOS 7.9

unfortunately the config in the containerd.io package has, since forever, had a bad configuration for kubernetes tools. The bad configuration is they install a version of the config for containerd that is only good for docker. This config needs to be replaced at least with the default containerd config… and you can modify it from there if you like.

“containerd config default > /etc/containerd/config.toml” will overwrite docker’s version of the config and replace it with containerd’s version of the config… which also works just fine for docker. Then restart containerd.