cri-o: CRI-O 'net.ipv4.ping_group_range' set failure

Description I try to deploy a kubernetes cluster of version v1.20.1. The environment config as following:

Node	OS	Kubernetes Version	CRI Engine
Master	Ubuntu 18.04 (4.15.0-130-generic x86_64)	v1.20.1	Docker 20.10.2
Minion01	Centos 8 (4.18.0-240.1.1.el8_3.x86_64)	v1.20.1	CRI-O 1.20.1
Minion02	Centos 7 (5.9.1-1.el7.elrepo.x86_64)	v1.20.1	Docker 20.10.2

Then, I reconfig /etc/crio/crio.conf to add net.ipv4.ping_group_range = 0 2147483647 in default_sysctls:

# List of default sysctls. If it is empty or commented out, only the sysctls
# defined in the container json file by the user/kube will be added.
default_sysctls = [
        "net.ipv4.ping_group_range = 0 2147483647",
]

by the way, the config format error in crio.conf.5.md on the github

Finally,

Steps to reproduce the issue:

Deploy a node with centos 8.
Install CRI-O.
Modify /etc/crio/crio.conf, add net.ipv4.ping_group_range in default_sysctls.
Restart CRI-O service.
Install kubeadm, kubectl and kubelet, then use kubeadm join to join the cluster.
Deploy a POD on the node of the cluster.

Describe the results you received: Got the pod deploy error:

Events:
  Type     Reason                  Age               From               Message
  ----     ------                  ----              ----               -------
  Normal   Scheduled               46s               default-scheduler  Successfully assigned default/busybox to k8s-minion-01
  Warning  FailedCreatePodSandBox  8s (x4 over 45s)  kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to pin namespaces [net ipc uts]: [pinns:e]: failed to open /proc/sys/net/ipv4/ping_group_range : No such file or directory
[pinns:e]: Failed to configure sysctls after unshare: No such file or directory

Describe the results you expected: Deploy a pod worked on the node.

Additional information you deem important (e.g. issue happens only occasionally): CNI is weave-kube 2.7.0

Output of crio --version:

crio version 1.20.0
Version:       1.20.0
GitCommit:     d388528dbed26b93c5bc1c89623607a1e597aa57
GitTreeState:  clean
BuildDate:     2021-01-11T20:46:36Z
GoVersion:     go1.14.12
Compiler:      gc
Platform:      linux/amd64
Linkmode:      dynamic

Additional environment details (AWS, VirtualBox, physical, etc.):

About this issue

Original URL
State: closed
Created 3 years ago
Comments: 17 (9 by maintainers)

Most upvoted comments

ah! the default_sysctls seem to be invalid. they’re specified as

# List of default sysctls. If it is empty or commented out, only the sysctls
# defined in the container json file by the user/kube will be added.
default_sysctls = [
        "net.ipv4.ping_group_range = 0 2147483647",
]

but It needs to not have the extra spaces:

# List of default sysctls. If it is empty or commented out, only the sysctls
# defined in the container json file by the user/kube will be added.
default_sysctls = [
        "net.ipv4.ping_group_range=0 2147483647",
]

haircommander on Feb 22, 2022