kubernetes: kube-proxy can't load xz compressed kernel ip_vs module

Is this a BUG REPORT or FEATURE REQUEST?: /kind bug

What happened: kube-proxy failed to load ip_vs.ko.xz module

What you expected to happen: compressed kernel module ip_vs could loaded by kube-proxy

How to reproduce it (as minimally and precisely as possible): This issue noticed on RHEL

Anything else we need to know?:

Environment:

Kubernetes version (use kubectl version): 1.9.2
Cloud provider or hardware configuration: Openstack
OS (e.g. from /etc/os-release): Red Hat Enterprise Linux Server 7.4 (Maipo)
Kernel (e.g. uname -a): Linux j-upgrade-worker-edge-01 3.10.0-693.2.2.el7.x86_64 #1 SMP Sat Sep 9 03:55:24 EDT 2017 x86_64 x86_64 x86_64 GNU/Linux
Install tools: from scratch
Others: modprobe ip_vs works fine on node.

insmod failed in kube-proxy container
# insmod /lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/net/netfilter/ipvs/ip_vs.ko.xz
insmod: ERROR: could not insert module /lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/net/netfilter/ipvs/ip_vs.ko.xz: Invalid module format

kube-proxy log:

[root@j-upgrade-worker-edge-01 /etc/modprobe.d]$ docker logs 71a413c7b015
I0307 03:32:57.394549       1 feature_gate.go:184] feature gates: map[]
time="2018-03-07T03:32:57Z" level=warning msg="Running modprobe ip_vs failed with message: `modprobe: ERROR: could not insert 'ip_vs': Exec format error\ninsmod /lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/net/netfilter/ipvs/ip_vs.ko.xz`, error: exit status 1"
time="2018-03-07T03:32:57Z" level=error msg="Could not get ipvs family information from the kernel. It is possible that ipvs is not enabled in your kernel. Native loadbalancing will not work until this is fixed."

About this issue

Original URL
State: closed
Created 6 years ago
Comments: 27 (6 by maintainers)

Most upvoted comments

@ReSearchITEng I load ip_vs at boot time. cat /etc/modules-load.d/ip_vs.conf ip_vs

jejer on Apr 2, 2018

Hi @dixudx I can load this module on node using modprobe and insmod:

[root@j-upgrade-worker-edge-01 ~]$ modprobe -r ip_vs
[root@j-upgrade-worker-edge-01 ~]$ lsmod|grep ip_vs
[root@j-upgrade-worker-edge-01 ~]$ modprobe ip_vs
[root@j-upgrade-worker-edge-01 ~]$ lsmod|grep ip_vs
ip_vs                 141092  0
nf_conntrack          133387  9 ip_vs,nf_nat,nf_nat_ipv4,nf_nat_ipv6,xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_netlink,nf_conntrack_ipv4,nf_conntrack_ipv6
libcrc32c              12644  3 ip_vs,nf_nat,nf_conntrack
[root@j-upgrade-worker-edge-01 ~]$ modprobe -r ip_vs
[root@j-upgrade-worker-edge-01 ~]$ insmod /lib/modules/3.10.0-693.2.2.el7.x86_64/kernel/net/netfilter/ipvs/ip_vs.ko.xz
[root@j-upgrade-worker-edge-01 ~]$ lsmod|grep ip_vs
ip_vs                 141092  0
nf_conntrack          133387  9 ip_vs,nf_nat,nf_nat_ipv4,nf_nat_ipv6,xt_conntrack,nf_nat_masquerade_ipv4,nf_conntrack_netlink,nf_conntrack_ipv4,nf_conntrack_ipv6
libcrc32c              12644  3 ip_vs,nf_nat,nf_conntrack
[root@j-upgrade-worker-edge-01 ~]$

If I manually load ip_vs before kube-proxy bringup, kube-proxy will not complain about ipvs.

jejer on Mar 7, 2018

@spencerhance

dcbw on May 30, 2019

same issue with kube-proxy-amd64:v1.11.2 on Centos 7.

fkpwolf on Aug 23, 2018

I am facing the same with the default 1.10 image: k8s.gcr.io/kube-proxy-amd64:v1.10.0

What are the options (short/long term)?

ReSearchITEng on Mar 30, 2018