kubernetes: IPset missing from gcr.io/google_containers/hyperkube-amd64
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug
What happened: Upgraded to Kuberenetes 1.9 and now IPVS mode in kube-proxy stopped working due to
I1218 09:59:28.446570 1 feature_gate.go:184] feature gates: map[SupportIPVSProxyMode:true]
E1218 09:59:28.463326 1 server_others.go:299] can't determine whether to use ipvs proxy, error: error getting ipset version, error: executable file not found in $PATH
I1218 09:59:28.464584 1 server_others.go:138] Using iptables Proxier
What you expected to happen:
Normal startup as it’s been working flawless in 1.8.5
How to reproduce it (as minimally and precisely as possible): Use kube-proxy with IPVS service mode on 1.9.0 hyperkube image from GCR
Anything else we need to know?:
If i exec into the running kube-proxy pod and do apt install ipset then restart the container kube-proxy starts in ipvs mode fine
Environment:
- Kubernetes version (use
kubectl version):
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.0", GitCommit:"925c127ec6b946659ad0fd596fa959be43f0cc05", GitTreeState:"clean", BuildDate:"2017-12-15T21:07:38Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.0", GitCommit:"925c127ec6b946659ad0fd596fa959be43f0cc05", GitTreeState:"clean", BuildDate:"2017-12-15T20:55:30Z", GoVersion:"go1.9.2", Compiler:"gc", Platform:"linux/amd64"}
- Cloud provider or hardware configuration: On-prem
- OS (e.g. from /etc/os-release): CoreOS Container Linux by CoreOS stable (1576.4.0)
- Kernel (e.g.
uname -a): 4.13.16-coreos-r2 - Install tools: Own
- Others:
docker version
Client:
Version: 17.09.0-ce
API version: 1.32
Go version: go1.8.4
Git commit: afdb6d4
Built: Tue Sep 26 22:24:58 2017
OS/Arch: linux/amd64
Server:
Version: 17.09.0-ce
API version: 1.32 (minimum version 1.12)
Go version: go1.8.4
Git commit: afdb6d4
Built: Tue Sep 26 22:24:58 2017
OS/Arch: linux/amd64
Experimental: false
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Comments: 15 (6 by maintainers)
Commits related to this issue
- Add ipset binary for IPVS, context: https://github.com/kubernetes/kubernetes/issues/57321 — committed to Fsero/kubernetes by deleted user 7 years ago
- Merge pull request #57648 from Fsero/bugfix/add_ipset_binary_for_ipvs Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a hr... — committed to fanzhangio/kubernetes by deleted user 6 years ago
- Add ipset binary for IPVS, context: https://github.com/kubernetes/kubernetes/issues/57321 — committed to dmlambea/kubernetes by deleted user 7 years ago
- Add ipset binary for IPVS, context: https://github.com/kubernetes/kubernetes/issues/57321 — committed to jingxu97/kubernetes by deleted user 7 years ago
@roffe @dixudx @ivanilves So
kube-proxy-amd64:v1.9.0does have ipset. So please use that. Howeverhyperkube-amd64:v1.9.0does not have ipset (see below).The commit above https://github.com/kubernetes/kubernetes/commit/787a55bb67ccd2da14aa6e7f91289c859beecb5f was merged well before we shipped v1.9.0, But hyperkube image does not use the debian-iptables, it uses debian-hyperkube-base image if you see https://github.com/kubernetes/kubernetes/blob/master/cluster/images/hyperkube/Makefile#L24
Anyone want to log a PR for hyperkube image to have ipset?
FYI: I’ve rebuilt hyperkube image for CoreOS https://hub.docker.com/r/ivanilves/hyperkube/ with
ipsetinstalled.Just for testing purposes. Everything works for me with it. Waiting for the next release. ☺️
Can we re-open this, #57648 only fixes this on debian, not amd64!
hyperkubeimage is based on https://github.com/kubernetes/kubernetes/tree/master/build/debian-hyperkube-base (which is based on https://github.com/kubernetes/kubernetes/tree/master/build/debian-base )kube-proxy(andflannel) has a separate image based on https://github.com/kubernetes/kubernetes/tree/master/build/debian-iptables But ashyperkubehas includedkube-proxysupport, IMO alsodebian-hyperkube-baseshould be extended byipsetcommand, after this https://github.com/kubernetes/kubernetes/blob/e378a2a6c3a51b2b016165e0c10a3dd01ce99cb0/build/debian-hyperkube-base/Dockerfile#L35 line.