kubernetes: kube-proxy ipvs: "Failed to add IPVS service" err="no such file or directory"

What happened?

Network is disconnected and then restored, kube-proxy log errs:

E1007 13:57:26.145807       1 proxier.go:1242] "Failed to sync service" err="no such file or directory" service="123.123.252.197:8989/TCP"
E1007 13:57:26.145840       1 proxier.go:2016] "Failed to add IPVS service" err="no such file or directory" svcName="zy/aiolcm-service-use"
E1007 13:57:26.145875       1 proxier.go:1593] "Failed to sync service" err="no such file or directory" service="194.246.5.29:31134/TCP"
E1007 13:57:26.145904       1 proxier.go:2016] "Failed to add IPVS service" err="no such file or directory" svcName="kube-system/metrics-server"
E1007 13:57:26.145924       1 proxier.go:1242] "Failed to sync service" err="no such file or directory" service="123.123.47.233:443/TCP"
E1007 13:57:26.145952       1 proxier.go:2016] "Failed to add IPVS service" err="no such file or directory" svcName="cnf/sc-gw-0-0-1:dualhttpsgw"
E1007 13:57:26.162009       1 proxier.go:1672] "Failed to get ipvs service" err="could not get IPVS services: no such file or directory"

As a result, the K8S service IP cannot be added to kube-ipvs0

I don’t know which file or directory no such file or directory refers to,

https://github.com/kubernetes/kubernetes/blob/c92036820499fedefec0f847e2054d824aea6cd1/pkg/util/ipvs/ipvs_linux.go#L67 returned the “no such file or directory” error message

What did you expect to happen?

no errs

How can we reproduce it (as minimally and precisely as possible)?

Cloud disk network is disconnected and then restore

Anything else we need to know?

No response

Kubernetes version

$ kubectl version
1.22.3
# paste output here

Cloud provider

no

OS version

# On Linux:
$ cat /etc/os-release
# paste output here
$ uname -a
# paste output here

# On Windows:
C:\> wmic os get Caption, Version, BuildNumber, OSArchitecture
# paste output here

Install tools

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, …) and versions (if applicable)

About this issue

  • Original URL
  • State: open
  • Created 9 months ago
  • Comments: 15 (15 by maintainers)

Most upvoted comments

The bug is likely not in moby in this case, but in the way the error is handled in kube-proxy. The bug in my issue has a work-around (open-write-read-checkResult). IMO we can live with the un-maintained moby until the nft proxier comes around.

/cc @aroradaman