kubernetes: kubectl drain uses 200% cpu.
Running :
kubectl drain node/XXX --force=true --delete-local-data=true --ignore-daemonsets
cpu usage goes to 2 cores @ 100%
What you expected to happen: Way lower cpu usage.
Environment:
- Kubernetes version (use
kubectl version
):
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:58:53Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16+", GitVersion:"v1.16.9-gke.6", GitCommit:"14bc8ad5f8c245f1240a8e4eab128c4d51bfeffe", GitTreeState:"clean", BuildDate:"2020-05-30T02:07:03Z", GoVersion:"go1.13.9b4", Compiler:"gc", Platform:"linux/amd64"}
- Cloud provider or hardware configuration: GKE
- OS (e.g:
cat /etc/os-release
):
NAME="Ubuntu"
VERSION="18.04.4 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.4 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic
- Kernel (e.g.
uname -a
): Linux felix-Latitude-7490 5.3.0-62-generic #56~18.04.1-Ubuntu SMP Wed Jun 24 16:17:03 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 1
- Comments: 16 (8 by maintainers)
Maybe I am wrong and it’s just some busy waiting. In my case I noticed it when draining nodes with many pods that cannot immediately be stopped because it would break the deployments maxUnavailable limits, so kubectl has to wait until it can stop all the pods on the draining node.