cilium: enable-bpf-clock-probe functionality is broken, "Cannot probe CONFIG_HZ", 1.12.0, 1.12.1

Is there an existing issue for this?

I have searched the existing issues

What happened?

It seems enable-bpf-clock-probe functionality is broken. On cilium-agent startup i see the error stating “Cannot probe CONFIG_HZ”. I logged-in into container, and tried to run

root@k8s-master-01:/home/cilium# cilium-probe-kernel-hz
Cannot set priority: Operation not permitted

Then i added CAP_SYS_NICE to relevant DaemonSet, and tried again

root@k8s-master-01:/home/cilium# cilium-probe-kernel-hz
No procfs support?
Cannot prep jiffies: Success

Then i checked contents of /proc/timer_list in container, and it was empty

root@k8s-master-01:/home/cilium# cat /proc/timer_list

BTW no such problem in 1.11.6

Br, Alex

Cilium Version

Client: 1.12.1 4c9a630 2022-08-15T16:29:39-07:00 go version go1.18.5 linux/amd64 Daemon: 1.12.1 4c9a630 2022-08-15T16:29:39-07:00 go version go1.18.5 linux/amd64

Kernel Version

Linux k8s-master-01 5.15.0-46-generic #49-Ubuntu SMP Thu Aug 4 18:03:25 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Kubernetes Version

WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version. Client Version: version.Info{Major:“1”, Minor:“24”, GitVersion:“v1.24.1”, GitCommit:“3ddd0f45aa91e2f30c70734b175631bec5b5825a”, GitTreeState:“clean”, BuildDate:“2022-05-24T12:26:19Z”, GoVersion:“go1.18.2”, Compiler:“gc”, Platform:“linux/amd64”} Kustomize Version: v4.5.4 Server Version: version.Info{Major:“1”, Minor:“24”, GitVersion:“v1.24.1”, GitCommit:“3ddd0f45aa91e2f30c70734b175631bec5b5825a”, GitTreeState:“clean”, BuildDate:“2022-05-24T12:18:48Z”, GoVersion:“go1.18.2”, Compiler:“gc”, Platform:“linux/amd64”}

Sysdump

cilium-sysdump-20220818-225109.zip

Relevant log output

level=info msg="Auto-disabling \"enable-bpf-clock-probe\" feature since KERNEL_HZ cannot be determined" error="Cannot probe CONFIG_HZ" subsys=daemon

Anything else?

No response

Code of Conduct

I agree to follow this project’s Code of Conduct

About this issue

Original URL
State: closed
Created 2 years ago
Comments: 27 (16 by maintainers)

Commits related to this issue

Mount host /boot into cilium-agent container It's required to run 'bpftool -j feature probe' during kernel feature probing. Control host /boot mount with values, allow to override host /boot path. F... — committed to agrevtsev/cilium by agrevtsev 2 years ago
Mount host /boot into cilium-agent container It's required to run 'bpftool -j feature probe' during kernel feature probing. Control host /boot mount with values, allow to override host /boot path. F... — committed to cilium/cilium by agrevtsev 2 years ago
Mount host /boot into cilium-agent container It's required to run 'bpftool -j feature probe' during kernel feature probing. Control host /boot mount with values, allow to override host /boot path. F... — committed to xmltiger/cilium by agrevtsev 2 years ago

Most upvoted comments

it seems that mounting host /boot into cilium-agent container allows probe code to run bpftool feature probe and it works in an unprivileged mode. So no need then to run probeSystemKernelHz() which requires `/proc/timer_list’ access. So, from inside the cilium-agent container

root@k8s-node-01:/home/cilium# bpftool feature probe | grep HZ
CONFIG_HZ is set to 250
root@k8s-node-01:/home/cilium# cat /proc/timer_list
root@k8s-node-01:/home/cilium#

agrevtcev on Aug 27, 2022

@ti-mo sorry for the late reply. Yep, /proc/schedstat is readable (0444), more important- it’s also readable from inside cilium-agent container. So go ahead friend ))

Br, Alexey

agrevtcev on Nov 4, 2022

or simply add /proc/timer_list in readonly path instead of masked path, any security issue to make that change?

vincentmli on Aug 21, 2022