ebpf: loading a program using constants as a non-root user returns "operation not permitted"

To reproduce, clone rakelimit at this branch: https://github.com/cloudflare/rakelimit/pull/22

go test -v -exec 'strace -f -e trace=bpf' -run TestAttackPropagation

That should give you:

generalisation_test.go:393: Can't create limiter: load BPF: field FilterIpv4: program filter_ipv4: map .rodata: can't freeze map: can't freeze map: operation not permitted

Plus a bunch of syscall traces, among which is the reason for the error:

[pid 59981] bpf(BPF_MAP_FREEZE, {map_fd=7}, 4) = -1 EPERM (Operation not permitted)

The error originates here: https://github.com/cilium/ebpf/blob/2e76685ddb33660f5cc13955b796d0e53d2ea662/map.go#L346-L348

There are two options: we’re doing something wrong and get EPERM for that reason, or freezing a map is a privileged operation (which is surprising). So it would be good to verify why we get EPERM in the first place, by looking at the kernel source.

A work around is to check whether we get EPERM from map.Freeze in the library and continue as if nothing happened in that case. This would be a bit of a bummer though, since freezing is useful to unprivileged programs as well.

Here is what has to happen behind the scenes for constants to work:

btf = bpf_load_btf(...)
rodata = bpf_map_create(..., btf)
bpf_map_freeze(rodata)
bpf_prog_load(..., rodata)

BTF is essentially metadata, see https://www.kernel.org/doc/html/latest/bpf/btf.html

The library currently does the following:

btf = bpf_load_btf(...) // returns EPERM, ignored
rodata = bpf_map_create(...) // without BTF, due to earlier EPERM!
bpf_map_freeze(rodata) // returns EPERM, causes the error described here
bpf_prog_load(..., rodata) // we don't get here

I initially assumed that we can fix the error by ignoring EPERM from bpf_map_freeze. If you do that, I suspect you will see that bpf_prog_load (that we don’t get to) will also return an error of some sort. This error is likely because we loaded the map without BTF.

So: allowing bpf_map_freeze is probably a simple change. However, it doesn’t allow using constants from an unprivileged user, since that requires being able to load BTF. Making BTF available for unprivileged users is a much bigger problem. @borkmann will know whether it’s feasible at all.

At the moment there is no clear scope for my GSoC yet. I was planning on doing a proposal about this https://github.com/cilium/cilium/issues/11014 here (probably the solution that would involve implementing a feature probe API in cilium/ebpf) which is unrelated to the current issue. But I need a PR to apply for a GSoC with cilium which is why I am here.

But related or not, happy to help investigating if I can! I will ask on slack if this can count towards a GSoC proposal though because otherwise I’ll also have to find something for that too.