bpftrace: Accessing pointers broken on LLVM <12
bpftrace --info
System
OS: Linux 5.6.4-arch1-1 #1 SMP PREEMPT Mon, 13 Apr 2020 12:21:19 +0000
Arch: x86_64
Build
version: v0.10.0-63-g4b37
LLVM: 10
foreach_sym: yes
unsafe uprobe: no
btf: no
bfd: yes
bpf_attach_kfunc: yes
Kernel helpers
probe_read: yes
probe_read_str: yes
probe_read_user: yes
probe_read_user_str: yes
probe_read_kernel: yes
probe_read_kernel_str: yes
get_current_cgroup_id: yes
send_signal: yes
override_return: yes
Kernel features
Instruction limit: 1000000
Loop support: yes
Map types
hash: yes
percpu hash: yes
array: yes
percpu array: yes
stack_trace: yes
perf_event_array: yes
Probe types
kprobe: yes
tracepoint: yes
perf_event: yes
kfunc: no
What reproduces the bug?
This works:
#include <linux/fs.h>
#include <linux/blk_types.h>
#include <linux/blkdev.h>
#include <linux/blk-cgroup.h>
#include <linux/cgroup-defs.h>
#include <linux/sched.h>
kprobe:blk_update_request
{
$req = (struct request *)arg0;
printf("cgroup name: %p\n", $req->bio->bi_blkg->blkcg->css.cgroup->kn->parent);
}
with output like this:
cgroup name: 0xffffa02fdd173480
cgroup name: 0xffffa02fdd173480
cgroup name: (nil)
cgroup name: (nil)
But this script does not work:
#include <linux/fs.h>
#include <linux/blk_types.h>
#include <linux/blkdev.h>
#include <linux/blk-cgroup.h>
#include <linux/cgroup-defs.h>
#include <linux/sched.h>
kprobe:blk_update_request
{
$req = (struct request *)arg0;
$kn = $req->bio->bi_blkg->blkcg->css.cgroup->kn;
printf("cgroup name: %p\n", $kn->parent);
}
with output like:
cgroup name: (nil)
cgroup name: (nil)
cgroup name: (nil)
cgroup name: (nil)
cgroup name: (nil)
cgroup name: (nil)
cgroup name: (nil)
cgroup name: (nil)
cgroup name: (nil)
cgroup name: (nil)
There are no non-nils.
I think there might have been some change w/ handling pointers already on the stack. Maybe it’s eliding a bpf_probe_read()? Could make sense cuz then kernel will fail the dereference and give you 0x0.
cc @mmisono
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 1
- Comments: 19 (8 by maintainers)
I filed a bug against llvm upstream: https://bugs.llvm.org/show_bug.cgi?id=47591
From https://llvm.org/docs/LangRef.html, llvm.lifetime.{start,end} encloses a region of lifetime of a memory object. From the above IR, I see
After
call void @llvm.lifetime.end.p0i8(i64 -1, i8* nonnull %15), the compiler is free to reuse the same stack location, and based on the above, the compiler is supposed to doloadfirst and then reuse the stack. This is exactly llvm7 asm code is doing.Need to do a little bit study on why llvm did the above reordering. I guess this might be a llvm bug. One comment mentioned it is after BPF DAG->DAG Pattern Instruction Selection. Let me take a further check and then will comment back.
Cannot reproduce this one and #1558 with the llvm12 build from #1716 \o/
The bug is fixed in llvm12. This LLVM patch https://reviews.llvm.org/D91833 fixed the problem and I added a test case (derived from this issue) in BPF https://reviews.llvm.org/D92451.