parca-agent: python: Process detection not working for dynamically linked Python

$ ls /usr/lib/debug/usr/lib64/
libpython3.11d.so.1.0-3.11.5-1.el8_9.x86_64.debug  libpython3.11d.so.1.0-3.11.5-1.el8_9.x86_64.debug-gdb.py  libpython3.11.so.1.0-3.11.5-1.el8_9.x86_64.debug  libpython3.11.so.1.0-3.11.5-1.el8_9.x86_64.debug-gdb.py    __pycache__  python3.11

About this issue

Original URL
State: closed
Created 7 months ago
Comments: 20 (13 by maintainers)

Most upvoted comments

https://github.com/parca-dev/parca-agent/pull/2267 should fix the issue, will share the container built off main with you as soon as it’s built

javierhonduco on Nov 20, 2023

Oh I think I see the bug now, I always that Golang returns nil if a key isn’t present in a map… Sorry about this!

javierhonduco on Nov 20, 2023

(We’ll soon be working on using the legend for filtering, which is I believe what you tried doing first as well)

brancz on Nov 20, 2023

You managed to hit exactly the time where we rolled out a very major change in the infrastructure, that caused a few failed requests, let me know if you continue to see this error.

brancz on Nov 20, 2023

cc @kakkoyun about the first one. Reporting the second one internally so the backend team can take a look. Thanks a lot again for the reports!

javierhonduco on Nov 20, 2023

This is all the log I have

[33mooooooooo.                                                  .o.                                            .[0m
[33m`888   `Y88.                                               .888.                                         .o8[0m
[33m 888   .d88'  .oooo.   oooo d8b  .ooooo.   .oooo.         .8"888.      .oooooooo  .ooooo.  ooo. .oo.   .o888oo[0m
[33m 888ooo88P'  `P  )88b  `888""8P d88' `"Y8 `P  )88b       .8' `888.    888' `88b  d88' `88b `888P"Y88b    888[0m
[33m 888          .oP"888   888     888        .oP"888      .88ooo8888.   888   888  888ooo888  888   888    888[0m
[33m 888         d8(  888   888     888   .o8 d8(  888     .8'     `888.  `88bod8P'  888    .o  888   888    888 .[0m
[33mo888o        `Y888""8o d888b    `Y8bod8P' `Y888""8o   o88o     o8888o `8oooooo.  `Y8bod8P' o888o o888o   "888"[0m
[33m                                                                      d"     YD[0m
[33m                                                                      "Y88888P'[0m
[33m[0m
level=info name=parca-agent ts=2023-11-20T11:46:44.250910139Z caller=main.go:509 msg="maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined"
level=info name=parca-agent ts=2023-11-20T11:46:44.357325456Z caller=main.go:553 msg="no config file provided, using default config"
level=info name=parca-agent ts=2023-11-20T11:46:44.363432701Z caller=main.go:611 msg="eBPF is supported and enabled by the host kernel"
name=parca-agent ts=2023-11-20T11:46:44.364829992Z caller=main.go:739 msg=starting... node=ip-172-30-36-152.ec2.internal store=grpc.polarsignals.com:443
level=info name=parca-agent ts=2023-11-20T11:46:44.372770429Z caller=cpu.go:229 component=cpu_profiler msg="loaded pyperf BPF module"
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1cdc15a]
goroutine 225 [running]:
github.com/parca-dev/parca-agent/pkg/pprof.(*Converter).addInterpreterLocation(0xc00257cf00, 0x2d6aea32f8379a)
	github.com/parca-dev/parca-agent/pkg/pprof/pprof.go:309 +0x7a
github.com/parca-dev/parca-agent/pkg/pprof.(*Converter).Convert(0xc00257cf00, {0xbc52b9774?, 0x3e5d6e0?}, {0xc0007e8400, 0x22, 0x2459388?})
	github.com/parca-dev/parca-agent/pkg/pprof/pprof.go:209 +0xb8f
github.com/parca-dev/parca-agent/pkg/profiler/cpu.(*CPU).Run(0xc0000fea80, {0x2b21690, 0xc002504060})
	github.com/parca-dev/parca-agent/pkg/profiler/cpu/cpu.go:877 +0x1ec7
main.run.func17.1({0x2b21690?, 0xc002504060?})
	github.com/parca-dev/parca-agent/cmd/parca-agent/main.go:1134 +0x3b
runtime/pprof.Do({0x2b216c8?, 0xc000792190?}, {{0xc002508000?, 0x2?, 0x0?}}, 0xc00008dec0)
	runtime/pprof/runtime.go:51 +0x9d
main.run.func17()
	github.com/parca-dev/parca-agent/cmd/parca-agent/main.go:1133 +0x4ec
github.com/oklog/run.(*Group).Run.func1({0xc0024ce300?, 0xc000a821c0?})
	github.com/oklog/run@v1.1.0/group.go:38 +0x29
created by github.com/oklog/run.(*Group).Run in goroutine 1
	github.com/oklog/run@v1.1.0/group.go:37 +0x67
level=error name=parca-agent ts=2023-11-20T11:47:48.4765783Z caller=main.go:366 msg="======================= unexpected error ======================="
level=error name=parca-agent ts=2023-11-20T11:47:48.476661713Z caller=main.go:367 msg="last stderr" last_stderr="level=info name=parca-agent ts=2023-11-20T11:46:44.250910139Z caller=main.go:509 msg=\"maxprocs: Leaving GOMAXPROCS=4: CPU quota undefined\"\nlevel=info name=parca-agent ts=2023-11-20T11:46:44.357325456Z caller=main.go:553 msg=\"no config file provided, using default config\"\nlevel=info name=parca-agent ts=2023-11-20T11:46:44.363432701Z caller=main.go:611 msg=\"eBPF is supported and enabled by the host kernel\"\nname=parca-agent ts=2023-11-20T11:46:44.364829992Z caller=main.go:739 msg=starting... node=ip-172-30-36-152.ec2.internal store=grpc.polarsignals.com:443\nlevel=info name=parca-agent ts=2023-11-20T11:46:44.372770429Z caller=cpu.go:229 component=cpu_profiler msg=\"loaded pyperf BPF module\"\npanic: runtime error: invalid memory address or nil pointer dereference\n[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1cdc15a]\n\ngoroutine 225 [running]:\ngithub.com/parca-dev/parca-agent/pkg/pprof.(*Converter).addInterpreterLocation(0xc00257cf00, 0x2d6aea32f8379a)\n\tgithub.com/parca-dev/parca-agent/pkg/pprof/pprof.go:309 +0x7a\ngithub.com/parca-dev/parca-agent/pkg/pprof.(*Converter).Convert(0xc00257cf00, {0xbc52b9774?, 0x3e5d6e0?}, {0xc0007e8400, 0x22, 0x2459388?})\n\tgithub.com/parca-dev/parca-agent/pkg/pprof/pprof.go:209 +0xb8f\ngithub.com/parca-dev/parca-agent/pkg/profiler/cpu.(*CPU).Run(0xc0000fea80, {0x2b21690, 0xc002504060})\n\tgithub.com/parca-dev/parca-agent/pkg/profiler/cpu/cpu.go:877 +0x1ec7\nmain.run.func17.1({0x2b21690?, 0xc002504060?})\n\tgithub.com/parca-dev/parca-agent/cmd/parca-agent/main.go:1134 +0x3b\nruntime/pprof.Do({0x2b216c8?, 0xc000792190?}, {{0xc002508000?, 0x2?, 0x0?}}, 0xc00008dec0)\n\truntime/pprof/runtime.go:51 +0x9d\nmain.run.func17()\n\tgithub.com/parca-dev/parca-agent/cmd/parca-agent/main.go:1133 +0x4ec\ngithub.com/oklog/run.(*Group).Run.func1({0xc0024ce300?, 0xc000a821c0?})\n\tgithub.com/oklog/run@v1.1.0/group.go:38 +0x29\ncreated by github.com/oklog/run.(*Group).Run in goroutine 1\n\tgithub.com/oklog/run@v1.1.0/group.go:37 +0x67\n"
level=error name=parca-agent ts=2023-11-20T11:47:48.47668844Z caller=main.go:368 msg="================================================================"
level=error name=parca-agent ts=2023-11-20T11:47:48.476701618Z caller=main.go:370 msg="about to report error to server"
level=info name=parca-agent ts=2023-11-20T11:47:48.896795684Z caller=main.go:395 msg="report sent successfully"

ethervoid on Nov 20, 2023