beyla: Beyla failing to capture HTTP calls
We are using Beyla to instrument a Java based application which is receiving calls from outside. The application is acting as a server and it gets called by a client (through ingress-nginx). The results have some discrepancies in them, because only a very small fraction of the calls the client is sending to the application are being captured by Beyla. We have tried running Beyla with BEYLA_PRINT_TRACES=true to log every single call/trace and this does not seem to show the missing calls - nor are they present as Prometheus metrics.
Beyla configuration
routes:
unmatched: heuristic
discovery:
services:
- exe_path: .*(zulu|elasticsearch).*
Service discovery does not seem to be the problem, as some calls are present - therefore the executable is correctly found and instrumented. Other than the Beyla logs missing the relevant traces, there are no abnormalities in the logs.
Here’s an example of calls from the application (from the client-side logs) vs. what is captured by Beyla (Beyla application logs).
There seems to be no pattern, neither in the length of the path nor in its content to explain why certain calls are not captured. Interestingly, on subsequent runs of the same test we also get different results, meaning a call that was not captured before is now suddenly present and vice versa.
About this issue
- Original URL
- State: closed
- Created 2 years ago
- Comments: 24 (13 by maintainers)
Thanks again Martijn! I put out a PR to allow us to load the debug BPF on 5.10. I’ll also repeat my tests on 5.10 to see if that makes a difference, perhaps the missing events issue is related to lower kernel versions…