beyla: Beyla traces not showing up

Hey! I’m running Beyla on a local kubernetes (colima) cluster on an M2 mac.

I’m following the example here with just a few tweaks to the beyla-config to enable debug logs and discovering k8s namespaces,

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: beyla
  name: beyla-config
data:
  beyla-config.yml: |
    # this is required to enable kubernetes discovery and metadata
    attributes:
      kubernetes:
        enable: true
    # this will provide automatic routes report while minimizing cardinality
    print_traces: true
    log_level: DEBUG
    routes:
      unmatched: heuristic
    discovery:
      services:
        - k8s_namespace: beyla
        - k8s_namespace: default
        - k8s_namespace: kube-system

From the logs I think beyla is able to watch other pods in different namespaces.

level=DEBUG msg="inserting pod" component=kube.Metadata informer=Pod name=beyla-lzgxh namespace=beyla uid=a4df9638-84ce-4b32-a3d8-b6ad2148f375 owner=k8s.daemonset.name:beyla node=colima startTime="2024-07-26 00:05:29 +0000 UTC" containerIDs=[5c9dbe4c8a9efd132c26dfee176765a6df8bfb6bf8fc4a258a49ebcbe55aeb17]

But I do not see any traces in Grafana Cloud. How should I debug further? I do see these logs about traces

time=2024-07-26T00:19:52.441Z level=DEBUG msg="new process" component=discover.WatcherKubeEnricher pid=1244
time=2024-07-26T00:19:52.441Z level=DEBUG msg="Found namespace" component=httpfltr.Tracer nsPid=pid:[4026531836]
time=2024-07-26T00:19:52.441Z level=DEBUG msg="new process" component=discover.WatcherKubeEnricher pid=2968
time=2024-07-26T00:19:52.441Z level=DEBUG msg="Found namespace" component=httpfltr.Tracer nsPid=pid:[4026531836]
time=2024-07-26T00:19:52.441Z level=DEBUG msg="can't get container info for PID" component=discover.WatcherKubeEnricher pid=2968 error="/proc/2968/cgroup: couldn't find any docker entry for process with PID 2968"

Do these logs mean beyla is able to receive the traces but unable to send it to Grafana cloud?

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 15 (7 by maintainers)

Most upvoted comments

@tbgbeansbot I don’t believe the print_traces changes are related - as a matter of fact, print_traces should still work (if it’s set, it will have the same effect as trace_printer=text, we just deprecated it and you should now get a warning when using it).

By the way, we’ve updated these docs - if I missed anything on that front, could you point me to it so that I can update it?