beyla: Unable to get prometheus metrics

Greetings! 👋🏻

I’m new here and have just started exploring Beyla, which seems quite promising 👍🏻 . We have a self-hosted Kubernetes (K8s) cluster and rely on a self-hosted Prometheus and Grafana (kube-prometheus stack) for metrics and dashboards, and it works great ☺️

Now, I’m trying to get my hands dirty with Beyla to see if we can leverage it for our internal applications. I attempted to follow the official Kubernetes documentation (https://grafana.com/docs/beyla/latest/setup/kubernetes/) and have tried the sidecar method with the following manifest (The only change is that I have removed the OTEL_EXPORTER_OTLP_ENDPOINT envVar as we don’t use grafana agent and have added BEYLA_PROMETHEUS_PORT envVar for having Prometheus scrape endpoint

apiVersion: apps/v1
kind: Deployment
metadata:
  name: goblog
  labels:
    app: goblog
spec:
  replicas: 2
  selector:
    matchLabels:
      app: goblog
  template:
    metadata:
      labels:
        app: goblog
    spec:
      # Required so the sidecar instrument tool can access the service process
      shareProcessNamespace: true
      containers:
        # Container for the instrumented service
        - name: goblog
          image: mariomac/goblog:dev
          imagePullPolicy: IfNotPresent
          command: ["/goblog"]
          env:
            - name: "GOBLOG_CONFIG"
              value: "/sample/config.yml"
          ports:
            - containerPort: 8443
              name: https
        # Sidecar container with Beyla - the eBPF auto-instrumentation tool
        - name: autoinstrument
          image: grafana/beyla:latest
          securityContext: # Privileges are required to install the eBPF probes
            runAsUser: 0
            capabilities:
              add:
                - SYS_ADMIN
          env:
            - name: BEYLA_OPEN_PORT # The internal port of the goblog application container
              value: "8443"
            - name: BEYLA_PROMETHEUS_PORT
               value: "8445"
            - name: BEYLA_LOG_LEVEL
               value: "DEBUG"

I tried to send some requests to the goblog service, but no matter how many requests the service receives, all I get from the metrics endpoint is:

 curl localhost:8445/metrics
# HELP promhttp_metric_handler_errors_total Total number of internal errors encountered by the promhttp metric handler.
# TYPE promhttp_metric_handler_errors_total counter
promhttp_metric_handler_errors_total{cause="encoding"} 0
promhttp_metric_handler_errors_total{cause="gathering"} 0

According to the documentation, the exporter metrics include a handful of HTTP metrics, but I’m not sure why they are missing. Do I need to take care of any additional configurations?. I would appreciate it if someone could provide guidance here. Thanks! 🙏🏻