strimzi-kafka-operator: [Question] Missing kafka_cluster_* metrics

I tried to look at the documentation but I couldn’t find anything related to how to export at-min-insync metrics. My prometheus doesn’t have kafka_cluster_* metrics (kafka_cluster_partition_atminisr and kafka_cluster_partition_underminisr) and some panels on kafka-exporter dashboard are not working. Am I missing something on Strimzi kafkaExporter definition?

About this issue

Original URL
State: closed
Created 4 years ago
Comments: 15 (5 by maintainers)

Most upvoted comments

Nice I fixed it here and I can see all the metrics on prometheus. The problem was that I didn’t have the metrics defined for Kafka, so I copied everything from the example and also prometheus was not scraping the kafka brokers, so I added the template prometheus annotations like this:

spec:
  kafka:
    ...
    metrics:
      lowercaseOutputName: true
      rules:
        ...
    template:
      pod:
        metadata:
          annotations:
            prometheus.io/scrape: true
            prometheus.io/port: 9404
            prometheus.io/path: /metrics

I’m closing this issue, thank you very much for the support @scholzj!

n5sgg on Sep 2, 2020

Prometheus scrape the kafka exporter /metrics endpoint right? Since I don’t have kafka_cluster_partition_atminisr and kafka_cluster_partition_underminisr on kafka exporter, prometheus won’t have them as well.

No. Your Prometheus has to scrape metrics from the Kafka Exporter as well as from Kafka brokers directly. I guess the name could be confusing. But Kafka Exporter does not expose the Kafka metrics. It is used to create additional metrics which Kafka it self doesn’t provide (with the main focus being consumer lag monitoring). So you need to have Prometheus scrape the Kafka metrics from Kafka + the additional metrics from Kafka Exporter.

scholzj on Sep 2, 2020