micrometer: High CPU usage while using KafkaStreamsMetrics
Describe the bug
We run spring-boot + kafka-streams with micrometer and KafkaStreamsMetrics. Metrics are exposed by using micrometer-registry-prometheus
After upgrading from micrometer 1.7.3 to 1.7.4 we notice high cpu-usage. If we disable KafkaStreamsMetrics the high cpu usage disappear. Also, downgrading to 1.7.3 makes the high cpu usage disappear.
The cpu usage increases while calling actuator/prometheus
. Our monitoring system polls this endpoint frequently and the CPU is therefore always high.
Have provided a small sample-app that reproduces the issue: https://github.com/JorgenRingen/micrometer_1_7_4_high_cpu_usage
Environment Kubernetes and locally.
- Micrometer version: 1.7.4
- Micrometer registry: prometheus
- OS: macOS, Linux
- Java version: openjdk version “11.0.7”
To Reproduce How to reproduce the bug: https://github.com/JorgenRingen/micrometer_1_7_4_high_cpu_usage Java-branch has a very basic spring-boot, kafka-streams, micrometer setup.
Expected behavior
While polling localhost:8080/acutator/prometheus
the CPU usage is very high on 1.7.4. Expected behavior is that this should barely be noticeable.
Additional context Slack thread: https://micrometer-metrics.slack.com/archives/C662HUJC9/p1632729969102400
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Reactions: 3
- Comments: 16 (9 by maintainers)
Commits related to this issue
- Fetch Kafka Metrics on a schedule not every time when a Gauge is polled fixes gh-2801 — committed to jonatan-ivanov/micrometer by jonatan-ivanov 3 years ago
- Testing if Kafka Metrics are indirectly updated not every time when a meter is polled. related: gh-2801 — committed to jonatan-ivanov/micrometer by jonatan-ivanov 3 years ago
@JorgenRingen I merged the fix in, should go out in the next patch releases (today).
Released, 1.7.5 should fix this, please give it a try.