krr: Krr 1.5.3 returns no results due to no metrics for PercentileCPULoader and MaxMemoryLoader

Describe the bug

I updated the tool to the 1.5.3 version. When I executed the simple strategy and it returned no results:

> krr simple --namespace <namespace> --selector="app = grafana"


 _____       _               _          _  _______  _____
|  __ \     | |             | |        | |/ /  __ \|  __ \
| |__) |___ | |__  _   _ ___| |_ __ _  | ' /| |__) | |__) |
|  _  // _ \| '_ \| | | / __| __/ _` | |  < |  _  /|  _  /
| | \ \ (_) | |_) | |_| \__ \ || (_| | | . \| | \ \| | \ \
|_|  \_\___/|_.__/ \__,_|___/\__\__,_| |_|\_\_|  \_\_|  \_\



Running Robusta's KRR (Kubernetes Resource Recommender) v1.5.3
Using strategy: Simple
Using formatter: table

[INFO] Using clusters: ['<cluster>']
on 0: [INFO] Listing scannable objects in <cluster>
on 0: [INFO] Connecting to Prometheus for <cluster> cluster
on 0: [INFO] Using Prometheus at https://<server>/api/v1/namespaces/<namespace>/services/prometheus-server-service:9091/proxy for cluster <cluster>   
on 0: [INFO] Prometheus found
on 0: [INFO] Prometheus connected successfully for <cluster> cluster
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment <namespace>/grafana-deployment/grafana
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment <namespace>/grafana-deployment/grafana
Calculating Recommendation |████████████████████████████████████████| 1 in 15.4s (0.07/s)



Simple Strategy

CPU request: 99.0% percentile, limit: unset
Memory request: max + 5.0%, limit: max + 5.0%

This strategy does not work with objects with HPA defined (Horizontal Pod Autoscaler).
If HPA is defined for CPU or Memory, the strategy will return "?" for that resource.

Learn more: https://github.com/robusta-dev/krr#algorithm

┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
┃ Number ┃ Namespace        ┃ Name             ┃ Pods ┃ Old Pods ┃ Type       ┃ Container ┃ CPU Diff ┃ CPU Requests     ┃ CPU Limits       ┃ Memory Diff ┃ Memory Requests  ┃ Memory Limits     ┃
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
│     1. │ <namespace>      │ grafana-deploym… │ 0    │ 0        │ Deployment │ grafana   │          │ 40m -> ? (No     │ unset -> ? (No   │             │ 75Mi -> ? (No    │ 90Mi -> ? (No     │
│        │                  │                  │      │          │            │           │          │ data)            │ data)            │             │ data)            │ data)             │
└────────┴──────────────────┴──────────────────┴──────┴──────────┴────────────┴───────────┴──────────┴──────────────────┴──────────────────┴─────────────┴──────────────────┴───────────────────┘
                                                                                         100 points - A

When I execute the same command for the same cluster with the 1.4.1 version, it works fine:

> krr simple --namespace monitor-operator --selector="app = grafana"                                                                               


 _____       _               _          _  _______  _____
|  __ \     | |             | |        | |/ /  __ \|  __ \
| |__) |___ | |__  _   _ ___| |_ __ _  | ' /| |__) | |__) |
|  _  // _ \| '_ \| | | / __| __/ _` | |  < |  _  /|  _  /
| | \ \ (_) | |_) | |_| \__ \ || (_| | | . \| | \ \| | \ \
|_|  \_\___/|_.__/ \__,_|___/\__\__,_| |_|\_\_|  \_\_|  \_\



Running Robusta's KRR (Kubernetes Resource Recommender) v1.4.1
Using strategy: Simple
Using formatter: table

[INFO] Using clusters: ['<cluster>']
[INFO] Listing scannable objects in <cluster>
[INFO] Found 1 objects across 1 namespaces in <cluster>
on 0: [INFO] Connecting to Prometheus for <cluster> cluster
on 0: [INFO] Using Prometheus at https://<server>/api/v1/namespaces/<namespace>/services/prometheus-server-service:9091/proxy for cluster <cluster>   
on 0: [INFO] Prometheus found
on 0: [INFO] Prometheus connected successfully for <cluster>  cluster
Calculating Recommendation |████████████████████████████████████████| 1/1 [100%] in 5.5s (0.18/s)



Simple Strategy

CPU request: 99.0% percentile, limit: unset
Memory request: max + 5.0%, limit: max + 5.0%

This strategy does not work with objects with HPA defined (Horizontal Pod Autoscaler).
If HPA is defined for CPU or Memory, the strategy will return "?" for that resource.

Learn more: https://github.com/robusta-dev/krr#algorithm

┏━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓    
┃ Number ┃ Namespace        ┃ Name               ┃ Pods ┃ Old Pods ┃ Type       ┃ Container ┃ CPU Diff ┃ CPU Requests     ┃ CPU Limits ┃ Memory Diff ┃ Memory Requests      ┃ Memory Limits ┃    
┡━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩    
│     1. │ <namespace>      │ grafana-deployment │ 1    │ 0        │ Deployment │ grafana   │ -35m     │ (-35m) 40m -> 5m │ unset      │ -15Mi       │ (-15Mi) 75Mi -> 60Mi │ 90Mi -> 60Mi  │    
└────────┴──────────────────┴────────────────────┴──────┴──────────┴────────────┴───────────┴──────────┴──────────────────┴────────────┴─────────────┴──────────────────────┴───────────────┘    
                                                                                       100 points - A    

To Reproduce Steps to reproduce the behavior:

  1. execute the simple strategy only with the namespace and selector parameters

Expected behavior

Recommendations should be calculated.

Desktop

  • OS: Microsoft Windows 11 Enterprise, 10.0.22621
  • Browser: Brave, 1.56.20 Chromium: 115.0.5790.171 (Official Build) (64-bit)

About this issue

  • Original URL
  • State: closed
  • Created 10 months ago
  • Reactions: 2
  • Comments: 15

Most upvoted comments

similar issue with Google Cloud Managed Prometheus, 1.60-alpha does not work for me. Had to downgrade to 1.4.1

kube_pod_owner does not exist as a metric.

Logs
_____       _               _          _  _______  _____
|  __ \     | |             | |        | |/ /  __ \|  __ \
| |__) |___ | |__  _   _ ___| |_ __ _  | ' /| |__) | |__) |
|  _  // _ \| '_ \| | | / __| __/ _` | |  < |  _  /|  _  /
| | \ \ (_) | |_) | |_| \__ \ || (_| | | . \| | \ \| | \ \
|_|  \_\___/|_.__/ \__,_|___/\__\__,_| |_|\_\_|  \_\_|  \_\



Running Robusta's KRR (Kubernetes Resource Recommender) v1.6.0
Using strategy: Simple
Using formatter: table

[DEBUG] Found 1 clusters: gke_XREDACTEDX
[DEBUG] Current cluster: gke_XREDACTEDX
[DEBUG] Configured clusters: []
[INFO] Using clusters: ['gke_XREDACTEDX']
on 0: [INFO] Listing scannable objects in gke_XREDACTEDX
on 0: [DEBUG] Namespaces: *
on 0: [DEBUG] Resources: *
on 0: [DEBUG] Listing Deployments in gke_XREDACTEDX
on 0: [DEBUG] Listing Rollouts in gke_XREDACTEDX
on 0: [DEBUG] Listing StatefulSets in gke_XREDACTEDX
on 0: [DEBUG] Listing DaemonSets in gke_XREDACTEDX
on 0: [DEBUG] Listing Jobs in gke_XREDACTEDX
on 0: [DEBUG] Found 6 Deployment in gke_XREDACTEDX
on 0: [DEBUG] Rollout API not available in gke_XREDACTEDX
on 0: [DEBUG] Found 0 DaemonSet in gke_XREDACTEDX
on 0: [DEBUG] Found 1 StatefulSet in gke_XREDACTEDX
on 0: [INFO] Connecting to Prometheus for gke_XREDACTEDX cluster
on 0: [INFO] Using Prometheus at https://monitoring.googleapis.com/v1/projects/XREDACTEDX/location/global/prometheus for cluster gke_XREDACTEDX
on 0: [INFO] Prometheus found
on 0: [INFO] Prometheus connected successfully for gke_XREDACTEDX cluster
on 0: [DEBUG] Adding historic pods for Deployment staging/balance-fetcher/balance-fetcher
on 0: [DEBUG] Found 0 Job in gke_XREDACTEDX
on 0: [DEBUG] Adding historic pods for Deployment staging/client/client
on 0: [DEBUG] Adding historic pods for Deployment staging/latest-prices-service/latest-prices-service
on 0: [DEBUG] Adding historic pods for Deployment staging/report-generator/report-generator
on 0: [DEBUG] Adding historic pods for Deployment staging/report-generator/chrome-puppet
on 0: [DEBUG] Adding historic pods for Deployment staging/server/server
on 0: [DEBUG] Adding historic pods for Deployment staging/server/cloud-sql-proxy-main
on 0: [DEBUG] Adding historic pods for Deployment staging/server/cloud-sql-proxy-prices
on 0: [DEBUG] Adding historic pods for Deployment staging/websocket-service/websocket-service
on 0: [DEBUG] Adding historic pods for StatefulSet staging/importer/importer
on 0: [DEBUG] No pods found for StatefulSet staging/importer/importer
on 0: [DEBUG] Gathering PercentileCPULoader metric for StatefulSet staging/importer/importer
on 0: [DEBUG] No pods found for Deployment staging/report-generator/report-generator
on 0: [DEBUG] Gathering PercentileCPULoader metric for Deployment staging/report-generator/report-generator
on 0: [DEBUG] No pods found for Deployment staging/latest-prices-service/latest-prices-service
on 0: [DEBUG] Gathering PercentileCPULoader metric for Deployment staging/latest-prices-service/latest-prices-service
on 0: [DEBUG] No pods found for Deployment staging/client/client
on 0: [DEBUG] Gathering PercentileCPULoader metric for Deployment staging/client/client
on 0: [DEBUG] No pods found for Deployment staging/balance-fetcher/balance-fetcher
on 0: [DEBUG] Gathering PercentileCPULoader metric for Deployment staging/balance-fetcher/balance-fetcher
on 0: [DEBUG] No pods found for Deployment staging/server/cloud-sql-proxy-main
on 0: [DEBUG] Gathering PercentileCPULoader metric for Deployment staging/server/cloud-sql-proxy-main
on 0: [DEBUG] No pods found for Deployment staging/server/cloud-sql-proxy-prices
on 0: [DEBUG] Gathering PercentileCPULoader metric for Deployment staging/server/cloud-sql-proxy-prices
on 0: [DEBUG] No pods found for Deployment staging/server/server
on 0: [DEBUG] Gathering PercentileCPULoader metric for Deployment staging/server/server
on 0: [DEBUG] No pods found for Deployment staging/websocket-service/websocket-service
on 0: [DEBUG] Gathering PercentileCPULoader metric for Deployment staging/websocket-service/websocket-service
on 0: [DEBUG] No pods found for Deployment staging/report-generator/chrome-puppet
on 0: [DEBUG] Gathering PercentileCPULoader metric for Deployment staging/report-generator/chrome-puppet
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment staging/balance-fetcher/balance-fetcher
on 0: [DEBUG] Gathering MaxMemoryLoader metric for Deployment staging/balance-fetcher/balance-fetcher
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment staging/websocket-service/websocket-service
on 0: [DEBUG] Gathering MaxMemoryLoader metric for Deployment staging/websocket-service/websocket-service
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment staging/report-generator/report-generator
on 0: [DEBUG] Gathering MaxMemoryLoader metric for Deployment staging/report-generator/report-generator
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for StatefulSet staging/importer/importer
on 0: [DEBUG] Gathering MaxMemoryLoader metric for StatefulSet staging/importer/importer
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment staging/server/cloud-sql-proxy-prices
on 0: [DEBUG] Gathering MaxMemoryLoader metric for Deployment staging/server/cloud-sql-proxy-prices
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment staging/latest-prices-service/latest-prices-service
on 0: [DEBUG] Gathering MaxMemoryLoader metric for Deployment staging/latest-prices-service/latest-prices-service
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment staging/client/client
on 0: [DEBUG] Gathering MaxMemoryLoader metric for Deployment staging/client/client
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment staging/report-generator/chrome-puppet
on 0: [DEBUG] Gathering MaxMemoryLoader metric for Deployment staging/report-generator/chrome-puppet
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment staging/server/server
on 0: [DEBUG] Gathering MaxMemoryLoader metric for Deployment staging/server/server
on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment staging/server/cloud-sql-proxy-main
on 0: [DEBUG] Gathering MaxMemoryLoader metric for Deployment staging/server/cloud-sql-proxy-main
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment staging/server/cloud-sql-proxy-prices
on 0: [DEBUG] Gathering CPUAmountLoader metric for Deployment staging/server/cloud-sql-proxy-prices
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment staging/report-generator/chrome-puppet
on 0: [DEBUG] Gathering CPUAmountLoader metric for Deployment staging/report-generator/chrome-puppet
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment staging/server/server
on 0: [DEBUG] Gathering CPUAmountLoader metric for Deployment staging/server/server
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment staging/websocket-service/websocket-service
on 0: [DEBUG] Gathering CPUAmountLoader metric for Deployment staging/websocket-service/websocket-service
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment staging/report-generator/report-generator
on 0: [DEBUG] Gathering CPUAmountLoader metric for Deployment staging/report-generator/report-generator
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment staging/client/client
on 0: [DEBUG] Gathering CPUAmountLoader metric for Deployment staging/client/client
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment staging/server/cloud-sql-proxy-main
on 0: [DEBUG] Gathering CPUAmountLoader metric for Deployment staging/server/cloud-sql-proxy-main
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment staging/balance-fetcher/balance-fetcher
on 0: [DEBUG] Gathering CPUAmountLoader metric for Deployment staging/balance-fetcher/balance-fetcher
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment staging/latest-prices-service/latest-prices-service
on 0: [DEBUG] Gathering CPUAmountLoader metric for Deployment staging/latest-prices-service/latest-prices-service
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for StatefulSet staging/importer/importer
on 0: [DEBUG] Gathering CPUAmountLoader metric for StatefulSet staging/importer/importer
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment staging/server/cloud-sql-proxy-prices
on 0: [DEBUG] Gathering MemoryAmountLoader metric for Deployment staging/server/cloud-sql-proxy-prices
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment staging/server/server
on 0: [DEBUG] Gathering MemoryAmountLoader metric for Deployment staging/server/server
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment staging/websocket-service/websocket-service
on 0: [DEBUG] Gathering MemoryAmountLoader metric for Deployment staging/websocket-service/websocket-service
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment staging/client/client
on 0: [DEBUG] Gathering MemoryAmountLoader metric for Deployment staging/client/client
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment staging/server/cloud-sql-proxy-main
on 0: [DEBUG] Gathering MemoryAmountLoader metric for Deployment staging/server/cloud-sql-proxy-main
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment staging/report-generator/report-generator
on 0: [DEBUG] Gathering MemoryAmountLoader metric for Deployment staging/report-generator/report-generator
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment staging/balance-fetcher/balance-fetcher
on 0: [DEBUG] Gathering MemoryAmountLoader metric for Deployment staging/balance-fetcher/balance-fetcher
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment staging/latest-prices-service/latest-prices-service
on 0: [DEBUG] Gathering MemoryAmountLoader metric for Deployment staging/latest-prices-service/latest-prices-service
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for StatefulSet staging/importer/importer
on 0: [DEBUG] Gathering MemoryAmountLoader metric for StatefulSet staging/importer/importer
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment staging/report-generator/chrome-puppet
on 0: [DEBUG] Gathering MemoryAmountLoader metric for Deployment staging/report-generator/chrome-puppet
on 0: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment staging/server/cloud-sql-proxy-prices
on 0: [DEBUG] Calculating recommendations for Deployment staging/server/cloud-sql-proxy-prices with 4 metrics
on 1: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment staging/websocket-service/websocket-service
on 1: [DEBUG] Calculating recommendations for Deployment staging/websocket-service/websocket-service with 4 metrics
on 2: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment staging/server/server
on 2: [DEBUG] Calculating recommendations for Deployment staging/server/server with 4 metrics
on 3: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment staging/balance-fetcher/balance-fetcher
on 3: [DEBUG] Calculating recommendations for Deployment staging/balance-fetcher/balance-fetcher with 4 metrics
on 4: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment staging/server/cloud-sql-proxy-main
on 4: [DEBUG] Calculating recommendations for Deployment staging/server/cloud-sql-proxy-main with 4 metrics
on 4: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment staging/latest-prices-service/latest-prices-service
on 4: [DEBUG] Calculating recommendations for Deployment staging/latest-prices-service/latest-prices-service with 4 metrics
on 6: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment staging/report-generator/report-generator
on 6: [DEBUG] Calculating recommendations for Deployment staging/report-generator/report-generator with 4 metrics
on 6: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment staging/client/client
on 6: [DEBUG] Calculating recommendations for Deployment staging/client/client with 4 metrics
on 8: [WARNING] Prometheus returned no MemoryAmountLoader metrics for StatefulSet staging/importer/importer
on 8: [DEBUG] Calculating recommendations for StatefulSet staging/importer/importer with 4 metrics
on 9: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment staging/report-generator/chrome-puppet
on 9: [DEBUG] Calculating recommendations for Deployment staging/report-generator/chrome-puppet with 4 metrics

Hi @LeaveMyYard. I apologize for the delay, but I was on vacation. I tested with 1.60-alpha and it does not work:

on 0: [WARNING] Prometheus returned no PercentileCPULoader metrics for Deployment <namespace>/grafana-deployment/grafana
on 0: [WARNING] Prometheus returned no MaxMemoryLoader metrics for Deployment <namespace>/grafana-deployment/grafana
on 0: [WARNING] Prometheus returned no CPUAmountLoader metrics for Deployment <namespace>/grafana-deployment/grafana
on 0: [WARNING] Prometheus returned no MemoryAmountLoader metrics for Deployment <namespace>/grafana-deployment/grafana

Hey, @agabrys

Can you try the code from #128 ?