serving: Autoscaler pod count metrics reported improperly
/area monitoring
What version of Knative?
HEAD
Expected Behavior
When I create a new ksvc and hit it with a single request, I expect to see at least one data point for desired_pods and requested_pods that is >0 for that revision.
Actual Behavior
The metric only sometimes gets reported as non-zero.
Steps to Reproduce the Problem
- Create a new revision
- Make a single request to that revision
- View metrics – using a backend with a longer reporting window, such as Stackdriver, increases the likelihood of reproducing the issue
- Observe that
desired_podsandrequested_podsmay be reported as 0
About this issue
- Original URL
- State: closed
- Created 5 years ago
- Comments: 28 (23 by maintainers)
Ah ok. ISTM adding a Max() would also work well here. I dont see an option for this on https://godoc.org/go.opencensus.io/stats/view#Aggregation although histogram does seem to support this - maybe thats the right fix?
Average I worry will solve this problem at the expense of making the information less useful in other ways. Really what were hitting here is you cant measure frequencies below your sampling rate and theres not any way around that.