ingress-nginx: Few metrics are missing in Ingress nginx

NGINX Ingress controller version: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.30.0

Kubernetes version (use kubectl version): Server Version: v1.14.10-gke.27

Environment:

Cloud provider or hardware configuration: GCP
OS (e.g. from /etc/os-release): Container OS
Kernel (e.g. uname -a): Linux nginx-ingress-controller-5485cb45c7-48qfp 4.14.138+ #1 SMP Tue Sep 3 02:58:08 PDT 2019 x86_64 Linux

What happened:

After fresh installed nginx ingress controller with metrics enabled. we are not able to see few metrics.

go_goroutines Number of goroutines that currently exist.
go_goroutines gauge
nginx_ingress_controller_bytes_sent The number of bytes sent to a client
nginx_ingress_controller_bytes_sent histogram
nginx_ingress_controller_ingress_upstream_latency_seconds Upstream service latency per Ingress
nginx_ingress_controller_ingress_upstream_latency_seconds summary
nginx_ingress_controller_request_duration_seconds The request processing time in milliseconds
nginx_ingress_controller_request_duration_seconds histogram
nginx_ingress_controller_request_size The request length (including request line, header, and request body)
nginx_ingress_controller_request_size histogram
nginx_ingress_controller_requests The total number of client requests.
nginx_ingress_controller_requests counter
nginx_ingress_controller_response_duration_seconds The time spent on receiving the response from the upstream server
nginx_ingress_controller_response_duration_seconds histogram
nginx_ingress_controller_response_size The response length (including request line, header, and request body)
nginx_ingress_controller_response_size histogram

What you expected to happen:

We are able to see above metrics in different cluster with same ingress nginx controller version, we are not sure why in this fresh install those are metrics are missing.

We also verified deployment, configmap and other setting, all are same for both installation but still it is not working for new installer.

How to reproduce it:

Install following helm chart with version 1.36.2

helm install stable/nginx-ingress --name ingress --set controller.metrics.enabled=true --version 1.36.2

after installed, go into ingress controller and check

curl localhost:10254/metrics

Additional information:

We also put load for ingress controller but still could not see those metrics.

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 1
Comments: 30 (13 by maintainers)

Most upvoted comments

If you didn’t specify ingress host, ingress status is as below:

$ kubectl get ingress
NAME                     CLASS     HOSTS              ADDRESS        PORTS   AGE
example-ingress   <none>   *                         10.8.201.111   80      5m36s

When HOSTS parameter is *, execute this if statement: https://github.com/kubernetes/ingress-nginx/blob/d82585917c845e598a2cd5d2c4291ce891b5e5b4/internal/ingress/metric/collectors/socket.go#L233-L235 So you can’t see some metrics such as nginx_ingress_controller_request

Bo0km4n on Jun 26, 2020