kubernetes: Kubelet cAdvisor API not working on windows nodes
What happened: When browsing to the cadvisor metrics API on a Windows node running contained, no metrics are returned about the underlying pods and containers.
curl -X GET https://<windows-node-ip>:10250/metrics/cadvisor --header "Authorization: Bearer $TOKEN" --insecure
# HELP cadvisor_version_info A metric with a constant '1' value labeled by kernel version, OS version, docker version, cadvisor version & cadvisor revision.
# TYPE cadvisor_version_info gauge
cadvisor_version_info{cadvisorRevision="",cadvisorVersion="",dockerVersion="",kernelVersion="10.0.17763.1697",osVersion="Windows Server 2019 Standard"} 1
# HELP container_scrape_error 1 if there was an error while getting container metrics, 0 otherwise
# TYPE container_scrape_error gauge
container_scrape_error 0
However running this on a Linux Node on the same cluster returns correctly
curl -X GET https://<linux-node-ip>:10250/metrics/cadvisor --header "Authorization: Bearer $TOKEN" --insecure
...
# HELP container_network_transmit_packets_total Cumulative count of packets transmitted
# TYPE container_network_transmit_packets_total counter
container_network_transmit_packets_total{container="",id="/",image="",interface="antrea-gw0",name="",namespace="",pod=""} 551458 1611166320528
container_network_transmit_packets_total{container="",id="/",image="",interface="eth0",name="",namespace="",pod=""} 2.226427e+06 1611166320528
container_network_transmit_packets_total{container="",id="/",image="",interface="genev_sys_6081",name="",namespace="",pod=""} 332835 1611166320528
...
What you expected to happen: Expect the container metrics to be returned.
How to reproduce it (as minimally and precisely as possible):
- Create a windows node using the containerd runtime (using kubernetes-sig/image-builder) and join cluster.
- Browse to the nodes cadvisor endpoint :10250/metrics/cadvisor or api/proxy/node/<node-name>/metrics/cadvisor endpoint
Anything else we need to know?:
Environment:
- Kubernetes version (use
kubectl version
): v1.19.1 - Cloud provider or hardware configuration:
- OS (e.g:
cat /etc/os-release
): Windows 2019 Standard LTSC - Kernel (e.g.
uname -a
): 10.0.17763.1697 - Install tools:
- Others: containerd://1.4.3
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 17 (9 by maintainers)
@jsturtevant updated the title to reflect this is a windows node issue not just containerd
/retitle Kubelet cAdvisor API not working with containerd on windows node