kubernetes: Kubelet cAdvisor API not working on windows nodes

What happened: When browsing to the cadvisor metrics API on a Windows node running contained, no metrics are returned about the underlying pods and containers.

curl -X GET https://<windows-node-ip>:10250/metrics/cadvisor --header "Authorization: Bearer $TOKEN" --insecure

# HELP cadvisor_version_info A metric with a constant '1' value labeled by kernel version, OS version, docker version, cadvisor version & cadvisor revision.
# TYPE cadvisor_version_info gauge
cadvisor_version_info{cadvisorRevision="",cadvisorVersion="",dockerVersion="",kernelVersion="10.0.17763.1697",osVersion="Windows Server 2019 Standard"} 1
# HELP container_scrape_error 1 if there was an error while getting container metrics, 0 otherwise
# TYPE container_scrape_error gauge
container_scrape_error 0

However running this on a Linux Node on the same cluster returns correctly

curl -X GET https://<linux-node-ip>:10250/metrics/cadvisor --header "Authorization: Bearer $TOKEN" --insecure
...
# HELP container_network_transmit_packets_total Cumulative count of packets transmitted
# TYPE container_network_transmit_packets_total counter
container_network_transmit_packets_total{container="",id="/",image="",interface="antrea-gw0",name="",namespace="",pod=""} 551458 1611166320528
container_network_transmit_packets_total{container="",id="/",image="",interface="eth0",name="",namespace="",pod=""} 2.226427e+06 1611166320528
container_network_transmit_packets_total{container="",id="/",image="",interface="genev_sys_6081",name="",namespace="",pod=""} 332835 1611166320528
...

What you expected to happen: Expect the container metrics to be returned.

How to reproduce it (as minimally and precisely as possible):

  • Create a windows node using the containerd runtime (using kubernetes-sig/image-builder) and join cluster.
  • Browse to the nodes cadvisor endpoint :10250/metrics/cadvisor or api/proxy/node/<node-name>/metrics/cadvisor endpoint

Anything else we need to know?:

Environment:

  • Kubernetes version (use kubectl version): v1.19.1
  • Cloud provider or hardware configuration:
  • OS (e.g: cat /etc/os-release): Windows 2019 Standard LTSC
  • Kernel (e.g. uname -a): 10.0.17763.1697
  • Install tools:
  • Others: containerd://1.4.3

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 17 (9 by maintainers)

Most upvoted comments

@jsturtevant updated the title to reflect this is a windows node issue not just containerd

/retitle Kubelet cAdvisor API not working with containerd on windows node