kubernetes: Pod readiness probe cannot be directed at specific IP family

What happened:

When trying to deploy an IPv6-only application in a IPv4-first dual-stack cluster, I noticed that the following readiness probe will fail because it is trying to probe the first pod IP, which is IPv4.

    readinessProbe:
      failureThreshold: 30
      httpGet:
        path: /healthz/ready
        port: 15021
        scheme: HTTP

What you expected to happen:

It should be configurable which of the pod IPs should be probed

How to reproduce it (as minimally and precisely as possible):

Deploy application that does not listen for or accept IPv4 requests for readiness probe (==only IPv6) in IPv4-first DS cluster

Anything else we need to know?:

I guess I can work around it by adding the host: ::1 option to httpGet, though I ran into another problem with my application and could not verify this.

Environment:

Kubernetes version (use kubectl version): v1.20.4

About this issue

Original URL
State: open
Created 3 years ago
Reactions: 2
Comments: 30 (21 by maintainers)

Most upvoted comments

Just for reference, in the meantime, there is a solution using the exec probe and curl …

aojea on Aug 10, 2021

@aojea Indeed it seems like a complex lift. As a workaround, I am using socat in a sidecar as follows:

      - args:
        - tcp6-listen:8888,ipv6only,fork
        - tcp4-connect:127.0.0.1:8888
        image: alpine/socat:1.7.4.2-r0
        imagePullPolicy: IfNotPresent
        name: socat

centromere on Jan 21, 2022

This was discussed here https://github.com/kubernetes/enhancements/pull/1975 and the decision was to avoid complicating the kubelet probe logic until there was more evidence …

If host is empty it uses the PodIP, and in this case it seems to be the IPv4 https://github.com/kubernetes/kubernetes/blob/9e8506eb1f8724a2dda4fa8178528c8dce4f05b3/pkg/kubelet/prober/prober.go#L162-L167

I guess I can work around it by adding the host: ::1 option to httpGet, though I ran into another problem with my application and could not verify this.

IIRC kubelet is the one to execute the GET, so it will do it against the host localhost, not the pod localhost.

@danwinship @thockin @khenidak what do you think?

maybe add an IPFamily field to v1.Probe and if IPfamily is empty it uses whatever is in status.PodIP , otherwise choose the corresponding one in status.PodIPs

aojea on Apr 21, 2021