TLDR, There are many moving parts in a K8s cluster that may use / depend on

• the "node.kubernetes.io/network-unavailable" condition and/or
• v1.TaintNodeNetworkUnavailable taint Condition/Taint couple

In ambiguous/ad-hoc sorts of ways. This makes it confusing for users and vendors to know when they should/shouldnt manipulate the values of such TCs.

Solution

• We may need to make more well defined taints/tolerations that say things precisely about network state. This might break systems relying heavily on the oversimplified “NetworkUnavailable” semantics we currently have.
• We need to deprecate, re-define, and document taints/tolerations for k8s networks… This is a several week or month process b/c it will require lots of water carrying across vendors / sig-docs/sig-net/sig-node.

By the way, lots of water-carrying here between vendors/sig-net, might be a good “first” issue for someone whose very eager to learn about the history/semantics of K8s networking and pod scheduling, and has many weeks to dedicate !

Detailed Summary

(9/12 Editing this issue to update it per comments…)

Many, many issues expose the complexity of node networking readiness.

• https://github.com/antrea-io/antrea/issues/2298
• https://github.com/flannel-io/flannel/issues/1431
• https://github.com/projectcalico/calico/issues/6165
• https://github.com/projectcalico/calico/issues/5233
• https://github.com/containernetworking/cni/issues/859
• https://github.com/containernetworking/cni/issues/628

There’s proposals to get CNI Status into the CNI specification. For now, the most urgent need appears to be to fully

• DEFINE the current taints, tolerations, and expectations that CNI, cloud provider, and others should have
• FILE bugs against Kuberentes and CNI/cloud provider implementations that may/may not use the taints/tolerations in Kubernetes properly
• TALK to the Multi-networking and CNI providers individually to figure out what the lowest common denominator for node networking readiness and tainting is
• WRITE a kep to update all of these taints, tolerations, and so on so that they are consistently aligned to the indended use (and deprecate the taints which don’t have a usable definition once we consider this broader world view of K8s networking)
• Original brainstorming doc… not relevant anymore but for historical reasons: https://docs.google.com/document/d/1xlZYVw2iu447b5Us8V8gRC5uigjrjtOCX7RgQ19Oe4/edit

Goals

I think the goals of this issue are something like:

• The CNI and CRI/Kubelet would have a well defined “taint namespace” (not literally, but, some sort of semantic boundary)… related specifically to when CNIs were ready to make pods
• There would be comprehensive guidance for CNIs and cloud controllers on how / when to/ when not to modify these taints conditions
• Taints/Conditions related to network readiness, would have clear well-defined meaning in multi-networking and multi-OS environments w/ docs on them
• There would be no cloud specific behaviour in K8s related to these taints and conditions
• Maybe this means we need to entirely rethink the value of high-level, networking related taints

### Tasks
- [ ] https://github.com/kubernetes/website/issues/43018
- [ ] https://github.com/kubernetes/website/issues/42917