kubernetes: TopologyAwareHints not work

What happened:

TopologyAwareHints not work

What you expected to happen:

TopologyAwareHints will work

How to reproduce it (as minimally and precisely as possible):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-test
  labels:
    app: nginx-test
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 1
  selector:
    matchLabels:
      app: nginx-test
  template:
    metadata:
      labels:
        app: nginx-test
      annotations:
        seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                    - key: app
                      operator: In
                      values:
                        - nginx-test
                topologyKey: kubernetes.io/hostname
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
        - effect: NoSchedule
          key: node-role.kubernetes.io/control-plane
        - key: "CriticalAddonsOnly"
          operator: "Exists"
      containers:
      - name: coredns
        image: nginx:alpine
        ports:
        - containerPort: 80

create 3 nginx pods in three nodes and create the svc use topologyKeys

apiVersion: v1
kind: Service
metadata:
  name: nginx-test
  labels:
    app: nginx-test
  annotations:
#    service.kubernetes.io/topology-aware-hints: auto
spec:
  selector:
    app: nginx-test
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  topologyKeys:
    - "kubernetes.io/hostname"
    - "*"
$ kubectl get svc,po -o wide
NAME                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE     SELECTOR
service/kubernetes   ClusterIP   172.26.0.1    <none>        443/TCP   4h13m   <none>
service/nginx-test   ClusterIP   172.26.70.8   <none>        80/TCP    10m     app=nginx-test

NAME                              READY   STATUS    RESTARTS   AGE   IP           NODE           NOMINATED NODE   READINESS GATES
pod/nginx-test-8675c7d7b4-6dfj6   1/1     Running   0          28m   172.27.5.4   10.93.86.126   <none>           <none>
pod/nginx-test-8675c7d7b4-rvxnp   1/1     Running   0          28m   172.27.2.3   10.93.86.123   <none>           <none>
pod/nginx-test-8675c7d7b4-xxndx   1/1     Running   0          28m   172.27.0.3   10.93.86.124   <none>           <none>

I curl 172.26.70.8 on node 10.93.86.124, through the nginx pods logs, Every nginx pod log can see the requested。 the demo Prefer Node Local Endpoints does not work, The load balancing of svc is always polled

Anything else we need to know?:

Environment:

  • Kubernetes version (use kubectl version): v1.21.0
  • Cloud provider or hardware configuration:
  • OS (e.g: cat /etc/os-release): CentOS Linux release 7.4.1708 (Core)
  • Kernel (e.g. uname -a): Linux centos74 3.10.0-693.el7.x86_64 #1 SMP Tue Aug 22 21:09:27 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools: bin
  • Network plugin and version (if this is a network-related bug): 0.8.7
  • Others:
$ systemctl cat kube-apiserver.service  | grep -i hint
  --feature-gates=TopologyAwareHints=true \
$ systemctl cat kube-controller-manager.service  | grep -i hint
  --feature-gates=TopologyAwareHints=true \
$ systemctl cat kube-proxy.service  | grep -i hint
  --feature-gates=TopologyAwareHints=true \
$ kubectl get nodes --show-labels
NAME           STATUS   ROLES         AGE     VERSION   LABELS
10.93.86.123   Ready    master,node   4h41m   v1.21.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=10.93.86.123,kubernetes.io/os=linux,node-role.kubernetes.io/master=true
10.93.86.124   Ready    master,node   4h41m   v1.21.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=10.93.86.124,kubernetes.io/os=linux,node-role.kubernetes.io/master=true
10.93.86.125   Ready    master,node   4h41m   v1.21.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=10.93.86.125,kubernetes.io/os=linux,node-role.kubernetes.io/master=true
10.93.86.126   Ready    node          4h41m   v1.21.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=10.93.86.126,kubernetes.io/os=linux,node-role.kubernetes.io/node=true
10.93.86.127   Ready    node          4h41m   v1.21.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=10.93.86.127,kubernetes.io/os=linux,node-role.kubernetes.io/node=true
10.93.86.128   Ready    node          4h41m   v1.21.0   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=10.93.86.128,kubernetes.io/os=linux,node-role.kubernetes.io/node=true

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 21 (13 by maintainers)

Most upvoted comments

@zhangguanzhang are your nodes labeled with the topology zone information? otherwise it will not work You can see an e2e test example here https://github.com/kubernetes/kubernetes/pull/100807/files#diff-01b7ace09b00e5b162a43775939eaccc12feda0494c0241cf669a30295f204a4

You can look at the command record I pasted above, I have already labeled the node

$ kubectl label node 10.93.86.123 topology.kubernetes.io/zone=zone
node/10.93.86.123 labeled
$ kubectl label node 10.93.86.124 topology.kubernetes.io/zone=zone
node/10.93.86.124 labeled
$ kubectl label node 10.93.86.125 topology.kubernetes.io/zone=zone
node/10.93.86.125 labeled
$ kubectl label node 10.93.86.126 topology.kubernetes.io/zone=zone
node/10.93.86.126 labeled
$ kubectl label node 10.93.86.127 topology.kubernetes.io/zone=zone
node/10.93.86.127 labeled
$ kubectl label node 10.93.86.128 topology.kubernetes.io/zone=zone
node/10.93.86.128 labeled