kubernetes: hairpin not set by kubelet with CNI plugin

Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.): No

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): kube-proxy clusterIP target hang

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one):

BUG REPORT

Kubernetes version (use kubectl version): 1.6.2

Environment:

• Cloud provider or hardware configuration: AWS
• OS (e.g. from /etc/os-release): coreos-1235.6.0
• Kernel (e.g. uname -a): 4.7.3-coreos-r2
• Install tools: cloudinit direct install
• Others:

What happened:

Try creating a service with a cluster IP. Then try accessing the service IP and port from within a target pod. It hangs. As far as I can tell, SYN but that is it.

Note that the same is true if you use type: NodePort and access the node IP and port.

What you expected to happen:

Should be able to access the service from any pod, whether one of the targets or a different one.

How to reproduce it (as minimally and precisely as possible):

Use the following simple service and deployment:

apiVersion: v1
kind: Service
metadata:
  name: nginx
  labels:
    name: nginx
spec:
  type: NodePort
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: http
    nodePort: 30100
  selector:
    instance: nginx
---
kind: Deployment
apiVersion: apps/v1beta1
metadata:
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      instance: nginx
  template:
    metadata:
      labels:
        name: nginx
        instance: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:alpine
        ports:
        - containerPort: 80
          name: http

Working pod:

exec into any pod other than the one created as part of the deployment:

1. curl <nginx_podIP>:80: works
2. curl <nginx_serviceIP>:80: works

Failing pod

exec into the pod created by the above deployment:

1. curl <nginx_podIP>:80: works
2. curl localhost:80: works
3. curl <nginx_serviceIP>:80: FAILS

Anything else we need to know:

It doesn’t matter if you use the same port for the service as for the pod or a different one, same name or different name, etc.

I suspect iptables weirdness, but…

Running 1.6.2 with weave networking. I was concerned it might be lack of --cluster-cidr on the kube-proxy, but that makes no difference.