ingress-nginx: proxy timeout annotations have no effect on nginx

I had the same problem and discovered that the following do not work:

nginx.ingress.kubernetes.io/proxy‑read‑timeout: 1800 nginx.ingress.kubernetes.io/proxy‑read‑timeout: 1800s nginx.ingress.kubernetes.io/proxy‑read‑timeout: “1800s”

What does work is:

nginx.ingress.kubernetes.io/proxy‑read‑timeout: “1800”

Adding client_body_timeout was the key fix for me here. This needs to be put in the documentation somewhere since it was hard to find

    nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
    nginx.ingress.kubernetes.io/server-snippet: "keepalive_timeout 3600s; grpc_read_timeout 3600s; grpc_send_timeout 3600s;client_body_timeout 3600s;"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"

After way to much trial and error and frustration; some tips that might work for others who end up here:

• nginx.ingress.kubernetes.io/proxy-connect-timeout did not work for me. Nothing changed in the nginx configuration in the ingress controller. No errors were shown. Removing the initial nginx. did work. Ending up with these annotations:

    ingress.kubernetes.io/proxy-connect-timeout: "600"
    ingress.kubernetes.io/proxy-read-timeout: "600"
    ingress.kubernetes.io/proxy-send-timeout: "600"
    ingress.kubernetes.io/send-timeout: "600"

• If you want to inspect what the end result, the nginx.conf, looks like. You can get it from the ingress controller pod. To access the ingress controller pod with kubectl you need to specify namespace when running commands since the controller doesn’t live in the default namespace. So like this:

$ kubectl get pods --all-namespaces
...
$ kubectl -n kube-system exec nginx-ingress-controller-138430828-pqb7q cat /etc/nginx/nginx.conf | tee nginx.test-ingress-export.conf

You have an incorrect char “-” in your annotations. In my configuration nginx.ingress.kubernetes.io/proxy-read-timeout is written and it works on the same version (0.10.2).

what is the approved solution for this… for me also its same… its getting CLIENT_DISCONNECTED exactly in 60sec, I have tried all options mentioned in this forum but not working… any solid clue to get it fixed?

I had to use the string version instead of the number version, any idea why this is?

From the first tip in the docs https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/

Annotation keys and values can only be strings. Other types, such as boolean or numeric values must be quoted, i.e. "true", "false", "100".

@Tim-Schwalbe I am using the helm chart as well, although a different version. It only worked with ConfigMaps.

Here are the steps that helped me. You need the name of the pod running the controller. Say nginx-ingress-controller-1234abcd

Make sure you’re running images from quay.io $ kubectl describe pod nginx-ingress-controller-1234abcd | grep Image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0 if it doesn’t start with quay.io the following steps may not be relevant.

Determine the name of the ConfigMap it reads all those properties from: $ kubectl describe pod nginx-ingress-controller-1234abcd | grep configmap= --configmap=default/nginx-ingress-controller

That means it reads from from a ConfigMap with name nginx-ingress-controller in the default namespace. Append such a ConfigMap to you Ingress yaml file:

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-ingress-controller
data:
  proxy-read-timeout: "234"
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: lb-ingress
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
  - http:
      paths:
      - path: /
        backend:
          serviceName: app-service
          servicePort: 8080

Properties you can add to the ConfigMap compiled in the table here: https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/configmap.md

The result in /etc/nginx/nginx.conf proxy_read_timeout 234s;

I hope that was helpful.

Bumping this: we hit a worse version of this problem when moving from 1.12.1 to 1.12.4. Apparently now if you have these invalid values (not specified as strings) all of your annotations are discarded. Seems like ‘kubectl apply’ with invalid annotations shouldn’t silently accept and discard these values.

To turn this around: is anyone actually able to have something communicate across a kubernetes cluster boundary with >60s idle time between packets? Perhaps using something else than nginx?

I had to use the string version instead of the number version, any idea why this is?

This breaks:

nginx.ingress.kubernetes.io/proxy-read-timeout: 300

This works:

nginx.ingress.kubernetes.io/proxy-read-timeout: "300"

For me this is not working. Anyone sees an issue? I am using the nginx helm chart: nginx-ingress-0.8.9

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: zalenium
  namespace: zalenium
  annotations:
    kubernetes.io/ingress.class: nginx
    ingress.kubernetes.io/auth-type: basic
    ingress.kubernetes.io/auth-secret: zalenium-basic-auth
    ingress.kubernetes.io/auth-realm: "Authentication Required"
    nginx.ingress.kubernetes.io/enable-cors: "true"
    nginx.ingress.kubernetes.io/cors-allow-methods: "*"
    nginx.ingress.kubernetes.io/cors-allow-origin: "*"
    nginx.ingress.kubernetes.io/proxy-connect-timeout: 3600
    nginx.ingress.kubernetes.io/proxy-send-timeout: 3600
    nginx.ingress.kubernetes.io/proxy-read-timeout: 3600
spec:
  rules:
  - host: "test.whatever"
    http:
      paths:
      - path: /
        backend:
          serviceName: zalenium
          servicePort: 4444

Closing. As @akaGelo commented you have an issue with the -. Is my fault. I am sure you copy/paste from the docs (a good thing) but in order to make readable the table the character was different Please check https://github.com/kubernetes/ingress-nginx/pull/2111

I’m really surprised to see that everyone is proposing solutions but their is not single final solution proposed and used by everyone. Why is it such a mess with ngnix? 504 error is just bugging us alot as well

The hyphens are not the normal hyphens - but just look like them.

What @akaGelo is trying to say is that if you use your browsers search option and put in a - then some of them will not be highlighted. These are those hyphens which are not the correct ones.

@yivo you’re missing the beginning of the annotation, you need nginx in front of ingress,

so instead of

ingress.kubernetes.io/proxy-read-timeout

you should have

nginx.ingress.kubernetes.io/proxy-read-timeout

Doesn’t work for me.

Okay so this:

    nginx.ingress.kubernetes.io/proxy-connect-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
    nginx.ingress.kubernetes.io/proxy-next-upstream-timeout: "600"

works but only after restarting the complete nginx controller deployment! It is somehow not picking up on ingress change! Seems like a bug to me.

I used k9s to shell into the pod and executed:

cat /etc/nginx/nginx.conf | grep proxy_.*_timeout

@jontro This definitely must be included at the front page of the documentation. This helped me(+ saved me) a lot. Thanks.

kubernetes.io/ingress.class: “gce”

I seem you are using the GCE ingress controller. This annotation only works in nginx

I ran into this as well. I’m assuming an integer is required for timeouts? I was using “5m” because Nginx docs seemed to show that I could. Changed to 300 and things worked great after that.

Oh now that’s seems very obvious ! Thanks guys, that was a pretty simple mistake. I’ll look into official documentation if we can improve that with the same type of character for working copy/paste.

@garagatyi Maybe you have the same problem ? You should also update your ingress revision.

I don’t get the point, annotations were tested one by one and - is an acceptable YAML value. Can you elaborate a bit more about an incorrect - in my annotations ?