ingress-nginx: Cannot get multi-path working
I have tried every configuration in the universe.
“work”: Can access all URLs without an error in multiple browsers and with curl.
“does not work”: Sends back 404 or 502 errors. If you refresh on some URLs with some configs, it can toggle between 3 different error results, two are different 404s.
I have two backend services. They work consistently with port-forward, LoadBalance in the service, and basic ingress without paths.
I get the root to map, but not any service below that in the path.
What I want is “/” to route to a node app in nginx and “/service2” and subsequent services to route to various services used by that app. I couldn’t even get “/service1” and “/service2” working without the root path, which matched the examples. I tried following every example, then experimented into the unknown. No matter what I did, no path other than the root one would map successfully to a service.
I am running K8S 1.6.7 on GKE. Having ingress.kubernetes.io/rewrite-target: / doesn’t help. When these errors occur, it is simply not getting to the service.
When I am inside a pod, I can curl port 80 of a service no problem. curl -v http://myservice returns the expected 200. When I get a 200, on the node app, I get a log from nginx.
Note that I did not create any controllers. I am just using gce. Nevertheless, I tried adding kubernetes.io/ingress.class: "gce" with no effect.
I spent over 8 hours trying every possibility I could think of. I deleted and re-created the ingress probably 100-200 times. I even deleted and re-created the cluster.
Going back to the original configuration, where you cannot access either service via the Ingress LB, you consistently get 404 when trying to access the service. I have noticed it is slightly different when I include a slash at the end. Without it, it returns this to curl:
< Content-Length: 0
< Date: Sat, 12 Aug 2017 17:42:38 GMT
< Via: 1.1 google
When I add a slash, I get that with the content default backend - 404. E.g.,
curl -v http://host/collector/
Here is the original multi-path config:
kind: Ingress
metadata:
name: ingress-combined
annotations:
ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.global-static-ip-name: sc-static-ip
kubernetes.io/ingress.class: "gce"
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: sc-admin
servicePort: 80
- path: /collector
backend:
serviceName: sc-collector
servicePort: 80
This basic config works no matter which service I point it to:
kind: Ingress
metadata:
name: basic-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: sc-static-ip
spec:
backend:
serviceName: sc-admin
servicePort: 80
It is only multi-path with anything other than path: / that does not work.
Thank you in advance for your help.
About this issue
- Original URL
- State: closed
- Created 7 years ago
- Reactions: 19
- Comments: 28 (4 by maintainers)
I found a workaround. Using these paths works:
Note that
http://host/teststill resolves to the first service, whereashttp://host/test/resolves to the second service. The services don’t have to be human friendly URLs, so that is not an issue.I clicked on every link in the Node app and couldn’t locate a single 404! YAY! curl seemed happy, too.
The question I have is how come I couldn’t find an example using a wildcard in the path in all the documentation on Ingress? The only reason I thought of it was because I figured out that GKE was using GCE URL Maps under the hood, and learned that it supports wild cards.
I did locate these discussions, which leave this as an open issue.
It has just been a long 10+ hour confusing ride due to documentation not being clear on this. And, to be sure, Google is #1 to earn credit here because I started out with and learned about Ingress from their GKE Setting up HTTP Load Balancing with Ingress documentation where their fanout example shows a root and a context under it pointing to two services WITHOUT asterisks! That was the example I started with, and the config I have been trying to get working. They really do need to update that page, and explain the impact of having or not having a wild card.
Hopefully, someone finds this issue in the first hour of their Ingress adventure. 😃
By default you get a GCE ingress controller running. Because of this:
kubernetes.io/ingress.class: "gce"annotation has no effectingress.kubernetes.io/rewrite-targetannotation is not supported by the GCE ingress controllerIngress version 0.22.0 or higher has changed the way how rewrite-target works. You’ll need to regex-match the path and add it to the rewrite-target.
nginx.ingress.kubernetes.io/rewrite-target: /$2andRefer to changelog here How to article here
In my case, I based my solution in this tutorial and I have the following:
The only way it worked for me was creating 2 ingress, because If I use
nginx.ingress.kubernetes.io/rewrite-target: /It breaks my backend that needs full url pathIf I use
nginx.ingress.kubernetes.io/rewrite-target: /$2It breaks my frontend, showing a blank home page.My solution:
I attached full ingress file yaml.
Hope it helps and thank you so much, this thread helped me a lot!
I am facing the same issue using Nginx Ingress on AWS. Unfortunately it took me 10+ hours to find this thread. 😦
@nimendra Thank you for leaving that comment! We too had this, which was working until we upgraded from an old ingress-nginx version (0.9.0-beta.11 -> 0.22.0):
After many hours, I found your comment and changed to this, which works:
@lemonshow remove
*from the pathsI sorted out by adding nginx.ingress.kubernetes.io/rewrite-target annotation. I’m using LetsEncrypt and not allowing http, but https.
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: nginx-ingress namespace: default annotations: certmanager.k8s.io/cluster-issuer: letsencrypt-prod ingress.kubernetes.io/service-upstream: “true” kubernetes.io/ingress.class: “nginx” kubernetes.io/tls-acme: “true” kubernetes.io/ingress.allow-http: “false” nginx.ingress.kubernetes.io/from-to-www-redirect: “true” nginx.ingress.kubernetes.io/ssl-redirect: “true” nginx.ingress.kubernetes.io/force-ssl-redirect: “true” nginx.ingress.kubernetes.io/rewrite-target: /
Refer to Step 6 : Serving Multiple Applications on a Load Balancer on this Google Kubernetes Engine tutorial on Setting up HTTP Load Balancing with Ingress. It exactly answers the questions raised in this thread.
To route path
/to backend servicewebon port8080and path/v2to backend serviceweb2on port8080, the spec section of the ingress yaml file will be:Ensure that backend services respond with a
200OKfor/request. Note thatweb2needs to respond with a200OKto a root path/( not/v2/). This is important for Google Load Balancer to perform a successful health check on the two services before making them available. If they do not pass the health checks, you would get a502Error.In our case, we have a
angularfrontend service with anexpressbackend. We want to angular to serve/and express to serve/api for the same host.So, in our
expressrequest handling file, we have the following added to respond with a200OKfor path/.Hope this helps.
Hi, I have the Nginx Ingress running without issues in Production and HTTPS by using LetsEncrypt certificates:
I set up a quick repo with the resources I used for the production environment. You need to replace the “test” subdomain with your own and set up the SSL certificates.
Hope it helps.
@LorenzoR no with ingress-nginx
It works now! Actually I had another issue after this where I was testing with NLB but I comment out https. I saw 308 redirect to https. Uncomment https, things start to work. thanks a lot,
@erik777 I got the same issue. The base works but the multi-path not. I am using Nginx ingress controller + AWS NLB and try to bring up my two services with the Ingress. It always routes to default and I am getting 404. Anyone used with AWS EKS with luck? Use rewrite did not help either. Any other suggestion? thanks,
apiVersion: extensions/v1beta1 kind: Ingress metadata: name: test-ingress namespace: local annotations: ingress.kubernetes.io/rewrite-target: ‘/’ spec: #backend: #serviceName: app1-service #servicePort: 80 rules:
In my case, I had two services, one running at / and another at /ui . I tried several combinations and the only one that worked with K8s 1.10 was
http://mydomain.com/* Service 1 Ingress kubernetes.io/ingress.class: “nginx” path: /
http://mydonain.com/ui/app/index.html Service 2 Ingress kubernetes.io/ingress.class: “nginx” nginx.ingress.kubernetes.io/rewrite-target: /ui path: /ui/*
Note that for service 2, my tomcat was also initialising at /ui , so the rewrite-target did the trick otherwise, I had to add two “ui” to my url like http://mydonain.com/ui/ui/app/index.html
I have a similar issue with Azure AKS (v1.9.6) and a nginx (nginx-ingress-controller-0.15.0) :
I want to have different paths to route requests to different backends.
The following configuration works, but when I request “https://xxx/iv-da/swagger”, the /swagger/v0/swagger.json AJAX call makes a request to “https://xxx/swagger/v0/swagger.json” without the “iv-da” context.
How could it keep the context for each backend’s inside requests ?
@felipecrescencio if you create 2 ingress, do you have to pay for 2 load balancers in GCP?
@dolphub I ended up doing this :
And for the swagger problem, it is inside the code that we need to force the context, it is not an ingress problem.
Hope it helps.