istio: Istio gateway not working with any other port except 80 and 443
Describe the bug Hello, We are using istio with istio auth enable and expose the istio ingress controller using NodePort.
NAME READY STATUS RESTARTS AGE IP NODE
grafana-6f6dff9986-sdqqh 1/1 Running 0 7d 172.16.0.3 node2.example.com
istio-citadel-7bdc7775c7-vfkl6 1/1 Running 0 7d 172.16.2.6 node1.example.com
istio-cleanup-old-ca-2snhr 0/1 Completed 0 7d 172.16.1.7 node3.example.com
istio-egressgateway-78dd788b6d-sttcs 1/1 Running 0 7d 172.16.1.3 node3.example.com
istio-ingressgateway-7dd84b68d6-2xf5m 1/1 Running 0 7d 172.16.0.5 node2.example.com
istio-ingressgateway-7dd84b68d6-kcm9b 1/1 Running 0 7d 172.16.2.5 node1.example.com
istio-ingressgateway-7dd84b68d6-krcxk 1/1 Running 0 7d 172.16.1.4 node3.example.com
istio-mixer-post-install-ckcfs 0/1 Completed 0 7d 172.16.1.2 node3.example.com
istio-pilot-d5bbc5c59-7ph4m 2/2 Running 1 7d 172.16.0.8 node2.example.com
istio-policy-64595c6fff-p22zh 2/2 Running 0 7d 172.16.0.4 node2.example.com
istio-sidecar-injector-645c89bc64-wmlxx 1/1 Running 0 7d 172.16.0.9 node2.example.com
istio-statsd-prom-bridge-949999c4c-bqwnx 1/1 Running 0 7d 172.16.0.2 node2.example.com
istio-telemetry-cfb674b6c-hcd5p 2/2 Running 0 7d 172.16.1.5 node3.example.com
istio-tracing-754cdfd695-2gbtx 1/1 Running 0 7d 172.16.0.7 node2.example.com
prometheus-86cb6dd77c-8thdj 1/1 Running 0 7d 172.16.0.6 node2.example.com
servicegraph-5849b7d696-vdx69 1/1 Running 0 7d 172.16.1.6 node3.example.com
Services:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
grafana ClusterIP 10.3.0.229 3000/TCP 7d
istio-citadel ClusterIP 10.3.0.199 <none> 8060/TCP,9093/TCP 7d
istio-egressgateway ClusterIP 10.3.0.57 <none> 80/TCP,443/TCP 7d
istio-ingressgateway NodePort 10.3.0.253 <none> 80:31380/TCP,443:31390/TCP,31400:31400/TCP 7d
istio-pilot ClusterIP 10.3.0.123 <none> 15003/TCP,15005/TCP,15007/TCP,15010/TCP,15011/TCP,8080/TCP,9093/TCP 7d
istio-policy ClusterIP 10.3.0.103 <none> 9091/TCP,15004/TCP,9093/TCP 7d
istio-sidecar-injector ClusterIP 10.3.0.251 <none> 443/TCP 7d
istio-statsd-prom-bridge ClusterIP 10.3.0.250 <none> 9102/TCP,9125/UDP 7d
istio-telemetry ClusterIP 10.3.0.170 <none> 9091/TCP,15004/TCP,9093/TCP,42422/TCP 7d
prometheus ClusterIP 10.3.0.169 <none> 9090/TCP 7d
servicegraph ClusterIP 10.3.0.144 <none> 8088/TCP 7d
tracing LoadBalancer 10.3.0.224 <pending> 80:31379/TCP 7d
zipkin ClusterIP 10.3.0.213 <none> 9411/TCP 7d
Issue is we are not able to use any other ports in istio gateway except standard Ports(80/443) for domain.
We are trying to call our domain with the port 5000 (test.com:5000) but it always gives the following error in ingress log [404 NOTFOUND]:
[2018-07-19T12:55:53.271Z] "GET / HTTP/1.1" 404 NR 0 0 3 - "192.168.X.X" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36" "e59d988c-e99f-9f9e-a6a1-23513dcf8e89" "test.com:5000" "-"
[2018-07-19T12:55:54.462Z] "GET / HTTP/1.1" 404 NR 0 0 0 - "192.168.X.X" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36" "be272d54-45ba-97a9-8573-bccaaf556f24" "test.com:5000" "-"
[2018-07-19T12:55:55.382Z] "GET / HTTP/1.1" 404 NR 0 0 0 - "192.168.X.X" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36" "dbeaa57e-3fee-9a43-a7a9-067fda01afae" "test.com:5000" "-"
[2018-07-19T12:56:08.721Z] "GET / HTTP/1.1" 404 NR 0 0 3 - "192.168.X.X" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36" "08640ff9-bc6a-981e-8902-cf9e57f08754" "test.com:5000" "-"
[2018-07-19T12:56:10.454Z] "GET / HTTP/1.1" 404 NR 0 0 0 - "192.168.X.X" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.181 Safari/537.36" "291fee82-bb13-9270-ace2-83946b824815" "test.com:5000" "-"
But when we tried using port 80 it works [200 OK]:
[2018-07-19T12:41:58.143Z] "GET / HTTP/1.1" 200 - 0 11 25 21 "192.168.X.X" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0" "23e000b5-2307-9ba9-8c6f-c4385e961ba2" "test.com" "172.16.1.58:5000"
[2018-07-19T12:42:16.852Z] "GET / HTTP/1.1" 200 - 0 11 55 24 "192.168.X.X" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:61.0) Gecko/20100101 Firefox/61.0" "4275d15d-ae9a-9476-a4f9-401b5ab96671" "test.com" "172.16.1.58:5000"
gateway.yaml:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: myistio-gateway
namespace: istio-system
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- test.com
port:
number: 5000
virtualservice.yaml:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: virtualservice-test
namespace: istio-system
spec:
hosts:
- test.com
gateways:
- myistio-gateway
http:
- match:
- uri:
prefix: /
route:
- destination:
host: service-test.ns-test.svc.cluster.local
port:
number: 5000
So are we missing something here?
Version Istio - 0.8 Kubernetes - 10.3 etcd - v3
Is Istio Auth enabled or not? Yes it is enabled
Environment We deployed kubernetes on coreos.
Are we missing something here?
Thanks in advance 😃
About this issue
- Original URL
- State: closed
- Created 6 years ago
- Comments: 25 (4 by maintainers)
Oh I totally forgot. If you want to expose new ports in the ingressgateway, you have to edit the helm chart (in values.yaml, we have a gateways section that lists the ports that the gateway service in Kubernetes should listen on. This will allow k8s to setup the load balancer properly with the appropriate ports to open up). And then you do the rest (adding new gateway server on the port, etc.)…
Put another way, if you want to open up new ports on the platform specific gateway service, you have ot make sure that these ports are accessible from outside. So, while your gateway definition would have caused Envoy to start listening on port 5000, the K8S load balancer object pointing to the gateway is still only accepting connections on 80/443 per the K8S service declaration (this stuff is happening at the Os level or setting up stuff at cloud provider level for load balancers). All of Istio stuff is confined to Envoy level.