istio: Readiness probe failed: HTTP probe failed with statuscode: 503
How to fix this - if possible?
Download istio 1.5.2 installed by
istioctl manifest apply --set profile=default --set values.gateways.istio-ingressgateway.type=NodePort
root@master ~]# kubectl describe pod istio-ingressgateway-6cf56d9598-ptqp7
.....
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned istio-system/istio-ingressgateway-6cf56d9598-ptqp7 to node1.yr
Normal Pulled 21m kubelet, node1.yr Container image "docker.io/istio/proxyv2:1.5.2" already present on machine
Normal Created 21m kubelet, node1.yr Created container istio-proxy
Normal Started 21m kubelet, node1.yr Started container istio-proxy
Warning Unhealthy 111s (x600 over 21m) kubelet, node1.yr Readiness probe failed: HTTP probe failed with statuscode: 503
[root@master ~]# kubectl get po
NAME READY STATUS RESTARTS AGE
istio-ingressgateway-6cf56d9598-ptqp7 0/1 Running 0 25m
istiod-c7757dcf7-2dpgn 1/1 Running 0 25m
prometheus-dfd976959-5jxxf 1/2 Running 0 25m
[root@master ~]#
[root@master ~]# kubectl get endpoints -n istio-system
NAME ENDPOINTS AGE
istio-ingressgateway 30m
istio-pilot 100.96.79.25:8080,100.96.79.25:15011,100.96.79.25:15010 + 3 more... 30m
istiod 100.96.79.25:15012,100.96.79.25:15017 30m
prometheus
Environment: Virutal Machine on private cloud
[root@master ~]# kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.1", GitCommit:"7879fc12a63337efff607952a323df90cdc7a335", GitTreeState:"clean", BuildDate:"2020-04-08T17:38:50Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.1", GitCommit:"7879fc12a63337efff607952a323df90cdc7a335", GitTreeState:"clean", BuildDate:"2020-04-08T17:30:47Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
[root@master ~]#
[root@master ~]# kubectl get po -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-6b4c6ff56f-w4n4w 1/1 Running 1 31h
calico-node-gks95 1/1 Running 1 31h
calico-node-q6fng 1/1 Running 1 31h
coredns-689857ddd7-942qw 1/1 Running 1 31h
coredns-689857ddd7-zw6bb 1/1 Running 1 31h
etcd-master.yr 1/1 Running 1 31h
kube-apiserver-master.yr 1/1 Running 1 31h
kube-controller-manager-master.yr 1/1 Running 1 31h
kube-proxy-6s6bd 1/1 Running 1 31h
kube-proxy-t9v48 1/1 Running 1 31h
kube-scheduler-master.yr 1/1 Running 1 31h
[root@master ~]#
Logs
[root@master ~]# kubectl logs istio-ingressgateway-6cf56d9598-ptqp7
2020-04-26T08:40:15.309590Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2020-04-26T08:40:17.309575Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2020-04-26T08:40:19.309387Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2020-04-26T08:40:21.309381Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2020-04-26T08:40:23.309601Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2020-04-26T08:40:25.309706Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
[Envoy (Epoch 0)] [2020-04-26 08:40:26.977][20][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:91] gRPC config stream closed: 14, no healthy upstream
[Envoy (Epoch 0)] [2020-04-26 08:40:26.977][20][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:54] Unable to establish new stream
2020-04-26T08:40:27.309739Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2020-04-26T08:40:29.309719Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
[Envoy (Epoch 0)] [2020-04-26 08:40:30.144][20][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:91] gRPC config stream closed: 14, no healthy upstream
[Envoy (Epoch 0)] [2020-04-26 08:40:30.144][20][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:54] Unable to establish new stream
2020-04-26T08:40:31.309331Z info Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Reactions: 7
- Comments: 19 (1 by maintainers)
I am facing same error. When i applyed this offical document configretion demo,this error had been resloved. https://istio.io/latest/zh/docs/tasks/security/authentication/authn-policy/ I guess this problem because of istio default require mutual TLS. when you applyed last config,please regenerate pod.
[root@hd1-msb-testk8scluster-m1]# kubectl apply -f - <<EOF apiVersion: “networking.istio.io/v1alpha3” kind: “DestinationRule” metadata: name: “default” namespace: “istio-system” spec: host: “*.local” trafficPolicy: tls: mode: ISTIO_MUTUAL EOF