istio: Readiness probe failed: HTTP probe failed with statuscode: 503

How to fix this - if possible?

kubectl describe pods istio-ingressgateway-66c8bbb77b-j99j6 -n istio-system
...
Events:
  Type     Reason     Age                       From                                                Message
  ----     ------     ----                      ----                                                -------
  Warning  Unhealthy  2m11s (x94307 over 2d4h)  kubelet, gke-mini-default-pool-15a06374-dftr  Readiness probe failed: HTTP probe failed with statuscode: 503
kubectl get pods -n istio-system                                                   
NAME                                    READY   STATUS    RESTARTS   AGE
istio-ingressgateway-57896bd575-cv2p9   0/1     Running   0          4h3m
istio-pilot-74c44c647f-hrz65            1/1     Running   0          4h3m

Environment: Google Kubernetes Engine(GKE)

kubectl version                                                                                                                                   130 ↵
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.9", GitCommit:"2e808b7cb054ee242b68e62455323aa783991f03", GitTreeState:"clean", BuildDate:"2020-01-18T23:33:14Z", GoVersion:"go1.12.12", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"15+", GitVersion:"v1.15.9-gke.24", GitCommit:"39e41a8d6b7221b901a95d3af358dea6994b4a40", GitTreeState:"clean", BuildDate:"2020-02-29T01:24:35Z", GoVersion:"go1.12.12b4", Compiler:"gc", Platform:"linux/amd64"}
istioctl version
client version: 1.5.1
control plane version: 1.5.1
data plane version: none

Installed from https://knative.dev/docs/install/installing-istio/#installing-istio-without-sidecar-injection:

# A lighter template, with just pilot/gateway.
# Based on install/kubernetes/helm/istio/values-istio-minimal.yaml
helm template --namespace=istio-system \
  --set prometheus.enabled=false \
  --set mixer.enabled=false \
  --set mixer.policy.enabled=false \
  --set mixer.telemetry.enabled=false \
  `# Pilot doesn't need a sidecar.` \
  --set pilot.sidecar=false \
  --set pilot.resources.requests.memory=128Mi \
  `# Disable galley (and things requiring galley).` \
  --set galley.enabled=false \
  --set global.useMCP=false \
  `# Disable security / policy.` \
  --set security.enabled=false \
  --set global.disablePolicyChecks=true \
  `# Disable sidecar injection.` \
  --set sidecarInjectorWebhook.enabled=false \
  --set global.proxy.autoInject=disabled \
  --set global.omitSidecarInjectorConfigMap=true \
  --set gateways.istio-ingressgateway.autoscaleMin=1 \
  --set gateways.istio-ingressgateway.autoscaleMax=2 \
  `# Set pilot trace sampling to 100%` \
  --set pilot.traceSampling=100 \
  --set global.mtls.auto=false \
  install/kubernetes/helm/istio \
  > ./istio-lean.yaml

kubectl apply -f istio-lean.yaml

Logs

kubectl -n istio-system  logs istio-ingressgateway-57896bd575-cv2p9
2020-04-04T07:47:52.944158Z	info	Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2020-04-04T07:47:54.943944Z	info	Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2020-04-04T07:47:56.944064Z	info	Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2020-04-04T07:47:58.945674Z	info	Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2020-04-04T07:48:00.944137Z	info	Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2020-04-04T07:48:02.944018Z	info	Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
2020-04-04T07:48:04.943958Z	info	Envoy proxy is NOT ready: config not received from Pilot (is Pilot running?): cds updates: 0 successful, 0 rejected; lds updates: 0 successful, 0 rejected
[Envoy (Epoch 0)] [2020-04-04 07:48:05.780][19][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:91] gRPC config stream closed: 14, no healthy upstream
[Envoy (Epoch 0)] [2020-04-04 07:48:05.780][19][warning][config] [bazel-out/k8-opt/bin/external/envoy/source/common/config/_virtual_includes/grpc_stream_lib/common/config/grpc_stream.h:54] Unable to establish new stream

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 15 (2 by maintainers)

Most upvoted comments

I’m facing the same problem, is there any update or work around to solve it?

Is there any update on this? Facing the same issue.

can you pasted the full log and kubectl get endpoints -n istio-system?

when i run kubectl get endpoints -n istio-system,

NAME ENDPOINTS AGE istio-ingressgateway 3d14h istiod 172.17.1.21:15012,172.17.1.21:15010,172.17.1.21:15017 + 1 more… 3d14h

what is wrong whit my instio-install. My k8s version=1.19.3,istio version=1.8.2