istio: timed out waiting for the condition Deployment/istio-system/istiod
- Installation
Expected behavior
Complete installation without errors
I tried to install Istio 1.5 on my local k8s cluster and encountered an error. By reading the code, it seems that it is caused by Deployment / istio-system / istiod
not ready. Install Istio 1.5 on katacoda
. This step only takes a few seconds, but on my local computer, He will keep Waiting for resources to become ready...
until he prompts 2020-04-02T21:51:57.048098Z error installer Failed to wait for resource: resources not ready after 10m0s: timed out waiting for the condition Deployment/istio-system/istiod
after 10 minutes.
Steps to reproduce the bug
install with the doc
[root@k1 istio-1.5.1]# export PATH=$PWD/bin:$PATH
[root@k1 istio-1.5.1]#
[root@k1 istio-1.5.1]# istioctl verify-install
Checking the cluster to make sure it is ready for Istio installation...
#1. Kubernetes-api
-----------------------
Can initialize the Kubernetes client.
Can query the Kubernetes API Server.
#2. Kubernetes-version
-----------------------
Istio is compatible with Kubernetes: v1.17.4.
#3. Istio-existence
-----------------------
Istio will be installed in the istio-system namespace.
#4. Kubernetes-setup
-----------------------
Can create necessary Kubernetes configurations: Namespace,ClusterRole,ClusterRoleBinding,CustomResourceDefinition,Role,ServiceAccoun t,Service,Deployments,ConfigMap.
#5. SideCar-Injector
-----------------------
This Kubernetes cluster supports automatic sidecar injection. To enable automatic sidecar injection see https://istio.io/docs/setup/ kubernetes/additional-setup/sidecar-injection/#deploying-an-app
-----------------------
Install Pre-Check passed! The cluster is ready for Istio installation.
[root@k1 istio-1.5.1]# istioctl manifest apply --set profile=demo
Detected that your cluster does not support third party JWT authentication. Falling back to less secure first party JWT. See https://istio.io/docs/ops/best-practices/security/#configure-third-party-service-account-tokens for details .
- Applying manifest for component Base...
✔ Finished applying manifest for component Base.
- Applying manifest for component Pilot...
✔ Finished applying manifest for component Pilot.
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
...........................
...........................
...........................
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
2020-04-02T21:51:57.048098Z error installer Failed to wait for resource: resources not ready after 10m0s: timed out waiting for the condition
Deployment/istio-system/istiod
- Applying manifest for component EgressGateways...
- Applying manifest for component IngressGateways...
- Applying manifest for component AddonComponents...
✔ Finished applying manifest for component EgressGateways.
✔ Finished applying manifest for component AddonComponents.
✔ Finished applying manifest for component IngressGateways.
Version (include the output of istioctl version --remote
and kubectl version
and helm version
if you used Helm)
[root@k1 istio-1.5.1]# istioctl version --remote
Error: 1 error occurred:
* error execing into istiod-646b6fcc6-9zp8k discovery container: unable to upgrade connection: container not found ("discovery")
[root@k1 istio-1.5.1]# kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T21:03:42Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T20:55:23Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
How was Istio installed?
Installing
Environment where bug was observed (cloud vendor, OS, etc)
local k8s cluster
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 33 (9 by maintainers)
Then for debugging the istiod I run to get the detailed info
kubectl describe pod istiod-5bcb74c764-n52gh -n istio-system
Then I find that I wrongly set the hub and tag parameters(–istio.test.hub --istio.test.tag ) in my command. After fix it, it succeed
Not sure if is related but I have the same issue with resource timeout, without creating any pods ( the deploy objects and rs were created).
The thing was I had namespaces istio-system and istio-operator manually created. After 2h of tries, I decided to delete everything. After I deleted the ns istio-system and istio-operator and run the commands install, worked:
[charts:(⎈ |kubernetes-admin@kubernetes:istio-system)]# istioctl install --readiness-timeout 10s This will install the default Istio profile into the cluster. Proceed? (y/N) y Detected that your cluster does not support third party JWT authentication. Falling back to less secure first party JWT. See https://istio.io/docs/ops/best-practices/security/#configure-third-party-service-account-tokens for details. ✔ Istio core installed ✔ Istiod installed ✔ Ingress gateways installed ✔ Installation complete
As a mention this is a local cluster build on VMs under VMware with kubeadm.
@laurentiuspurba Yes,I am able to resolve the issues.Some pods are failing due to low memory and no of cpu .Just increase the memory and no of cpu usage.Then it will be installed.
I encountered the same issue, but it was the lack of CPU and Memory value in my case (using Minikube).
Istio’s document says:
Installation went well after setting CPUs and Memory as designated above.
Found the actual issue:
The issue occurs if namespaces istio-system is labeled with “istio-injection=enabled”
Just remove the label and try again: kubectl label ns istio-system istio-injection-
I was able to upgrade from
1.4.3
->1.4.10
->1.5.8
->1.6.4
. I could not have done it without help from Istio community (Prune and Vito).Prior upgrading to other version, make sure to add newline character on ca-cert.pem as describe here.
1.4.3
->1.4.10
: I did the upgrade usingistioctl experimental upgrade -f IstioControlPlane.yaml
--force
1.4.10
->1.5.8
:kubectl -n istio-system delete service/istio-galley deployment.apps/istio-galley
kubectl delete validatingwebhookconfiguration.admissionregistration.k8s.io/istio-galley
istioctl upgrade -f IstioOperator.yaml
to upgrade (--force
might be required)kubectl -n istio-system delete deployment istio-citadel istio-galley istio-pilot istio-policy istio-sidecar-injector istio-telemetry
kubectl -n istio-system delete service istio-citadel istio-policy istio-sidecar-injector istio-telemetry
kubectl -n istio-system delete horizontalpodautoscaler.autoscaling/istio-pilot horizontalpodautoscaler.autoscaling/istio-telemetry
kubectl -n istio-system delete pdb istio-citadel istio-galley istio-pilot istio-policy istio-sidecar-injector istio-telemetry
kubectl -n istio-system delete deployment istiocoredns
kubectl -n istio-system delete service istiocoredns
1.5.8
to1.64
IstioOperator.yaml
Version 1.5.8 file to support Version 1.6.4istioctl upgrade -f IstioOperator.yaml
to upgrade (--force
might be required)