istio: Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "validation.istio.io": Post https://istiod.istio-system.svc:443/validate?timeout=30s:
Bug description Try to install last 1.5.0 version over Kubernetes 1.7, and consistently failed with error: Error from server (InternalError): error when creating “STDIN”: Internal error occurred: failed calling webhook “validation.istio.io”: Post https://istiod.istio-system.svc:443/validate?timeout=30s: dial tcp 10.111.86.18:443: connect: no route to host (repeated 2 times)
[ ] Configuration Infrastructure [ ] Docs [X ] Installation [ ] Networking [ ] Performance and Scalability [ ] Policies and Telemetry [ ] Security [ ] Test and Release [ ] User Experience [ ] Developer Infrastructure
Expected behavior Expect to see istio deployed over kubernetes.
Steps to reproduce the bug istioctl manifest apply --set addonComponents.grafana.enabled=true
Version (include the output of istioctl version --remote
and kubectl version
and helm version
if you used Helm)
istioctl version: 1.5.0.
kubectl version:
kubectl version
Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-11T18:14:22Z", GoVersion:"go1.13.6", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.4", GitCommit:"8d8aa39598534325ad77120c120a22b3a990b5ea", GitTreeState:"clean", BuildDate:"2020-03-12T20:55:23Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"linux/amd64"}
How was Istio installed? curl -L https://istio.io/downloadIstio | sh -
Environment where bug was observed (cloud vendor, OS, etc) Installation on premise - Centos 7
Full error output:
istioctl manifest apply --set addonComponents.grafana.enabled=true
Detected that your cluster does not support third party JWT authentication. Falling back to less secure first party JWT. See https://istio.io/docs/ops/best-practices/security/#configure-third-party-service-account-tokens for details.
- Applying manifest for component Base...
✔ Finished applying manifest for component Base.
- Applying manifest for component Pilot...
✔ Finished applying manifest for component Pilot.
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
Waiting for resources to become ready...
- Applying manifest for component IngressGateways...
- Applying manifest for component AddonComponents...
2020-03-19T12:39:04.197433Z error installer error running kubectl: exit status 1
✘ Finished applying manifest for component AddonComponents.
2020-03-19T12:39:07.010758Z error installer error running kubectl: exit status 1
✘ Finished applying manifest for component IngressGateways.
Component AddonComponents - manifest apply returned the following errors:
Error: error running kubectl: exit status 1
Error detail:
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "validation.istio.io": Post https://istiod.istio-system.svc:443/validate?timeout=30s: dial tcp 10.111.86.18:443: connect: no route to host (repeated 1 times)
clusterrole.rbac.authorization.k8s.io/prometheus-istio-system created
clusterrolebinding.rbac.authorization.k8s.io/prometheus-istio-system created
serviceaccount/prometheus created
configmap/istio-grafana created
configmap/istio-grafana-configuration-dashboards-citadel-dashboard created
configmap/istio-grafana-configuration-dashboards-galley-dashboard created
configmap/istio-grafana-configuration-dashboards-istio-mesh-dashboard created
configmap/istio-grafana-configuration-dashboards-istio-performance-dashboard created
configmap/istio-grafana-configuration-dashboards-istio-service-dashboard created
configmap/istio-grafana-configuration-dashboards-istio-workload-dashboard created
configmap/istio-grafana-configuration-dashboards-mixer-dashboard created
configmap/istio-grafana-configuration-dashboards-pilot-dashboard created
configmap/prometheus created
deployment.apps/grafana created
deployment.apps/prometheus created
service/grafana created
service/prometheus created
Component IngressGateways - manifest apply returned the following errors:
Error: error running kubectl: exit status 1
Error detail:
Error from server (InternalError): error when creating "STDIN": Internal error occurred: failed calling webhook "validation.istio.io": Post https://istiod.istio-system.svc:443/validate?timeout=30s: dial tcp 10.111.86.18:443: connect: no route to host (repeated 2 times)
serviceaccount/istio-ingressgateway-service-account created
deployment.apps/istio-ingressgateway created
poddisruptionbudget.policy/ingressgateway created
role.rbac.authorization.k8s.io/istio-ingressgateway-sds created
rolebinding.rbac.authorization.k8s.io/istio-ingressgateway-sds created
horizontalpodautoscaler.autoscaling/istio-ingressgateway created
service/istio-ingressgateway created
✘ Errors were logged during apply operation. Please check component installation logs above.
Error: failed to apply manifests: errors were logged during apply operation
Thanks.
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 15 (2 by maintainers)
i have same issue. k8s: 1.17.3 istio: 1.5.1
A little different:istio-ingressgateway is running. But the log of kube-apiserver still reports errors:
the log of istiod reports info:
@feliperfmarques I am facing a similar issue with Centos 7 on-premise development. There is no firewall running and automatic sidecar injection for the pod is also failing.
kubernetes version: 1.18.1 istio version : 1.5.2
Any help much appreciated.
Error: Kube-api server:
– Failed calling webhook, failing open validation.istio.io: failed calling webhook “validation.istio.io”: Post https://istiod.istio-system.svc:443/validate?timeout=30s: context deadline exceeded
validationController Not ready to switch validation to fail-closed: dummy invalid config not rejected validationController validatingwebhookconfiguration istiod-istio-system (failurePolicy=Ignore, resourceVersion=205973) is up-to-date. No change required. validationController Reconcile(enter): retry dry-run creation of invalid config
It’s a flannel issue. For now able to fix the issue by changing the flannel backend to host-gw from vxlan. Not sure why it didn’t work with vxlan. Thanks for @nustiueudinastea (Alex) for debugging the issue.
This issue can also to be a problem of webhook validation. For webhook works, firewall rule needs setting to port 15017 instead of 9443 on Istio 1.5, as mentioned here #19532 (comment). Try add this firewall rule.