istio: Istio-Proxy returning UH / no healthy upstream at startup for STRICT_DNS clusters
Is this the right place to submit this?
- This is not a security vulnerability or a crashing bug
- This is not a question about how to use Istio
Bug Description
We have around 3-4k STRICT_DNS clusters (via ServiceEntries), and observed during startup istio-proxy returns a bunch of UH / no healthy upstream, which subsequently subside.
Our assumption is - Given the large amount of STRICT_DNS clusters, istio-proxy is taking time to complete the DNS resolution, and until then it doesn’t show the endpoints for such clusters (checked with istioctl proxy-config endpoints command), and hence returns a UH / no healthy upstream.
Istio-proxy should not call itself ready if it is not able to handle calls to STRICT_DNS clusters, as it has not completed DNS resolution for them.
Questions:
- Can istio-proxy ensure all DNS resolutions are complete before taking any traffic?
- Can it do runtime DNS resolution if it was not already done, and avoid UH / no healthy upstream errors?
Version
$ istioctl version
client version: 1.17.4
control plane version: 1.17-dca7c
data plane version: 1.17-dca7c (11 proxies)
╰─$ k version 11:03AM
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.3", GitCommit:"25b4e43193bcda6c7328a6d147b1fb73a33f1598", GitTreeState:"clean", BuildDate:"2023-06-14T09:47:38Z", GoVersion:"go1.20.5", Compiler:"gc", Platform:"darwin/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"25+", GitVersion:"v1.25.16-eks-8cb36c9", GitCommit:"3a3ea80e673d7867f47bdfbccd4ece7cb5f4a83a", GitTreeState:"clean", BuildDate:"2023-11-22T21:53:22Z", GoVersion:"go1.20.10", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.27) and server (1.25) exceeds the supported minor version skew of +/-1
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 41 (40 by maintainers)
No, I think its just based on the interval not “on demand” like this.