istio: Filter Chain Match Trie logic leads to unexpected matches leading to various bugs

Bug description

We find that Istio seems to cause outbound HTTPS traffic issues in some of our Kubernetes clusters. For eg:- We have an application that makes HTTPS calls to the NewRelic and AWS API. The behaviour is inconsistent in that the outbound HTTPS call works in some of our Kubernetes clusters. We are trying to understand what the root cause is.

In an effort to find the root cause, we are trying to compare the istio proxy configs from Kubernetes clusters that have and don’t have this issue. Since the istio-proxy config dumps are huge JSONs. we request help to narrow down on the relevant JSON fields/structs for configuring outbound HTTPS requests.

 time="2020-07-30T23:11:49Z" level=error msg="RequestError: send request failed\ncaused by: Get \"https://route53.amazonaws.com/2013-04-01/hostedzone\": http: server gave HTTP response to HTTPS client"

[ ] Docs [ ] Installation [X] Networking [ ] Performance and Scalability [ ] Extensions and Telemetry [ ] Security [ ] Test and Release [X] User Experience [ ] Developer Infrastructure

Expected behavior

Steps to reproduce the bug

Just inject istio-proxy to these pods that make outbound HTTPS calls

Version (include the output of istioctl version --remote and kubectl version and helm version if you used Helm)

v1.6.3

How was Istio installed?

Helm

Environment where bug was observed (cloud vendor, OS, etc)

AWS

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 6
Comments: 28 (22 by maintainers)

Most upvoted comments

Opened envoy bug: https://github.com/envoyproxy/envoy/issues/12572

howardjohn on Aug 10, 2020

I just want to add that this is rather concerning. The customer workload which had this bad port naming was not even meant to be a part of the service mesh (no istio-injection enabled in its namespace) and yet it affected the entire mesh. We can’t afford to have such a mishap in production especially given its likely due to envoy’s hot reloads.

I see two problems here

i) Why should just ONE service workload that’s not istio-meshed compromise the entire service network? ii) port.Name is intended to be largely informational and not of much consequence Kubernetes-wise. Why is Istio’s proxy configuration so reliant on this field port.Name?

tariq1890 on Aug 7, 2020

port.Name very early on becomes server.Protocol which is handled differently in listener.go

On Thu, Aug 6, 2020, 6:09 PM Tariq Ibrahim notifications@github.com wrote:

@rshriram https://github.com/rshriram Could you point out where the logic for port.Name is being used here? Trying to understand how the port Naming affects listener creation

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/istio/istio/issues/26079#issuecomment-670266104, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEYGXO5DMBQVO5KLTKXXBDR7NH6JANCNFSM4PTT5WAQ .

howardjohn on Aug 7, 2020