istio: missing lbEndpoint for second cluster in istio egressgateway proxy

Is this the right place to submit this?

  • This is not a security vulnerability or a crashing bug
  • This is not a question about how to use Istio

Bug Description

Missing lbEndpoint for second cluster in istio egressgateway proxy

STEPS:

  • BringUp two k8s cluster and install 1.20.4 istio on the both
  • Deployed say shopping in cluster1 which will connect to catalog service that is running in cluster2
  • Configured the needed service entries, destinationrules and gateways

Service Entry:

% k get se -n default        nsxsm.catalog  -o yaml
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
  creationTimestamp: "2024-03-21T09:12:41Z"
  generation: 1
  labels:
    cluster: eekkss
    createdby: test
  name: nsxsm.catalog
  namespace: default
  resourceVersion: "5908383"
  uid: 91b0df7f-0efd-4f18-be0d-b1ef76fede05
spec:
  addresses:
  - XXX.XXX.XXX.XXX
  endpoints:
  - address: istio-egressgateway.istio-system
    ports:
      http-catalog: 443
    weight: 1
  exportTo:
  - .
  hosts:
  - catalog.acme.com
  location: MESH_INTERNAL
  ports:
  - name: http-catalog
    number: 8082
    protocol: HTTP
  resolution: DNS

Destination Rule:

% k get dr -n istio-system   nsxsm.gns.acmetest.com -o yaml
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  creationTimestamp: "2024-03-21T10:31:21Z"
  generation: 1
  labels:
    createdby: test
  name: nsxsm.gns.acme.com
  namespace: istio-system
  resourceVersion: "5928249"
  uid: 83ce0e72-16b0-461f-8f5a-d3e880e546a3
spec:
  host: '*.acme.com'
  trafficPolicy:
    loadBalancer:
      localityLbSetting:
        enabled: false
      simple: ROUND_ROBIN
    outlierDetection:
      baseEjectionTime: 120s
      consecutive5xxErrors: 1
      consecutiveGatewayErrors: 1
      interval: 1s
      maxEjectionPercent: 1
      minHealthPercent: 1
    tls:
      mode: ISTIO_MUTUAL

`

**gateway : Egressgateway**

`
% k get gw -n istio-system   istio-egressgateway  -o yaml
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  creationTimestamp: "2024-03-21T11:36:53Z"
  generation: 1
  labels:
    app: gateways
    createdby: allspark
    release: istio
  name: istio-egressgateway
  namespace: istio-system
  resourceVersion: "5944911"
  uid: 4e1b90e3-09e5-4c94-aee4-68c96220440a
spec:
  selector:
    app: istio-egressgateway
    istio: egressgateway
  servers:
  - hosts:
    - '*.acme.com'
    port:
      name: tls-multicluster-listener
      number: 443
      protocol: TLS
    tls:
      mode: AUTO_PASSTHROUGH

Expected:

The shopping service in cluster1 should able to connect to catalog service in cluster2

Actual: Receiving 503 error in shopping when tries to connect to catalog. Same config worked with istio 1.19.3

Istio Egress Proxy config:

% istioctl proxy-config cluster -i istio-system istio-egressgateway-9f4bb8b8d-4pq9c.istio-system --fqdn catalog.acme.com -o json                 
[
    {
        "name": "outbound|8082||catalog.acme.com",
        "type": "STRICT_DNS",
        "connectTimeout": "10s",
        "loadAssignment": {
            "clusterName": "outbound|8082||catalog.acme.com",
            "endpoints": [
                {
                    "locality": {},
                    "lbEndpoints": [
                        {
                            "endpoint": {
                                "address": {
                                    "socketAddress": {
                                        "address": “XXX.XXX.XXX.XXX”,
                                        "portValue": 443
                                    }
                                }
                            },
                            "metadata": {
                                "filterMetadata": {
                                    "istio": {
                                        "workload": ";;;;Kubernetes"
                                    }
                                }
                            },
                            "loadBalancingWeight": 1
                        },
                        {
                            "endpoint": {
                                "address": {
                                    "socketAddress": {
                                        "address": "istio-egressgateway.istio-system",
                                        "portValue": 443
                                    }
                                }
                            },
                            "metadata": {
                                "filterMetadata": {
                                    "istio": {
                                        "workload": ";;;;Kubernetes"
                                    }
                                }
                            },
                            "loadBalancingWeight": 1
                        }
                    ],
                    "loadBalancingWeight": 2
                }
            ]
        },
        ………
	………
                "sni": "outbound_.8082_._.catalog.acme.com"
            }
        },
       ………
	………
    },

    {
        "name": "outbound_.8082_._.catalog.acme.com",
        "type": "STRICT_DNS",
        "connectTimeout": "10s",
        "loadAssignment": {
            "clusterName": "outbound_.8082_._.catalog.acme.com",
            "endpoints": [
                {
                    "locality": {}                               <<<<<<<<<<<<<<. LbEndpoints missing here
                }
            ]
        },
       ………
	………
]

Version

% istioctl version
client version: 1.20.0
control plane version: 1.20.4
data plane version: 1.20.4


 % kubectl version

Client Version: v1.26.10"
Kustomize Version: v4.5.7
Server Version: v1.26.13-eks-508b6b3

Additional Information

No response

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 37 (15 by maintainers)

Most upvoted comments

In ServiceEntry registry, we make endpoint converted from SE as tls disabled.

func (s *Controller) convertServiceEntryToInstances(cfg config.Config, services []*model.Service) []*model.ServiceInstance {