external-dns: Multiple Istio Gateways (--istio-ingress-gateway) not working as expected

@LorbusChris I think this is still an issue. There is a work around, but it involves deploying multiple versions of the external-dns chart, further the behavior deviates from how the external-dns chart works elsewhere. When I used it a single install with multiple ingresses, it published the proper ip to proper hostname. And I think I saw that --namespace flag got removed now that multiple gateways are supported it didn’t really make sense.

Now that multiple istio gateways are supported when I added 2 gateways:

Containers:
  external-dns:
    Container ID:  docker://49e1f0db4316267a5ff8bcd363359c160b8aee942e680a48cea55cb5ad3c0d09
    Image:         registry.opensource.zalan.do/teapot/external-dns:v0.5.12
    Image ID:      docker-pullable://registry.opensource.zalan.do/teapot/external-dns@sha256:f395ba72e53d9c1e8851461af82da3bd215240bacddde3e7843058dc5cea76c2
    Port:          7979/TCP
    Host Port:     0/TCP
    Args:
      --log-level=info
      --policy=upsert-only
      --provider=google
      --registry=txt
      --interval=1m
      --txt-owner-id=sandbox
      --source=service
      --source=ingress
      --source=istio-gateway
      --istio-ingress-gateway=istio-system/istio-gateway-private
      --istio-ingress-gateway=istio-system/istio-gateway-public

Lets make up some ips for the Gatway’s Services (type LB): istio-gateway-private: 10.0.0.1 istio-gateway-public: 35.0.0.1

I have 2 dns entries one on each gateway… lets call them private.example.com and public.example.com

Both entries get both dns entries so: private.example.com A 10.0.0.1, 35.0.0.1 public.example.com A 10.0.0.1, 35.0.0.1

I don’t think this is what anyone would want or expect, so for now I still need to use the annotation (think I forgot to say thanks @julientyro for that) and have currently 2 deploys of the dns chart, pretty sure I need to deploy a third gateway so that will make it 3 copies of the dns chart.

Anyways my 2 cents is this should still be addressed.

@LorbusChris I can make an attempt at making PR to address @nesl247 proposes which will inspect the Gateway object and add all the ingressgateways specified in its spec.selector if we are all in agreement

Running into this as well. We have the default (public) and secondary (private) istio gateways. When external-dns creates it’s DNS entry, it includes both the default and secondary istio gateway IPs. It should only include the IPs from whatever gateways are specified on the Gateway resources via the spec.selector object.

I’m not sure if Github notifies when an issue is referenced, but I opened a PR earlier today (#1154) that does exactly what @crhuber described here.

@LorbusChris

We also seeing the same behaviour but for a different use case. We have multiple gateway services ie: istio-ingressgateway-a and istio-ingressgateway-b and would like the dns records to point to the appropriate gateway service as specified in the Gateway resource spec.selector

We have applied the following args

-  --istio-ingress-gateway=istio-system/istio-ingressgateway-a
-  --istio-ingress-gateway=istio-system/istio-ingressgateway-b

We have multiple Gateway objects

kind: Gateway
metadata:
  name: foo
  namespace: foo
spec:
  selector:
    istio: istio-ingressgateway-a
---
kind: Gateway
metadata:
  name: bar
  namespace: bar
spec:
  selector:
    istio: istio-ingressgateway-b

At this point we are seeing the dns record for Gateway bar pointing to the Gateway Controller/Service istio-ingressgateway-a or vice-versa. The records seem to be created at random to the multiple gateway controllers

We’ve annotated our Gateways with labels to group them (e.g: type=private) Then, if you want a specific instance of external-dns to only load these Gateway configs, add flag --annotation-filter=type=private to your deployment options.

I’ve been having the same issues described above with external-dns and multiple Istio ingress-gateways causing incorrect A records pointing at each.

There are already a couple of workarounds above but I’ve found another which works for me and may be useful for others. It only requires a single deployment of external-dns.

TL;DR

• Don’t use the Istio external-dns source
• Use the service source along with ingress gateway service annotations to create an A record for you IG.
• Use crd source to create CNAME records for your Istio Gateways pointing at your A records created above.

And in a bit more detail…

Run external-dns with the following args.

...
 - args:
    ....
    - --registry=txt
    - --txt-prefix=cname_
    - --crd-source-apiversion=externaldns.k8s.io/v1alpha1
    - --crd-source-kind=DNSEndpoint
    - --source=service
    - --source=crd
...

We use helm to deploy Istio (Doesn’t everyone?) and on our ingress-gateway definition we then add the following to create the service annotations.

  istio-customig:
    enabled: true
    type: LoadBalancer
    serviceAnnotations:
      external-dns.alpha.kubernetes.io/hostname: <your A record hostname>
...

Then create your CNAME as you deploy your service…

apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint
metadata:
  name: {{ $key }}
spec:
  endpoints:
  - dnsName: <your CNAME record>
    recordTTL: 180
    recordType: CNAME
    targets:
    - <your A record hostname>

This seems to work well. We are using a helm chart for external-dns deployment so actually set these settings via…

...
sources:
- service
- ingress
- crd
rbac:
  create: true
metrics: 
  enabled: true
crd:
  create: true
  apiversion: externaldns.k8s.io/v1alpha1
  kind: DNSEndpoint
txtPrefix: cname_

If you don’t use this ensure your RBAC is update correctly. This is also a useful reference… https://github.com/kubernetes-incubator/external-dns/blob/master/docs/contributing/crd-source.md

Hope this is useful

It worked for me creating two external-dns with --annotation-filter like:

Private external-dns

- --istio-ingress-gateway=istio-system/istio-ingressgateway
- --annotation-filter=type notin (public)

Private gateway (without annotation because I’m filtering with type notin (public))

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: private-test
spec:
  selector:
    istio: ingressgateway

Public external-dns

- --istio-ingress-gateway=istio-system/istio-ingressgateway-public
- --annotation-filter=type=public

Public gateway (with annotation type: public)

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: public-test
  annotations:
    type: public
spec:
  selector:
    istio: ingressgateway-public

Yep. I had to do a not in (values, here) instead, which I really don’t like.

@LorbusChris actually what I was thinking of with namespace is istioNamespace and it was actually your comment 😃

https://github.com/kubernetes-incubator/external-dns/pull/907#issuecomment-473313973

Anyways I updated the title, its wordy b/c I really didn’t know how to describe it concisely. Open to suggestions.

@LorbusChris that’s it… the annotation-filter from external-dns refers to the application gateway annotation, now it’s working, thanks! For reference, that’s how the Gateway looks like now:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  annotations:
    istio-type: internal
  name: test-istio-internal
  namespace: default
spec:
  selector:
    istio: internalgateway
  servers:
  - hosts:
    - test-internal.example.com
    port:
      name: http
      number: 80
      protocol: HTTP

@luizferreira094 can you try using an annotation instead of a label on your Gateway object? Also, I don’t think labeling the IngressGateway objects is necessary.