external-dns: v0.7.3 regression from v0.7.2 when using istio-gateway source (all records DELETEd)

What happened:

Upgrading to v0.7.3 deleted all created DNS entries sourced from Istio gateway (–source=istio-gateway) Downgrading back to v0.7.2 with no other changes recreated the entries deleted by v0.7.3

What you expected to happen:

Upgrade works with existing configuration.

How to reproduce it (as minimally and precisely as possible):

  1. Have a working config using istio-gateway as source, external-dns v0.7.2
  2. Upgrade to external-dns v0.7.3
  3. Witness all entries created from istio-gateway source deleted by external-dns

External-dns config:

      --txt-owner-id=REDACTED
      --domain-filter=REDACTED
      --source=service
      --source=ingress
      --source=istio-gateway
      --provider=aws
      --aws-zone-type=public
      --registry=txt

External-dns logs from v0.7.2 before upgrade:

external-dns-565d8dd4c7-zq8vn external-dns time="2020-08-10T19:32:13Z" level=info msg="config: {Master: KubeConfig: RequestTimeout:30s IstioIngressGatewayServices:[] ContourLoadBalancerService:heptio-contour/contour SkipperRouteGroupVersion:zalando.org/v1 Sources:[service ingress istio-gateway] Namespace: AnnotationFilter: FQDNTemplate: CombineFQDNAndAnnotation:false IgnoreHostnameAnnotation:false Compatibility: PublishInternal:false PublishHostIP:false AlwaysPublishNotReadyAddresses:false ConnectorSourceServer:localhost:8080 Provider:aws GoogleProject: GoogleBatchChangeSize:1000 GoogleBatchChangeInterval:1s DomainFilter:[REDACTED] ExcludeDomains:[] ZoneIDFilter:[] AlibabaCloudConfigFile:/etc/kubernetes/alibaba-cloud.json AlibabaCloudZoneType: AWSZoneType:public AWSZoneTagFilter:[] AWSAssumeRole: AWSBatchChangeSize:1000 AWSBatchChangeInterval:1s AWSEvaluateTargetHealth:true AWSAPIRetries:3 AWSPreferCNAME:false AzureConfigFile:/etc/kubernetes/azure.json AzureResourceGroup: AzureSubscriptionID: AzureUserAssignedIdentityClientID: CloudflareProxied:false CloudflareZonesPerPage:50 CoreDNSPrefix:/skydns/ RcodezeroTXTEncrypt:false AkamaiServiceConsumerDomain: AkamaiClientToken: AkamaiClientSecret: AkamaiAccessToken: InfobloxGridHost: InfobloxWapiPort:443 InfobloxWapiUsername:admin InfobloxWapiPassword: InfobloxWapiVersion:2.3.1 InfobloxSSLVerify:true InfobloxView: InfobloxMaxResults:0 DynCustomerName: DynUsername: DynPassword: DynMinTTLSeconds:0 OCIConfigFile:/etc/kubernetes/oci.yaml InMemoryZones:[] OVHEndpoint:ovh-eu PDNSServer:http://localhost:8081 PDNSAPIKey: PDNSTLSEnabled:false TLSCA: TLSClientCert: TLSClientCertKey: Policy:sync Registry:txt TXTOwnerID:REDACTED TXTPrefix: TXTSuffix: Interval:1m0s Once:false DryRun:false UpdateEvents:false LogFormat:text MetricsAddress::7979 LogLevel:info TXTCacheInterval:0s ExoscaleEndpoint:https://api.exoscale.ch/dns ExoscaleAPIKey: ExoscaleAPISecret: CRDSourceAPIVersion:externaldns.k8s.io/v1alpha1 CRDSourceKind:DNSEndpoint ServiceTypeFilter:[] CFAPIEndpoint: CFUsername: CFPassword: RFC2136Host: RFC2136Port:0 RFC2136Zone: RFC2136Insecure:false RFC2136TSIGKeyName: RFC2136TSIGSecret: RFC2136TSIGSecretAlg: RFC2136TAXFR:false RFC2136MinTTL:0s NS1Endpoint: NS1IgnoreSSL:false TransIPAccountName: TransIPPrivateKeyFile:}"
external-dns-565d8dd4c7-zq8vn external-dns time="2020-08-10T19:32:13Z" level=info msg="Instantiating new Kubernetes client"
external-dns-565d8dd4c7-zq8vn external-dns time="2020-08-10T19:32:13Z" level=info msg="Using inCluster-config based on serviceaccount-token"
external-dns-565d8dd4c7-zq8vn external-dns time="2020-08-10T19:32:13Z" level=info msg="Created Kubernetes client https://100.64.0.1:443"
external-dns-565d8dd4c7-zq8vn external-dns time="2020-08-10T19:32:22Z" level=info msg="Desired change: CREATE kafdrop.REDACTED A [Id: /hostedzone/REDACTED]"
...REDACTED...
external-dns-565d8dd4c7-zq8vn external-dns time="2020-08-10T19:32:23Z" level=info msg="24 record(s) in zone REDACTED. [Id: /hostedzone/REDACTED] were successfully updated"
external-dns-565d8dd4c7-zq8vn external-dns time="2020-08-10T19:33:23Z" level=info msg="All records are already up to date"

External-dns logs from v0.7.3 after upgrade:

external-dns-568764f9cb-48djw external-dns time="2020-08-10T19:57:19Z" level=info msg="config: {APIServerURL: KubeConfig: RequestTimeout:30s IstioIngressGatewayServices:[] ContourLoadBalancerService:heptio-contour/contour SkipperRouteGroupVersion:zalando.org/v1 Sources:[service ingress istio-gateway] Namespace: AnnotationFilter: FQDNTemplate: CombineFQDNAndAnnotation:false IgnoreHostnameAnnotation:false Compatibility: PublishInternal:false PublishHostIP:false AlwaysPublishNotReadyAddresses:false ConnectorSourceServer:localhost:8080 Provider:aws GoogleProject: GoogleBatchChangeSize:1000 GoogleBatchChangeInterval:1s DomainFilter:[REDACTED] ExcludeDomains:[] ZoneIDFilter:[] AlibabaCloudConfigFile:/etc/kubernetes/alibaba-cloud.json AlibabaCloudZoneType: AWSZoneType:public AWSZoneTagFilter:[] AWSAssumeRole: AWSBatchChangeSize:1000 AWSBatchChangeInterval:1s AWSEvaluateTargetHealth:true AWSAPIRetries:3 AWSPreferCNAME:false AzureConfigFile:/etc/kubernetes/azure.json AzureResourceGroup: AzureSubscriptionID: AzureUserAssignedIdentityClientID: CloudflareProxied:false CloudflareZonesPerPage:50 CoreDNSPrefix:/skydns/ RcodezeroTXTEncrypt:false AkamaiServiceConsumerDomain: AkamaiClientToken: AkamaiClientSecret: AkamaiAccessToken: InfobloxGridHost: InfobloxWapiPort:443 InfobloxWapiUsername:admin InfobloxWapiPassword: InfobloxWapiVersion:2.3.1 InfobloxSSLVerify:true InfobloxView: InfobloxMaxResults:0 DynCustomerName: DynUsername: DynPassword: DynMinTTLSeconds:0 OCIConfigFile:/etc/kubernetes/oci.yaml InMemoryZones:[] OVHEndpoint:ovh-eu OVHApiRateLimit:20 PDNSServer:http://localhost:8081 PDNSAPIKey: PDNSTLSEnabled:false TLSCA: TLSClientCert: TLSClientCertKey: Policy:sync Registry:txt TXTOwnerID:REDACTED TXTPrefix: TXTSuffix: Interval:1m0s Once:false DryRun:false UpdateEvents:false LogFormat:text MetricsAddress::7979 LogLevel:info TXTCacheInterval:0s ExoscaleEndpoint:https://api.exoscale.ch/dns ExoscaleAPIKey: ExoscaleAPISecret: CRDSourceAPIVersion:externaldns.k8s.io/v1alpha1 CRDSourceKind:DNSEndpoint ServiceTypeFilter:[] CFAPIEndpoint: CFUsername: CFPassword: RFC2136Host: RFC2136Port:0 RFC2136Zone: RFC2136Insecure:false RFC2136TSIGKeyName: RFC2136TSIGSecret: RFC2136TSIGSecretAlg: RFC2136TAXFR:false RFC2136MinTTL:0s NS1Endpoint: NS1IgnoreSSL:false TransIPAccountName: TransIPPrivateKeyFile: DigitalOceanAPIPageSize:50}"
external-dns-568764f9cb-48djw external-dns time="2020-08-10T19:57:19Z" level=info msg="Instantiating new Kubernetes client"
external-dns-568764f9cb-48djw external-dns time="2020-08-10T19:57:19Z" level=info msg="Using inCluster-config based on serviceaccount-token"
external-dns-568764f9cb-48djw external-dns time="2020-08-10T19:57:19Z" level=info msg="Created Kubernetes client https://100.64.0.1:443"
external-dns-568764f9cb-48djw external-dns time="2020-08-10T19:57:28Z" level=info msg="Desired change: DELETE kafdrop.REDACTED A [Id: /hostedzone/REDACTED]"
...REDACTED...
external-dns-568764f9cb-48djw external-dns time="2020-08-10T19:57:29Z" level=info msg="24 record(s) in zone REDACTED. [Id: /hostedzone/REDACTED] were successfully updated"
external-dns-568764f9cb-48djw external-dns time="2020-08-10T19:58:28Z" level=info msg="All records are already up to date"

Anything else we need to know?:

Environment:

  • External-DNS version (use external-dns --version): v0.7.3
  • DNS provider: AWS Route53

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 17 (9 by maintainers)

Most upvoted comments

@tariq1890

Using Istio 1.6.7 deployed with the Istio Operator on an AWS EKS Cluster.

Labels of istio-ingressgateway:

app=istio-ingressgateway
install.operator.istio.io/owning-resource=cluster-name-istiocontrolplane
install.operator.istio.io/owning-resource-namespace=istio-system
istio=ingressgateway
operator.istio.io/component=IngressGateways
operator.istio.io/managed=Reconcile
operator.istio.io/version=1.6.7
release=istio

Istio 1.4.10

Labels on keycloak gateway:

    labels:
      app: keycloak
      applications.argoproj.io/app-name: keycloak
      version: "1.0"