coredns: k8s_external plugin doesn't work with AWS ELB/NLB

When running a Kubernetes cluster on AWS (with AWS cloud provider) k8s_external plugin returns an empty response with just AUTHORITY SECTION included.

What happened: I deployed a k8s_external plugin on an AWS-based k8s cluster, queried an external domain and got the following response:

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 581
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;my-svc.default.foo.com. IN	A

;; AUTHORITY SECTION:
foo.com.	5	IN	SOA	ns1.dns.foo.com. hostmaster.dns.foo.com. 12345 7200 1800 86400 5

This happens because the AWS ELB assigns a DNS alias instead of just an IP address.

In order to differentiate between AWS and other clouds, the loadbalancer’s status field looks different:

status:
  loadBalancer:
    ingress:
    - hostname: uuid.eu-west-2.elb.amazonaws.com

Compared to, for example, GKE:

status:
  loadBalancer:
    ingress:
    - ip: 35.123.9.321

What you expected to happen: I was expecting that k8s_external would return a CNAME that points to the uuid.eu-west-2.elb.amazonaws.com, e.g.

;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57815
;; flags: qr rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 1b5b8301e38cd740 (echoed)
;; QUESTION SECTION:
;my-svc.default.foo.com. IN A

;; ANSWER SECTION:
my-svc.default.foo.com. 30 IN CNAME uuid.eu-west-2.elb.amazonaws.com.

How to reproduce it (as minimally and precisely as possible):

  1. Create an EKS cluster
  2. Create a service type:Loadbalancer
  3. Add the following to Corefile’s default zone k8s_external foo.com
  4. Query the domain service.namespace.foo.com

Anything else we need to know?: I was able to relatively easy patch the kubernetes and k8s_external plugins to return a CNAME in this case. I was wondering if it’s something you may consider for a PR, since I’m not sure if this has been considered and dismissed before.

--- a/plugin/k8s_external/msg_to_dns.go
+++ b/plugin/k8s_external/msg_to_dns.go
@@ -18,7 +18,8 @@ func (e *External) a(services []msg.Service, state request.Request) (records []d
 
                switch what {
                case dns.TypeCNAME:
-                       // can't happen
+                       rr := s.NewCNAME(state.QName(), s.Host)
+                       records = append(records, rr)
 
                case dns.TypeA:
                        if _, ok := dup[s.Host]; !ok {
diff --git a/plugin/kubernetes/object/service.go b/plugin/kubernetes/object/service.go
index 295715e2..0fdbc2bf 100644
--- a/plugin/kubernetes/object/service.go
+++ b/plugin/kubernetes/object/service.go
@@ -62,6 +62,7 @@ func toService(skipCleanup bool, obj interface{}) interface{} {
        li := copy(s.ExternalIPs, svc.Spec.ExternalIPs)
        for i, lb := range svc.Status.LoadBalancer.Ingress {
                s.ExternalIPs[li+i] = lb.IP
+               s.ExternalIPs[li+i] = lb.Hostname
        }

Environment:

  • the version of CoreDNS: docker.io/coredns/coredns:1.6.9
  • Corefile:
.:53 {
    errors
    log
    health {
        lameduck 5s
    }
    ready
    kubernetes cluster.local in-addr.arpa ip6.arpa {
        pods insecure
        fallthrough in-addr.arpa ip6.arpa
        ttl 30
    }
    prometheus 0.0.0.0:9153
    forward . /etc/resolv.conf
    cache 30
    loop
    reload
    loadbalance
    k8s_external foo.com

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 15 (15 by maintainers)

Most upvoted comments

right, got it. so am I ok to open a PR for this?

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#loadbalanceringress-v1-core

The API docs don’t strictly say that only ONE of ip or hostname will be populated… but it kinda-sorta implies it. So perhaps we can safely assume that, and thus simply prefer one, ignoring the other (e.g. if both hostname and ip are there, use the ip, and ignore the hostname).