coredns: k8s_external plugin doesn't work with AWS ELB/NLB
When running a Kubernetes cluster on AWS (with AWS cloud provider) k8s_external plugin returns an empty response with just AUTHORITY SECTION included.
What happened:
I deployed a k8s_external
plugin on an AWS-based k8s cluster, queried an external domain and got the following response:
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 581
;; flags: qr rd; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;my-svc.default.foo.com. IN A
;; AUTHORITY SECTION:
foo.com. 5 IN SOA ns1.dns.foo.com. hostmaster.dns.foo.com. 12345 7200 1800 86400 5
This happens because the AWS ELB assigns a DNS alias instead of just an IP address.
In order to differentiate between AWS and other clouds, the loadbalancer’s status field looks different:
status:
loadBalancer:
ingress:
- hostname: uuid.eu-west-2.elb.amazonaws.com
Compared to, for example, GKE:
status:
loadBalancer:
ingress:
- ip: 35.123.9.321
What you expected to happen:
I was expecting that k8s_external
would return a CNAME that points to the uuid.eu-west-2.elb.amazonaws.com
, e.g.
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57815
;; flags: qr rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
; COOKIE: 1b5b8301e38cd740 (echoed)
;; QUESTION SECTION:
;my-svc.default.foo.com. IN A
;; ANSWER SECTION:
my-svc.default.foo.com. 30 IN CNAME uuid.eu-west-2.elb.amazonaws.com.
How to reproduce it (as minimally and precisely as possible):
- Create an EKS cluster
- Create a service type:Loadbalancer
- Add the following to Corefile’s default zone
k8s_external foo.com
- Query the domain
service.namespace.foo.com
Anything else we need to know?:
I was able to relatively easy patch the kubernetes
and k8s_external
plugins to return a CNAME in this case. I was wondering if it’s something you may consider for a PR, since I’m not sure if this has been considered and dismissed before.
--- a/plugin/k8s_external/msg_to_dns.go
+++ b/plugin/k8s_external/msg_to_dns.go
@@ -18,7 +18,8 @@ func (e *External) a(services []msg.Service, state request.Request) (records []d
switch what {
case dns.TypeCNAME:
- // can't happen
+ rr := s.NewCNAME(state.QName(), s.Host)
+ records = append(records, rr)
case dns.TypeA:
if _, ok := dup[s.Host]; !ok {
diff --git a/plugin/kubernetes/object/service.go b/plugin/kubernetes/object/service.go
index 295715e2..0fdbc2bf 100644
--- a/plugin/kubernetes/object/service.go
+++ b/plugin/kubernetes/object/service.go
@@ -62,6 +62,7 @@ func toService(skipCleanup bool, obj interface{}) interface{} {
li := copy(s.ExternalIPs, svc.Spec.ExternalIPs)
for i, lb := range svc.Status.LoadBalancer.Ingress {
s.ExternalIPs[li+i] = lb.IP
+ s.ExternalIPs[li+i] = lb.Hostname
}
Environment:
- the version of CoreDNS: docker.io/coredns/coredns:1.6.9
- Corefile:
.:53 {
errors
log
health {
lameduck 5s
}
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
pods insecure
fallthrough in-addr.arpa ip6.arpa
ttl 30
}
prometheus 0.0.0.0:9153
forward . /etc/resolv.conf
cache 30
loop
reload
loadbalance
k8s_external foo.com
About this issue
- Original URL
- State: closed
- Created 4 years ago
- Comments: 15 (15 by maintainers)
right, got it. so am I ok to open a PR for this?
https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.13/#loadbalanceringress-v1-core
The API docs don’t strictly say that only ONE of ip or hostname will be populated… but it kinda-sorta implies it. So perhaps we can safely assume that, and thus simply prefer one, ignoring the other (e.g. if both hostname and ip are there, use the ip, and ignore the hostname).