unbound: exceeded the maximum nameserver nxdomains

It however seems that exceeded the maximum nameserver nxdomains massively appears for otherwise normal domains after a temporary network connectivity loss (it seems unbound does not distinguish here between actual authorative NXDOMAIN and temporary network unavailability) and marks all domains which it tried to access while network was down as NXDOMAIN.

This leads to admin needing to restart unbound server manually after every WLAN/network connectivity issue, which makes it basically unusable as caching DNS server.

Debian Bullseye GNU/Linux, unbound 1.13.2-1 (using forward-tls-upstream: yes, forward-first: yes, forward-addr:)

e.g.

Aug 25 03:31:00  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:00  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:00  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:00  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:00  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:00  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:03  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:03  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:03  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:03  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:03  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:06  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:06  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:06  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:06  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:06  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:09  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:09  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:09  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:09  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:09  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:09  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:09  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:09  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:09  unbound: [1536:0] error: SSL_handshake syscall: No route to host
Aug 25 03:31:09  wpa_supplicant[2138]: wlan0: SME: Trying to authenticate with xx:xx:xx:xx:xx:xx (SSID='whatever' freq=2437 MHz)
Aug 25 03:31:10  wpa_supplicant[2138]: wlan0: Trying to associate with xx:xx:xx:xx:xx:xx (SSID='whatever' freq=2437 MHz)
Aug 25 03:31:10  wpa_supplicant[2138]: wlan0: Associated with xx:xx:xx:xx:xx:xx
Aug 25 03:31:10  wpa_supplicant[2138]: wlan0: CTRL-EVENT-SUBNET-STATUS-UPDATE status=0
Aug 25 03:31:10  wpa_supplicant[2138]: wlan0: WPA: Key negotiation completed with xx:xx:xx:xx:xx:xx [PTK=CCMP GTK=CCMP]
Aug 25 03:31:10  wpa_supplicant[2138]: wlan0: CTRL-EVENT-CONNECTED - Connection to xx:xx:xx:xx:xx:xx completed [id=0 id_str=]
Aug 25 03:31:13  unbound: [1536:0] error: SSL_handshake syscall: Connection refused
Aug 25 03:31:13  unbound: [1536:0] error: SSL_handshake syscall: Connection refused
Aug 25 03:31:13  unbound: [1536:0] error: SSL_handshake syscall: Connection refused
Aug 25 03:31:16  unbound: [1536:0] error: SERVFAIL <jabber.fsfe.org. A IN>: exceeded the maximum nameserver nxdomains
[... many other addresses removed for privacy reasons...]
Aug 25 03:33:31  unbound: [1536:0] error: SERVFAIL <shavar.services.mozilla.com. A IN>: exceeded the maximum nameserver nxdomains
Aug 25 03:33:39  unbound: [1536:0] error: SERVFAIL <push.services.mozilla.com. A IN>: exceeded the maximum nameserver nxdomains
Aug 25 03:33:39  unbound: [1536:0] error: SERVFAIL <push.services.mozilla.com. AAAA IN>: exceeded the maximum nameserver nxdomains
Aug 25 03:34:30  unbound: [1536:0] error: SERVFAIL <duckduckgo.com. A IN>: exceeded the maximum nameserver nxdomains
Aug 25 03:34:30  unbound: [1536:0] error: SERVFAIL <duckduckgo.com. AAAA IN>: exceeded the maximum nameserver nxdomains

What it would also make sense to me in this case is for a SERVFAIL from Unbound to be treated as a temporary failure for a nameserver on a list and the email process to continue after some retries with some kind of agreement from the rest of the services. Does one service being unavailable (in this case barracuda) kills the whole thing?

Kills? No. Just delayed the delivery of message. Probably for something from 15 minutes to an hour.

Right now many persons treats email like something that should be delivered instantly, they sends code to confirm your identity or similar things with TTL like 15 minutes.

And such unbound behaviour makes email setup useless for that case.

Frankly speaking it requires quite a time to discover that behaviour of unbound which is seems unusual in comparing with another DNS client.

Do you need this in order for Unbound to not reply with SERVFAIL and eventually get an answer?

Yes, that’s what other DNS clients do without any special tweaking

Are there any other solutions?

Yes, other resolvers.
Unfortunately unbound is unusable on servers and border gateways due to hard-coding on number of sends. Because of this, we removed it from all servers.

I have observed the same issue.

Today i’ve updated my openwrt and afterwards i had to restart unbound inside my container…

Sep 08 07:20:26 unbound[837545:0] error: SERVFAIL <www.iana.org. AAAA IN>: exceeded the maximum number of sends
Sep 08 07:20:26 unbound[837545:0] error: SERVFAIL <www.iana.org. AAAA IN>: exceeded the maximum number of sends
Sep 08 07:20:28 unbound[837545:1] error: SERVFAIL <cocoapi.bmwgroup.com. AAAA IN>: exceeded the maximum number of sends
Sep 08 07:20:41 unbound[837545:1] error: SERVFAIL <nv2-namain.netatmo.net. A IN>: exceeded the maximum number of sends
Sep 08 07:20:41 unbound[837545:1] error: SERVFAIL <nv2-namain.netatmo.net. A IN>: exceeded the maximum number of sends
Sep 08 07:20:45 unbound[837545:0] error: SERVFAIL <diag.meethue.com. AAAA IN>: exceeded the maximum nameserver nxdomains
Sep 08 07:21:02 unbound[837545:0] info: validation failure <account.xiaomi.com. AAAA IN>: key for validation xiaomi.com. is marked as invalid
Sep 08 07:21:02 unbound[837545:0] info: validation failure <account.xiaomi.com. AAAA IN>: key for validation xiaomi.com. is marked as invalid
Sep 08 07:21:11 unbound[837545:1] info: validation failure <account.xiaomi.com. AAAA IN>: key for validation xiaomi.com. is marked as invalid
Sep 08 07:21:11 unbound[837545:1] info: validation failure <account.xiaomi.com. AAAA IN>: key for validation xiaomi.com. is marked as invalid
Sep 08 07:21:11 unbound[837545:1] info: validation failure <account.xiaomi.com. AAAA IN>: key for validation xiaomi.com. is marked as invalid
Sep 08 07:21:11 unbound[837545:1] info: validation failure <account.xiaomi.com. AAAA IN>: key for validation xiaomi.com. is marked as invalid
Sep 08 07:21:11 unbound[837545:1] info: validation failure <account.xiaomi.com. AAAA IN>: key for validation xiaomi.com. is marked as invalid
Sep 08 07:21:11 unbound[837545:1] info: validation failure <account.xiaomi.com. AAAA IN>: key for validation xiaomi.com. is marked as invalid
Sep 08 07:21:12 unbound[837545:1] info: validation failure <account.xiaomi.com. AAAA IN>: key for validation xiaomi.com. is marked as invalid
Sep 08 07:21:12 unbound[837545:1] info: validation failure <account.xiaomi.com. AAAA IN>: key for validation xiaomi.com. is marked as invalid
Sep 08 07:21:20 unbound[837545:1] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. no server to query nameserver addresses not usable
Sep 08 07:21:20 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. AAAA IN>: all servers for this domain failed, at zone wall-box.com. no server to query nameserver addresses not usable
Sep 08 07:21:20 unbound[837545:1] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: exceeded the maximum number of sends
Sep 08 07:21:20 unbound[837545:1] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: exceeded the maximum number of sends
Sep 08 07:21:20 unbound[837545:1] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:26 unbound[837545:1] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:27 unbound[837545:1] error: SERVFAIL <rt.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:37 unbound[837545:1] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:37 unbound[837545:1] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:40 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:40 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:40 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:40 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:40 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:40 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:40 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:40 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. A IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:56 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. AAAA IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:56 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. AAAA IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:56 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. AAAA IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:56 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. AAAA IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:56 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. AAAA IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:56 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. AAAA IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:56 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. AAAA IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:56 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. AAAA IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:56 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. AAAA IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:21:56 unbound[837545:0] error: SERVFAIL <iot.telemetry.wall-box.com. AAAA IN>: all servers for this domain failed, at zone wall-box.com. upstream server timeout
Sep 08 07:22:06 unbound[837545:1] error: SERVFAIL <api.wall-box.com. AAAA IN>: all servers for this domain failed, at zone wall-box.com. no server to query nameserver addresses not usable

after restart, everything was fine again…

When looking up the addresses for nameservers, unbound encounters too many NXDOMAIN responses for those lookup and stops to avoid causing a denial of service. The domain has a long list of NS records, and those domains perhaps also have lists of NS records. All of those, or a lot of them, have no addresses and thus do not work. While trying to resolve the domain unbound is recursively looking up those nameservers and the nameservers to lookup those nameservers, and this is taking too much resources.

The domain should not have such a long list of nameservers that do not have addresses. Or nameservers for the nameservers that have no addresses. There was a CVE for that a while ago about this resource consumption causing issues, too many queries, too much resource usage on the DNS server. This error stops the resource usage.