tailscale: macOS: When using an Exit Node and `--accept-dns=false` all system DNS resolution is broken because no resolver is set

What is the issue?

Per #12231 I am running Tailscale with --accept-dns=false as a workaround for a bug in Tailscale not updating my local DNS resolver.

However, I’ve discovered that with --accept-dns=false now Exit Nodes are effectively broken. It seems that even when MagicDNS is off Tailscale will setup 100.100.100.100 as a resolver when using an Exit Node, however turning off accept-dns prevents this. As a result no resolver is set on the system! This can be observed by seeing that /etc/resolv.conf is empty.

Steps to reproduce

  • Be on macOS
  • Run tailscale set --accept-dns=false to prevent Tailscale from setting any system DNS settings.
  • Switch to using an exit node
  • Observe that all system DNS resolution is broken. Try running ping www.google.com.
  • Run cat /etc/resolv.conf to observe that no nameserver is set.
  • Now run tailscale set --accept-dns=true to observer that now a nameserver is set and DNS resolution is working again.

The above behavior seems identical whether MagicDNS is enabled or not.

Possibly related to my other DNS+macOS bug report #12231, at least in how its related to Tailscale taking over all system DNS.

Are there any recent changes that introduced the issue?

I am a new Tailscale user

OS

macOS

OS version

14.3.1

Tailscale version

1.66.4 tailscale commit: 067defc641850791142cc83a44dadae29709344a

Other software

I’m running Little Snitch but all the testing done above was done with Little Snitch off. (however it was still installed on the system, so that might be contributing).

I also ensured that iCloud Private Relay was off (both system wide and the Wi-Fi network specific “Limit IP Address tracking” option).

Bug report

BUG-cafb7d6238f79c6992e84aac8608faa66cc8d12fb844958915c46f6c20dd3592-20240525193733Z-1d4fc7fafa5df705

About this issue

  • Original URL
  • State: closed
  • Created 2 years ago
  • Comments: 19 (8 by maintainers)

Most upvoted comments

Status update: we have built and are currently verifying/testing a new UI pane in the DNS settings view to specify custom DNS (or DoH!) servers to be used for tunnel-bound traffic when Tailscale DNS is disabled. The settings will also be remotely configurable via MDM. I expect this new feature will land in an unstable build some time early next week.

Screenshot 2024-05-31 at 10 36 58

@gyoza we’re working on it 👍🏻

Yeah, I was surprised to hear this would be related to a client upgrade on the exit node. The problem appears to be entirely in our macOS client. It’s not easy to solve. I could tell the client to look up the DNS server used by your external network interface, and use that whenever, but this would break any setup where a DNS configuration profile is being used.

Temporary workaround

To unblock you while I come up with something better, it appears possible to use scutil to explicitly set the DNS servers to be used by the VPN interface. Run the following after picking an exit node in the Tailscale menu:

$ sudo scutil
> list

The above will print a list of services, find the UUID of the Tailscale service (it should be contained in the only line that ends in VPN, if Tailscale is the only VPN installed), for instance: subKey [155] = State:/Network/Service/DB5586F7-6051-4364-BDC7-0E7F18C0A323/VPN

Then, do the following to set the DNS servers for the Tailscale service:

$ sudo scutil
> open
> d.init
> d.add ServerAddresses * 8.8.8.8 9.9.9.9
> set State:/Network/Service/TAILSCALE_SERVICE_UUID/DNS
> quit

scutil --dns should now list 8.8.8.8 and 9.9.9.9 as the resolvers used for the utun interface.

Currently, one way to solve this bug would be to add an option to let admins configure these fallback DNS servers with a single CLI command or MDM-deployable setting. For instance:

defaults write io.tailscale.ipn.macos FallbackDNSServers -array 8.8.8.8 9.9.9.9

But I’d rather spend some extra time on it and come up with something that works out of the box if possible.

Yeah, the current implementation doesn’t react to network changes. We certainly need to do so, and we’ll look into adding this to a future release.

Shipped in unstable v1.77.139, landing for everyone in the next v1.78.

Note that /etc/resolv.conf is not always the right place to identify what DNS resolver a macOS system is using. Prefer using scutil --dns

The issue is that scutil --dns is showing a resolver that apparently isn’t being used and only /etc/resolv.conf hints that something is amiss. But now I realize that I should be looking for a resolver on the Tailscale VPN interface.

Thanks for this awesome workaround solution using scutil. I love learning more about macOS networking internals. To make it easier, here’s a version of your solution with a little more automation to make it require no interaction. (Finding the UUID was trickier for me since I have many other VPN interfaces).

# Get the Tailscale service UUID
TS_UUID="$(scutil --nc list | awk '/io.tailscale.ipn.macos/ {print $3}')"

# pipe the scutil interactive commands in directly
sudo scutil <<EOF 
open
d.init
d.add ServerAddresses * 1.1.1.1
set State:/Network/Service/$TS_UUID/DNS
quit
EOF

edit: I’ve kept having to use this workaround often so I created a little shell function for it: https://gist.github.com/varenc/971796720a31c3b16e6c99903794bc1e