tailscale: Fedora default `firewalld` rules block Tailscale forwarding (exit node, subnet routing)
What is the issue?
Hello Tailscale team,
I’m splitting out a new issue from #2654. I’m still experiencing the issue of a Tailscale exit node on Fedora not forwarding any traffic, not even pings (so not a DNS issue). Extending @bexelbie’s investigation in that issue got me to discover that Fedora’s default firewalld configuration places the tailscale0 interface in the default zone, which prohibits IP forwarding.
Steps to reproduce
- Turn forwarding on:
net.ipv4.ip_forward = 1,net.ipv6.conf.all.forwarding = 1in/etc/sysctl.conf tailscale up --advertise-exit-nodeip a: thetailscale0interface is 100.103.143.10 and the LAN interface is 192.168.12.183firewall-cmd --zone=FedoraWorkstation --list-interfaces:tailscale0is in theFedoraWorkstationzonenft list ruleset: see below, observe that thehook forwardchain intable inet firewalldhas a higher priority than Tailscale’s chain and rejects the packet- Enable the exit node in the Tailscale web UI
- On client, select the exit node
ping 100.103.143.10, the exit node’stailscale0interface. Observe ping replyping 192.168.0.25, the exit node’s LAN interface. Observe ping replyping 1.1.1.1, on the Internet via the exit node. Observe reply from100.103.143.10: Destination net unreachable- On the exit node,
firewall-cmd --zone=trusted --change-interface=tailscale0: movestailscale0to thetrustedzone - The
firewalldreload eats Tailscale’snftablestables completely, so dotailscale downandtailscale up --advertise-exit-nodeagain nft list ruleset: see below, observe that in thefirewalldchains, the forwarded packets to output interfacetailscale0are nowaccept, allowing them to fall through to Tailscale’s chains- Wait ten seconds …
- From client,
ping 1.1.1.1. Observe ping reply
Here are the relevant chains from the nftables ruleset, before moving tailscale0 to the trusted zone:
table ip filter {
chain FORWARD {
type filter hook forward priority filter; policy accept;
counter packets 12369 bytes 7336248 jump ts-forward
}
chain ts-forward {
iifname "tailscale0" counter packets 7114 bytes 6307854 meta mark set 0x40000
mark 0x40000 counter packets 7114 bytes 6307854 accept
oifname "tailscale0" ip saddr 100.64.0.0/10 counter packets 0 bytes 0 drop
oifname "tailscale0" counter packets 5255 bytes 1028394 accept
}
}
table inet firewalld {
chain filter_FORWARD {
type filter hook forward priority filter + 10; policy accept;
ct state { established, related } accept
ct status dnat accept
iifname "lo" accept
ip6 daddr { ::/96, ::ffff:0.0.0.0/96, 2002::/24, 2002:a00::/24, 2002:7f00::/24, 2002:a9fe::/32, 2002:ac10::/28, 2002:c0a8::/32, 2002:e000::/19 } reject with icmpv6 addr-unreachable
jump filter_FORWARD_ZONES
ct state invalid drop
reject with icmpx admin-prohibited
}
chain filter_FORWARD_POLICIES_pre {
}
chain filter_FORWARD_ZONES {
iifname "tailscale0" goto filter_FWD_FedoraWorkstation
iifname "wlp1s0" goto filter_FWD_FedoraWorkstation
goto filter_FWD_FedoraWorkstation
}
chain filter_FORWARD_POLICIES_post {
}
chain filter_FWD_FedoraWorkstation {
jump filter_FORWARD_POLICIES_pre
jump filter_FWD_FedoraWorkstation_pre
jump filter_FWD_FedoraWorkstation_log
jump filter_FWD_FedoraWorkstation_deny
jump filter_FWD_FedoraWorkstation_allow
jump filter_FWD_FedoraWorkstation_post
jump filter_FORWARD_POLICIES_post
reject with icmpx admin-prohibited
}
chain filter_FWD_FedoraWorkstation_pre {
}
chain filter_FWD_FedoraWorkstation_log {
}
chain filter_FWD_FedoraWorkstation_deny {
}
chain filter_FWD_FedoraWorkstation_allow {
}
chain filter_FWD_FedoraWorkstation_post {
}
}
And here’s the same after moving tailscale0 to the trusted zone (note the oifname "tailscale0" accept line):
table ip filter {
chain FORWARD {
type filter hook forward priority filter; policy accept;
counter packets 12749 bytes 7385739 jump ts-forward
}
chain ts-forward {
iifname "tailscale0" counter packets 7363 bytes 6335629 meta mark set 0x40000
mark 0x40000 counter packets 7363 bytes 6335629 accept
oifname "tailscale0" ip saddr 100.64.0.0/10 counter packets 0 bytes 0 drop
oifname "tailscale0" counter packets 5386 bytes 1050110 accept
}
}
table inet firewalld {
chain filter_FORWARD {
type filter hook forward priority filter + 10; policy accept;
ct state { established, related } accept
ct status dnat accept
iifname "lo" accept
ip6 daddr { ::/96, ::ffff:0.0.0.0/96, 2002::/24, 2002:a00::/24, 2002:7f00::/24, 2002:a9fe::/32, 2002:ac10::/28, 2002:c0a8::/32, 2002:e000::/19 } reject with icmpv6 addr-unreachable
jump filter_FORWARD_ZONES
ct state invalid drop
reject with icmpx admin-prohibited
}
chain filter_FORWARD_POLICIES_pre {
}
chain filter_FORWARD_ZONES {
iifname "tailscale0" goto filter_FWD_trusted
iifname "wlp1s0" goto filter_FWD_FedoraWorkstation
goto filter_FWD_FedoraWorkstation
}
chain filter_FORWARD_POLICIES_post {
}
chain filter_FWD_FedoraWorkstation {
jump filter_FORWARD_POLICIES_pre
jump filter_FWD_FedoraWorkstation_pre
jump filter_FWD_FedoraWorkstation_log
jump filter_FWD_FedoraWorkstation_deny
jump filter_FWD_FedoraWorkstation_allow
jump filter_FWD_FedoraWorkstation_post
jump filter_FORWARD_POLICIES_post
reject with icmpx admin-prohibited
}
chain filter_FWD_FedoraWorkstation_pre {
}
chain filter_FWD_FedoraWorkstation_log {
}
chain filter_FWD_FedoraWorkstation_deny {
}
chain filter_FWD_FedoraWorkstation_allow {
}
chain filter_FWD_FedoraWorkstation_post {
}
chain filter_FWD_trusted {
jump filter_FORWARD_POLICIES_pre
jump filter_FWD_trusted_pre
jump filter_FWD_trusted_log
jump filter_FWD_trusted_deny
jump filter_FWD_trusted_allow
jump filter_FWD_trusted_post
jump filter_FORWARD_POLICIES_post
accept
}
chain filter_FWD_trusted_pre {
}
chain filter_FWD_trusted_log {
}
chain filter_FWD_trusted_deny {
}
chain filter_FWD_trusted_allow {
oifname "tailscale0" accept
}
chain filter_FWD_trusted_post {
}
}
Proposed solution
Add the Tailscale interface to the trusted firewalld zone, or advise Fedora users to do so manually.
Are there any recent changes that introduced the issue?
No response
OS
Linux
OS version
Fedora Workstation 35
Tailscale version
1.24.2
Bug report
BUG-678e86f9848a0db601cdddf4a45c5646b7904f0ed4aae82f95707f4f172686df-20220508054813Z-4f5a6aa9ff539e00
About this issue
- Original URL
- State: open
- Created 5 years ago
- Reactions: 1
- Comments: 15
I do the following on CentOS/Fedora/EL
to make it work with firewalld
This is what worked for me - at some point the commands for setting up firewalld for routing changed.
I used https://forums.rockylinux.org/t/configure-firewalld-for-routing-traffic-between-internal-and-external/7384/2, and I’m running the machine as a router using these instructions anyway. so there might be some addtional steps involved
First, you’ll want to enable packet forwarding. Edit /etc/sysctl.conf and uncomment
net.ipv4.ip_forward=1andnet.ipv6.conf.all.forwarding = 1. run sysctl -p as root or sudo to activate these changesI’m setting up firewalld to run tailscale as its own zone (so in theory you can set its own firewall rules, and in practice it dosen’t clobber my main network)
I’m assuming wan as the wan interface, and that its already set to the external zone, and able to connect out . If not you can set it as such with
firewall-cmd --zone=external --add-interface=wan --permanentI’ve also set it to be the default zone.Assuming those things- the following firewall-cmd commands should work.
I could connect to the internet via my exit node after running this:
im still having problems when following this, it does seem to be a nf_tables issues however after testing an older rocky instance
I actually experienced this issue working fine “by default” on Fedora Workstation, but not on OpenSUSE Tumbleweed
I put “by default” in quotes because I started this Fedora install with Fedora 34 and having been rolling forward since, so defaults could have changed but to my knowledge I have never touched the firewall settings. The OpenSUSE Tumbleweed install was very recent; I do not recall which install options I selected but I tend to stick to defaults for stuff I am not familiar with, like firewalls.
Note that I was not attempting exit node or subnet routing work, I was attempting to access a service across the tailnet
With Tailscale running on both machines: I was able to reach an echo server I was running on 0.0.0.0:8080 on Fedora, from the OpenSUSE machine I was not able to reach the same service running on the OpenSUSE machine, from any other machine
In all cases the service appeared in the Tailscale Services list
Fedora output which was already working:
OpenSUSE output which was not working:
The only things differing were that Fedora had the ports open, and
forward: no, while OpenSUSE had no ports open andforward: yesOpening the UDP port on the OpenSUSE public zone did not fix it, I assume because I was running an HTTP server? Opening the TCP port did fix it
Having forward yes vs. no did not matter on OpenSUSE, though if I understand how that works,
forward: nomay have still blocked things if I wanted the requests to be let through to the docker firewall zoneHere’s the beginning of the nearly 10-year-old thread on Fedora mailing list with a bunch of extensive arguing about this policy: https://lists.fedoraproject.org/pipermail/devel/2014-December/205010.html
Adding some notes here:
(FYI I use the internal zone not ‘trusted’ but same idea)
and then:
is indeed the fix, and the extra aggravating part comes when you need to reboot – these commands have to be run every time.