tailscale: Tailscale client fails to connect when using mwan3 with static IP on primary WAN interface
What is the issue?
We are experiencing an issue with Tailscale connectivity when using OpenWRT as our gateway OS with mwan3 for multi-WAN setup. The problem occurs specifically when using a static IP on the primary WAN interface (wanA) while the secondary interface (wanB) uses a dynamic IP.
- Multi-WAN: Enabled using mwan3
- WAN Configuration:
- wanA: Static IP
- wanB: Dynamic IP
When the above configuration is used, the Tailscale client fails to connect, resulting in the following error:
# Health check:
# - not logged in, last login error=fetch control key: Get "https://controlplane.tailscale.com/key?v=85": dial tcp [2a05:d014:386:202:30f5:de62:8eed:5c47]:443: connect: network is unreachable
unexpected state: NoState
The issue only occurs with a static IP on wanA. Using dynamic IPs on both interfaces works correctly. OpenWRT networking functions as expected:
root@gw-di-01:/# mwan3 status
Interface status:
interface wan is offline 00h:00m:00s, uptime 00h:04m:23s and tracking is active
interface wanb is online 00h:04m:03s, uptime 00h:04m:19s and tracking is active
root@gw-di-01:/# ping [google.ch](http://google.ch/)
PING [google.ch](http://google.ch/) (142.250.203.99): 56 data bytes
64 bytes from [142.250.203.99](http://142.250.203.99/): seq=0 ttl=111 time=3.034 ms
64 bytes from [142.250.203.99](http://142.250.203.99/): seq=1 ttl=111 time=2.983 ms
64 bytes from [142.250.203.99](http://142.250.203.99/): seq=2 ttl=111 time=3.019 ms
^C
--- [google.ch](http://google.ch/) ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 2.983/3.012/3.034 ms
root@gw-di-01:/# curl https://controlplane.tailscale.com/key?v=85
{
"legacyPublicKey": "mkey:[redacted]",
"publicKey": "mkey:[redacted]"
}
Removing the static IP route immediately resolves the Tailscale connectivity issue:
ip route del default via [redacted]
It appears that Tailscale is not respecting the host’s networking configuration when a static IP is used on the primary WAN interface in conjunction with mwan3.
Related OpenWRT forum post: https://forum.openwrt.org/t/tailscale-and-mwan3-conflict-on-openwrt-connection-issues-and-routing-problems/205743
Any help is greatly appreciated.
Steps to reproduce
Using a OpenWRT gateway with 3 interfaces:
- WAN A (static IP)
- WAN B (DHCP)
- LAN
…and a default failover-configuration of mwan3 is enough to reproduce the issue.
It’s also not relevant what Tailscale version we’re using: The issue occurs up to 1.70.
OS
Linux
OS version
OpenWrt 23.05.2
Tailscale version
1.58.2-1.70
Bug report
root@gw-di-01:/# tailscale netcheck --verbose
2024/09/04 12:53:41 No DERP map from tailscaled; using default.
2024/09/04 12:53:41 netcheck: sent to 43.245.48.220:3478
2024/09/04 12:53:41 netcheck: sent to 148.163.220.129:3478
2024/09/04 12:53:41 netcheck: sent to 102.67.167.245:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.242.187:3478
2024/09/04 12:53:41 netcheck: sent to 43.245.49.105:3478
2024/09/04 12:53:41 netcheck: sent to 103.6.84.152:3478
2024/09/04 12:53:41 netcheck: sent to 199.38.181.104:3478
2024/09/04 12:53:41 netcheck: sent to 68.183.90.120:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.252.65:3478
2024/09/04 12:53:41 netcheck: sent to 208.83.234.151:3478
2024/09/04 12:53:41 netcheck: sent to 185.34.3.232:3478
2024/09/04 12:53:41 netcheck: sent to 209.177.158.246:3478
2024/09/04 12:53:41 netcheck: sent to 209.177.156.94:3478
2024/09/04 12:53:41 netcheck: sent to 103.84.155.178:3478
2024/09/04 12:53:41 netcheck: sent to 176.58.92.144:3478
2024/09/04 12:53:41 netcheck: sent to 102.67.165.90:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.243.135:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.244.245:3478
2024/09/04 12:53:41 netcheck: sent to 176.58.93.248:3478
2024/09/04 12:53:41 netcheck: sent to 45.159.98.196:3478
2024/09/04 12:53:41 netcheck: sent to 162.248.221.199:3478
2024/09/04 12:53:41 netcheck: sent to 176.58.90.147:3478
2024/09/04 12:53:41 netcheck: sent to 185.40.234.219:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.240.161:3478
2024/09/04 12:53:41 netcheck: sent to 45.159.97.144:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.240.121:3478
2024/09/04 12:53:41 netcheck: sent to 43.245.49.83:3478
2024/09/04 12:53:41 netcheck: sent to 208.111.40.12:3478
2024/09/04 12:53:41 netcheck: sent to 45.159.97.61:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.242.28:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.252.134:3478
2024/09/04 12:53:41 netcheck: sent to 148.163.220.134:3478
2024/09/04 12:53:41 netcheck: sent to 209.177.158.15:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.243.229:3478
2024/09/04 12:53:41 netcheck: sent to 176.58.90.207:3478
2024/09/04 12:53:41 netcheck: sent to 209.177.145.120:3478
2024/09/04 12:53:41 netcheck: sent to 205.147.105.30:3478
2024/09/04 12:53:41 netcheck: sent to 102.67.165.185:3478
2024/09/04 12:53:41 netcheck: sent to 208.83.233.233:3478
2024/09/04 12:53:41 netcheck: sent to 103.84.155.188:3478
2024/09/04 12:53:41 netcheck: sent to 176.58.93.147:3478
2024/09/04 12:53:41 netcheck: sent to 185.40.234.113:3478
2024/09/04 12:53:41 netcheck: sent to 43.245.48.50:3478
2024/09/04 12:53:41 netcheck: sent to 45.159.98.253:3478
2024/09/04 12:53:41 netcheck: sent to 185.34.3.207:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.248.83:3478
2024/09/04 12:53:41 netcheck: sent to 176.58.88.183:3478
2024/09/04 12:53:41 netcheck: sent to 162.248.221.215:3478
2024/09/04 12:53:41 netcheck: sent to 68.183.90.120:3478
2024/09/04 12:53:41 netcheck: sent to 102.67.167.37:3478
2024/09/04 12:53:41 netcheck: sent to 185.34.3.75:3478
2024/09/04 12:53:41 netcheck: sent to 103.84.155.46:3478
2024/09/04 12:53:41 netcheck: sent to 45.159.97.233:3478
2024/09/04 12:53:41 netcheck: sent to 102.67.165.36:3478
2024/09/04 12:53:41 netcheck: sent to 162.248.221.248:3478
2024/09/04 12:53:41 netcheck: sent to 43.245.48.250:3478
2024/09/04 12:53:41 netcheck: sent to 176.58.92.254:3478
2024/09/04 12:53:41 netcheck: sent to 176.58.93.154:3478
2024/09/04 12:53:41 netcheck: sent to 209.177.156.197:3478
2024/09/04 12:53:41 netcheck: sent to 43.245.49.144:3478
2024/09/04 12:53:41 netcheck: sent to 208.111.40.216:3478
2024/09/04 12:53:41 netcheck: sent to 45.159.98.145:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.242.204:3478
2024/09/04 12:53:41 netcheck: sent to 208.111.34.178:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.243.141:3478
2024/09/04 12:53:41 netcheck: sent to 205.147.105.78:3478
2024/09/04 12:53:41 netcheck: sent to 192.73.240.132:3478
2024/09/04 12:53:41 netcheck: sent to 208.72.155.133:3478
2024/09/04 12:53:41 netcheck: sent to 199.38.182.118:3478
2024/09/04 12:53:41 netcheck: sent to 148.163.220.210:3478
2024/09/04 12:53:41 netcheck: sent to 199.38.181.93:3478
2024/09/04 12:53:41 netcheck: sent to 176.58.90.104:3478
2024/09/04 12:53:41 netcheck: sent to 185.40.234.77:3478
2024/09/04 12:53:41 netcheck: sent to 68.183.90.120:3478
2024/09/04 12:53:41 netcheck: sent to 102.67.167.188:3478
2024/09/04 12:53:41 netcheck: [v2] checkCaptivePortal url="http://derp23b.tailscale.com/generate_204" status_code=204 valid_response=true
2024/09/04 12:53:44 netcheck: hairCheck done
2024/09/04 12:53:44 netcheck: portMap done
2024/09/04 12:53:44 netcheck: netcheck: UDP is blocked, trying HTTPS
2024/09/04 12:53:44 netcheck: UDP is blocked, trying ICMP
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of sin (3): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of tor (21): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of hnl (24): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of dfw (9): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of lhr (8): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of ams (14): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of waw (22): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of nyc (1): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of mia (16): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of sfo (2): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of hkg (20): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of sea (10): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of ord (12): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of par (18): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of syd (5): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of sao (11): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of dbi (23): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of fra (4): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of den (13): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of tok (7): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of nai (25): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of mad (19): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of lax (17): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of jnb (15): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] measuring ICMP latency of blr (6): context deadline exceeded
2024/09/04 12:53:45 netcheck: [v1] netcheck: measuring HTTPS latency of blr (6): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of lax (17): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of fra (4): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of syd (5): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of dbi (23): dial tcp6 [2a00:dd80:3f:100::76f]:443: connect: network is unreachable
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of lhr (8): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of jnb (15): dial tcp6 [2c0f:edb0:0:10::963]:443: connect: network is unreachable
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of sao (11): dial tcp6 [2607:f740:1::211]:443: connect: network is unreachable
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of mia (16): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of hnl (24): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of waw (22): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of mad (19): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of ams (14): dial tcp6 [2a00:dd80:3c::807]:443: connect: network is unreachable
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of nyc (1): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of ord (12): dial tcp6 [2607:f740:e::811]:443: connect: network is unreachable
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of den (13): dial tcp6 [2607:f740:16::640]:443: connect: network is unreachable
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of par (18): dial tcp6 [2a00:dd80:3e::363]:443: connect: network is unreachable
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of hkg (20): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of sea (10): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of sfo (2): dial tcp6 [2607:f740:0:3f::287]:443: connect: network is unreachable
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of sin (3): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of tor (21): dial tcp6 [2607:f740:50::1d1]:443: connect: network is unreachable
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of tok (7): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of dfw (9): dial tcp6 [2607:f740:100::c05]:443: connect: network is unreachable
2024/09/04 12:53:46 netcheck: [v1] netcheck: measuring HTTPS latency of nai (25): context deadline exceeded
2024/09/04 12:53:46 netcheck: [v1] report: udp=false v4=false icmpv4=false v6=false v6os=true mapvarydest= hair= portmap= captiveportal=false derp=0
2024/09/04 12:53:46 netcheck: GetReport took 5.001s; err=<nil>
Report:
* UDP: false
* IPv4: (no addr found)
* IPv6: no, but OS has support
* MappingVariesByDestIP:
* HairPinning:
* PortMapping:
* CaptivePortal: false
* Nearest DERP: unknown (no response to latency probes)
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 1
- Comments: 19 (2 by maintainers)
Thanks. I’ve finally managed to have some availability and replaced mwan3 with your script there.
Just in case, as you don’t have issues enabled there I’ve seen
eth1is hardcoded https://github.com/DigitecGalaxus/mesh-network/blob/master/wan_failover/files/wan-failover.sh#L133 in case you want to fix it.@e-minguez I’m working in the same team as @qdrop17 , you can find the script here: https://github.com/DigitecGalaxus/mesh-network/blob/master/wan_failover/files/wan-failover.sh
We are currently using a workaround through the mwan3.user file to manage this issue. This solution involves dynamically modifying the main routing table and adding a new route with a lower metric to force traffic through the functioning interface:
However, this approach compromises several core mwan3 features, particularly the ability to load-balance traffic between multiple WAN connections.
We’ll test @junx964’s workaround, though we doubt it will solve our issue. While adding a longer sleep during Tailscale initialization helped with the first start, connections still drop during failover on static IP setups.
After further investigation, we believe we’ve identified the root cause of the issue. Tailscale relies on the “main” routing table, which becomes dysfunctional during a failover event with a static IP. Unlike DHCP, where the route is discarded along with the invalid lease, the route persists with a static IP.
We’ve found that manually deleting the faulty default route in the “main” routing table (via /etc/mwan3.user) allows Tailscale to immediately reconnect using wanB. However, this approach breaks mwan3 functionality, as mwan3track stops working correctly.
While we can restore wanA by manually restarting mwan3 (/etc/init.d/mwan restart), we haven’t yet found a satisfactory automatic workaround that doesn’t involve completely managing the routing table through custom shell scripts.
We’re seeking advice on how to achieve compatibility between Tailscale and mwan3. We suspect this might be related to the fwmarks used by mwan3. Any insights on how to resolve this conflict or suggestions for a more pragmatic solution that doesn’t deviate significantly from the default configuration would be greatly appreciated.