core: WireGuard endpoints no Internet access unless service manually restarted (NAT not being applied)
Important notices
Before you add a new report, we ask you kindly to acknowledge the following:
- [*] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
- [*] I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue
Describe the bug
After OPNsense boot, WireGuard endpoints don’t have Internet access. Traffic passes through OPNsense out to WAN interface and nothing appears to be blocked, but it seems NAT is not being applied so nothing comes back from the Internet. Last known working OPNsense version was 23.1.11_2. Updated to 23.7.5 and noticed the issue.
Forum topic about the issue. At least one other user reported what appears to be the same issue.
To Reproduce
Steps to reproduce the behavior:
- Restart OPNsense
- Connect WireGuard endpoint
- Try to access Internet or ping any domain/IP, no answer
- Restart WireGuard service
- Everything works
Expected behavior
WireGuard clients should have Internet access after OPNsense boot.
Relevant log files
OPNsense fresh boot, before manual WG restart:
wan 2023-10-01T15:43:55 10.101.80.1 1.1.1.1 icmp let out anything from firewall host itself
WGxInternet 2023-10-01T15:43:55 10.101.80.1 1.1.1.1 icmp Allow WGxInternet to Internet
After manual WG restart:
wan 2023-10-01T15:44:56 192.168.61.10 1.1.1.1 icmp let out anything from firewall host itself (force gw)
WGxInternet 2023-10-01T15:44:56 10.101.80.1 1.1.1.1 icmp Allow WGxInternet to Internet
10.101.80.1 -> WireGuard endpoint IP 192.168.61.10 -> WAN interface IP (upstream gateway 192.168.61.1)
Additional context
Communication between WireGuard endpoints and OPNsense works without issue. It is only when WG traffic has to go out through the WAN interface that the issue occurs.
Environment
OPNsense 23.7.5-amd64 FreeBSD 13.2-RELEASE-p3 OpenSSL 1.1.1w 11 Sep 2023
About this issue
- Original URL
- State: closed
- Created 9 months ago
- Comments: 16 (8 by maintainers)
Commits related to this issue
- VPN: WireGuard - hook wireguard empty devices during bootup, using wireguard_devices() plugin system. This should make sure services and components, such as the firewall, are able to use the device b... — committed to opnsense/core by AdSchellevis 9 months ago
- VPN: WireGuard - hook wireguard empty devices during bootup, using wireguard_devices() plugin system. This should make sure services and components, such as the firewall, are able to use the device b... — committed to secureonelabs/plugins by AdSchellevis 9 months ago
I’ve put together a janky but lightweight workaround here. The workaround essentially attempts to reload all rules after a minute has passed, and works rather well on my environment.
Ok, I think the CARP VHID feature is interfering. But as I said it’s risk free as it simply won’t modify files in that case.