k3s: Ubuntu 21.04 - vxlan failing to route

Environmental Info: K3s Version: 1.21.5+k3s2

NAME       STATUS   ROLES                       AGE   VERSION        INTERNAL-IP      EXTERNAL-IP   OS-IMAGE       KERNEL-VERSION      CONTAINER-RUNTIME
k3s-85fc   Ready    control-plane,etcd,master   18m   v1.21.5+k3s2   134.122.5.183    <none>        Ubuntu 21.04   5.11.0-18-generic   containerd://1.4.11-k3s1
k3s-a648   Ready    <none>                      18m   v1.21.5+k3s2   104.131.58.161   <none>        Ubuntu 21.04   5.11.0-18-generic   containerd://1.4.11-k3s1
k3s-ad66   Ready    <none>                      18m   v1.21.5+k3s2   143.198.24.211   <none>        Ubuntu 21.04   5.11.0-18-generic   containerd://1.4.11-k3s1

Node(s) CPU architecture, OS, and Version: Linux k3s-85fc 5.11.0-18-generic #19-Ubuntu SMP Fri May 7 14:22:03 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration: As per above. 3 nodes, 1 master and 2 workers. This is a fresh install of k3s on Ubuntu 21.04.

Describe the bug: With a fresh install of k3s on Ubuntu 21.04 vxlan is not working. AKA pods are not able to talk to each other across flannel. I have stopped and disabled ufw.

Steps To Reproduce:

  • build 3 - 21.04 - vms on Digitalocean.
  • use k3sup k3sup install --ip $server --user $user --k3s-extra-args '--no-deploy traefik --debug' --cluster --k3s-channel $k3s_channel --local-path ~/.kube/config
  • Installed K3s:

Expected behavior: pods talk / ping across the nodes.

Actual behavior: no ping

Additional context / logs: journalct does not show any logs. net.ipv4.ip_forward = 1 is enabled.

pings are not working. root@k3s-8fea:~# ip a list cni0 | grep -w inet inet 10.42.0.1/24 brd 10.42.0.255 scope global cni0

root@k3s-aaaa:~# ping 10.42.0.1
PING 10.42.0.1 (10.42.0.1) 56(84) bytes of data.
^C
--- 10.42.0.1 ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 1019ms

Updated both servers

Backporting no . This seems to be tide to Ubuntu 21.04. This works on 20.10.

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Comments: 26 (14 by maintainers)

Commits related to this issue

Most upvoted comments

For anyone who stumbles on this issue while using the latest version of ubuntu 21.10, vxlan modules were moved by upstream to a separate package: linux-modules-extra-raspi. Installing these should solve your issues.

v1.22.3+k3s1 works!

Because #3863 linked to this issue:

On Debian Bullseye iptables is set to nftables so I had to change it:

$ update-alternatives --set iptables /usr/sbin/iptables-legacy