metallb: metallb sending wrong arp when using multiple ethernet interfaces

When having multiple (2) ethernet interfaces, metallb Version 0.9.3 sometimes uses the wrong interface/MAC-address for the arp-tables.

Example:

Client outside the kubernetes cluster:

Every 2.0s: arp -a | grep '94:XX:28:XX:26:70'
? (10.139.0.149) at 94:XX:28:XX:26:70 [ether] on ens160
? (192.168.73.92) at 94:XX:28:XX:26:70 [ether] on ens192
? (10.139.0.130) at 94:XX:28:XX:26:70 [ether] on ens160

Only the second line is the correct one here, while 192.168.xxx is the address of the kubernetes-node. The first and the third line are metallb-interfaces, which belong to the wrong MAC-address here (you see, that this is another interface).

Deleting arp-caches, pods and re-deploy the services was not a solution.

As a workaround, I downgraded metallb to version v0.8.1. After clearing the arp-Cache on the client again, my problem was solved:

# arp -a | grep 94:XX:28:XX:26:70
? (192.168.73.92) at 94:XX:28:XX:26:70 [ether] on ens192
# arp -a | grep 149
? (10.139.0.149) at f4:XX:43:4e:XX:64 [ether] on ens160
  • MetalLB 0.9.3
  • Kubernetes Version 1.17.1
  • NetworkAddon flannel with edited config:
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "host-gw"
      }
    }
  • Whether you’ve configured kube-proxy for iptables or ipvs mode

I think this is kube-proxy - the cluster was initiated by kubeadm, so I did not do anything like that manually.

About this issue

  • Original URL
  • State: closed
  • Created 4 years ago
  • Comments: 15 (7 by maintainers)

Most upvoted comments

@rata Thanks for your answer. You asked:

Do you any problem when you connect to the external service IP assigned by MetalLB?

yes, of course! As I wrote:

As you see, the client in the following diagram cannot reach the metallb-IP anymore. The only solution was to downgrade metallb to 0.8.1 again.

@rata wrote

This is only relevant if you are using IPVS mode. You said you didn’t know what you were using in the report. So, you would need to check. My guess, though, is that you are not using IPVS mode as that is probably not the default in kubeadm and the ARP behavior you see. But, the you would really need to check 😃

As I wrote, mode is defined as "", which is IPtables -> https://stackoverflow.com/questions/56493651/enable-ipvs-mode-in-kube-proxy-on-a-ready-kubernetes-local-cluster

So this seems not to be the cause.