talos: Unexpected behavior at network configuration
Bug Report
Description
I have an eno1 interface and tried to configure a bridge using the following configuration:
interfaces:
- interface: br0
addresses:
- <IP>/24
bridge:
stp:
enabled: true
interfaces:
- eno1
routes:
- network: 0.0.0.0/0
gateway: <GW>
After applying the configuration, the network on the node went down, and I observed the following logs:
user: warning: [2025-03-23T16:44:02.086634404Z]: [talos] updating bridge settings {"component": "controller-runtime", "controller": "network.LinkSpecController",
"link": "br0", "old": "{STP:{Enabled:false} VLAN:{FilteringEnabled:false}}", "new": "{STP:{Enabled:true} VLAN:{FilteringEnabled:false}}"}
user: warning: [2025-03-23T16:44:02.088479404Z]: [talos] updated bridge settings {"component": "controller-runtime", "controller": "network.LinkSpecController",
"link": "br0"}
user: warning: [2025-03-23T16:44:02.124169404Z]: [talos] controller failed {"component": "controller-runtime", "controller": "network.RouteSpecController", "error":
"1 error occurred:\n\t* error adding route: netlink receive: network is unreachable, message {Family:2 DstLength:0 SrcLength:0 Tos:0 Table:0 Protocol:4 Scope:0 Type:
1 Flags:0 Attributes:{Dst:<nil> Src:<nil> Gateway:<GW> OutIface:124 Priority:1024 Table:254 Mark:0 Pref:<nil> Expires:<nil> Metrics:<nil> Multipath:[]}}
\n\n"}
user: warning: [2025-03-23T16:44:02.124237404Z]: [talos] restarting controller in 358.522766ms {"component": "controller-runtime", "controller":
"network.RouteSpecController"}
user: warning: [2025-03-23T16:44:02.124413404Z]: [talos] brought link up/down {"component": "controller-runtime", "controller": "network.LinkSpecController",
"link": "br0", "up": true}
user: warning: [2025-03-23T16:44:02.124413404Z]: [talos] brought link up/down {"component": "controller-runtime", "controller": "network.LinkSpecController",
"link": "br0", "up": true}
kern: info: [2025-03-23T16:44:02.214999404Z]: br0: port 1(eno1) entered blocking state
kern: info: [2025-03-23T16:44:02.215008404Z]: br0: port 1(eno1) entered disabled state
kern: info: [2025-03-23T16:44:02.215031404Z]: tg3 0000:02:00.0 eno1: entered allmulticast mode
SUBSYSTEM=pci
DEVICE=+pci:0000:02:00.0
kern: info: [2025-03-23T16:44:02.215099404Z]: tg3 0000:02:00.0 eno1: entered promiscuous mode
SUBSYSTEM=pci
DEVICE=+pci:0000:02:00.0
user: warning: [2025-03-23T16:44:02.215324404Z]: [talos] enslaved/unslaved link {"component": "controller-runtime", "controller": "network.LinkSpecController",
"link": "eno1", "parent": "br0"}
user: warning: [2025-03-23T16:44:02.377119404Z]: [talos] brought link up/down {"component": "controller-runtime", "controller": "network.LinkSpecController",
"link": "eno1", "up": true}
user: warning: [2025-03-23T16:44:02.386869404Z]: [talos] no suitable node IP found, please make sure .machine.kubelet.nodeIP filters and pod/service subnets are set
up correctly {"component": "controller-runtime", "controller": "k8s.NodeIPController"}
user: warning: [2025-03-23T16:44:02.483385404Z]: [talos] controller starting {"component": "controller-runtime", "controller": "network.RouteSpecController"}
user: warning: [2025-03-23T16:44:02.485942404Z]: [talos] created route {"component": "controller-runtime", "controller": "network.RouteSpecController",
"destination": "default", "gateway": <GW>, "table": "main", "link": "eno1", "priority": 0, "family": "inet4"}
user: warning: [2025-03-23T16:44:02.486094404Z]: [talos] created route {"component": "controller-runtime", "controller": "network.RouteSpecController",
"destination": "default", "gateway": GW, "table": "main", "link": "br0", "priority": 1024, "family": "inet4"}
kern: info: [2025-03-23T16:44:05.841838404Z]: tg3 0000:02:00.0 eno1: Link is up at 1000 Mbps, full duplex
SUBSYSTEM=pci
DEVICE=+pci:0000:02:00.0
kern: info: [2025-03-23T16:44:05.841869404Z]: tg3 0000:02:00.0 eno1: Flow control is on for TX and on for RX
SUBSYSTEM=pci
DEVICE=+pci:0000:02:00.0
kern: info: [2025-03-23T16:44:05.841877404Z]: tg3 0000:02:00.0 eno1: EEE is disabled
SUBSYSTEM=pci
DEVICE=+pci:0000:02:00.0
kern: info: [2025-03-23T16:44:05.842069404Z]: br0: port 1(eno1) entered blocking state
kern: info: [2025-03-23T16:44:05.842080404Z]: br0: port 1(eno1) entered listening state
kern: info: [2025-03-23T16:44:08.593861404Z]: tg3 0000:02:00.0 eno1: Link is down
SUBSYSTEM=pci
DEVICE=+pci:0000:02:00.0
kern: info: [2025-03-23T16:44:08.595875404Z]: br0: port 1(eno1) entered disabled state
user: warning: [2025-03-23T16:44:08.639567404Z]: [talos] no suitable node IP found, please make sure .machine.kubelet.nodeIP filters and pod/service subnets are set
up correctly {"component": "controller-runtime", "controller": "k8s.NodeIPController"}
user: warning: [2025-03-23T16:44:16.463060404Z]: [talos] running image cleanup {"component": "controller-runtime", "controller": "runtime.CRIImageGCController"}
user: warning: [2025-03-23T16:44:16.463942404Z]: [talos] image is referenced, skipping garbage collection {"component": "controller-runtime", "controller":
"runtime.CRIImageGCController", "image": "gcr.io/etcd-development/etcd:v3.5.19"}
user: warning: [2025-03-23T16:44:16.463998404Z]: [talos] image is referenced, skipping garbage collection {"component": "controller-runtime", "controller":
"runtime.CRIImageGCController", "image": "gcr.io/etcd-development/etcd@sha256:c415170328bbb2cb497c79dbd8494b1c8d58886dccf95fbfdd3845c13c6d1ca3"}
user: warning: [2025-03-23T16:44:16.464028404Z]: [talos] image is referenced, skipping garbage collection {"component": "controller-runtime", "controller":
"runtime.CRIImageGCController", "image": "ghcr.io/siderolabs/kubelet:v1.32.3"}
user: warning: [2025-03-23T16:44:16.465685404Z]: [talos] image is referenced, skipping garbage collection {"component": "controller-runtime", "controller":
"runtime.CRIImageGCController", "image": "ghcr.io/siderolabs/kubelet@sha256:0b4bea893ea2eb8e41b5d7bc1dc04359060d5e32aba5f609d0c4994590a074a2"}
user: warning: [2025-03-23T16:44:16.465730404Z]: [talos] image is referenced, skipping garbage collection {"component": "controller-runtime", "controller":
"runtime.CRIImageGCController", "image": "sha256:0b4bea893ea2eb8e41b5d7bc1dc04359060d5e32aba5f609d0c4994590a074a2"}
user: warning: [2025-03-23T16:44:16.465757404Z]: [talos] image is referenced, skipping garbage collection {"component": "controller-runtime", "controller":
"runtime.CRIImageGCController", "image": "sha256:c415170328bbb2cb497c79dbd8494b1c8d58886dccf95fbfdd3845c13c6d1ca3"}
user: warning: [2025-03-23T16:44:20.661100404Z]: [talos] serving static pod manifests {"component": "controller-runtime", "controller":
"k8s.StaticPodServerController", "size": 12390}
Before configuring the bridge, I was using a static IP address with this configuration:
interfaces:
- interface: eno0
addresses:
- <IP>/24
routes:
- network: 0.0.0.0/0
gateway: <GW>
However, I believe the eno0 interface in the configuration is incorrect because I do not have this interface. Interestingly, if I change it to a value like eno1000, the configuration applies successfully.
Additionally, I applied a bridge configuration with eno0 (which does not actually exist), and it was created successfully. However, when debugging with bridge-utils, I noticed that the bridge had no interfaces attached. At the same time, eno1 was mysteriously brought up. Since my provider does not offer DHCP, I don’t think it could have been configured automatically.
Environment
- Talos version: 1.9.5
- Kubernetes version: 1.32.3
- Platform: bare metal, amd64
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 1
- Comments: 15 (15 by maintainers)
Hi, I’ve resolved the issue, but I have a few interesting observations:
I tried setting
dhcp: falsefor an interface without any routes, buteno1still had routes assigned. I expected Talos to remove all routes from the interface in this case.My current configuration looks like this:
I expect Talos to disable DHCP on
eno1, but I’m still seeing logs like:I doesn’t expect DHCP requests when it disabled by config.
To sum up, I think the issue was a conflict between DHCP and static configuration. It seems like DHCP tried to assign an address or route to
eno1. Then, when the bridge was configured, both DHCP and my static config were trying to apply the same address/routes, leading to conflicts.I’m not sure what the issue is, the only thing I can think of is that the link is not brought up by Talos when it’s part of the bridge, you can try adding “empty”
- interface: eno1.You need first to understand what are the links you have, including their physical link status
talosctl get links, and configure them accordingly. You don’t need a bridge usually.Talos can’t validate link names in the machine configuration, as links might appear and disappear, so the configuration will be applied whenever the link pops up.