cilium: Failed to set sysctl needed by BPF bandwidth manager
Bug report
General Information
-
Cilium version (run
cilium version)1.9 -
Kernel version (run
uname -a)Linux unknown000c29c41271 5.3.18-24.34-default #1 SMP Thu Oct 29 22:18:30 UTC 2020 (89a1d25) x86_64 x86_64 x86_64 GNU/Linux -
Orchestration system version in use (e.g.
kubectl version, …)
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2+k3s1", GitCommit:"1d4adb0301b9a63ceec8cabb11b309e061f43d5f", GitTreeState:"clean", BuildDate:"2021-01-14T23:52:37Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.2+k3s1", GitCommit:"1d4adb0301b9a63ceec8cabb11b309e061f43d5f", GitTreeState:"clean", BuildDate:"2021-01-14T23:52:37Z", GoVersion:"go1.15.5", Compiler:"gc", Platform:"linux/amd64"}
-
Link to relevant artifacts (policies, deployments scripts, …) https://raw.githubusercontent.com/cilium/cilium/v1.9/install/kubernetes/quick-install.yaml
-
Generate and upload a system zip:
curl -sLO https://git.io/cilium-sysdump-latest.zip && python cilium-sysdump-latest.zip
cilium-sysdump-20210524-111215-quick-start.zip
How to reproduce the issue
- kubectl create -f https://raw.githubusercontent.com/cilium/cilium/v1.9/install/kubernetes/quick-install.yaml
cilium-agent Logs
level=info msg="Skipped reading configuration file" reason="Config File \"ciliumd\" Not Found in \"[/root]\"" subsys=config
level=info msg="Started gops server" address="127.0.0.1:9890" subsys=daemon
level=info msg="Memory available for map entries (0.003% of 1514762240B): 3786905B" subsys=config
level=info msg="option bpf-ct-global-tcp-max set by dynamic sizing to 131072" subsys=config
level=info msg="option bpf-ct-global-any-max set by dynamic sizing to 65536" subsys=config
level=info msg="option bpf-nat-global-max set by dynamic sizing to 131072" subsys=config
level=info msg="option bpf-neigh-global-max set by dynamic sizing to 131072" subsys=config
level=info msg="option bpf-sock-rev-map-max set by dynamic sizing to 65536" subsys=config
level=info msg=" --agent-health-port='9876'" subsys=daemon
level=info msg=" --agent-labels=''" subsys=daemon
level=info msg=" --allow-icmp-frag-needed='true'" subsys=daemon
level=info msg=" --allow-localhost='auto'" subsys=daemon
level=info msg=" --annotate-k8s-node='true'" subsys=daemon
level=info msg=" --api-rate-limit='map[]'" subsys=daemon
level=info msg=" --arping-refresh-period='5m0s'" subsys=daemon
level=info msg=" --auto-create-cilium-node-resource='true'" subsys=daemon
level=info msg=" --auto-direct-node-routes='false'" subsys=daemon
level=info msg=" --blacklist-conflicting-routes='false'" subsys=daemon
level=info msg=" --bpf-compile-debug='false'" subsys=daemon
level=info msg=" --bpf-ct-global-any-max='262144'" subsys=daemon
level=info msg=" --bpf-ct-global-tcp-max='524288'" subsys=daemon
level=info msg=" --bpf-ct-timeout-regular-any='1m0s'" subsys=daemon
level=info msg=" --bpf-ct-timeout-regular-tcp='6h0m0s'" subsys=daemon
level=info msg=" --bpf-ct-timeout-regular-tcp-fin='10s'" subsys=daemon
level=info msg=" --bpf-ct-timeout-regular-tcp-syn='1m0s'" subsys=daemon
level=info msg=" --bpf-ct-timeout-service-any='1m0s'" subsys=daemon
level=info msg=" --bpf-ct-timeout-service-tcp='6h0m0s'" subsys=daemon
level=info msg=" --bpf-fragments-map-max='8192'" subsys=daemon
level=info msg=" --bpf-lb-acceleration='disabled'" subsys=daemon
level=info msg=" --bpf-lb-algorithm='random'" subsys=daemon
level=info msg=" --bpf-lb-maglev-hash-seed='JLfvgnHc2kaSUFaI'" subsys=daemon
level=info msg=" --bpf-lb-maglev-table-size='16381'" subsys=daemon
level=info msg=" --bpf-lb-map-max='65536'" subsys=daemon
level=info msg=" --bpf-lb-mode='snat'" subsys=daemon
level=info msg=" --bpf-map-dynamic-size-ratio='0.0025'" subsys=daemon
level=info msg=" --bpf-nat-global-max='524288'" subsys=daemon
level=info msg=" --bpf-neigh-global-max='524288'" subsys=daemon
level=info msg=" --bpf-policy-map-max='16384'" subsys=daemon
level=info msg=" --bpf-root=''" subsys=daemon
level=info msg=" --bpf-sock-rev-map-max='262144'" subsys=daemon
level=info msg=" --certificates-directory='/var/run/cilium/certs'" subsys=daemon
level=info msg=" --cgroup-root=''" subsys=daemon
level=info msg=" --cluster-id=''" subsys=daemon
level=info msg=" --cluster-name='default'" subsys=daemon
level=info msg=" --clustermesh-config='/var/lib/cilium/clustermesh/'" subsys=daemon
level=info msg=" --cmdref=''" subsys=daemon
level=info msg=" --config=''" subsys=daemon
level=info msg=" --config-dir='/tmp/cilium/config-map'" subsys=daemon
level=info msg=" --conntrack-gc-interval='0s'" subsys=daemon
level=info msg=" --crd-wait-timeout='5m0s'" subsys=daemon
level=info msg=" --datapath-mode='veth'" subsys=daemon
level=info msg=" --debug='false'" subsys=daemon
level=info msg=" --debug-verbose=''" subsys=daemon
level=info msg=" --device=''" subsys=daemon
level=info msg=" --devices=''" subsys=daemon
level=info msg=" --direct-routing-device=''" subsys=daemon
level=info msg=" --disable-cnp-status-updates='true'" subsys=daemon
level=info msg=" --disable-conntrack='false'" subsys=daemon
level=info msg=" --disable-endpoint-crd='false'" subsys=daemon
level=info msg=" --disable-envoy-version-check='false'" subsys=daemon
level=info msg=" --disable-iptables-feeder-rules=''" subsys=daemon
level=info msg=" --dns-max-ips-per-restored-rule='1000'" subsys=daemon
level=info msg=" --egress-masquerade-interfaces=''" subsys=daemon
level=info msg=" --egress-multi-home-ip-rule-compat='false'" subsys=daemon
level=info msg=" --enable-auto-protect-node-port-range='true'" subsys=daemon
level=info msg=" --enable-bandwidth-manager='false'" subsys=daemon
level=info msg=" --enable-bpf-clock-probe='true'" subsys=daemon
level=info msg=" --enable-bpf-masquerade='true'" subsys=daemon
level=info msg=" --enable-bpf-tproxy='false'" subsys=daemon
level=info msg=" --enable-endpoint-health-checking='true'" subsys=daemon
level=info msg=" --enable-endpoint-routes='false'" subsys=daemon
level=info msg=" --enable-external-ips='true'" subsys=daemon
level=info msg=" --enable-health-check-nodeport='true'" subsys=daemon
level=info msg=" --enable-health-checking='true'" subsys=daemon
level=info msg=" --enable-host-firewall='false'" subsys=daemon
level=info msg=" --enable-host-legacy-routing='false'" subsys=daemon
level=info msg=" --enable-host-port='true'" subsys=daemon
level=info msg=" --enable-host-reachable-services='false'" subsys=daemon
level=info msg=" --enable-hubble='true'" subsys=daemon
level=info msg=" --enable-identity-mark='true'" subsys=daemon
level=info msg=" --enable-ip-masq-agent='false'" subsys=daemon
level=info msg=" --enable-ipsec='false'" subsys=daemon
level=info msg=" --enable-ipv4='true'" subsys=daemon
level=info msg=" --enable-ipv4-fragment-tracking='true'" subsys=daemon
level=info msg=" --enable-ipv6='false'" subsys=daemon
level=info msg=" --enable-ipv6-ndp='false'" subsys=daemon
level=info msg=" --enable-k8s-api-discovery='false'" subsys=daemon
level=info msg=" --enable-k8s-endpoint-slice='true'" subsys=daemon
level=info msg=" --enable-k8s-event-handover='false'" subsys=daemon
level=info msg=" --enable-l7-proxy='true'" subsys=daemon
level=info msg=" --enable-local-node-route='true'" subsys=daemon
level=info msg=" --enable-local-redirect-policy='false'" subsys=daemon
level=info msg=" --enable-monitor='true'" subsys=daemon
level=info msg=" --enable-node-port='false'" subsys=daemon
level=info msg=" --enable-policy='default'" subsys=daemon
level=info msg=" --enable-remote-node-identity='true'" subsys=daemon
level=info msg=" --enable-selective-regeneration='true'" subsys=daemon
level=info msg=" --enable-session-affinity='true'" subsys=daemon
level=info msg=" --enable-svc-source-range-check='true'" subsys=daemon
level=info msg=" --enable-tracing='false'" subsys=daemon
level=info msg=" --enable-well-known-identities='false'" subsys=daemon
level=info msg=" --enable-xt-socket-fallback='true'" subsys=daemon
level=info msg=" --encrypt-interface=''" subsys=daemon
level=info msg=" --encrypt-node='false'" subsys=daemon
level=info msg=" --endpoint-interface-name-prefix='lxc+'" subsys=daemon
level=info msg=" --endpoint-queue-size='25'" subsys=daemon
level=info msg=" --endpoint-status=''" subsys=daemon
level=info msg=" --envoy-log=''" subsys=daemon
level=info msg=" --exclude-local-address=''" subsys=daemon
level=info msg=" --fixed-identity-mapping='map[]'" subsys=daemon
level=info msg=" --flannel-master-device=''" subsys=daemon
level=info msg=" --flannel-uninstall-on-exit='false'" subsys=daemon
level=info msg=" --force-local-policy-eval-at-source='true'" subsys=daemon
level=info msg=" --gops-port='9890'" subsys=daemon
level=info msg=" --host-reachable-services-protos='tcp,udp'" subsys=daemon
level=info msg=" --http-403-msg=''" subsys=daemon
level=info msg=" --http-idle-timeout='0'" subsys=daemon
level=info msg=" --http-max-grpc-timeout='0'" subsys=daemon
level=info msg=" --http-normalize-path='true'" subsys=daemon
level=info msg=" --http-request-timeout='3600'" subsys=daemon
level=info msg=" --http-retry-count='3'" subsys=daemon
level=info msg=" --http-retry-timeout='0'" subsys=daemon
level=info msg=" --hubble-disable-tls='false'" subsys=daemon
level=info msg=" --hubble-event-queue-size='0'" subsys=daemon
level=info msg=" --hubble-flow-buffer-size='4095'" subsys=daemon
level=info msg=" --hubble-listen-address=':4244'" subsys=daemon
level=info msg=" --hubble-metrics=''" subsys=daemon
level=info msg=" --hubble-metrics-server=''" subsys=daemon
level=info msg=" --hubble-socket-path='/var/run/cilium/hubble.sock'" subsys=daemon
level=info msg=" --hubble-tls-cert-file='/var/lib/cilium/tls/hubble/server.crt'" subsys=daemon
level=info msg=" --hubble-tls-client-ca-files='/var/lib/cilium/tls/hubble/client-ca.crt'" subsys=daemon
level=info msg=" --hubble-tls-key-file='/var/lib/cilium/tls/hubble/server.key'" subsys=daemon
level=info msg=" --identity-allocation-mode='crd'" subsys=daemon
level=info msg=" --identity-change-grace-period='5s'" subsys=daemon
level=info msg=" --install-iptables-rules='true'" subsys=daemon
level=info msg=" --ip-allocation-timeout='2m0s'" subsys=daemon
level=info msg=" --ip-masq-agent-config-path='/etc/config/ip-masq-agent'" subsys=daemon
level=info msg=" --ipam='cluster-pool'" subsys=daemon
level=info msg=" --ipsec-key-file=''" subsys=daemon
level=info msg=" --iptables-lock-timeout='5s'" subsys=daemon
level=info msg=" --iptables-random-fully='false'" subsys=daemon
level=info msg=" --ipv4-node='auto'" subsys=daemon
level=info msg=" --ipv4-pod-subnets=''" subsys=daemon
level=info msg=" --ipv4-range='auto'" subsys=daemon
level=info msg=" --ipv4-service-loopback-address='169.254.42.1'" subsys=daemon
level=info msg=" --ipv4-service-range='auto'" subsys=daemon
level=info msg=" --ipv6-cluster-alloc-cidr='f00d::/64'" subsys=daemon
level=info msg=" --ipv6-mcast-device=''" subsys=daemon
level=info msg=" --ipv6-node='auto'" subsys=daemon
level=info msg=" --ipv6-pod-subnets=''" subsys=daemon
level=info msg=" --ipv6-range='auto'" subsys=daemon
level=info msg=" --ipv6-service-range='auto'" subsys=daemon
level=info msg=" --ipvlan-master-device='undefined'" subsys=daemon
level=info msg=" --join-cluster='false'" subsys=daemon
level=info msg=" --k8s-api-server=''" subsys=daemon
level=info msg=" --k8s-force-json-patch='false'" subsys=daemon
level=info msg=" --k8s-heartbeat-timeout='30s'" subsys=daemon
level=info msg=" --k8s-kubeconfig-path=''" subsys=daemon
level=info msg=" --k8s-namespace='kube-system'" subsys=daemon
level=info msg=" --k8s-require-ipv4-pod-cidr='false'" subsys=daemon
level=info msg=" --k8s-require-ipv6-pod-cidr='false'" subsys=daemon
level=info msg=" --k8s-service-cache-size='128'" subsys=daemon
level=info msg=" --k8s-service-proxy-name=''" subsys=daemon
level=info msg=" --k8s-sync-timeout='3m0s'" subsys=daemon
level=info msg=" --k8s-watcher-endpoint-selector='metadata.name!=kube-scheduler,metadata.name!=kube-controller-manager,metadata.name!=etcd-operator,metadata.name!=gcp-controller-manager'" subsys=daemon
level=info msg=" --k8s-watcher-queue-size='1024'" subsys=daemon
level=info msg=" --keep-config='false'" subsys=daemon
level=info msg=" --kube-proxy-replacement='probe'" subsys=daemon
level=info msg=" --kube-proxy-replacement-healthz-bind-address=''" subsys=daemon
level=info msg=" --kvstore=''" subsys=daemon
level=info msg=" --kvstore-connectivity-timeout='2m0s'" subsys=daemon
level=info msg=" --kvstore-lease-ttl='15m0s'" subsys=daemon
level=info msg=" --kvstore-opt='map[]'" subsys=daemon
level=info msg=" --kvstore-periodic-sync='5m0s'" subsys=daemon
level=info msg=" --label-prefix-file=''" subsys=daemon
level=info msg=" --labels=''" subsys=daemon
level=info msg=" --lib-dir='/var/lib/cilium'" subsys=daemon
level=info msg=" --log-driver=''" subsys=daemon
level=info msg=" --log-opt='map[]'" subsys=daemon
level=info msg=" --log-system-load='false'" subsys=daemon
level=info msg=" --masquerade='true'" subsys=daemon
level=info msg=" --max-controller-interval='0'" subsys=daemon
level=info msg=" --metrics=''" subsys=daemon
level=info msg=" --monitor-aggregation='medium'" subsys=daemon
level=info msg=" --monitor-aggregation-flags='all'" subsys=daemon
level=info msg=" --monitor-aggregation-interval='5s'" subsys=daemon
level=info msg=" --monitor-queue-size='0'" subsys=daemon
level=info msg=" --mtu='0'" subsys=daemon
level=info msg=" --nat46-range='0:0:0:0:0:FFFF::/96'" subsys=daemon
level=info msg=" --native-routing-cidr=''" subsys=daemon
level=info msg=" --node-port-acceleration='disabled'" subsys=daemon
level=info msg=" --node-port-algorithm='random'" subsys=daemon
level=info msg=" --node-port-bind-protection='true'" subsys=daemon
level=info msg=" --node-port-mode='snat'" subsys=daemon
level=info msg=" --node-port-range='30000,32767'" subsys=daemon
level=info msg=" --policy-audit-mode='false'" subsys=daemon
level=info msg=" --policy-queue-size='100'" subsys=daemon
level=info msg=" --policy-trigger-interval='1s'" subsys=daemon
level=info msg=" --pprof='false'" subsys=daemon
level=info msg=" --preallocate-bpf-maps='false'" subsys=daemon
level=info msg=" --prefilter-device='undefined'" subsys=daemon
level=info msg=" --prefilter-mode='native'" subsys=daemon
level=info msg=" --prepend-iptables-chains='true'" subsys=daemon
level=info msg=" --prometheus-serve-addr=''" subsys=daemon
level=info msg=" --proxy-connect-timeout='1'" subsys=daemon
level=info msg=" --proxy-prometheus-port='0'" subsys=daemon
level=info msg=" --read-cni-conf=''" subsys=daemon
level=info msg=" --restore='true'" subsys=daemon
level=info msg=" --sidecar-istio-proxy-image='cilium/istio_proxy'" subsys=daemon
level=info msg=" --single-cluster-route='false'" subsys=daemon
level=info msg=" --skip-crd-creation='false'" subsys=daemon
level=info msg=" --socket-path='/var/run/cilium/cilium.sock'" subsys=daemon
level=info msg=" --sockops-enable='false'" subsys=daemon
level=info msg=" --state-dir='/var/run/cilium'" subsys=daemon
level=info msg=" --tofqdns-dns-reject-response-code='refused'" subsys=daemon
level=info msg=" --tofqdns-enable-dns-compression='true'" subsys=daemon
level=info msg=" --tofqdns-endpoint-max-ip-per-hostname='50'" subsys=daemon
level=info msg=" --tofqdns-idle-connection-grace-period='0s'" subsys=daemon
level=info msg=" --tofqdns-max-deferred-connection-deletes='10000'" subsys=daemon
level=info msg=" --tofqdns-min-ttl='0'" subsys=daemon
level=info msg=" --tofqdns-pre-cache=''" subsys=daemon
level=info msg=" --tofqdns-proxy-port='0'" subsys=daemon
level=info msg=" --tofqdns-proxy-response-max-delay='100ms'" subsys=daemon
level=info msg=" --trace-payloadlen='128'" subsys=daemon
level=info msg=" --tunnel='vxlan'" subsys=daemon
level=info msg=" --version='false'" subsys=daemon
level=info msg=" --write-cni-conf-when-ready=''" subsys=daemon
level=info msg=" _ _ _" subsys=daemon
level=info msg=" ___|_| |_|_ _ _____" subsys=daemon
level=info msg="| _| | | | | | |" subsys=daemon
level=info msg="|___|_|_|_|___|_|_|_|" subsys=daemon
level=info msg="Cilium 1.9.7 f993696 2021-05-12T18:21:30-07:00 go version go1.15.12 linux/amd64" subsys=daemon
level=info msg="cilium-envoy version: 82a70d56bf324287ced3129300db609eceb21d10/1.17.3/Distribution/RELEASE/BoringSSL" subsys=daemon
level=info msg="clang (10.0.0) and kernel (5.3.18) versions: OK!" subsys=linux-datapath
level=info msg="linking environment: OK!" subsys=linux-datapath
level=info msg="Detected mounted BPF filesystem at /sys/fs/bpf" subsys=bpf
level=info msg="Parsing base label prefixes from default label list" subsys=labels-filter
level=info msg="Parsing additional label prefixes from user inputs: []" subsys=labels-filter
level=info msg="Final label prefixes to be used for identity evaluation:" subsys=labels-filter
level=info msg=" - reserved:.*" subsys=labels-filter
level=info msg=" - :io.kubernetes.pod.namespace" subsys=labels-filter
level=info msg=" - :io.cilium.k8s.namespace.labels" subsys=labels-filter
level=info msg=" - :app.kubernetes.io" subsys=labels-filter
level=info msg=" - !:io.kubernetes" subsys=labels-filter
level=info msg=" - !:kubernetes.io" subsys=labels-filter
level=info msg=" - !:.*beta.kubernetes.io" subsys=labels-filter
level=info msg=" - !:k8s.io" subsys=labels-filter
level=info msg=" - !:pod-template-generation" subsys=labels-filter
level=info msg=" - !:pod-template-hash" subsys=labels-filter
level=info msg=" - !:controller-revision-hash" subsys=labels-filter
level=info msg=" - !:annotation.*" subsys=labels-filter
level=info msg=" - !:etcd_node" subsys=labels-filter
level=info msg="Using autogenerated IPv4 allocation range" subsys=node v4Prefix=10.156.0.0/16
level=info msg="Initializing daemon" subsys=daemon
level=info msg="Establishing connection to apiserver" host="https://10.43.0.1:443" subsys=k8s
level=info msg="Connected to apiserver" subsys=k8s
level=info msg="Trying to auto-enable \"enable-node-port\", \"enable-external-ips\", \"enable-host-reachable-services\", \"enable-host-port\", \"enable-session-affinity\" features" subsys=daemon
level=warning msg="Session affinity for host reachable services needs kernel 5.7.0 or newer to work properly when accessed from inside cluster: the same service endpoint will be selected from all network namespaces on the host." subsys=daemon
level=info msg="BPF host routing is only available in native routing mode. Falling back to legacy host routing (enable-host-legacy-routing=true)." subsys=daemon
level=info msg="Inheriting MTU from external network interface" device=eth0 ipAddr=192.168.1.156 mtu=1500 subsys=mtu
level=info msg="Restored services from maps" failed=0 restored=6 subsys=service
level=info msg="Reading old endpoints..." subsys=daemon
level=info msg="No old endpoints found." subsys=daemon
level=info msg="Envoy: Starting xDS gRPC server listening on /var/run/cilium/xds.sock" subsys=envoy-manager
level=error msg="Command execution failed" cmd="[iptables -t mangle -n -L CILIUM_PRE_mangle]" error="exit status 1" subsys=iptables
level=warning msg="iptables: No chain/target/match by that name." subsys=iptables
level=info msg="Waiting until all Cilium CRDs are available" subsys=k8s
level=info msg="All Cilium CRDs have been found and are available" subsys=k8s
level=info msg="Creating or updating CiliumNode resource" node=unknown000c29c41271 subsys=nodediscovery
level=info msg="Retrieved node information from cilium node" nodeName=unknown000c29c41271 subsys=k8s
level=info msg="Received own node information from API server" ipAddr.ipv4=192.168.1.156 ipAddr.ipv6="<nil>" k8sNodeIP=192.168.1.156 labels="map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:k3s beta.kubernetes.io/os:linux k3s.io/hostname:unknown000c29c41271 k3s.io/internal-ip:192.168.1.156 kubernetes.io/arch:amd64 kubernetes.io/hostname:unknown000c29c41271 kubernetes.io/os:linux node-role.kubernetes.io/control-plane:true node-role.kubernetes.io/etcd:true node-role.kubernetes.io/master:true node.kubernetes.io/instance-type:k3s]" nodeName=unknown000c29c41271 subsys=k8s v4Prefix=10.0.0.0/24 v6Prefix="<nil>"
level=info msg="k8s mode: Allowing localhost to reach local endpoints" subsys=daemon
level=info msg="Using auto-derived devices for BPF node port" devices="[eth0]" directRoutingDevice=eth0 subsys=daemon
level=info msg="Enabling k8s event listener" subsys=k8s-watcher
level=info msg="Removing stale endpoint interfaces" subsys=daemon
level=info msg="Skipping kvstore configuration" subsys=daemon
level=info msg="Restored router address from node_config" file=/var/run/cilium/state/globals/node_config.h ipv4=10.0.0.115 ipv6="<nil>" subsys=node
level=info msg="Initializing node addressing" subsys=daemon
level=info msg="Initializing cluster-pool IPAM" subsys=ipam v4Prefix=10.0.0.0/24 v6Prefix="<nil>"
level=info msg="Restoring endpoints..." subsys=daemon
level=info msg="Waiting until all pre-existing resources related to policy have been received" subsys=k8s-watcher
level=info msg="Endpoints restored" failed=0 restored=0 subsys=daemon
level=info msg="Addressing information:" subsys=daemon
level=info msg=" Cluster-Name: default" subsys=daemon
level=info msg=" Cluster-ID: 0" subsys=daemon
level=info msg=" Local node-name: unknown000c29c41271" subsys=daemon
level=info msg=" Node-IPv6: <nil>" subsys=daemon
level=info msg=" External-Node IPv4: 192.168.1.156" subsys=daemon
level=info msg=" Internal-Node IPv4: 10.0.0.115" subsys=daemon
level=info msg=" IPv4 allocation prefix: 10.0.0.0/24" subsys=daemon
level=info msg=" Loopback IPv4: 169.254.42.1" subsys=daemon
level=info msg=" Local IPv4 addresses:" subsys=daemon
level=info msg=" - 192.168.1.156" subsys=daemon
level=info msg=" - 10.0.0.115" subsys=daemon
level=info msg="Creating or updating CiliumNode resource" node=unknown000c29c41271 subsys=nodediscovery
level=info msg="Adding local node to cluster" node="{unknown000c29c41271 default [{InternalIP 192.168.1.156} {CiliumInternalIP 10.0.0.115}] 10.0.0.0/24 <nil> 10.0.0.40 <nil> 0 local 0 map[beta.kubernetes.io/arch:amd64 beta.kubernetes.io/instance-type:k3s beta.kubernetes.io/os:linux k3s.io/hostname:unknown000c29c41271 k3s.io/internal-ip:192.168.1.156 kubernetes.io/arch:amd64 kubernetes.io/hostname:unknown000c29c41271 kubernetes.io/os:linux node-role.kubernetes.io/control-plane:true node-role.kubernetes.io/etcd:true node-role.kubernetes.io/master:true node.kubernetes.io/instance-type:k3s] 6}" subsys=nodediscovery
level=info msg="Annotating k8s node" subsys=daemon v4CiliumHostIP.IPv4=10.0.0.115 v4Prefix=10.0.0.0/24 v4healthIP.IPv4=10.0.0.40 v6CiliumHostIP.IPv6="<nil>" v6Prefix="<nil>" v6healthIP.IPv6="<nil>"
level=info msg="Initializing identity allocator" subsys=identity-cache
level=info msg="Cluster-ID is not specified, skipping ClusterMesh initialization" subsys=daemon
level=info msg="Setting up BPF datapath" bpfClockSource=ktime bpfInsnSet=v2 subsys=datapath-loader
level=info msg="Setting sysctl" subsys=datapath-loader sysParamName=net.core.bpf_jit_enable sysParamValue=1
level=info msg="Setting sysctl" subsys=datapath-loader sysParamName=net.ipv4.conf.all.rp_filter sysParamValue=0
level=info msg="Setting sysctl" subsys=datapath-loader sysParamName=kernel.unprivileged_bpf_disabled sysParamValue=1
level=info msg="Setting sysctl" subsys=datapath-loader sysParamName=kernel.timer_migration sysParamValue=0
level=info msg="All pre-existing resources related to policy have been received; continuing" subsys=k8s-watcher
level=error msg="Command execution failed" cmd="[/var/lib/cilium/bpf/init.sh /var/lib/cilium/bpf /var/run/cilium/state 10.0.0.115 <nil> vxlan eth0 <nil> <nil> 1500 true true false /var/run/cilium/cgroupv2 /sys/fs/bpf true false v2 eth0=0x9c01a8c0 <nil> 128 false]" error="exit status 2" subsys=datapath-loader
level=warning msg="+ set -o pipefail" subsys=datapath-loader
level=warning msg="++ command -v cilium-map-migrate" subsys=datapath-loader
level=warning msg="+ [[ ! -n /usr/bin/cilium-map-migrate ]]" subsys=datapath-loader
level=warning msg="+ rm /var/run/cilium/state/encap.state" subsys=datapath-loader
level=warning msg="+ true" subsys=datapath-loader
level=warning msg="+ DIR=/run/cilium/state/globals" subsys=datapath-loader
level=warning msg="+ case \"${MODE}\" in" subsys=datapath-loader
level=warning msg="+ HOST_DEV1=cilium_host" subsys=datapath-loader
level=warning msg="+ HOST_DEV2=cilium_net" subsys=datapath-loader
level=warning msg="+ setup_veth_pair cilium_host cilium_net" subsys=datapath-loader
level=warning msg="+ local -r NAME1=cilium_host" subsys=datapath-loader
level=warning msg="+ local -r NAME2=cilium_net" subsys=datapath-loader
level=warning msg="++ cut -d ' ' -f 2" subsys=datapath-loader
level=warning msg="++ ip link show cilium_host type veth" subsys=datapath-loader
level=warning msg="+ '[' cilium_host@cilium_net: '!=' cilium_host@cilium_net: ']'" subsys=datapath-loader
level=warning msg="+ setup_dev cilium_host" subsys=datapath-loader
level=warning msg="+ local -r NAME=cilium_host" subsys=datapath-loader
level=warning msg="+ ip link set cilium_host up" subsys=datapath-loader
level=warning msg="+ '[' '<nil>' '!=' '<nil>' ']'" subsys=datapath-loader
level=warning msg="+ '[' 10.0.0.115 '!=' '<nil>' ']'" subsys=datapath-loader
level=warning msg="+ echo 1" subsys=datapath-loader
level=warning msg="+ echo 0" subsys=datapath-loader
level=warning msg="+ echo 1" subsys=datapath-loader
level=warning msg="+ echo 0" subsys=datapath-loader
level=warning msg="+ setup_dev cilium_net" subsys=datapath-loader
level=warning msg="+ local -r NAME=cilium_net" subsys=datapath-loader
level=warning msg="+ ip link set cilium_net up" subsys=datapath-loader
level=warning msg="+ '[' '<nil>' '!=' '<nil>' ']'" subsys=datapath-loader
level=warning msg="+ '[' 10.0.0.115 '!=' '<nil>' ']'" subsys=datapath-loader
level=warning msg="+ echo 1" subsys=datapath-loader
level=warning msg="+ echo 0" subsys=datapath-loader
level=warning msg="+ echo 1" subsys=datapath-loader
level=warning msg="+ echo 0" subsys=datapath-loader
level=warning msg="+ ip link set cilium_host arp off" subsys=datapath-loader
level=warning msg="+ ip link set cilium_net arp off" subsys=datapath-loader
level=warning msg="+ ip link set cilium_host mtu 1500" subsys=datapath-loader
level=warning msg="+ ip link set cilium_net mtu 1500" subsys=datapath-loader
level=warning msg="+ case \"${MODE}\" in" subsys=datapath-loader
level=warning msg="+ sed -i '/^#.*CILIUM_NET_MAC.*$/d' /var/run/cilium/state/globals/node_config.h" subsys=datapath-loader
level=warning msg="++ grep ether" subsys=datapath-loader
level=warning msg="++ awk '{print $2}'" subsys=datapath-loader
level=warning msg="++ ip link show cilium_net" subsys=datapath-loader
level=warning msg="+ CILIUM_NET_MAC=d6:68:ae:82:4e:5b" subsys=datapath-loader
level=warning msg="++ mac2array d6:68:ae:82:4e:5b" subsys=datapath-loader
level=warning msg="++ echo '{0xd6,0x68,0xae,0x82,0x4e,0x5b}'" subsys=datapath-loader
level=warning msg="+ CILIUM_NET_MAC='{0xd6,0x68,0xae,0x82,0x4e,0x5b}'" subsys=datapath-loader
level=warning msg="+ sed -i '/^#.*CILIUM_NET_MAC.*$/d' /var/run/cilium/state/globals/node_config.h" subsys=datapath-loader
level=warning msg="+ echo '#ifndef CILIUM_NET_MAC'" subsys=datapath-loader
level=warning msg="+ echo '#define CILIUM_NET_MAC { .addr = {0xd6,0x68,0xae,0x82,0x4e,0x5b}}'" subsys=datapath-loader
level=warning msg="+ echo '#endif /* CILIUM_NET_MAC */'" subsys=datapath-loader
level=warning msg="+ sed -i '/^#.*HOST_IFINDEX.*$/d' /var/run/cilium/state/globals/node_config.h" subsys=datapath-loader
level=warning msg="++ cat /sys/class/net/cilium_net/ifindex" subsys=datapath-loader
level=warning msg="+ HOST_IDX=72" subsys=datapath-loader
level=warning msg="+ echo '#define HOST_IFINDEX 72'" subsys=datapath-loader
level=warning msg="+ sed -i '/^#.*HOST_IFINDEX_MAC.*$/d' /var/run/cilium/state/globals/node_config.h" subsys=datapath-loader
level=warning msg="++ grep ether" subsys=datapath-loader
level=warning msg="++ awk '{print $2}'" subsys=datapath-loader
level=warning msg="++ ip link show cilium_host" subsys=datapath-loader
level=warning msg="+ HOST_MAC=3e:9d:06:54:3a:d0" subsys=datapath-loader
level=warning msg="++ mac2array 3e:9d:06:54:3a:d0" subsys=datapath-loader
level=warning msg="++ echo '{0x3e,0x9d,0x06,0x54,0x3a,0xd0}'" subsys=datapath-loader
level=warning msg="+ HOST_MAC='{0x3e,0x9d,0x06,0x54,0x3a,0xd0}'" subsys=datapath-loader
level=warning msg="+ echo '#define HOST_IFINDEX_MAC { .addr = {0x3e,0x9d,0x06,0x54,0x3a,0xd0}}'" subsys=datapath-loader
level=warning msg="+ sed -i '/^#.*CILIUM_IFINDEX.*$/d' /var/run/cilium/state/globals/node_config.h" subsys=datapath-loader
level=warning msg="++ cat /sys/class/net/cilium_host/ifindex" subsys=datapath-loader
level=warning msg="+ CILIUM_IDX=73" subsys=datapath-loader
level=warning msg="+ echo '#define CILIUM_IFINDEX 73'" subsys=datapath-loader
level=warning msg="++ awk '{print $1}'" subsys=datapath-loader
level=warning msg="++ cat /proc/sys/net/ipv4/ip_local_port_range" subsys=datapath-loader
level=warning msg="+ CILIUM_EPHEMERAL_MIN=32768" subsys=datapath-loader
level=warning msg="+ echo '#define EPHEMERAL_MIN 32768'" subsys=datapath-loader
level=warning msg="+ '[' true = true ']'" subsys=datapath-loader
level=warning msg="+ MAC_BY_IFINDEX_MACRO='#define NATIVE_DEV_MAC_BY_IFINDEX(IFINDEX) ({ \\" subsys=datapath-loader
level=warning msg="\tunion macaddr __mac = {.addr = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}; \\" subsys=datapath-loader
level=warning msg="\tswitch (IFINDEX) { \\\\\\n'" subsys=datapath-loader
level=warning msg="+ MAC_BY_IFINDEX_MACRO_END='\t} \\" subsys=datapath-loader
level=warning msg="\t__mac; })'" subsys=datapath-loader
level=warning msg="+ for NATIVE_DEV in ${NATIVE_DEVS//;/ }" subsys=datapath-loader
level=warning msg="++ cat /sys/class/net/eth0/ifindex" subsys=datapath-loader
level=warning msg="+ IDX=2" subsys=datapath-loader
level=warning msg="++ grep ether" subsys=datapath-loader
level=warning msg="++ awk '{print $2}'" subsys=datapath-loader
level=warning msg="++ ip link show eth0" subsys=datapath-loader
level=warning msg="+ MAC=00:0c:29:c4:12:71" subsys=datapath-loader
level=warning msg="++ mac2array 00:0c:29:c4:12:71" subsys=datapath-loader
level=warning msg="++ echo '{0x00,0x0c,0x29,0xc4,0x12,0x71}'" subsys=datapath-loader
level=warning msg="+ MAC='{0x00,0x0c,0x29,0xc4,0x12,0x71}'" subsys=datapath-loader
level=warning msg="+ MAC_BY_IFINDEX_MACRO='#define NATIVE_DEV_MAC_BY_IFINDEX(IFINDEX) ({ \\" subsys=datapath-loader
level=warning msg="\tunion macaddr __mac = {.addr = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}; \\" subsys=datapath-loader
level=warning msg="\tswitch (IFINDEX) { \\\\\\n\tcase 2: {union macaddr __tmp = {.addr = {0x00,0x0c,0x29,0xc4,0x12,0x71}}; __mac=__tmp;} break; \\\\\\n'" subsys=datapath-loader
level=warning msg="+ MAC_BY_IFINDEX_MACRO='#define NATIVE_DEV_MAC_BY_IFINDEX(IFINDEX) ({ \\" subsys=datapath-loader
level=warning msg="\tunion macaddr __mac = {.addr = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}; \\" subsys=datapath-loader
level=warning msg="\tswitch (IFINDEX) { \\\\\\n\tcase 2: {union macaddr __tmp = {.addr = {0x00,0x0c,0x29,0xc4,0x12,0x71}}; __mac=__tmp;} break; \\\\\\n\t} \\" subsys=datapath-loader
level=warning msg="\t__mac; })'" subsys=datapath-loader
level=warning msg="+ echo -e '#define NATIVE_DEV_MAC_BY_IFINDEX(IFINDEX) ({ \\" subsys=datapath-loader
level=warning msg="\tunion macaddr __mac = {.addr = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0}}; \\" subsys=datapath-loader
level=warning msg="\tswitch (IFINDEX) { \\\\\\n\tcase 2: {union macaddr __tmp = {.addr = {0x00,0x0c,0x29,0xc4,0x12,0x71}}; __mac=__tmp;} break; \\\\\\n\t} \\" subsys=datapath-loader
level=warning msg="\t__mac; })'" subsys=datapath-loader
level=warning msg="+ case \"${MODE}\" in" subsys=datapath-loader
level=warning msg="+ '[' '<nil>' '!=' '<nil>' ']'" subsys=datapath-loader
level=warning msg="+ '[' 10.0.0.115 '!=' '<nil>' ']'" subsys=datapath-loader
level=warning msg="++ ip -4 addr show to 10.0.0.115 dev cilium_host" subsys=datapath-loader
level=warning msg="+ '[' -n '73: cilium_host@cilium_net: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000" subsys=datapath-loader
level=warning msg=" inet 10.0.0.115/32 scope link cilium_host" subsys=datapath-loader
level=warning msg=" valid_lft forever preferred_lft forever' ']'" subsys=datapath-loader
level=warning msg="+ move_local_rules" subsys=datapath-loader
level=warning msg="+ '[' 10.0.0.115 '!=' '<nil>' ']'" subsys=datapath-loader
level=warning msg="+ move_local_rules_af 'ip -4'" subsys=datapath-loader
level=warning msg="+ IP='ip -4'" subsys=datapath-loader
level=warning msg="++ ip -4 rule list" subsys=datapath-loader
level=warning msg="+ '[' -z '9:\tfrom all fwmark 0x200/0xf00 lookup 2004 " subsys=datapath-loader
level=warning msg="10:\tfrom all fwmark 0xa00/0xf00 lookup 2005 " subsys=datapath-loader
level=warning msg="100:\tfrom all lookup local " subsys=datapath-loader
level=warning msg="32766:\tfrom all lookup main " subsys=datapath-loader
level=warning msg="32767:\tfrom all lookup default ' ']'" subsys=datapath-loader
level=warning msg="+ grep 100" subsys=datapath-loader
level=warning msg="+ grep 'lookup local'" subsys=datapath-loader
level=warning msg="+ ip -4 rule list" subsys=datapath-loader
level=warning msg="100:\tfrom all lookup local " subsys=datapath-loader
level=warning msg="+ ip -4 rule del from all lookup local pref 0" subsys=datapath-loader
level=warning msg="+ true" subsys=datapath-loader
level=warning msg="++ grep 'lookup local'" subsys=datapath-loader
level=warning msg="++ wc -l" subsys=datapath-loader
level=warning msg="++ ip -4 rule list" subsys=datapath-loader
level=warning msg="+ '[' 1 -eq 0 ']'" subsys=datapath-loader
level=warning msg="+ '[' '<nil>' '!=' '<nil>' ']'" subsys=datapath-loader
level=warning msg="+ setup_proxy_rules" subsys=datapath-loader
level=warning msg="+ '[' vxlan = ipvlan ']'" subsys=datapath-loader
level=warning msg="+ from_ingress_rulespec='fwmark 0xA00/0xF00 pref 10 lookup 2005'" subsys=datapath-loader
level=warning msg="+ to_proxy_rulespec='fwmark 0x200/0xF00 pref 9 lookup 2004'" subsys=datapath-loader
level=warning msg="+ '[' 10.0.0.115 '!=' '<nil>' ']'" subsys=datapath-loader
level=warning msg="++ ip -4 rule list" subsys=datapath-loader
level=warning msg="+ '[' -n '9:\tfrom all fwmark 0x200/0xf00 lookup 2004 " subsys=datapath-loader
level=warning msg="10:\tfrom all fwmark 0xa00/0xf00 lookup 2005 " subsys=datapath-loader
level=warning msg="100:\tfrom all lookup local " subsys=datapath-loader
level=warning msg="32766:\tfrom all lookup main " subsys=datapath-loader
level=warning msg="32767:\tfrom all lookup default ' ']'" subsys=datapath-loader
level=warning msg="++ ip -4 rule list fwmark 0x200/0xF00 pref 9 lookup 2004" subsys=datapath-loader
level=warning msg="+ '[' -z '9:\tfrom all fwmark 0x200/0xf00 lookup 2004 ' ']'" subsys=datapath-loader
level=warning msg="+ '[' false = true ']'" subsys=datapath-loader
level=warning msg="++ ip -4 rule list fwmark 0xA00/0xF00 pref 10 lookup 2005" subsys=datapath-loader
level=warning msg="+ '[' -z '10:\tfrom all fwmark 0xa00/0xf00 lookup 2005 ' ']'" subsys=datapath-loader
level=warning msg="+ ip route replace table 2004 local 0.0.0.0/0 dev lo" subsys=datapath-loader
level=warning msg="+ '[' false = true ']'" subsys=datapath-loader
level=warning msg="+ ip route replace table 2005 10.0.0.115/32 dev cilium_host" subsys=datapath-loader
level=warning msg="+ ip route replace table 2005 default via 10.0.0.115" subsys=datapath-loader
level=warning msg="+ case \"${MODE}\" in" subsys=datapath-loader
level=warning msg="+ '[' '<nil>' '!=' '<nil>' ']'" subsys=datapath-loader
level=warning msg="+ ip -6 rule del fwmark 0x200/0xF00 pref 9 lookup 2004" subsys=datapath-loader
level=warning msg="+ true" subsys=datapath-loader
level=warning msg="+ ip -6 rule del fwmark 0xA00/0xF00 pref 10 lookup 2005" subsys=datapath-loader
level=warning msg="+ true" subsys=datapath-loader
level=warning msg="+ sed -i /ENCAP_GENEVE/d /var/run/cilium/state/globals/node_config.h" subsys=datapath-loader
level=warning msg="+ sed -i /ENCAP_VXLAN/d /var/run/cilium/state/globals/node_config.h" subsys=datapath-loader
level=warning msg="+ '[' vxlan = vxlan ']'" subsys=datapath-loader
level=warning msg="+ echo '#define ENCAP_VXLAN 1'" subsys=datapath-loader
level=warning msg="+ '[' vxlan = vxlan -o vxlan = geneve ']'" subsys=datapath-loader
level=warning msg="+ ENCAP_DEV=cilium_vxlan" subsys=datapath-loader
level=warning msg="+ ip link show cilium_vxlan" subsys=datapath-loader
level=warning msg="74: cilium_vxlan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000" subsys=datapath-loader
level=warning msg=" link/ether 3a:18:41:e0:20:e3 brd ff:ff:ff:ff:ff:ff" subsys=datapath-loader
level=warning msg="+ ip link set cilium_vxlan mtu 1500" subsys=datapath-loader
level=warning msg="+ setup_dev cilium_vxlan" subsys=datapath-loader
level=warning msg="+ local -r NAME=cilium_vxlan" subsys=datapath-loader
level=warning msg="+ ip link set cilium_vxlan up" subsys=datapath-loader
level=warning msg="+ '[' '<nil>' '!=' '<nil>' ']'" subsys=datapath-loader
level=warning msg="+ '[' 10.0.0.115 '!=' '<nil>' ']'" subsys=datapath-loader
level=warning msg="+ echo 1" subsys=datapath-loader
level=warning msg="+ echo 0" subsys=datapath-loader
level=warning msg="+ echo 1" subsys=datapath-loader
level=warning msg="+ echo 0" subsys=datapath-loader
level=warning msg="++ cat /sys/class/net/cilium_vxlan/ifindex" subsys=datapath-loader
level=warning msg="+ ENCAP_IDX=74" subsys=datapath-loader
level=warning msg="+ sed -i '/^#.*ENCAP_IFINDEX.*$/d' /var/run/cilium/state/globals/node_config.h" subsys=datapath-loader
level=warning msg="+ echo '#define ENCAP_IFINDEX 74'" subsys=datapath-loader
level=warning msg="+ CALLS_MAP=cilium_calls_overlay_2" subsys=datapath-loader
level=warning msg="+ COPTS='-DSECLABEL=2 -DFROM_ENCAP_DEV=1'" subsys=datapath-loader
level=warning msg="+ '[' true = true ']'" subsys=datapath-loader
level=warning msg="+ COPTS='-DSECLABEL=2 -DFROM_ENCAP_DEV=1 -DDISABLE_LOOPBACK_LB'" subsys=datapath-loader
level=warning msg="+ bpf_load cilium_vxlan '-DSECLABEL=2 -DFROM_ENCAP_DEV=1 -DDISABLE_LOOPBACK_LB' ingress bpf_overlay.c bpf_overlay.o from-overlay cilium_calls_overlay_2" subsys=datapath-loader
level=warning msg="+ DEV=cilium_vxlan" subsys=datapath-loader
level=warning msg="+ OPTS='-DSECLABEL=2 -DFROM_ENCAP_DEV=1 -DDISABLE_LOOPBACK_LB'" subsys=datapath-loader
level=warning msg="+ WHERE=ingress" subsys=datapath-loader
level=warning msg="+ IN=bpf_overlay.c" subsys=datapath-loader
level=warning msg="+ OUT=bpf_overlay.o" subsys=datapath-loader
level=warning msg="+ SEC=from-overlay" subsys=datapath-loader
level=warning msg="+ CALLS_MAP=cilium_calls_overlay_2" subsys=datapath-loader
level=warning msg="++ grep ether" subsys=datapath-loader
level=warning msg="++ awk '{print $2}'" subsys=datapath-loader
level=warning msg="++ ip link show cilium_vxlan" subsys=datapath-loader
level=warning msg="+ NODE_MAC=3a:18:41:e0:20:e3" subsys=datapath-loader
level=warning msg="++ mac2array 3a:18:41:e0:20:e3" subsys=datapath-loader
level=warning msg="++ echo '{0x3a,0x18,0x41,0xe0,0x20,0xe3}'" subsys=datapath-loader
level=warning msg="+ NODE_MAC='{.addr={0x3a,0x18,0x41,0xe0,0x20,0xe3}}'" subsys=datapath-loader
level=warning msg="+ OPTS='-DSECLABEL=2 -DFROM_ENCAP_DEV=1 -DDISABLE_LOOPBACK_LB -DNODE_MAC={.addr={0x3a,0x18,0x41,0xe0,0x20,0xe3}} -DCALLS_MAP=cilium_calls_overlay_2'" subsys=datapath-loader
level=warning msg="+ bpf_compile bpf_overlay.c bpf_overlay.o obj '-DSECLABEL=2 -DFROM_ENCAP_DEV=1 -DDISABLE_LOOPBACK_LB -DNODE_MAC={.addr={0x3a,0x18,0x41,0xe0,0x20,0xe3}} -DCALLS_MAP=cilium_calls_overlay_2'" subsys=datapath-loader
level=warning msg="+ IN=bpf_overlay.c" subsys=datapath-loader
level=warning msg="+ OUT=bpf_overlay.o" subsys=datapath-loader
level=warning msg="+ TYPE=obj" subsys=datapath-loader
level=warning msg="+ EXTRA_OPTS='-DSECLABEL=2 -DFROM_ENCAP_DEV=1 -DDISABLE_LOOPBACK_LB -DNODE_MAC={.addr={0x3a,0x18,0x41,0xe0,0x20,0xe3}} -DCALLS_MAP=cilium_calls_overlay_2'" subsys=datapath-loader
level=warning msg="+ clang -O2 -target bpf -std=gnu89 -nostdinc -emit-llvm -Wall -Wextra -Werror -Wshadow -Wno-address-of-packed-member -Wno-unknown-warning-option -Wno-gnu-variable-sized-type-not-at-end -Wdeclaration-after-statement -I. -I/run/cilium/state/globals -I/var/lib/cilium/bpf -I/var/lib/cilium/bpf/include -D__NR_CPUS__=128 -DENABLE_ARP_RESPONDER=1 -DSECLABEL=2 -DFROM_ENCAP_DEV=1 -DDISABLE_LOOPBACK_LB '-DNODE_MAC={.addr={0x3a,0x18,0x41,0xe0,0x20,0xe3}}' -DCALLS_MAP=cilium_calls_overlay_2 -c /var/lib/cilium/bpf/bpf_overlay.c -o -" subsys=datapath-loader
level=warning msg="+ llc -march=bpf -mcpu=v2 -mattr=dwarfris -filetype=obj -o bpf_overlay.o" subsys=datapath-loader
level=warning msg="+ tc qdisc replace dev cilium_vxlan clsact" subsys=datapath-loader
level=warning msg="Error: Specified qdisc not found." subsys=datapath-loader
level=warning msg="+ true" subsys=datapath-loader
level=warning msg="++ grep -v 'pref 1 bpf chain 0 $\\|pref 1 bpf chain 0 handle 0x1'" subsys=datapath-loader
level=warning msg="++ tc filter show dev cilium_vxlan ingress" subsys=datapath-loader
level=warning msg="+ '[' -z '' ']'" subsys=datapath-loader
level=warning msg="+ cilium-map-migrate -s bpf_overlay.o" subsys=datapath-loader
level=warning msg="+ set +e" subsys=datapath-loader
level=warning msg="+ tc filter replace dev cilium_vxlan ingress prio 1 handle 1 bpf da obj bpf_overlay.o sec from-overlay" subsys=datapath-loader
level=warning msg="Error: Parent Qdisc doesn't exists." subsys=datapath-loader
level=warning msg="We have an error talking to the kernel, -1" subsys=datapath-loader
level=warning msg="+ RETCODE=2" subsys=datapath-loader
level=warning msg="+ set -e" subsys=datapath-loader
level=warning msg="+ cilium-map-migrate -e bpf_overlay.o -r 2" subsys=datapath-loader
level=warning msg="+ return 2" subsys=datapath-loader
level=error msg="Error while initializing daemon" error="exit status 2" subsys=daemon
level=fatal msg="Error while creating daemon" error="exit status 2" subsys=daemon
About this issue
- Original URL
- State: closed
- Created 3 years ago
- Comments: 26 (14 by maintainers)
Hi @jamesongithub
The JeOS kernel is indeed mising
fqandfq_codelmodules. We didn’t notice that, because both openSUSE Kubic, and SUSE CaaSP are using the default kernel flavors (kernel-defaultpackage), not the minimal one which comes with JeOS (kernel-default-basepackage).I will try to get those modules also in JeOS flavor, but it might take time.
For now, as a workaround, I would suggest installing the default kernel on all nodes with:
Sorry for the inconvenience and late reaction
sure, attached.
config-5.3.18-24.46-default.txt
yeah so the sysdump tool attempts to run this:
kubectl exec -n kube-system cilium-mh9kn -c cilium-agent cilium-bugtoolkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] – [COMMAND] instead. error: unable to upgrade connection: container not found (“cilium-agent”)but the cilium-agent is in crashloopbackup so it’s gone when sysdump runs. the logs posted earlier was from the cilium-agent container
trying to run the container directly doesn’t work – requires some env vars i guess
crictl run ccda7ef2b0849 cilium-bugtoolFATA[2021-05-26T13:29:37.128835757-07:00] running container: load podSandboxConfig: config at cilium-bugtool not foundlogs from sysdump