microk8s: microks8 flannel - subnet.env not found

Hello,

My server :

Ubuntu 19.10 microk8s 1.16

I have a problem with microk8s and flannel.

When a run this command :

microk8s.kubectl -n kube-system describe pod kubernetes-dashboard-5c848cc544-dnq66

I have this log :

Warning FailedCreatePodSandBox 15m kubelet, monkeydryzen Failed create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "a5f4e184e1fd364b99dee83b32715ca63e3acba3dc6c3df1d338c15f1be3f126": open /var/snap/microk8s/common/run/flannel/subnet.env: no such file or directory

“flannel” folder do not exist “subnet.env” file do not exist

When a run this command :

microk8s.kubectl get all --all-namespaces

I have this log :

`NAMESPACE NAME READY STATUS RESTARTS AGE kube-system pod/dashboard-metrics-scraper-687667bb6c-4cbwb 0/1 ContainerCreating 0 20m kube-system pod/heapster-v1.5.2-5c58f64f8b-48sww 0/4 ContainerCreating 0 20m kube-system pod/kubernetes-dashboard-5c848cc544-dnq66 0/1 ContainerCreating 0 20m kube-system pod/monitoring-influxdb-grafana-v4-6d599df6bf-5vkmg 0/2 ContainerCreating 0 20m

NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE default service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 26m kube-system service/dashboard-metrics-scraper ClusterIP 10.152.183.161 <none> 8000/TCP 20m kube-system service/heapster ClusterIP 10.152.183.233 <none> 80/TCP 20m kube-system service/kubernetes-dashboard ClusterIP 10.152.183.224 <none> 443/TCP 20m kube-system service/monitoring-grafana ClusterIP 10.152.183.205 <none> 80/TCP 20m kube-system service/monitoring-influxdb ClusterIP 10.152.183.246 <none> 8083/TCP,8086/TCP 20m

NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE kube-system deployment.apps/dashboard-metrics-scraper 0/1 1 0 20m kube-system deployment.apps/heapster-v1.5.2 0/1 1 0 20m kube-system deployment.apps/kubernetes-dashboard 0/1 1 0 20m kube-system deployment.apps/monitoring-influxdb-grafana-v4 0/1 1 0 20m

NAMESPACE NAME DESIRED CURRENT READY AGE kube-system replicaset.apps/dashboard-metrics-scraper-687667bb6c 1 1 0 20m kube-system replicaset.apps/heapster-v1.5.2-5c58f64f8b 1 1 0 20m kube-system replicaset.apps/kubernetes-dashboard-5c848cc544 1 1 0 20m kube-system replicaset.apps/monitoring-influxdb-grafana-v4-6d599df6bf 1 1 0 20m`

Have you a idea ?

Thank you

About this issue

Original URL
State: closed
Created 4 years ago
Reactions: 1
Comments: 15 (2 by maintainers)

Most upvoted comments

I have the same issue with: microk8s v1.18 Ubuntu 18.04

This issue emerged after the last microk8s update. Before that everything was working great…

Edit:

when I manually create /var/snap/microk8s/common/run/flannel/subnet.env:

FLANNEL_NETWORK=10.1.0.0/16
FLANNEL_SUBNET=10.1.28.1/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=false

Everything is working again. Any idea, why it’s not created automatically?

jonasscherer on Apr 1, 2020

OK i have confirmed, https://github.com/coreos/flannel/issues/1365 is the same issue. flanneld crashes on my system.

the fix is to compile flannel w/ golang v1.15+

$ curl -sSL https://github.com/coreos/flannel/releases/download/v0.13.0/flannel-v0.13.0-linux-amd64.tar.gz | tar Cxzv /tmp/ flanneld
flanneld
$ /tmp/flanneld -version
fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0xe0 pc=0x7ffb199db96e]

donbowman on Dec 8, 2020

Also observing this problem. For me the culprit is the flannel service crasing w/ sigsegv. microk8s.inspect attached inspection-report-20201208_134629.tar.gz

uname -a
Linux cube 5.10.0-051000rc6-generic #202011291930 SMP Mon Nov 30 00:36:46 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.10
DISTRIB_CODENAME=groovy
DISTRIB_DESCRIPTION="Ubuntu 20.10"

microk8s                 v1.18.9 (via snap)

cat /var/snap/microk8s/1702/args/flanneld 
--iface=""
--etcd-endpoints=https://127.0.0.1:12379
--etcd-cafile=${SNAP_DATA}/certs/ca.crt
--etcd-certfile=${SNAP_DATA}/certs/server.crt
--etcd-keyfile=${SNAP_DATA}/certs/server.key
--subnet-file=${SNAP_COMMON}/run/flannel/subnet.env

It may match https://github.com/coreos/flannel/issues/1365

Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691638]: PrevNode.Value: {"Network": "10.1.0.0/16", "Backend": {"Type": "vxlan"}}
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691645]: {"Network": "10.1.0.0/16", "Backend": {"Type": "vxlan"}}
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: fatal error: unexpected signal during runtime execution
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: [signal SIGSEGV: segmentation violation code=0x1 addr=0xe0 pc=0x7fde247df96e]
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: runtime stack:
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: runtime.throw(0x19a3a14, 0x2a)
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/runtime/panic.go:616 +0x81
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: runtime.sigpanic()
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/runtime/signal_unix.go:372 +0x28e
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: goroutine 1 [syscall, locked to thread]:
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: runtime.cgocall(0x14432e0, 0xc420095c50, 0x19a184f)
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/runtime/cgocall.go:128 +0x64 fp=0xc420095c08 sp=0xc420095bd0 pc=0x402304
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: os/user._Cfunc_mygetpwuid_r(0x0, 0xc4200c4d20, 0x45b9460, 0x400, 0xc42000e048, 0x0)
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         _cgo_gotypes.go:170 +0x4d fp=0xc420095c50 sp=0xc420095c08 pc=0x6c637d
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: os/user.lookupUnixUid.func1.1(0x0, 0xc4200c4d20, 0x45b9460, 0x400, 0xc42000e048, 0xc420095ce0)
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/os/user/cgo_lookup_unix.go:100 +0x13f fp=0xc420095c90 sp=0xc420095c50 pc=0x6c79ef
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: os/user.lookupUnixUid.func1(0xc420095cb0)
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/os/user/cgo_lookup_unix.go:100 +0x50 fp=0xc420095cd0 sp=0xc420095c90 pc=0x6c7a90
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: os/user.retryWithBuffer(0xc420086a70, 0xc420095dc0, 0xc420086a70, 0x0)
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/os/user/cgo_lookup_unix.go:253 +0x3b fp=0xc420095d30 sp=0xc420095cd0 pc=0x6c6ebb
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: os/user.lookupUnixUid(0x0, 0x0, 0x0, 0x0)
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/os/user/cgo_lookup_unix.go:96 +0x130 fp=0xc420095df8 sp=0xc420095d30 pc=0x6c6800
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: os/user.current(0xc420095e60, 0x10, 0xc420095e58)
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/os/user/cgo_lookup_unix.go:49 +0x27 fp=0xc420095e28 sp=0xc420095df8 pc=0x6c6697
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: os/user.Current.func1()
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/os/user/lookup.go:11 +0x22 fp=0xc420095e50 sp=0xc420095e28 pc=0x6c7832
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: sync.(*Once).Do(0x27a14c0, 0x19f7218)
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/sync/once.go:44 +0xbe fp=0xc420095e88 sp=0xc420095e50 pc=0x46c45e
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: os/user.Current(0xc42003846c, 0x4, 0xc42003846c)
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/os/user/lookup.go:11 +0x3d fp=0xc420095eb8 sp=0xc420095e88 pc=0x6c5f5d
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: github.com/coreos/flannel/vendor/github.com/golang/glog.init.1()
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /go/src/github.com/coreos/flannel/vendor/github.com/golang/glog/glog_file.go:63 +0x39 fp=0xc420095f10 sp=0xc420095eb8 pc=0x6cc4f9
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: github.com/coreos/flannel/vendor/github.com/golang/glog.init()
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         <autogenerated>:1 +0x1c2 fp=0xc420095f58 sp=0xc420095f10 pc=0x6cd212
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: main.init()
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         <autogenerated>:1 +0x87 fp=0xc420095f88 sp=0xc420095f58 pc=0x1441b87
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: runtime.main()
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/runtime/proc.go:186 +0x1ca fp=0xc420095fe0 sp=0xc420095f88 pc=0x42be9a
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: runtime.goexit()
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/runtime/asm_amd64.s:2361 +0x1 fp=0xc420095fe8 sp=0xc420095fe0 pc=0x456f71
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: goroutine 5 [syscall]:
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: os/signal.signal_recv(0x0)
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/runtime/sigqueue.go:139 +0xa6
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: os/signal.loop()
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/os/signal/signal_unix.go:22 +0x22
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: created by os/signal.init.0
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /usr/local/go/src/os/signal/signal_unix.go:28 +0x41
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: goroutine 6 [chan receive]:
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: github.com/coreos/flannel/vendor/github.com/golang/glog.(*loggingT).flushDaemon(0x27a1f40)
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /go/src/github.com/coreos/flannel/vendor/github.com/golang/glog/glog.go:879 +0x8b
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]: created by github.com/coreos/flannel/vendor/github.com/golang/glog.init.0
Dec 08 13:43:28 cube microk8s.daemon-flanneld[2691559]:         /go/src/github.com/coreos/flannel/vendor/github.com/golang/glog/glog.go:410 +0x203
Dec 08 13:43:28 cube systemd[1]: snap.microk8s.daemon-flanneld.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
Dec 08 13:43:28 cube systemd[1]: snap.microk8s.daemon-flanneld.service: Failed with result 'exit-code'.
Dec 08 13:43:28 cube systemd[1]: snap.microk8s.daemon-flanneld.service: Scheduled restart job, restart counter is at 45.

donbowman on Dec 8, 2020