weave: Weave as k8s AddOn does not run on worker nodes

I have been banging my head on this (or something like it) for several hours now. I am trying to do something that should be really simple: start a kube cluster with just weavenet networking. As simple as:

kubectl apply -f https://git.io/weave-kube-1.6

And yet:

  1. No weave containers are run on the worker node (kubelet host)
  2. kubelet never enters ready state, message just like @weitzj 's : Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized

I do not know if the two are related - or if it is connected to https://github.com/weaveworks/weave/issues/2826 - but I just cannot get “simple one-step install” to be, well, simple. 😃

About this issue

  • Original URL
  • State: open
  • Created 7 years ago
  • Reactions: 2
  • Comments: 48 (40 by maintainers)

Most upvoted comments

One more piece of the puzzle. SELinux?

Apr 05 08:06:37 ip-10-50-22-42.ec2.internal kubelet[29296]: I0405 08:06:37.633880   29296 kuberuntime_manager.go:384] No ready sandbox for pod "weave-net-rc838_kube-system(1c6182e5-1964-11e7-b1af-0e94e95c9de0)" can be found. Need to start a new one
Apr 05 08:06:37 ip-10-50-22-42.ec2.internal kubelet[29296]: I0405 08:06:37.634046   29296 kuberuntime_manager.go:458] Container {Name:weave Image:weaveworks/weave-kube:1.9.4 Command:[/home/weave/launch.sh] Args:[] WorkingDir: Ports:[] EnvFrom:[] Env:[] Resources:{Limits:map[] Requests:map[cpu:{i:{value:10 scale:-3} d:{Dec:<nil>} s:10m Format:DecimalSI}]} VolumeMounts:[{Name:weavedb ReadOnly:false MountPath:/weavedb SubPath:} {Name:cni-bin ReadOnly:false MountPath:/host/opt SubPath:} {Name:cni-bin2 ReadOnly:false MountPath:/host/home SubPath:} {Name:cni-conf ReadOnly:false MountPath:/host/etc SubPath:} {Name:dbus ReadOnly:false MountPath:/host/var/lib/dbus SubPath:} {Name:lib-modules ReadOnly:false MountPath:/lib/modules SubPath:} {Name:weave-net-token-82nn3 ReadOnly:true MountPath:/var/run/secrets/kubernetes.io/serviceaccount SubPath:}] LivenessProbe:&Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/status,Port:6784,Host:127.0.0.1,Scheme:HTTP,HTTPHeaders:[],},TCPSocket:nil,},InitialDelaySeconds:30,TimeoutSeconds:1,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,} ReadinessProbe:nil Lifecycle:nil TerminationMessagePath:/dev/termination-log TerminationMessagePolicy:File ImagePullPolicy:IfNotPresent SecurityContext:&SecurityContext{Capabilities:nil,Privileged:*true,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,} Stdin:false StdinOnce:false TTY:false} is dead, but RestartPolicy says that we should restart it.
Apr 05 08:06:37 ip-10-50-22-42.ec2.internal kubelet[29296]: I0405 08:06:37.634091   29296 kuberuntime_manager.go:458] Container {Name:weave-npc Image:weaveworks/weave-npc:1.9.4 Command:[] Args:[] WorkingDir: Ports:[] EnvFrom:[] Env:[] Resources:{Limits:map[] Requests:map[cpu:{i:{value:10 scale:-3} d:{Dec:<nil>} s:10m Format:DecimalSI}]} VolumeMounts:[{Name:weave-net-token-82nn3 ReadOnly:true MountPath:/var/run/secrets/kubernetes.io/serviceaccount SubPath:}] LivenessProbe:nil ReadinessProbe:nil Lifecycle:nil TerminationMessagePath:/dev/termination-log TerminationMessagePolicy:File ImagePullPolicy:IfNotPresent SecurityContext:&SecurityContext{Capabilities:nil,Privileged:*true,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,} Stdin:false StdinOnce:false TTY:false} is dead, but RestartPolicy says that we should restart it.
Apr 05 08:06:37 ip-10-50-22-42.ec2.internal kernel: SELinux: mount invalid.  Same superblock, different security settings for (dev mqueue, type mqueue)
Apr 05 08:06:37 ip-10-50-22-42.ec2.internal containerd[1341]: time="2017-04-05T08:06:37.835787158Z" level=error msg="containerd: start container" error="oci runtime error: container_linux.go:247: starting container process caused \"process_linux.go:359: container init caused \\\"write /proc/self/task/14661/attr/exec: invalid argument\\\"\"\n" id=dcafd3ef35a51333219b32148f7afa1aa733fe978066402f0d0627f703df0f1c
Apr 05 08:06:37 ip-10-50-22-42.ec2.internal dockerd[1342]: time="2017-04-05T08:06:37.836307141Z" level=error msg="Create container failed with error: invalid header field value \"oci runtime error: container_linux.go:247: starting container process caused \\\"process_linux.go:359: container init caused \\\\\\\"write /proc/self/task/14661/attr/exec: invalid argument\\\\\\\"\\\"\\n\""

But no idea how to resolve this. Does weave+kube+coreos not work as a combo?