minikube: minikube with xhyve: Error starting stopped host: Something went wrong running an SSH command!

Same here:

Minikube version: v0.18.0

Environment:

• OS: Mac OS Sierra 10.12.4
• VM Driver: xhyve

Installed minikube and docker-machine-driver-xhyve yesterday. Starting minikube for the first time went smoothly. Today, when I wanted to restart minikube I got:

[12:04|obergner@Olafs-iMac|tmp ]$ minikube start
Starting local Kubernetes cluster...
Starting VM...
E0423 12:04:37.164969    6696 start.go:116] Error starting host: Error starting stopped host: Something went wrong running an SSH command!
command : echo -e "#/bin/bash\nsudo mkdir -p /Users\nsudo mount -t 9p -o version=9p2000 -o trans=virtio -o uname=obergner -o dfltuid=$(id -u docker) -o dfltgid=50 -o access=any host /Users" | sudo tee /var/lib/boot2docker/bootlocal.sh && sudo chmod +x /var/lib/boot2docker/bootlocal.sh && /var/lib/boot2docker/bootlocal.sh
err     : exit status 32
output  : #/bin/bash
sudo mkdir -p /Users
sudo mount -t 9p -o version=9p2000 -o trans=virtio -o uname=obergner -o dfltuid=1001 -o dfltgid=50 -o access=any host /Users
mount: host is already mounted or /Users busy
       host is already mounted on /Users

.

 Retrying.
E0423 12:04:37.233056    6696 start.go:122] Error starting host:  Error starting stopped host: Something went wrong running an SSH command!
command : echo -e "#/bin/bash\nsudo mkdir -p /Users\nsudo mount -t 9p -o version=9p2000 -o trans=virtio -o uname=obergner -o dfltuid=$(id -u docker) -o dfltgid=50 -o access=any host /Users" | sudo tee /var/lib/boot2docker/bootlocal.sh && sudo chmod +x /var/lib/boot2docker/bootlocal.sh && /var/lib/boot2docker/bootlocal.sh
err     : exit status 32
output  : #/bin/bash
sudo mkdir -p /Users
sudo mount -t 9p -o version=9p2000 -o trans=virtio -o uname=obergner -o dfltuid=1001 -o dfltgid=50 -o access=any host /Users
mount: host is already mounted or /Users busy
       host is already mounted on /Users

Reverting to minikube’s default VM driver fixes the problem:

[12:18|obergner@Olafs-iMac|tmp ]$ minikube config unset vm-driver
[12:18|obergner@Olafs-iMac|tmp ]$ minikube start
Starting local Kubernetes cluster...
Starting VM...
SSH-ing files into VM...
Setting up certs...
Starting cluster components...
Connecting to cluster...
Setting up kubeconfig...
Kubectl is now configured to use the cluster.
[12:19|obergner@Olafs-iMac|tmp ]$

@belldina I run this command, and it works. kubectl config use-context minikube

v0.3.3 was released; Homebrew PR submitted: https://github.com/Homebrew/homebrew-core/pull/15079

Result after installing from Homebrew via the PR:

λ minikube start
Starting local Kubernetes v1.6.4 cluster...
Starting VM...
Moving files into cluster...
Setting up certs...
Starting cluster components...
Connecting to cluster...
Setting up kubeconfig...
Kubectl is now configured to use the cluster.

λ minikube stop
Stopping local Kubernetes cluster...
Machine stopped.

λ minikube start
Starting local Kubernetes v1.6.4 cluster...
Starting VM...
Moving files into cluster...
Setting up certs...
Starting cluster components...
Connecting to cluster...
Setting up kubeconfig...
Kubectl is now configured to use the cluster.

👍

Anytime I’ve run into this issue, I’ve been able to successfully work around it via the context switch posted a above: kubectl config use-context minikube minikube start --vm-driver=xhyve

and if that doesn’t work, then ( more drastically, since it deletes all state, containers, etc… ): minikube delete followed by the commands above…

Of course, this is likely more destructive that other possible solutions, but I mainly use minikube and its embedded docker repo as a staging area, so deleting/re-creating them is rarely a problem for that use case.

I can confirm that @samhalpa workaround works as I was having the same problem with the same versions reported in the original issue above.

Minikube version v0.19.1 still affected by this issue.

I also see this issue pop up if I forget to reset my kubectl config to the minikube context since I have multiple contexts around… so remember to jump back to minikube context, i.e. kubectl config use-context minikube minikube start --vm-driver=xhyve

@r2d4 I don’t. With fresh install:

minikube start --vm-driver=xhyve
// do some work for several hours
minikube stop
// next day or just later
minikube start --vm-driver=xhyve
// this error happens

Closing this, it should be fixed with the latest version of docker-machine-driver-xhyve https://github.com/zchee/docker-machine-driver-xhyve/releases/tag/v0.3.3

Please read https://github.com/kubernetes/minikube/issues/1400#issuecomment-298506787. The issue is fixed in the xhyve driver, however there has not been a release of for that binary yet.

@belldina Right, deleting the cluster always solves it but, like you said, while practically often easy to deal with, isn’t really a solution to this problem in the true sense. It’s the minikube equivalent of “unplug it and plug it back in.” Obviously, it shouldn’t happen in the first place. 😃

@belldina You might want to check out https://github.com/ahmetb/kubectx by @ahmetb. Pretty cool tool to help you switch between your contexts.

Same here, Was using kube-solo then switched to minikube. However, if I run sudo chown root:wheel /usr/local/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve and sudo chmod u+s /usr/local/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve

before the

minikube start --vm-driver xhyve

command, I can spin up the cluster.

Ah, this was actually my fault from a PR in the xhyve driver. Previously we were calling

//Start()
...
d.setupMounts
return d.waitForIP()
...

I switched these since there was sometimes a race condition in setting up the mounts before we were able to get an IP. However, it looks like on a start/stop, the mount already exists and d.setupMounts was silently throwing an error, which we are now catching.

https://github.com/zchee/docker-machine-driver-xhyve/pull/161/files

We should probably ignore the error from setup mounts, even though that would mask actual errors.