minikube: xhyve driver not working anymore on MacOS 10.12 sierra

Minikube version (use minikube version): 0.10.0

Environment:

  • OS (e.g. from /etc/os-release): MacOS 10.12 Sierra
  • VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): xhyve
  • Docker version (e.g. docker -v): Docker version 1.12.2-rc1, build 45bed2c, experimental (because of Sierra, but also tried previous and stable version, all Docker for Mac)
  • Install tools:
  • Others:

What happened: minikube start --vm-driver=xhyve Starting local Kubernetes cluster... E0930 11:28:13.241885 36469 start.go:85] Error starting host: Error creating host: Error creating machine: Error in driver during machine creation: Machine didn't return an IP after 120 seconds, aborting. Retrying.

What you expected to happen: Starting local Kubernetes cluster... Running pre-create checks... Creating machine... Starting local Kubernetes cluster... or similar

How to reproduce it (as minimally and precisely as possible): Run minikube start --vm-driver=xhyve on an environment similar to the one above.

Anything else do we need to know: The vbox driver works, so it must be something with the xhyve or docker-machine. Also looking at the error. Installed versions are: docker-machine version 0.8.2, build e18a919 docker-machine-driver-xhyve-0.2.3 The latter hasn’t been updated since August.

About this issue

  • Original URL
  • State: closed
  • Created 8 years ago
  • Reactions: 9
  • Comments: 38 (15 by maintainers)

Most upvoted comments

I don’t see this problem and can run minikube successfully with everything installed by homebrew:

  • macOS 10.12 (16A323)
  • Homebrew 1.0.5
  • xhyve: stable 0.2.0
  • docker-machine: stable 0.8.2
  • docker-machine-driver-xhyve: stable 0.2.3

Installation procedure for xhyve/docker-machine/docker-machine-driver-xhyve:

$ brew update
$ brew install --HEAD xhyve
$ brew install docker-machine-driver-xhyve
$ sudo chown root:wheel $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
$ sudo chmod u+s $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve

Maybe it’s an issue with docker-machine-driver-xhyve when installed with GO? GO in my environment is also installed via homebrew:

  • go version go1.7.1 darwin/amd64

I was having this same issue. Clearing out .minikube ( rm -rf ~/.minikube ) fixed it for me. I was then able to minikube start with the xhyve driver.

@zchee your latest version, built the way you’ve described (i.e. make) works fine for me with macOS 10.12 (16A323) whereas the previous version (via brew install) did not work.

minikube delete; rm -rf ~/.minikube; did the trick for me.

I can confirm this – had the same issue and it’s the docker-machine-driver-xhyve that crashes. This is due to incompatibilities introduced with macOS Sierra and the plugin binary you have doesn’t work on Sierra anymore. The repo is here: https://github.com/zchee/docker-machine-driver-xhyve. You can’t go the Homebrew route because that will download the incompatible binary again. The only thing you can do is get your local Go environment up-to-date so that you can recompile the plugin from source.

Or I could upload the fixed plugin binary somewhere. Alas, that would mean you trusting me.

@shakefu I had the same error message, which was caused by an incomplete VirtualBox uninstallation. Removing /usr/local/bin/VBoxManage resolved my problem, and I was able to use the Xhyve driver successfully.

For those looking for a solution, here’s how you can get going with minikube + xhyve right now.

brew install Caskroom/cask/minikube
brew install docker-machine 
brew install docker-machine-driver-xhyve --HEAD
sudo chown root:wheel $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve\
sudo chmod u+s $(brew --prefix)/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
minikube start --vm-driver=xhyve

@puja108 Sorry 🙇


@mboersma As your posted error log,

Docker Machine xhyve Driver Version 0.2.3 (HEAD)

It seems its binary is not installed by the

brew install docker-machine-driver-xhyve --HEAD

Maybe you installed by go get. (or, you have two docker-machine-driver-xhyve binary)

The short answer: I’m sorry to take up your time, retry

go get -u -d github.com/zchee/docker-machine-driver-xhyve
cd $GOPATH/src/github.com/zchee/docker-machine-driver-xhyve

# just in case
git submodule update --init

# build and install 
make install
# or if want verbose
make install V=1

I was added small trick to Version variable.

  1. If use go get, commit version uses to https://github.com/zchee/docker-machine-driver-xhyve/blob/master/xhyve/version.go#L8
  2. If use brew ... --HEAD, uses to https://github.com/Homebrew/homebrew-core/blob/master/Formula/docker-machine-driver-xhyve.rb#L38 actually HomebrewHEAD-8b542eb(current latest).
  3. If use make, HEAD will change to commit hash. https://github.com/zchee/docker-machine-driver-xhyve/blob/master/Makefile#L81

So, HEAD is maybe go get.

And now, my driver has been not supported go get install if want to use 9p folder share. Needs make. README.md has not yet been updated. It was my fault. very sorry…

And, _why does not support go get?_. The short answer is go get does not support build submodules. In this case, lib9p. e.g. Not support automatically run the go generate command.(maybe for the security issue)

Also, If not add -tags lib9p for go command , cgo did not built the pci_virtio_9p.c. https://github.com/zchee/docker-machine-driver-xhyve/blob/master/vendor/github.com/zchee/libhyperkit/pci_virtio_9p.c#L1

So, I think you’re binary not static linked lib9p.a object file. or not added -tags lib9p. Therefore occur

pci slot 5:0: unknown device "virtio-9p"

Please try

nm $(which docker-machine-driver-xhyve) | grep _l9p

If anything does not appear, it is not linked.

I’m having this problem as well, but the error I’m getting is quite a bit different, I think. The “SSH command” seems to be misinterpreted as exit 0.

Error Log

➜  ~ minikube start --vm-driver=xhyve --v=5 --show-libmachine-logs
Starting local Kubernetes cluster...
Found binary path at /usr/local/bin/docker-machine-driver-xhyve
Launching plugin server for driver xhyve
Plugin server listening at address 127.0.0.1:52020
() DBG | operation not supported by device
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetState
Waiting for SSH to be available...
Getting to WaitForSSH function...
(minikube) Calling .GetSSHHostname
(minikube) DBG |
Error getting ssh command 'exit 0' : IP not found for MAC  in DHCP leases

Homebrew Version of Docker Machine Xhyve Driver

➜  ~ brew info docker-machine-driver-xhyve
docker-machine-driver-xhyve: stable 0.3.1 (bottled), HEAD
Docker Machine driver for xhyve
https://github.com/zchee/docker-machine-driver-xhyve
/usr/local/Cellar/docker-machine-driver-xhyve/0.3.1 (3 files, 13.2M) *
  Poured from bottle on 2016-12-14 at 09:12:09
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/docker-machine-driver-xhyve.rb
==> Dependencies
Build: go ✘
Required: opam ✔, libev ✔
Recommended: docker-machine ✔
==> Requirements
Build: git ✔
Required: macOS >= 10.10 ✔
==> Options
--without-docker-machine
	Build without docker-machine support
--without-qcow2
	Do not support qcow2 disk image format
--HEAD
	Install HEAD version
==> Caveats
This driver requires superuser privileges to access the hypervisor. To
enable, execute
    sudo chown root:wheel /usr/local/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve
    sudo chmod u+s /usr/local/opt/docker-machine-driver-xhyve/bin/docker-machine-driver-xhyve

@mboersma Hmm… 😦

OK, I’ll debug it at tomorrow(today? now AM5:30 in Japan lol) will going to bed… Sorry, please wait a moment.

I rebuilt docker-machine-xhyve-binary with the above patch, but it ends up the same:

...
(minikube) Generate UUID....
(minikube) DBG | Generated UUID: 274755CE-F166-4A09-B5FF-3500E3D33468
(minikube) DBG | Running command: /usr/local/bin/docker-machine-driver-xhyve [xhyve -A -U 274755CE-F166-4A09-B5FF-3500E3D33468 -c 1 -m 1024M -l com1,autopty -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -s 3:0,ahci-cd,/Users/matt/.minikube/machines/minikube/boot2docker.iso -s 4:0,ahci-hd,/dev/rdisk4 -f kexec,/Users/matt/.minikube/machines/minikube/vmlinuz64,/Users/matt/.minikube/machines/minikube/initrd.img,loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 base host=minikube -M]
(minikube) Convert UUID to MAC address...
(minikube) DBG | Converted MAC address: 2a:dc:42:6c:24:b3
(minikube) Starting minikube...
(minikube) DBG | [xhyve -A -U 274755CE-F166-4A09-B5FF-3500E3D33468 -c 1 -m 1024M -l com1,autopty -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -s 3:0,ahci-cd,/Users/matt/.minikube/machines/minikube/boot2docker.iso -s 4:0,ahci-hd,/dev/rdisk4 -f kexec,/Users/matt/.minikube/machines/minikube/vmlinuz64,/Users/matt/.minikube/machines/minikube/initrd.img,loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 base host=minikube -F /Users/matt/.minikube/machines/minikube/minikube.pid -s 5,virtio-9p,host=/Users]
(minikube) Waiting for VM to come online...
(minikube) DBG | 
(minikube) DBG | Not there yet 1/60, error: IP not found for MAC 2a:dc:42:6c:24:b3 in DHCP leases
(minikube) DBG | operation not supported by device
(minikube) Waiting on a pseudo-terminal to be ready... done
(minikube) Hook up your terminal emulator to /dev/ttys004 in order to connect to your VM
(minikube) DBG | 
(minikube) DBG | Not there yet 2/60, error: IP not found for MAC 2a:dc:42:6c:24:b3 in DHCP leases
(minikube) DBG | 
(minikube) DBG | Not there yet 3/60, error: IP not found for MAC 2a:dc:42:6c:24:b3 in DHCP leases
(minikube) DBG | 
(minikube) DBG | Not there yet 4/60, error: IP not found for MAC 2a:dc:42:6c:24:b3 in DHCP leases
(minikube) DBG | rdmsr to register 0x34 on vcpu 0
(minikube) DBG | 
...

@mboersma try adding --v=5 --show-libmachine-logs to your start command. That will print out debug level logs from the driver. That might help figure out the issue

Holler if you need help. 😃

Sent from my iPhone SE

On Oct 4, 2016, at 10:00 AM, Puja notifications@github.com wrote:

@annismckenzie thanks, that makes sense, will try recompiling the plugin from source.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.