minishift: minishift start fails with various errors when ~/.kube exists

General information

• Minishift version: v1.26.1+1e20f27
• OS: macOS
• Hypervisor: VirtualBox

May be related to #1827

Steps to reproduce

This is a pretty hard to reproduce and strange error but I wanted to post something as I’ve been battling with this for over a week now, only just finding out the fix for it yesterday.

To begin, I think this may have something to do with an existing minikube, kubectx, or oc setup conflicting with minishift. I was using all of these tools prior to installing minishift.

When I first started, the main errors I was getting like:

-- Starting OpenShift cluster -- Running 'oc' with: 'cluster up --base-dir /var/lib/minishift/base --image 'openshift/origin-${component}:v3.10.0' --public-hostname 192.168.99.100 --routing-suffix 192.168.99.100.nip.io'
..............................................................................................................................Error during 'cluster up' execution: Error starting the cluster. ssh command error:
command : /var/lib/minishift/bin/oc cluster up --base-dir /var/lib/minishift/base --image 'openshift/origin-${component}:v3.10.0' --public-hostname 192.168.99.100 --routing-suffix 192.168.99.100.nip.io
err     : exit status 1

and

E1115 04:17:43.418019 2450 run_self_hosted.go:571] API server error: Get https://192.168.99.100:8443/healthz?timeout=32s: dial tcp 192.168.99.100:8443: connect: connection refused ()

After a few days of trying everything (stopping, deleting, restarting, deleting ~/.minisihft etc). I restarted my PC and managed to get it to run through. BUT there was another error at the end of the output:

Could not set oc CLI context for 'minishift' profile: Error during setting 'minishift' as active profile: The specified path to the kube config '/Users/adam/.minishift/machines/minishift_kubeconfig' does not exist

I found other issues similar to this, but nothing with that exact error.

I decided to try my luck and see if the cluster would work anyway, and it seemed that it did. The developer user could create deploymentconfigs, routes, etc and I could test an application that I was using, so I didn’t look too much more into it. However, if I stopped my cluster I would run into those connection errors again, resulting in having to delete the vm and start again.

It wasn’t until I was trying to test out https://heptio.github.io/ark/v0.10.0/ that I saw that things weren’t quite right. When trying to login as system:admin I was getting the same errors as #2601 in that it wasn’t recognising system:admin as a service account.

What finally fixed all of this, and allowed both the minishift_kubeconfig file to be written, and fixed all the connection issues and allowed my to login with system:admin was to:

minishift stop
minishift delete --force --clear-cache
mv ~/.kube ~/.kube.old
minishift start

I think the main thing that fixed it there is moving the whole .kube folder out. Interestingly I had tried mv ~/.kube/config ~/.kube/config.old before this and that did not fix it. So it was something other than the config file in that folder that was messing with minishift.

I still have that old folder, and the old config but I’m not sure what would help in diagnosing this issue.

I don’t know if this is anecdotal and some weird issue with my setup, but there seems to be a lot of similar issues around this kind of stuff but nothing exactly like this.

Let me know if you would like any more logs or file contents.