minikube: hyperv: "minikube stop" fails (Shutting down 0% until it eventually times out)

BUG REPORT Please provide the following details:

Environment: Windows 10 Pro 1803

Minikube version (use minikube version): v0.28.0

  • OS (e.g. from /etc/os-release): Windows 10 Pro 1803
  • VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): Hyper-v
  • ISO version (e.g. cat ~/.minikube/machines/minikube/config.json | grep -i ISO or minikube ssh cat /etc/VERSION): v0.28.0
  • Install tools:
  • Others:

What happened: Minikube stop fails to stop minikube VM

What you expected to happen: VM stops

How to reproduce it (as minimally and precisely as possible): With running minikube, perform minikube stop

Output of minikube logs (if applicable):

Anything else do we need to know: Clean install of Minikube 0.28.0 (deleted .minikube folder) Command sits with powershell overlay showing Shutting down 0% until it eventually times out Trying to shutdown via Hyper-V manager/Powershell doesn’t work either Opening VM in hyper-v shows command prompt - may not be relevant? image

About this issue

  • Original URL
  • State: closed
  • Created 6 years ago
  • Reactions: 49
  • Comments: 52 (18 by maintainers)

Most upvoted comments

Quite annoying indeed. I’ve found

$ minikube ssh
$ sudo poweroff

to be the fastest workaround.

You can combine the commands to issue a shutdown in one line:

minikube ssh "sudo poweroff"

Same environment and same problem here. Also, if I try to stop minikube from Hyper-V Manager I get the following error:

image

Still happening here as well…

OS: Windows 10 Pro (17134) VM-Driver: HyperV Minikube: v.0.31.0 (just updated a couple of days again)

Using minikube ssh to do a shutdown is getting pretty old… Any chance of a fix for the new year?

Is there an update to this issue?

Experienced this with minikube version: v0.32.0

Thanks for the minikube ssh | sudo poweroff workaround for this issue - even telling Hyper-V to shut the device down did not work (Presumably because it was in a frozen // not responding state).

Is there any further information those of us who have this problem can provide to assist in resolving? I’ve not seen anything particularly nasty in the event logs or similar…


Edit I have noticed that the sudo shutdown in minikube 0.27.0 does an immediate shutdown, whereas in minikube 0.28.0, it schedules the shutdown for one minute in the future. I don’t know if the infrastructure issues a sudo shutdown in the background, but if it does and gets into a hang or race condition with itself re-issuing the command, it will never actually shutdown…

Downgrading minikube from 0.28.2 to 0.27 solves everything. No need to remove directory ~/.minikube, just run minikube delete before starting again.

minikube delete
minikube config set bootstraper localkube
minikube config set kubernetes-version v1.10.0
minikube config set vm-driver hyperv
minikube config set hyperv-virtual-switch <your-vswitch-name>

Downgrading with chocolatey:

choco install -y --force minikube --version 0.27

Start

minikube start

i can confirm this is occuring with windows 10 pro 1703 with minikube .28 as well. My error message is:

C:\WINDOWS\system32>minikube stop minikube Stopping local Kubernetes cluster… Error stopping machine: Error stopping host: minikube: exit status 1

I will try to update this pc and see if that works, if not i will use the suggested downgrade for minikube.

upgrading to 1803 with Minikube v28 had the same results downgrading to Minikube v27 works as normal

yeah, I deleted my ~/.minikube folder + minikube virtual machine and run process : minikube start --vm-driver=x --kubernetes-version=“vx.xx.x” and works 👍

Still happening on v0.35.0

Same here:

OS: Windows 10 Pro (17134) VM-Driver: HyperV Minikube: v.0.30.0

I can shut it down with: minikube ssh "sudo poweroff"

I downgraded to 0.27 as follows:

minikube ssh "sudo poweroff"
minikube delete
choco uninstall minikube
choco install minikube --version 0.27
minikube start --vm-driver hyperv --hyperv-virtual-switch "Primary Virtual Switch"

I can then stop it with: minikube stop

Same problem here

Windows 10 Pro 1903 Docker v18.09.2 Kubectl v1.15.0 Minikube v1.2.0

Output from minikube stop:

* Stopping "minikube" in hyperv ...
* Stopping "minikube" in hyperv ...
* Stopping "minikube" in hyperv ...
* Stopping "minikube" in hyperv ...
* Stopping "minikube" in hyperv ...

X Unable to stop VM: Temporary Error: Stop: minikube: exit status 1
Temporary Error: Stop: minikube: exit status 1
Temporary Error: Stop: minikube: exit status 1
Temporary Error: Stop: minikube: exit status 1
Temporary Error: Stop: minikube: exit status 1

* Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
  - https://github.com/kubernetes/minikube/issues/new

Output from minikube stop --alsologtostderr -v=8:

I0702 14:50:50.622640   20504 notify.go:128] Checking for updates...                                                                                            * Stopping "minikube" in hyperv ...                                                                                                                             Stopping "minikube"...
[executing ==>] : C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM minikube ).state
[stdout =====>] : Running

[stderr =====>] :
[executing ==>] : C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive Hyper-V\Stop-VM minikube
[stdout =====>] :
[stderr =====>] : Hyper-V\Stop-VM : Der Status von "minikube" konnte nicht geändert werden.
In Zeile:1 Zeichen:1
+ Hyper-V\Stop-VM minikube
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Stop-VM], VirtualizationException
    + FullyQualifiedErrorId : OperationFailed,Microsoft.HyperV.PowerShell.Commands.StopVM
                                                                                                                                                                                                                                                                                                                                I0702 15:00:52.605171   20504 utils.go:126] error: Temporary Error: Stop: minikube: exit status 1 - sleeping 1s                                                 I0702 15:00:53.611320   20504 utils.go:115] retry loop 1                                                                                                        * Stopping "minikube" in hyperv ...                                                                                                                             Stopping "minikube"...
[executing ==>] : C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM minikube ).state
[stdout =====>] : Running

[stderr =====>] :
[executing ==>] : C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive Hyper-V\Stop-VM minikube
[stdout =====>] :
[stderr =====>] : Hyper-V\Stop-VM : Der Status von "minikube" konnte nicht geändert werden.
In Zeile:1 Zeichen:1
+ Hyper-V\Stop-VM minikube
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Stop-VM], VirtualizationException
    + FullyQualifiedErrorId : OperationFailed,Microsoft.HyperV.PowerShell.Commands.StopVM
                                                                                                                                                                                                                                                                                                                                I0702 15:10:55.198539   20504 utils.go:126] error: Temporary Error: Stop: minikube: exit status 1 - sleeping 1s                                                 I0702 15:10:56.201604   20504 utils.go:115] retry loop 2                                                                                                        * Stopping "minikube" in hyperv ...                                                                                                                             Stopping "minikube"...
[executing ==>] : C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM minikube ).state
[stdout =====>] : Running

[stderr =====>] :
[executing ==>] : C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive Hyper-V\Stop-VM minikube
[stdout =====>] :
[stderr =====>] : Hyper-V\Stop-VM : Der Status von "minikube" konnte nicht geändert werden.
In Zeile:1 Zeichen:1
+ Hyper-V\Stop-VM minikube
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Stop-VM], VirtualizationException
    + FullyQualifiedErrorId : OperationFailed,Microsoft.HyperV.PowerShell.Commands.StopVM


I0702 15:20:57.826478   20504 utils.go:126] error: Temporary Error: Stop: minikube: exit status 1 - sleeping 1s
I0702 15:20:58.831673   20504 utils.go:115] retry loop 3
* Stopping "minikube" in hyperv ...
Stopping "minikube"...
[executing ==>] : C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM minikube ).state
[stdout =====>] : Running

[stderr =====>] :
[executing ==>] : C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive Hyper-V\Stop-VM minikube
[stdout =====>] :
[stderr =====>] : Hyper-V\Stop-VM : Der Status von "minikube" konnte nicht geändert werden.
In Zeile:1 Zeichen:1
+ Hyper-V\Stop-VM minikube
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Stop-VM], VirtualizationException
    + FullyQualifiedErrorId : OperationFailed,Microsoft.HyperV.PowerShell.Commands.StopVM


I0702 15:31:00.199707   20504 utils.go:126] error: Temporary Error: Stop: minikube: exit status 1 - sleeping 1s
I0702 15:31:01.204811   20504 utils.go:115] retry loop 4
* Stopping "minikube" in hyperv ...
Stopping "minikube"...
[executing ==>] : C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive ( Hyper-V\Get-VM minikube ).state
[stdout =====>] : Running

[stderr =====>] :
[executing ==>] : C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -NonInteractive Hyper-V\Stop-VM minikube
[stdout =====>] :
[stderr =====>] : Hyper-V\Stop-VM : Der Status von "minikube" konnte nicht geändert werden.
In Zeile:1 Zeichen:1
+ Hyper-V\Stop-VM minikube
+ ~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Stop-VM], VirtualizationException
    + FullyQualifiedErrorId : OperationFailed,Microsoft.HyperV.PowerShell.Commands.StopVM


I0702 15:41:02.817565   20504 utils.go:126] error: Temporary Error: Stop: minikube: exit status 1 - sleeping 1s
W0702 15:41:03.819005   20504 exit.go:100] Unable to stop VM: Temporary Error: Stop: minikube: exit status 1
Temporary Error: Stop: minikube: exit status 1
Temporary Error: Stop: minikube: exit status 1
Temporary Error: Stop: minikube: exit status 1
Temporary Error: Stop: minikube: exit status 1

X Unable to stop VM: Temporary Error: Stop: minikube: exit status 1
Temporary Error: Stop: minikube: exit status 1
Temporary Error: Stop: minikube: exit status 1
Temporary Error: Stop: minikube: exit status 1
Temporary Error: Stop: minikube: exit status 1

* Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
  - https://github.com/kubernetes/minikube/issues/new

I have exactly the same issue on Windows 10 pro. Stopping the cluster with poweroff worked, but then the cluster couldn’t be started again. I had to uninstall, remove ~/.minikube and install version 0.27 for everything to work.

Just had a scare with this when the kube wouldn’t stop. Just followed the simple steps from the tutorial. Thought a computer restart would do it, but it got stuck at “Restarting your computer” for about 10 minutes. Luckily, though, in the end, it actually restarted.

Experienced this with minikube version: v0.32.0

Same, but after reading further up it seems the developers already know this. For anyone going to the bottom of the thread first, see these workarounds:

Minkube uses docker-machin to issue a shutdown using the hypervisor management tools. this would be acpi call. Surely minikube could use an sshcommand instead, but the issue is with submachine… And this library is in maintenance mode. I also haven’t seen this issue with our fork yet… Which we are in the process of adding to minikube. I’ll keep an eye on this issue and see if the library change fixes this.

On Aug 23, 2018 09:39, “Felix Mattick” notifications@github.com wrote:

Could minikube just be updated to run that command for minikube stop?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/kubernetes/minikube/issues/2914#issuecomment-415248809, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAHZqijtlxNuNFf-pAjewnl8leEiwSmks5uTgfCgaJpZM4Utd8V .

Just rolled up a minishift environment, based on minishift version v1.21.0+a8c8b37, and that is capable of successfully shutting down using minishift stop.

Based on other comments, I also tried the Stop-VM minikube command mentioned above. It puts a 0% progress-bar in the PowerShell window, and nothing continues. Looking in the SSH, I see there is a long-running -bash command:

$ sudo ps all
F   UID   PID  PPID PRI  NI    VSZ   RSS WCHAN  STAT TTY        TIME COMMAND
4     0  2052     1  20   0   9240   448 -      Ss+  tty1       0:00 /sbin/getty
4     0  2069     1  20   0   9240   452 -      Ss+  ttyS0      0:00 /sbin/getty
0  1001  4065  4064  20   0  16236  3432 -      Ss   pts/0      0:00 -bash
4     0  4267  4065  20   0  20316   688 -      R+   pts/0      0:00 ps all

That wasn’t there prior to the minikube stop command being triggered.

@gbraad I’ve certainly waited for over 1min (in fact over 5mins) and it doesn’t shutdown. My suspicions were around how Hyper-V interacts with the Linux VM but I couldn’t find anything useful to help with this. I did experiment with disabling the hyper-v integration services which includes an option for Guest OS shutdown and disabling this causes an immediate error.

image

I’m preparing for a talk tomorrow so don’t want to mess with my minikube installation today but can try and have another go at this later this week.