colima: 0.6.7 - Initialization of base image fails / stuck at "expanding disk" - macOS Intel with vmtype vz and virtioifs

Description

After updating to colima 0.6.7 something did not work upon start, so I did the usual

colima stop
colima delete
colima prune -a

# Even did additionally:
brew remove colima lima qemu
brew install colima
brew autoremove
# reboot and reinstall

Then I did a colima start with vm-type vz and virtiofs and the new image failed to load successfully. It got stuck at expanding disk image. After this happened the mac got stuck on shutdown/reboot as well and only a force shutdown worked.

I checked the folder ~/.colima/_lima/colima and it seems the image conversion might have failed or was missing a step as it was called diffdisk.<random>.tmp instead of just diffdisk.

I tried not specifying the vz type and virtiofs and it started successfully in QEMU and sshfs mode.

I then uninstalled 0.6.7 and manually downloaded 0.6.6, did a colima start there and it worked fine. I then removed the 0.6.6 colima binary and installed 0.6.7 through brew again and now a start works as well. so it seems the issuue is indeed with 0.6.7 and image initialization. maybe the image has a bug?

I originally had 13.x installed and tried the upgrade to 14.x to check if that might solve the issue. it did not, so the issue occured in both scenarios.

Version

colima version 0.6.7 git commit: ba1be00e9aec47f2c1ffdacfb7e428e465f0b58a

runtime: docker arch: x86_64 client: v24.0.7 server: v24.0.7

limactl version 0.19.0

Operating System

  • macOS Intel <= 13 (Ventura)
  • macOS Intel >= 14 (Sonoma)
  • Apple Silicon <= 13 (Ventura)
  • Apple Silicon >= 14 (Sonoma)
  • Linux

Output of colima status

INFO[0000] colima is running using macOS Virtualization.Framework INFO[0000] arch: x86_64 INFO[0000] runtime: docker INFO[0000] mountType: virtiofs

Reproduction Steps

see description.

Expected behaviour

colima starts even after delete on intel mac with 0.6.7

Additional context

No response

About this issue

  • Original URL
  • State: closed
  • Created 3 years ago
  • Reactions: 8
  • Comments: 43 (10 by maintainers)

Most upvoted comments

Same issue here on an older intel MacOS Ventura 13.6.4, with colima 0.6.7, if i recall correctly the version 0.6.6 had the same issue. I was able to circumvent the issue by downgrading to colima 0.6.5, start the vm, stop it and then upgrade

Here is how if it helps someone

brew uninstall colima && rm -rf ~/.colima # Removing the folder just in case I don´t think it is necessary

# this is the formulae for the version 0.6.5
curl https://raw.githubusercontent.com/Homebrew/homebrew-core/c3c2a6daa274d5f4b9cc7bb3d25b3cfde50b4e0a/Formula/c/colima.rb -o /tmp/colima.rb 

brew install --formulae /tmp/colima.rb
colima start --vm-type=vz --mount-type=virtiofs --disk 60
colima stop

brew update && brew upgrade

The only solution I found was to temporarily downgrade to 0.6.6, do the start there, then reinstall 0.6.7.

So something like this - probably not all deletions etc. are necessary but to be safe I did them multiple times:

colima stop -f
# Reboot macOS - warning most likely macOS will freeze during reboot and require a force shutdown.
colima delete
colima prune -a
rm -rf ~/.lima
rm -rf ~/.colima
rm -rf ~/Library/Caches/colima
rm -rf ~/Library/Caches/lima
brew uninstall colima
curl -o ~/colima -L https://github.com/abiosoft/colima/releases/download/v0.6.6/colima-Darwin-x86_64
chmod +x ~/colima
~/colima version
check that it is indeed version 0.6.6
~/colima delete
~/colima prune -a
~/colima start --vm-type vz --mount-type virtiofs ….  # add all necessary parameters.
~/colima stop
rm ~/colima
brew install colima
colima start
colima version

v0.6.9 has been released and should be available on homebrew shortly.

This has been fixed in the latest head. brew install --head colima.

It seems the issue is due to the way the Colima disk image was compressed. With compression disabled, it works as desired. The only downside is heavier download during first startup.

What fixed my issues was brew uninstall colima lima qemu rm -rf ~/.colima ~/.lima restarted macOS ‘brew install --HEAD colima’

This worked for me on an M3 with 14.3.1 (23D60).

I am seeing the same error on an M2 air with 0.6.8. Is there any workaround for this?

I had the same issue and using brew install --head colima fixed it, any chances of a new release soon?

Yeah, over the weekend.

Same error with colima 0.6.8 on M3 Pro on Sonoma 14.3. Well, it was hanging on the “expanding disk” step when I did colima start immediately after brew install colima. Not sure about the virtualization settings.

Colima now does this instead https://github.com/abiosoft/colima/issues/930#issuecomment-1855365096 as a permanent fix and would be part of the imminent next release.

The disk image downloads are smaller again but without the disk expansion issues.

Anyone can try the development version with brew install --head colima.

I also found that I was able to get past this by using --disk 60 instead of --disk 100. I have plenty of disk. I have no idea why that would be an issue.

I also noted that during several failures I saw that there were zombie instances of limactl that could not be killed, so when I saw them (ps -ef |grep lima) I had to restart to get rid of them.

Works on M1 Pro, but doesn’t work on M3 for me

Stopped working on M1 as well. During this phase: > 3.50 GiB / 3.50 GiB (100.00%) 393.94 MiB/stime=“2024-02-12T10:07:40+01:00” level=info msg=“Expanding to 100GiB”

It eats all my HDD space. My settings is:

# Default: 2
cpu: 4
 
# Size of the disk in GiB to be allocated to the virtual machine.
# NOTE: changing this has no effect after the virtual machine has been created.
# Default: 60
disk: 100
 
# Size of the memory in GiB to be allocated to the virtual machine.
# Default: 2
memory: 8
 
# Architecture of the virtual machine (x86_64, aarch64, host).
# Default: host
arch: aarch64
 
# Container runtime to be used (docker, containerd).
# Default: docker
runtime: docker

...
 
# Virtual Machine type (qemu, vz)
# NOTE: this is macOS 13 only. For Linux and macOS <13.0, qemu is always used.
#
# vz is macOS virtualization framework and requires macOS 13
#
# Default: qemu
vmType: vz

# Utilise rosetta for amd64 emulation (requires m1 mac and vmType `vz`)
# Default: false
rosetta: true
 
# Volume mount driver for the virtual machine (virtiofs, 9p, sshfs).
#
# virtiofs is limited to macOS and vmType `vz`. It is the fastest of the options.
#
# 9p is the recommended and the most stable option for vmType `qemu`.
#
# sshfs is faster than 9p but the least reliable of the options (when there are lots
# of concurrent reads or writes).
#
# Default: virtiofs (for vz), sshfs (for qemu)
mountType: virtiofs```

Can confirm that using brew install --HEAD colima worked for a few people in my organization.

It seems to specifically be people that are on M2/3 that are affected. Intel and M1 seems fine for the most part.

Using brew install --HEAD colima worked for me on a fresh M3

Same issue on Apple Silicon M2 running MacOS Sonoma 14.3.1 and Colima 0.6.8 but working fine with Colima 0.6.5.

When having this problem I also see caching behavior: Using cache "/Users/rfay/Library/Caches/lima/download/by-url-sha256/63f6e459cc61986ae586c5402c4e6e95ddc9073986f2a766ff662f19e0151407/data"

So best to rm -rf ~/Library/Caches/lima as well.

None of the above solutions worked for me (M2 Mac). I switched to Lima, which worked on the first try.

limactl start \
  --name=default \
  --cpus=4 \
  --memory=8 \
  --vm-type=vz \
  --rosetta \
  --mount-type=virtiofs \
  --mount-writable \
  --network=vzNAT \
  template://docker

Got the same issue

@abiosoft I did one more test and the following did indeed work (using limactl manually):

download and convert image manually to raw using qemu-img

curl -o ubuntu-23.10-minimal-cloudimg-amd64.qcow2 -L https://github.com/abiosoft/colima-core/releases/download/v0.6.7/ubuntu-23.10-minimal-cloudimg-amd64.qcow2
qemu-img convert ubuntu-23.10-minimal-cloudimg-amd64.qcow2 ubuntu-23.10-minimal-cloudimg-amd64.raw

and then using the following lime config I was able to successfully create the instance:

images:
  - location: './ubuntu-23.10-minimal-cloudimg-amd64.raw'
    arch: 'x86_64'
vmType: 'vz'
mountType: 'virtiofs'

so I guess the issue is indeed lima qcow2 auto-conversion to raw image.