colima: Colima doesn't start after OS update, fails with Waiting for the essential requirement 1 of 5: "ssh"

Description

After upgrading my mac to Ventura, colima start fails with below error.

`INFO[0000] starting colima INFO[0000] runtime: docker INFO[0000] preparing network … context=vm INFO[0000] starting … context=vm

[hostagent] Waiting for the essential requirement 1 of 5: “ssh” [hostagent] Waiting for the essential requirement 1 of 5: “ssh” [hostagent] Waiting for the essential requirement 1 of 5: “ssh” [hostagent] Waiting for the essential requirement 1 of 5: “ssh” [hostagent] Waiting for the essential requirement 1 of 5: “ssh” did not receive an event with the “running” status FATA[0600] error starting vm: error at ‘starting’: exit status 1`

`colima list

PROFILE STATUS ARCH CPUS MEMORY DISK RUNTIME ADDRESS

default Running x86_64 2 4GiB 60GiB

delete Stopped x86_64 8 22GiB 60GiB

list Stopped x86_64 8 22GiB 60GiB`

Below is the error in stderr.log

{"level":"debug","msg":"Creating iso file /Users/test/.lima/colima/cidata.iso","time":"2023-02-21T16:48:52+01:00"} {"level":"debug","msg":"Using /var/folders/wt/f_t9xy6s0f173r1qyfw27m_c0000gr/T/diskfs_iso1691064567 as workspace","time":"2023-02-21T16:48:52+01:00"} {"level":"debug","msg":"OpenSSH version 9.0.1 detected","time":"2023-02-21T16:48:52+01:00"} {"level":"debug","msg":"AES accelerator seems available, prioritizing openssh.com","time":"2023-02-21T16:48:52+01:00"} {"level":"debug","msg":"firmware candidates = [/Users/test/.local/share/qemu/edk2-x86_64-code.fd /Users/test/.colima/_wrapper/***/share/qemu/edk2-x86_64-code.fd /usr/share/OVMF/OVMF_CODE.fd /usr/share/qemu/ovmf-x86_64-code.bin /usr/share/edk2-ovmf/x64/OVMF_CODE.fd]","time":"2023-02-21T16:48:53+01:00"} {"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see \"/Users/test/.lima/colima/serial.log\")","time":"2023-02-21T16:48:53+01:00"} {"level":"debug","msg":"qCmd.Args: [/Users/test/.colima/_wrapper/***/bin/qemu-system-x86_64 -m 4096 -cpu qemu64 -machine q35,vmport=off -accel tcg,thread=multi,tb-size=512 -global ICH9-LPC.disable_s3=1 -smp 2,sockets=1,cores=2,threads=1 -drive if=pflash,format=raw,readonly=on,file=/Users/test/.colima/_wrapper/***/edk2-x86_64-code.fd -boot order=d,splash-time=0,menu=on -drive file=/Users/test/.lima/colima/basedisk,media=cdrom,readonly=on -drive file=/Users/test/.lima/colima/diffdisk,if=virtio,discard=on -cdrom /Users/test/.lima/colima/cidata.iso -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:51407-:22 -device virtio-net-pci,netdev=net0,mac=*** -device virtio-rng-pci -display none -device virtio-vga -device virtio-keyboard-pci -device virtio-mouse-pci -device qemu-xhci,id=usb-bus -parallel none -chardev socket,id=char-serial,path=/Users/test/.lima/colima/serial.sock,server=on,wait=off,logfile=/Users/test/.lima/colima/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/test/.lima/colima/qmp.sock,server=on,wait=off -qmp chardev:char-qmp -name lima-colima -pidfile /Users/test/.lima/colima/qemu.pid]","time":"2023-02-21T16:48:53+01:00"} {"level":"info","msg":"Waiting for the essential requirement 1 of 5: \"ssh\"","time":"2023-02-21T16:48:53+01:00"} {"level":"debug","msg":"executing script \"ssh\"","time":"2023-02-21T16:48:53+01:00"} {"level":"debug","msg":"executing ssh for script \"ssh\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/test/.lima/_config/user\" -o IdentityFile=\"/Users/test/.ssh/id_rsa\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"***" -o User=test -o ControlMaster=auto -o ControlPath=\"/Users/test/.lima/colima/ssh.sock\" -o ControlPersist=5m -p 51407 127.0.0.1 -- /bin/bash]","time":"2023-02-21T16:48:53+01:00"} {"level":"debug","msg":"stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 51407: Connection refused\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 51407: Connection refused\\r\\n\": exit status 255","time":"2023-02-21T16:48:53+01:00"} {"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: -accel tcg,thread=multi,tb-size=512: warning: Guest expects a stronger memory ordering than the host provides","time":"2023-02-21T16:48:53+01:00"} {"level":"debug","msg":"qemu[stderr]: This may cause strange/hard to debug errors","time":"2023-02-21T16:48:53+01:00"} {"level":"info","msg":"Waiting for the essential requirement 1 of 5: \"ssh\"","time":"2023-02-21T16:49:03+01:00"} {"level":"debug","msg":"executing script \"ssh\"","time":"2023-02-21T16:49:03+01:00"} {"level":"debug","msg":"executing ssh for script \"ssh\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/test/.lima/_config/user\" -o IdentityFile=\"/Users/test/.ssh/id_rsa\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm@openssh.com,aes256-gcm@openssh.com\" -o User=staliyan001 -o ControlMaster=auto -o ControlPath=\"/Users/test/.lima/colima/ssh.sock\" -o ControlPersist=5m -p 51407 127.0.0.1 -- /bin/bash]","time":"2023-02-21T16:49:03+01:00"} {"level":"debug","msg":"stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\nConnection reset by 127.0.0.1 port 51407\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"kex_exchange_identification: read: Connection reset by peer\\r\\nConnection reset by 127.0.0.1 port 51407\\r\\n\": exit status 255","time":"2023-02-21T16:50:17+01:00"} {"level":"info","msg":"Waiting for the essential requirement 1 of 5: \"ssh\"","time":"2023-02-21T16:50:27+01:00"} {"level":"debug","msg":"executing script \"ssh\"","time":"2023-02-21T16:50:27+01:00"} {"level":"debug","msg":"executing ssh for script \"ssh\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/test/.lima/_config/user\" -o IdentityFile=\"/Users/test/.ssh/id_rsa\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"***" -o User=test -o ControlMaster=auto -o ControlPath=\"/Users/test/.lima/colima/ssh.sock\" -o ControlPersist=5m -p 51407 127.0.0.1 -- /bin/bash]","time":"2023-02-21T16:50:27+01:00"}

Version

Colima Version:0.5.2 Lima Version:0.14.2 Qemu Version:7.2.0

Operating System

  • macOS Intel <= 12 (Monterrey)
  • macOS Intel >= 13 (Ventura)
  • macOS M1 <= 12 (Monterrey)
  • macOS M1 >= 13 (Ventura)
  • Linux

Output of colima status

colima status FATA[0074] error retrieving current runtime: empty value

Reproduction Steps

1.colima start --cpu 8 --memory 22 --arch amd 2. 3.

Expected behaviour

Colima should start without errors and colima status should not throw any errors

Additional context

I tried reinstalling colima with --HEAD, still the same error. Also it starts with command colima start --arch aarch64, but obviously i get below error while starting my container. The requested image’s platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested

About this issue

  • Original URL
  • State: open
  • Created a year ago
  • Comments: 43 (9 by maintainers)

Most upvoted comments

I regularly experience the same issue on an M1 Mac (ARM arch.) with no emulation. Occassionally, deleting the colima profile colima delete default and then colima start is sometimes enough to get the SSH part to work. Typically though, I’ve found that removing ~/.lima and ~/.colima and then colima start again works but this might be a fluke. It’s a bit disappointing that I need to rebuild all my docker containers regularly. This problem also seems to happen every time I reboot my machine so might leave it running for a few days 😉.

Hopefully this helps someone in the meantime.

Edit: just to ask, do I need QEMU if I’m running m1 based images on colima? Curious to learn a bit more about how Colima works.

+4
vsjoshtait on Apr 17, 2023

I’ve also had the issue and found that it doesn’t happen if you delete any prior ‘profile’ in the ~/.lima folder. It should only contain the _config file, nothing else, and everything works smoothly. Here are the steps I followed: A. clearing the ‘profile’ folders in ~/.lima

  1. rm -rf ~/.colima (if folder exists)
  2. cd ~/.lima
  3. mv _config …/
  4. rm -rf *
  5. mv …/_config . B. Starting a fresh instance
  6. colima start --cpu 4 --memory 8 ( no need for the arch options )
  7. colima status (to check that it all works)

A new folder ~/.colima is created with configs and .sock.

Hope this help.

+4
tomwaits on Apr 11, 2023

@ultimike have they tried stopping with the --force flag? Are they emulating x86 on m1 device?

colima stop --force
+2
abiosoft on Mar 27, 2023

I had several issues setting up Colima.

Colima Version: 0.6.6 Docker Version: 24.0.7 OS: Mac M3 running Sonoma 14.1.1

Issue 1

Cannot run docker cli commands due to not being able to connect to Docker daemon. Error: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Attempted Fix Which did NOT Work: Set environment variable DOCKER_HOST: export DOCKER_HOST="unix://$HOME/.colima/default/docker.sock"

Issue 2

Starting colima is stuck in Waiting for the essential requirement 1 of 4: "ssh" Error:

$ colima start
INFO[0000] starting colima
INFO[0000] runtime: docker
INFO[0001] creating and starting ...                     context=vm
> [hostagent] Starting QEMU (hint: to watch the boot progress, see "~/.colima/_lima/colima/serial*.log")
> SSH Local Port: 53987
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"

The Fix

What eventually worked for me is:

  1. Delete default colima profile: colima delete default
  2. Delete ~/.colima: rm -rf ~/.colima
  3. Start colima with vm-type parameter: colima start --vm-type vz
  4. Colima status
    $ colima status
    INFO[0000] colima is running using macOS Virtualization.Framework
    INFO[0000] arch: aarch64
    INFO[0000] runtime: docker
    INFO[0000] mountType: virtiofs
    INFO[0000] socket: unix://$HOME/.colima/default/docker.sock

P.S.

I installed docker, docker-compose, docker-buildx using Homebrew BEFORE installing colima

This works for me:

Delete default colima profile: colima delete default Delete ~/.colima: rm -rf ~/.colima Start colima with vm-type parameter: colima start --vm-type vz

+1
ItsCosmas on Dec 11, 2023

I had several issues setting up Colima.

Colima Version: 0.6.6 Docker Version: 24.0.7 OS: Mac M3 running Sonoma 14.1.1

Issue 1

Cannot run docker cli commands due to not being able to connect to Docker daemon. Error: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Attempted Fix Which did NOT Work: Set environment variable DOCKER_HOST: export DOCKER_HOST="unix://$HOME/.colima/default/docker.sock"

Issue 2

Starting colima is stuck in Waiting for the essential requirement 1 of 4: "ssh" Error:

$ colima start
INFO[0000] starting colima
INFO[0000] runtime: docker
INFO[0001] creating and starting ...                     context=vm
> [hostagent] Starting QEMU (hint: to watch the boot progress, see "~/.colima/_lima/colima/serial*.log")
> SSH Local Port: 53987
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"

The Fix

What eventually worked for me is:

  1. Delete default colima profile: colima delete default
  2. Delete ~/.colima: rm -rf ~/.colima
  3. Start colima with vm-type parameter: colima start --vm-type vz
  4. Colima status
    $ colima status
    INFO[0000] colima is running using macOS Virtualization.Framework
    INFO[0000] arch: aarch64
    INFO[0000] runtime: docker
    INFO[0000] mountType: virtiofs
    INFO[0000] socket: unix://$HOME/.colima/default/docker.sock

P.S.

I installed docker, docker-compose, docker-buildx using Homebrew BEFORE installing colima

+1
kimberlyamandalu on Nov 27, 2023

@tdy your issue seems to be something else, likely related to QEMU.

Can you try using the vz driver?

colima start --vm-type vz
+1
abiosoft on Nov 13, 2023
Read more comments on GitHub
← colima: Cannot chown or chmod bind mounted files/dirs
colima: Colima hangs a short while after starting →