colima: Error starting with --network-address

Description

Attempting to start --network-address fails. Additional starts without the flag fail with a different error message. Removing ~/.colima allows for a successful start, without the network address.

Version

Colima Version: 0.4.6 Lima Version: 0.12.0, 0.13.0 Qemu Version: 7.1.0

Operating System

  • macOS Intel
  • macOS M1
  • Linux

Reproduction Steps

  1. colima start --network-address

Expected behaviour

Colima starts correctly.

Additional context

First start:

INFO[0000] starting colima
INFO[0000] runtime: docker
INFO[0000] preparing network ...                         context=vm
WARN[0015] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status default], output: "time=\"2022-10-27T09:57:15-04:00\" level=fatal msg=\"pid file not found: stat /Users/joel.johnson/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
INFO[0015] creating and starting ...                     context=vm
WARN[0015] error setting up routable IP address: vmnet ptp socket file not found: stat /Users/joel.johnson/.colima/default/daemon/vmnet.ptp: no such file or directory
> Using cache "/Users/joel.johnson/Library/Caches/lima/download/by-url-sha256/e9bac04e9bdb31be4d3de1506d97eb60d59d9ad1a2d97f2b21f760e06f3e4408/data"
> [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/joel.johnson/.lima/colima/serial.log")
> SSH Local Port: 57347
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] QEMU has exited
> exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/joel.johnson/.lima/colima/ha.stderr.log")
FATA[0017] error starting vm: error at 'creating and starting': exit status 1

ha.stderr.log:

{"level":"debug","msg":"Creating iso file /Users/joel.johnson/.lima/colima/cidata.iso","time":"2022-10-27T09:57:15-04:00"}
{"level":"debug","msg":"Using /var/folders/1n/fvnyc01554s0240g36s382jh6nvfqs/T/diskfs_iso413631167 as workspace","time":"2022-10-27T09:57:15-04:00"}
{"level":"debug","msg":"firmware candidates = [/Users/joel.johnson/.local/share/qemu/edk2-aarch64-code.fd /Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/share/qemu/edk2-aarch64-code.fd /usr/share/AAVMF/AAVMF_CODE.fd /usr/share/qemu-efi-aarch64/QEMU_EFI.fd]","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"OpenSSH version 8.6.1 detected","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com","time":"2022-10-27T09:57:17-04:00"}
{"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see \"/Users/joel.johnson/.lima/colima/serial.log\")","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"qCmd.Args: [/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/bin/qemu-system-aarch64 -m 2048 -cpu host -machine virt,accel=hvf,highmem=off -smp 2,sockets=1,cores=2,threads=1 -drive if=pflash,format=raw,readonly=on,file=/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/share/qemu/edk2-aarch64-code.fd -boot order=d,splash-time=0,menu=on -drive file=/Users/joel.johnson/.lima/colima/basedisk,media=cdrom,readonly=on -drive file=/Users/joel.johnson/.lima/colima/diffdisk,if=virtio,discard=on -cdrom /Users/joel.johnson/.lima/colima/cidata.iso -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:57347-:22 -device virtio-net-pci,netdev=net0,mac=52:55:55:56:07:0a -device virtio-rng-pci -display none -vga none -device ramfb -device qemu-xhci,id=usb-bus -device usb-kbd,bus=usb-bus.0 -device usb-mouse,bus=usb-bus.0 -parallel none -chardev socket,id=char-serial,path=/Users/joel.johnson/.lima/colima/serial.sock,server=on,wait=off,logfile=/Users/joel.johnson/.lima/colima/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/joel.johnson/.lima/colima/qmp.sock,server=on,wait=off -qmp chardev:char-qmp -name lima-colima -pidfile /Users/joel.johnson/.lima/colima/qemu.pid]","time":"2022-10-27T09:57:17-04:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 5: \"ssh\"","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"qemu[stderr]: time=\"2022-10-27T09:57:17-04:00\" level=fatal msg=\"dial unix /Users/joel.johnson/.colima/default/daemon/gvproxy.sock: connect: no such file or directory\"","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2022-10-27T09:57:17-04:00"}
{"error":"exit status 1","level":"info","msg":"QEMU has exited","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"executing ssh for script \"ssh\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/joel.johnson/.lima/_config/user\" -o IdentityFile=\"/Users/joel.johnson/.ssh/id_ed25519\" -o IdentityFile=\"/Users/joel.johnson/.ssh/id_rsa\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm@openssh.com,aes256-gcm@openssh.com\" -o User=lima -o ControlMaster=auto -o ControlPath=\"/Users/joel.johnson/.lima/colima/ssh.sock\" -o ControlPersist=5m -p 57347 127.0.0.1 -- /bin/bash]","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 57347: Connection refused\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 57347: Connection refused\\r\\n\": exit status 255","time":"2022-10-27T09:57:17-04:00"}

Second start:

INFO[0000] starting colima
INFO[0000] runtime: docker
INFO[0000] preparing network ...                         context=vm
WARN[0015] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status default], output: "time=\"2022-10-27T10:01:35-04:00\" level=fatal msg=\"pid file not found: stat /Users/joel.johnson/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
WARN[0015] error setting up routable IP address: vmnet ptp socket file not found: stat /Users/joel.johnson/.colima/default/daemon/vmnet.ptp: no such file or directory
INFO[0015] starting ...                                  context=vm
> Using the existing instance "colima"
> [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/joel.johnson/.lima/colima/serial.log")
> SSH Local Port: 57370
> [hostagent] QEMU has exited
> exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/joel.johnson/.lima/colima/ha.stderr.log")
FATA[0017] error starting vm: error at 'starting': exit status 1

which is obviously due to the colima.yaml being updated with the --network-address flag.

About this issue

  • Original URL
  • State: open
  • Created 2 years ago
  • Reactions: 4
  • Comments: 50 (12 by maintainers)

Most upvoted comments

We were having this issue and discovered that having a multipass VM running causes this behavior. I assume that other VM hosts could cause a similar conflict.

I am also having the same problem, my logs are exactly the same however wiping the directories and reinstalling from HEAD works “intermittently” as described above but to the negative. Meaning: if I blow it all away and start over sometimes it errors out completely and sometimes it starts with no routable IP, I am not able to get a valid IP at all.

@abiosoft This works! Thank you! Steps followed: clean everything:

brew uninstall qemu lima colima docker docker-compose
brew autoremove
brew cleanup
sudo rm -rf /opt/colima
rm -rf /opt/homebrew/etc/gnutls
rm -rf /opt/homebrew/etc/unbound
rm -rf .lima .colima .docker

starts from scratch:

brew install qemu lima docker docker-compose
mkdir -p ~/.docker/cli-plugins
ln -sfn /opt/homebrew/opt/docker-compose/bin/docker-compose ~/.docker/cli-plugins/docker-compose
brew install --HEAD colima
colima start --arch x86_64 --memory 4 --network-driver slirp --network-address oraxe
colima list                                                                         
PROFILE    STATUS     ARCH      CPUS    MEMORY    DISK     RUNTIME    ADDRESS
oraxe      Running    x86_64    2       4GiB      60GiB    docker     192.168.106.2

Hello guys,

To whom it may concern:

colima version 0.5.5, arch: aarch64, macbook air m2, vz/virtiofs

With --network-address option set, colima starts a virtual machine, and bridges col0 interface with bridge100, which has ip address 192.168.106.1. It seems that bridge100 interface is a special one used for “internet sharing”, which means that there should be a DHCP server running. However, if for some reason “internet sharing” is disabled, then there is no DHCP server on bridge100 interface, and therefore, lima’s virtual machine is unable to acquire IP address, and therefore k3s kubernetes fails to start, which causes colima error “The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?”

The solution is to start dhcp server on bridge100 serving 192.168.106.* range, e.g., dnsmasq, which might require disabling macos builtin server bootps.plist, due to port conflict.

Certainly not the fix, but this works as a workaround for me when colima status shows no Network Address:

  1. Start your colima VM with whatever command you are using (ensuring you include --network-address option (Wait for it to start) I confirm it’s started with no network address by running:
> colima status
INFO[0000] colima is running using macOS Virtualization.Framework
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: virtiofs
INFO[0000] address:
INFO[0000] socket: unix:///Users/username/.colima/default/docker.sock
  1. On my Mac, there should now be a network bridge adapter created - mine is always bridge100 - grab it’s IP address by running:
> ifconfig bridge100 | grep "inet "
	inet 192.168.106.1 netmask 0xffffff00 broadcast 192.168.106.255
  1. colima ssh

(you are now in SSH inside VM)

  1. sudo su -
  2. Edit the file /etc/network/interfaces so that it looks similar to the below, using the IP address returned in step 2 as the value for gateway:
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

auto col0
iface col0 inet static
   address 192.168.106.2
   netmask 255.255.255.0
   gateway 192.168.106.1
  1. Run ifdown col0
  2. Run ifup col0
  3. Exit ssh in colima by pressing CTRL + D twice

This results in the VM having the IP 192.168.106.2 - validated by again running colima status:

> colima status
INFO[0000] colima is running using macOS Virtualization.Framework
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: virtiofs
INFO[0000] address: 192.168.106.2
INFO[0000] socket: unix:///Users/username/.colima/default/docker.sock

For me the issue persists M1 Ventura, latest colima version

WARN[0181] error setting up reachable IP address: vmnet socket file not found: stat /…/.colima/default/daemon/vmnet.sock: no such file or directory

Instructions havent changed the result.

Hi! I have the same error when using --network-address

colima start --network-address
INFO[0000] starting colima                              
INFO[0000] runtime: docker                              
INFO[0000] preparing network ...                         context=vm
WARN[0016] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status default], output: "time=\"2023-04-10T23:27:19-03:00\" level=fatal msg=\"pid file not found: stat /Users/jandresen/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
WARN[0016] error setting up reachable IP address: vmnet socket file not found: stat /Users/jandresen/.colima/default/daemon/vmnet.sock: no such file or directory 
INFO[0016] starting ...                                  context=vm
INFO[0039] provisioning ...                              context=docker
INFO[0040] starting ...                                  context=docker
INFO[0046] done                                         

colima list
PROFILE    STATUS     ARCH       CPUS    MEMORY    DISK     RUNTIME    ADDRESS
default    Running    aarch64    2       2GiB      60GiB    docker     

I am running macOS Monterey 12.4 on an M1 Mac.

Hi same problem. on a m1 mac os v.12 after clean install like codrutP mentioned. I get an ip adress on the vm. After stop and start i get the error the docker pid. is not in the directory. I only at the first run get a password promt.

` user@SIT-SMBP-C2FR2D ~ % colima start oraxe --very-verbose TRAC[0000] cmd [“limactl” “info”]
TRAC[0000] cmd [“limactl” “list” “colima-oraxe” “–json”] INFO[0000] starting colima [profile=oraxe]
INFO[0000] runtime: docker
TRAC[0000] cmd [“limactl” “list” “colima-oraxe” “–json”] INFO[0000] preparing network … context=vm INFO[0000] dependencies missing for setting up reachable IP address context=vm INFO[0000] sudo password may be required context=vm TRAC[0000] cmd int [“sudo” “mkdir” “-p” “/etc/sudoers.d”] Password: TRAC[0010] cmd int [“sudo” “sh” “-c” “cat > /etc/sudoers.d/colima”] TRAC[0010] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] TRAC[0010] cmd [“/opt/homebrew/bin/colima” “daemon” “start” “oraxe” “–vmnet” “–verbose”] TRAC[0010] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] TRAC[0013] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] TRAC[0017] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] TRAC[0020] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] TRAC[0023] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] TRAC[0026] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] WARN[0026] error setting up network dependencies: error at ‘preparing network’: error running [/opt/homebrew/bin/colima daemon status oraxe], output: “time="2023-02-02T11:29:58+01:00" level=fatal msg="pid file not found: stat /Users/user/.colima/oraxe/daemon/daemon.pid: no such file or directory"”, err: “exit status 1” context=vm WARN[0026] error setting up reachable IP address: vmnet socket file not found: stat /Users/user/.colima/oraxe/daemon/vmnet.sock: no such file or directory INFO[0026] starting … context=vm TRAC[0026] cmd [“limactl” “start” “colima-oraxe”]

Using the existing instance “colima-oraxe” [hostagent] Starting QEMU (hint: to watch the boot progress, see “/Users/user/.lima/colima-oraxe/serial.log”) SSH Local Port: 57052 [hostagent] Waiting for the essential requirement 1 of 5: “ssh” [hostagent] Waiting for the essential requirement 1 of 5: “ssh” [hostagent] The essential requirement 1 of 5 is satisfied [hostagent] Waiting for the essential requirement 2 of 5: “user session is ready for ssh” [hostagent] The essential requirement 2 of 5 is satisfied [hostagent] Waiting for the essential requirement 3 of 5: “sshfs binary to be installed” [hostagent] The essential requirement 3 of 5 is satisfied [hostagent] Waiting for the essential requirement 4 of 5: “/etc/fuse.conf (/etc/fuse3.conf) to contain "user_allow_other"” [hostagent] The essential requirement 4 of 5 is satisfied [hostagent] Waiting for the essential requirement 5 of 5: “the guest agent to be running” [hostagent] The essential requirement 5 of 5 is satisfied [hostagent] Mounting "/Users/user” on "/Users/user” [hostagent] Mounting “/tmp/colima-oraxe” on “/tmp/colima-oraxe” [hostagent] Waiting for the final requirement 1 of 1: “boot scripts must have finished” [hostagent] Forwarding “/var/run/docker.sock” (guest) to “/Users/user/.colima/oraxe/docker.sock” (host) [hostagent] Forwarding “/run/lima-guestagent.sock” (guest) to “/Users/user/.lima/colima-oraxe/ga.sock” (host) [hostagent] The final requirement 1 of 1 is satisfied READY. Run limactl shell colima-oraxe to open the shell. TRAC[0068] cmd [“lima” “sudo” “cat” “/etc/hosts”]
TRAC[0068] cmd [“lima” “sudo” “cat” “/etc/hosts”]
INFO[0068] provisioning … context=docker TRAC[0068] cmd [“lima” “sudo” “mkdir” “-p” “/etc/docker”] TRAC[0068] cmd int [“lima” “sudo” “sh” “-c” “cat > /etc/docker/daemon.json”] TRAC[0068] cmd [“docker” “context” “inspect” “colima-oraxe”] TRAC[0068] cmd [“docker” “context” “create” “colima-oraxe” “–description” “colima [profile=oraxe]” “–docker” “host=unix:///Users/user/.colima/oraxe/docker.sock”] colima-oraxe Successfully created context “colima-oraxe” TRAC[0068] cmd [“docker” “context” “use” “colima-oraxe”] colima-oraxe Current context is now “colima-oraxe” Warning: DOCKER_HOST environment variable overrides the active context. To use “colima-oraxe”, either set the global --context flag, or unset DOCKER_HOST environment variable. INFO[0068] starting … context=docker TRAC[0068] cmd [“lima” “sudo” “service” “docker” “start”]

  • /var/log/docker.log: creating file
  • /var/log/docker.log: correcting owner
  • Starting Docker Daemon … [ ok ] TRAC[0068] cmd [“lima” “sudo” “docker” “info”]
    TRAC[0074] cmd [“lima” “sudo” “docker” “info”]
    TRAC[0074] cmd [“lima” “sudo” “cat” “/etc/colima/colima.json”] TRAC[0074] cmd [“lima” “sudo” “mkdir” “-p” “/etc/colima”] TRAC[0074] cmd [“lima” “sudo” “mkdir” “-p” “/etc/colima”] TRAC[0074] cmd int [“lima” “sudo” “sh” “-c” “cat > /etc/colima/colima.json”] TRAC[0074] cmd [“lima” “sudo” “cat” “/etc/colima/colima.json”] TRAC[0074] cmd [“lima” “sudo” “mkdir” “-p” “/etc/colima”] TRAC[0074] cmd [“lima” “sudo” “mkdir” “-p” “/etc/colima”] TRAC[0074] cmd int [“lima” “sudo” “sh” “-c” “cat > /etc/colima/colima.json”] INFO[0074] done
    TRAC[0074] cmd [“limactl” “list” “–json”]
    TRAC[0074] cmd [“limactl” “show-ssh” “–format” “config” “colima-oraxe”] TRAC[0074] cmd [“limactl” “list” “colima-oraxe” “–json”] `

After restart. ` PROFILE STATUS ARCH CPUS MEMORY DISK RUNTIME ADDRESS

oraxe Running x86_64 2 2GiB 60GiB docker
`

deamon.log time="2023-02-02T11:20:16+01:00" level=info msg="- - - - - - - - - - - - - - -" time="2023-02-02T11:20:16+01:00" level=info msg="daemon started by colima" time="2023-02-02T11:20:16+01:00" level=info msg="Run/usr/bin/pkill -F /Users/user/.colima/oraxe/daemon/daemon.pidto kill the daemon" pidfile_open: Resource temporarily unavailable time="2023-02-02T11:20:16+01:00" level=error msg="error starting vmnet: error running vmnet: exit status 1" time="2023-02-02T11:20:16+01:00" level=info msg="terminate signal received" time="2023-02-02T11:20:16+01:00" level=fatal msg="context canceled" time="2023-02-02T11:23:09+01:00" level=info msg="- - - - - - - - - - - - - - -"

@PARENTELF I am struggling with properly reproducing this as well.

It seems to happen (for me) in one of the following scenarios.

  • Having another virtual machine software running that is utilising vmnet networking, e.g. VMWare, Parallel, UTMApp
  • Changing network connections (wired or wifi)

The main issue is that the behaviour is not consistent, it works most time but when it breaks I am usually unable to resolve it until a macOS restart.

Since you are on macOS 13 Ventura, you can give the VZ driver a try --vm-type vz. While VZ is more experimental, the network address is more stable.

colima delete # required for switching vm types
colima start --vm-type vz --network-address

PS: kindly update to latest head as it includes a fix to a regression introduced in HEAD-fe625fb

It seems like there may be an issue with the default profile.

If I run the following (profile = main), I can get --network-address to work.

brew remove colima
brew install --HEAD colima
colima delete default
colima start --runtime containerd --network-address main
colima list
PROFILE    STATUS     ARCH       CPUS    MEMORY    DISK     RUNTIME       ADDRESS
main       Running    aarch64    2       2GiB      60GiB    containerd    192.168.106.4

If I don’t specify a profile name and start Colima (colima start --runtime containerd --network-address), then it throws this error:

INFO[0000] starting colima
INFO[0000] runtime: containerd
INFO[0000] preparing network ...                         context=vm
WARN[0015] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status default], output: "time=\"2022-12-29T17:29:16-06:00\" level=fatal msg=\"pid file not found: stat /Users/jonathan/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
INFO[0015] creating and starting ...                     context=vm
WARN[0015] error setting up reachable IP address: vmnet socket file not found: stat /Users/jonathan/.colima/default/daemon/vmnet.sock: no such file or directory

Having a similar problem… Clearing and reinstalling/updating (with dependencies) didn’t help.

I have the same issue on Ventura M1. The colima version with --HEAD option colima does start but there is no IP address assigned.