colima: Error starting with --network-address
Description
Attempting to start --network-address
fails. Additional starts without the flag fail with a different error message. Removing ~/.colima
allows for a successful start, without the network address.
Version
Colima Version: 0.4.6 Lima Version: 0.12.0, 0.13.0 Qemu Version: 7.1.0
Operating System
- macOS Intel
- macOS M1
- Linux
Reproduction Steps
colima start --network-address
Expected behaviour
Colima starts correctly.
Additional context
First start:
INFO[0000] starting colima
INFO[0000] runtime: docker
INFO[0000] preparing network ... context=vm
WARN[0015] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status default], output: "time=\"2022-10-27T09:57:15-04:00\" level=fatal msg=\"pid file not found: stat /Users/joel.johnson/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1" context=vm
INFO[0015] creating and starting ... context=vm
WARN[0015] error setting up routable IP address: vmnet ptp socket file not found: stat /Users/joel.johnson/.colima/default/daemon/vmnet.ptp: no such file or directory
> Using cache "/Users/joel.johnson/Library/Caches/lima/download/by-url-sha256/e9bac04e9bdb31be4d3de1506d97eb60d59d9ad1a2d97f2b21f760e06f3e4408/data"
> [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/joel.johnson/.lima/colima/serial.log")
> SSH Local Port: 57347
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] QEMU has exited
> exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/joel.johnson/.lima/colima/ha.stderr.log")
FATA[0017] error starting vm: error at 'creating and starting': exit status 1
ha.stderr.log:
{"level":"debug","msg":"Creating iso file /Users/joel.johnson/.lima/colima/cidata.iso","time":"2022-10-27T09:57:15-04:00"}
{"level":"debug","msg":"Using /var/folders/1n/fvnyc01554s0240g36s382jh6nvfqs/T/diskfs_iso413631167 as workspace","time":"2022-10-27T09:57:15-04:00"}
{"level":"debug","msg":"firmware candidates = [/Users/joel.johnson/.local/share/qemu/edk2-aarch64-code.fd /Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/share/qemu/edk2-aarch64-code.fd /usr/share/AAVMF/AAVMF_CODE.fd /usr/share/qemu-efi-aarch64/QEMU_EFI.fd]","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"OpenSSH version 8.6.1 detected","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com","time":"2022-10-27T09:57:17-04:00"}
{"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see \"/Users/joel.johnson/.lima/colima/serial.log\")","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"qCmd.Args: [/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/bin/qemu-system-aarch64 -m 2048 -cpu host -machine virt,accel=hvf,highmem=off -smp 2,sockets=1,cores=2,threads=1 -drive if=pflash,format=raw,readonly=on,file=/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/share/qemu/edk2-aarch64-code.fd -boot order=d,splash-time=0,menu=on -drive file=/Users/joel.johnson/.lima/colima/basedisk,media=cdrom,readonly=on -drive file=/Users/joel.johnson/.lima/colima/diffdisk,if=virtio,discard=on -cdrom /Users/joel.johnson/.lima/colima/cidata.iso -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:57347-:22 -device virtio-net-pci,netdev=net0,mac=52:55:55:56:07:0a -device virtio-rng-pci -display none -vga none -device ramfb -device qemu-xhci,id=usb-bus -device usb-kbd,bus=usb-bus.0 -device usb-mouse,bus=usb-bus.0 -parallel none -chardev socket,id=char-serial,path=/Users/joel.johnson/.lima/colima/serial.sock,server=on,wait=off,logfile=/Users/joel.johnson/.lima/colima/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/joel.johnson/.lima/colima/qmp.sock,server=on,wait=off -qmp chardev:char-qmp -name lima-colima -pidfile /Users/joel.johnson/.lima/colima/qemu.pid]","time":"2022-10-27T09:57:17-04:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 5: \"ssh\"","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"qemu[stderr]: time=\"2022-10-27T09:57:17-04:00\" level=fatal msg=\"dial unix /Users/joel.johnson/.colima/default/daemon/gvproxy.sock: connect: no such file or directory\"","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2022-10-27T09:57:17-04:00"}
{"error":"exit status 1","level":"info","msg":"QEMU has exited","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"executing ssh for script \"ssh\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/joel.johnson/.lima/_config/user\" -o IdentityFile=\"/Users/joel.johnson/.ssh/id_ed25519\" -o IdentityFile=\"/Users/joel.johnson/.ssh/id_rsa\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm@openssh.com,aes256-gcm@openssh.com\" -o User=lima -o ControlMaster=auto -o ControlPath=\"/Users/joel.johnson/.lima/colima/ssh.sock\" -o ControlPersist=5m -p 57347 127.0.0.1 -- /bin/bash]","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 57347: Connection refused\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 57347: Connection refused\\r\\n\": exit status 255","time":"2022-10-27T09:57:17-04:00"}
Second start:
INFO[0000] starting colima
INFO[0000] runtime: docker
INFO[0000] preparing network ... context=vm
WARN[0015] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status default], output: "time=\"2022-10-27T10:01:35-04:00\" level=fatal msg=\"pid file not found: stat /Users/joel.johnson/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1" context=vm
WARN[0015] error setting up routable IP address: vmnet ptp socket file not found: stat /Users/joel.johnson/.colima/default/daemon/vmnet.ptp: no such file or directory
INFO[0015] starting ... context=vm
> Using the existing instance "colima"
> [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/joel.johnson/.lima/colima/serial.log")
> SSH Local Port: 57370
> [hostagent] QEMU has exited
> exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/joel.johnson/.lima/colima/ha.stderr.log")
FATA[0017] error starting vm: error at 'starting': exit status 1
which is obviously due to the colima.yaml being updated with the --network-address
flag.
About this issue
- Original URL
- State: open
- Created 2 years ago
- Reactions: 4
- Comments: 50 (12 by maintainers)
We were having this issue and discovered that having a
multipass
VM running causes this behavior. I assume that other VM hosts could cause a similar conflict.I am also having the same problem, my logs are exactly the same however wiping the directories and reinstalling from HEAD works “intermittently” as described above but to the negative. Meaning: if I blow it all away and start over sometimes it errors out completely and sometimes it starts with no routable IP, I am not able to get a valid IP at all.
@abiosoft This works! Thank you! Steps followed: clean everything:
starts from scratch:
Hello guys,
To whom it may concern:
colima version 0.5.5, arch: aarch64, macbook air m2, vz/virtiofs
With
--network-address
option set, colima starts a virtual machine, and bridges col0 interface with bridge100, which has ip address 192.168.106.1. It seems that bridge100 interface is a special one used for “internet sharing”, which means that there should be a DHCP server running. However, if for some reason “internet sharing” is disabled, then there is no DHCP server on bridge100 interface, and therefore, lima’s virtual machine is unable to acquire IP address, and therefore k3s kubernetes fails to start, which causes colima error “The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?”The solution is to start dhcp server on bridge100 serving 192.168.106.* range, e.g., dnsmasq, which might require disabling macos builtin server
bootps.plist
, due to port conflict.Certainly not the fix, but this works as a workaround for me when
colima status
shows no Network Address:--network-address
option (Wait for it to start) I confirm it’s started with no network address by running:bridge100
- grab it’s IP address by running:colima ssh
(you are now in SSH inside VM)
sudo su -
/etc/network/interfaces
so that it looks similar to the below, using the IP address returned in step 2 as the value forgateway
:ifdown col0
ifup col0
CTRL + D
twiceThis results in the VM having the IP
192.168.106.2
- validated by again runningcolima status
:For me the issue persists M1 Ventura, latest colima version
WARN[0181] error setting up reachable IP address: vmnet socket file not found: stat /…/.colima/default/daemon/vmnet.sock: no such file or directory
Instructions havent changed the result.
Hi! I have the same error when using --network-address
I am running macOS Monterey 12.4 on an M1 Mac.
Hi same problem. on a m1 mac os v.12 after clean install like codrutP mentioned. I get an ip adress on the vm. After stop and start i get the error the docker pid. is not in the directory. I only at the first run get a password promt.
` user@SIT-SMBP-C2FR2D ~ % colima start oraxe --very-verbose TRAC[0000] cmd [“limactl” “info”]
TRAC[0000] cmd [“limactl” “list” “colima-oraxe” “–json”] INFO[0000] starting colima [profile=oraxe]
INFO[0000] runtime: docker
TRAC[0000] cmd [“limactl” “list” “colima-oraxe” “–json”] INFO[0000] preparing network … context=vm INFO[0000] dependencies missing for setting up reachable IP address context=vm INFO[0000] sudo password may be required context=vm TRAC[0000] cmd int [“sudo” “mkdir” “-p” “/etc/sudoers.d”] Password: TRAC[0010] cmd int [“sudo” “sh” “-c” “cat > /etc/sudoers.d/colima”] TRAC[0010] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] TRAC[0010] cmd [“/opt/homebrew/bin/colima” “daemon” “start” “oraxe” “–vmnet” “–verbose”] TRAC[0010] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] TRAC[0013] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] TRAC[0017] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] TRAC[0020] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] TRAC[0023] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] TRAC[0026] cmd [“/opt/homebrew/bin/colima” “daemon” “status” “oraxe”] WARN[0026] error setting up network dependencies: error at ‘preparing network’: error running [/opt/homebrew/bin/colima daemon status oraxe], output: “time="2023-02-02T11:29:58+01:00" level=fatal msg="pid file not found: stat /Users/user/.colima/oraxe/daemon/daemon.pid: no such file or directory"”, err: “exit status 1” context=vm WARN[0026] error setting up reachable IP address: vmnet socket file not found: stat /Users/user/.colima/oraxe/daemon/vmnet.sock: no such file or directory INFO[0026] starting … context=vm TRAC[0026] cmd [“limactl” “start” “colima-oraxe”]
After restart. ` PROFILE STATUS ARCH CPUS MEMORY DISK RUNTIME ADDRESS
oraxe Running x86_64 2 2GiB 60GiB docker
`
deamon.log
time="2023-02-02T11:20:16+01:00" level=info msg="- - - - - - - - - - - - - - -" time="2023-02-02T11:20:16+01:00" level=info msg="daemon started by colima" time="2023-02-02T11:20:16+01:00" level=info msg="Run
/usr/bin/pkill -F /Users/user/.colima/oraxe/daemon/daemon.pidto kill the daemon" pidfile_open: Resource temporarily unavailable time="2023-02-02T11:20:16+01:00" level=error msg="error starting vmnet: error running vmnet: exit status 1" time="2023-02-02T11:20:16+01:00" level=info msg="terminate signal received" time="2023-02-02T11:20:16+01:00" level=fatal msg="context canceled" time="2023-02-02T11:23:09+01:00" level=info msg="- - - - - - - - - - - - - - -"
@PARENTELF it comes from here https://github.com/abiosoft/colima/blob/main/environment/vm/lima/lima.go#L119 which calls then here https://github.com/abiosoft/colima/blob/1c55a0979c8d958578dfd9691957fb3958135c80/daemon/process/gvproxy/gvproxy.go#L208-L213.
@msankhala try removing
/opt/colima
as suggested here https://github.com/abiosoft/colima/issues/590@PARENTELF I am struggling with properly reproducing this as well.
It seems to happen (for me) in one of the following scenarios.
The main issue is that the behaviour is not consistent, it works most time but when it breaks I am usually unable to resolve it until a macOS restart.
Since you are on macOS 13 Ventura, you can give the VZ driver a try
--vm-type vz
. WhileVZ
is more experimental, the network address is more stable.PS: kindly update to latest head as it includes a fix to a regression introduced in
HEAD-fe625fb
It seems like there may be an issue with the
default
profile.If I run the following (profile =
main
), I can get--network-address
to work.If I don’t specify a profile name and start Colima (
colima start --runtime containerd --network-address
), then it throws this error:Having a similar problem… Clearing and reinstalling/updating (with dependencies) didn’t help.
I have the same issue on Ventura M1. The colima version with --HEAD option colima does start but there is no IP address assigned.