lima: Experiencing slow network under VZ virtualization

Description

Lima: HEAD-0da3240 Colima: version=HEAD-cf522e8,driver=vz,mounts=virtiofs MacOS: Ventura 13.1

I’m running HEAD of lima and colima and I think I’m still seeing something similar to this issue https://github.com/lima-vm/lima/issues/1285 frequently . The most often occurrence of this issue for me is when pulling many docker images while starting a docker compose project. The command hangs and sometimes times out producing a message like:

Error response from daemon: Head "********************": dial tcp: lookup gcr.io on 192.168.5.3:53: read udp 192.168.5.15:40202->192.168.5.3:53: i/o timeout
Error: exit status 18

In the host agent logs ha.stderr.log I see many of the following message: {"level":"debug","msg":"Stopping udp proxy (read udp 192.168.5.2:58193: i/o timeout)","time":"2023-01-26T01:12:39-05:00"}

About this issue

Original URL
State: closed
Created a year ago
Reactions: 11
Comments: 40 (19 by maintainers)

Most upvoted comments

This still appears to be occurring for us. It seems like the issue appears to get worse the longer the vm is running.

terev on May 1, 2023

This issue should be fixed in latest master. https://github.com/lima-vm/lima/commit/336c66922cff7d88b0cf0c78f0aabe66b3406b2c

Do give a try and let us know if you face similar issues

balajiv113 on Sep 3, 2023

@aaronlehmann Looks like iptables are not updated properly for old instance. I will check if it can be fixed

balajiv113 on Sep 6, 2023

@balajiv113 Unfortunately I’m unable to share the exact compose project. I was considering trying to reproduce with many projects from https://github.com/docker/awesome-compose though that may run into docker hub rate limits.

terev on Sep 5, 2023

@balajiv113 Thanks, DNS and network in general seem to be quick now, and also no more udp timeout errors in logs.

herrernst on Sep 4, 2023

Yeah turning the host resolver off for user-v2 shows significant latency improvement too.

for i in $(seq 1 10); do curl -s -w 'lookup: %{time_namelookup} total: %{time_total}\n' -o /dev/null https://www.google.com ; done
lookup: 0.033342 total: 0.290496
lookup: 0.000825 total: 0.266588
lookup: 0.000858 total: 0.270565
lookup: 0.000861 total: 0.268383
lookup: 0.000876 total: 0.277635
lookup: 0.000808 total: 0.242906
lookup: 0.000641 total: 0.260983
lookup: 0.000769 total: 0.239489
lookup: 0.000787 total: 0.247760
lookup: 0.000792 total: 0.241116

terev on May 4, 2023

Are there downsides to responding to DNS from the gateway?

Not really, which is why we should be switching to it. AFAIK the only piece missing right now is to be able to define additional static names (like host.lima.internal, or additional entries from lima.yaml).

jandubois on Jan 31, 2023

slowness of the UDP proxy expected

Not sure,. Currently investigating this. Will update if i find something

Does this have to do with differences between VZ and QEMU’s network setup

@terev Exactly this. The way network is developed/configured in thr framework used is entirely different just the working is more/less the same

balajiv113 on Jan 28, 2023