kvm-guest-drivers-windows: netkvm: Control message timeout causes corrupt VQ
Control messages sent to device when link disconnected. When link reconnect, device reads corrupted messages, responds with errors.
[0]0EF8.0F0C::04/06/2025-11:07:07.255 [netkvm.sys] PostLinkStateIndicating Disconnected
[1]0004.0144::04/06/2025-11:07:07.256 [netkvm.sys] ParaNdis_OnOidSetNetworkAddressesaddress 0, type 3, len 28 (0000000000000000FE80000000000000FC9D26FFFE79127204000020)
[0]0004.00C8::04/06/2025-11:07:07.256 [netkvm.sys] ParaNdis_OnOidSetNetworkAddressesReceived IP address 169.254.209.116
[0]0004.00D0::04/06/2025-11:07:10.552 [netkvm.sys] CParaNdisCX::SendControlMessageCParaNdisCX::SendControlMessage - ERROR: get_buf failed
[1]0004.00D0::04/06/2025-11:07:13.475 [netkvm.sys] CParaNdisCX::SendControlMessageCParaNdisCX::SendControlMessage - ERROR: get_buf failed
[0]0004.00D0::04/06/2025-11:07:14.150 [netkvm.sys] CParaNdisCX::FireDPC[CParaNdisCX::FireDPC] message 4
[0]0004.00D0::04/06/2025-11:07:14.312 [netkvm.sys] CParaNdisCX::FireDPC[CParaNdisCX::FireDPC] message 4
[0]0004.00D0::04/06/2025-11:07:14.312 [netkvm.sys] CParaNdisCX::FireDPC[CParaNdisCX::FireDPC] message 4
[0]0004.00D0::04/06/2025-11:07:14.312 [netkvm.sys] CParaNdisCX::SendControlMessageCParaNdisCX::SendControlMessage - ERROR: error 1 returned for class 1
[0]0004.00D0::04/06/2025-11:07:14.313 [netkvm.sys] CParaNdisCX::FireDPC[CParaNdisCX::FireDPC] message 4
[0]0004.00D0::04/06/2025-11:07:14.313 [netkvm.sys] PostLinkStateIndicating Connected
[0]06D0.07AC::04/06/2025-11:07:14.326 [netkvm.sys] CParaNdisCX::FireDPC[CParaNdisCX::FireDPC] message 4
[0]0004.00C8::04/06/2025-11:07:15.514 [netkvm.sys] ParaNdis_OnOidSetNetworkAddressesaddress 0, type 3, len 28 (0000000000000000FE80000000000000FC9D26FFFE79127204000020)
[1]0004.00C0::04/06/2025-11:07:15.532 [netkvm.sys] CParaNdisCX::SendControlMessageCParaNdisCX::SendControlMessage - ERROR: error 1 returned for class 0
[1]0004.00C0::04/06/2025-11:07:15.532 [netkvm.sys] CParaNdisCX::SendControlMessageCParaNdisCX::SendControlMessage - ERROR: error 1 returned for class 0
[1]0004.00C0::04/06/2025-11:07:15.532 [netkvm.sys] CParaNdisCX::SendControlMessageCParaNdisCX::SendControlMessage - ERROR: error 1 returned for class 1
[0]0000.0000::04/06/2025-11:07:15.532 [netkvm.sys] CParaNdisCX::FireDPC[CParaNdisCX::FireDPC] message 4
[0]0004.00D0::04/06/2025-11:07:15.532 [netkvm.sys] CParaNdisCX::FireDPC[CParaNdisCX::FireDPC] message 4
[0]0000.0000::04/06/2025-11:07:15.533 [netkvm.sys] CParaNdisCX::FireDPC[CParaNdisCX::FireDPC] message 4
[1]0004.00D0::04/06/2025-11:07:17.514 [netkvm.sys] ParaNdis_OnOidSetNetworkAddressesReceived IP address 169.254.209.116
Reproduce: unplug + plug cable netkvm-ctrl-msg-send-when-unpluged.log
VQ corruption can also be caused by device slow response to control messages (more then 0.5 seconds) netkvm-ctrl-msg-send-timout.log
Host:
- Disto: Ubuntu 22.04 LTS
- Kernel version: 5.15.0-131-generic
- QEMU version: 6.2.0
QEMU command line
/usr/bin/qemu-system-x86_64 \
-name guest=Win10_Ent_2019_LTSC_x64,debug-threads=on \
-S \
-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/var/lib/libvirt/qemu/domain-2-Wi/master-key.aes"}' \
-machine pc-i440fx-jammy,usb=off,dump-guest-core=off,memory-backend=pc.ram \
-accel kvm \
-cpu 'core2duo,ibpb=on,md-clear=on,spec-ctrl=on,ssbd=on,hv-relaxed=on,hv-vapic=on,hv-spinlocks=0x1fff,hv-vendor-id=NVP Hv' \
-m 8192 \
-object '{"qom-type":"memory-backend-ram","id":"pc.ram","size":8589934592}' \
-overcommit mem-lock=off \
-smp 2,sockets=2,cores=1,threads=1 \
-uuid 3642cec1-17a1-4a13-b35f-f115fd121609 \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=33,server=on,wait=off \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-global kvm-pit.lost_tick_policy=delay \
-no-hpet \
-no-shutdown \
-global PIIX4_PM.disable_s3=1 \
-global PIIX4_PM.disable_s4=1 \
-boot strict=on \
-device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x4.0x7 \
-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x4 \
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x4.0x1 \
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x4.0x2 \
-device ahci,id=sata0,bus=pci.0,addr=0x5 \
-blockdev '{"driver":"file","filename":"/images/004.img","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \
-blockdev '{"node-name":"libvirt-1-format","read-only":false,"driver":"qcow2","file":"libvirt-1-storage","backing":null}' \
-device ide-hd,bus=sata0.0,drive=libvirt-1-format,id=sata0-0-0,bootindex=1 \
-netdev tap,fd=34,id=hostnet0 \
-device e1000,netdev=hostnet0,id=net0,mac=13:63:59:15:40:17,bus=pci.0,addr=0x3 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-device usb-tablet,id=input0,bus=usb.0,port=1 \
-audiodev '{"id":"audio1","driver":"none"}' \
-vnc 127.0.0.1:0,audiodev=audio1 \
-device VGA,id=video0,vgamem_mb=16,bus=pci.0,addr=0x2 \
-device vfio-pci,host=0000:0f:04.4,id=hostdev0,bus=pci.0,addr=0x7 \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
- libvirt version: 8.0.0
- libvirt XML file:
libvirt XML file
<domain type='kvm'>
<name>Win10_Ent_2019_LTSC_x64</name>
<uuid>3642cec1-17a1-4a13-b35f-f115fd121609</uuid>
<memory unit='KiB'>8388608</memory>
<currentMemory unit='KiB'>8388608</currentMemory>
<vcpu placement='static'>2</vcpu>
<os>
<type arch='x86_64' machine='pc-i440fx-jammy'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<hyperv mode='custom'>
<relaxed state='on'/>
<vapic state='on'/>
<spinlocks state='on' retries='8191'/>
<vendor_id state='on' value='NVP Hv'/>
</hyperv>
</features>
<cpu mode='custom' match='exact' check='partial'>
<model fallback='allow'>core2duo</model>
<feature policy='require' name='ibpb'/>
<feature policy='require' name='md-clear'/>
<feature policy='require' name='spec-ctrl'/>
<feature policy='require' name='ssbd'/>
</cpu>
<clock offset='utc'>
<timer name='rtc' tickpolicy='catchup'/>
<timer name='pit' tickpolicy='delay'/>
<timer name='hpet' present='no'/>
</clock>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>destroy</on_crash>
<pm>
<suspend-to-mem enabled='no'/>
<suspend-to-disk enabled='no'/>
</pm>
<devices>
<emulator>/usr/bin/qemu-system-x86_64</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2'/>
<source file='/images/004.img'/>
<target dev='sda' bus='sata'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<controller type='usb' index='0' model='ich9-ehci1'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x7'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci1'>
<master startport='0'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0' multifunction='on'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci2'>
<master startport='2'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x1'/>
</controller>
<controller type='usb' index='0' model='ich9-uhci3'>
<master startport='4'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x2'/>
</controller>
<controller type='pci' index='0' model='pci-root'/>
<controller type='sata' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
</controller>
<interface type='bridge'>
<mac address='13:63:59:15:40:17'/>
<source bridge='br0'/>
<model type='e1000'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
<serial type='pty'>
<target type='isa-serial' port='0'>
<model name='isa-serial'/>
</target>
</serial>
<console type='pty'>
<target type='serial' port='0'/>
</console>
<input type='tablet' bus='usb'>
<address type='usb' bus='0' port='1'/>
</input>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='vnc' port='-1' autoport='yes'>
<listen type='address'/>
</graphics>
<audio id='1' type='none'/>
<video>
<model type='vga' vram='16384' heads='1' primary='yes'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x0000' bus='0x0f' slot='0x04' function='0x4'/>
</source>
<address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
</hostdev>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
</memballoon>
</devices>
</domain>
VM:
- Windows version: Windows 10 Enterprise LTSC 1809 build 17763.6293
- Which driver has a problem: netkvm
- Driver version or commit hash that was used to build the driver: 0.1.266 (100.100.104.26600)
About this issue
- Original URL
- State: open
- Created 2 years ago
- Comments: 36
@ybendito drop3 seems to solve the issue with driver disable
@YanVugenfirer , You are correct on “mac address” setting on physical link down scenario. This must be supported.
I tend to disagree that driver picks an arbitrary timeout value. When spec has not defined any such limit driver does not get to choose it.
It is not that hw device does not function properly. Sometimes the device may take time. This is not a real time (time bound) device. Nor CVQ has timeout defined.
I dont think Eli’s explanation of ‘link down’ was incorrect. There are some internal link within the device which can go down due to some thermal or other issues. And those link takes time to recovers sometimes after few seconds. It is not the “ethernet link” that goes down. I believe this was the cause of confusion.
Hence, the expectation is the driver be resilient to such intermittent long time that happens once in a while (it is not that every single CVQ commands takes 5 sec always).