kubevirt: Libvirtd crash on virt-launcher during host reboot

/kind bug

What happened: Libvirtd crash on virt-launcher during host reboot.

What you expected to happen: No Crash is expected.

How to reproduce it (as minimally and precisely as possible): Launch a VM and restart host-os on vmi instance running.

Anything else we need to know?: Backtrace:

[root@0e711a7d997a ~]# gdb /usr/sbin/libvirtd test_vm-default_defa-1.core GNU gdb (GDB) Fedora 9.1-7.fc32 Copyright © 2020 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type “show copying” and “show warranty” for details. This GDB was configured as “x86_64-redhat-linux-gnu”. Type “show configuration” for configuration details. For bug reporting instructions, please see: http://www.gnu.org/software/gdb/bugs/. Find the GDB manual and other documentation resources online at: http://www.gnu.org/software/gdb/documentation/.

For help, type “help”. Type “apropos word” to search for commands related to “word”… Reading symbols from /usr/sbin/libvirtd… Reading symbols from /usr/lib/debug/usr/sbin/libvirtd-6.6.0-8.fc32.x86_64.debug… [New LWP 119] [New LWP 68] [New LWP 64] [New LWP 66] [New LWP 53] [New LWP 50] Missing separate debuginfo for /lib64/libvirt-lxc.so.0 Try: dnf --enablerepo=‘debug’ install /usr/lib/debug/.build-id/b9/b372ea199dcb3feac3890f7f849d43ed28c163.debug Missing separate debuginfo for /lib64/libvirt-qemu.so.0 Try: dnf --enablerepo=‘debug’ install /usr/lib/debug/.build-id/dc/621fa6e7416a90cab4f81ee670324b0bbea1aa.debug Missing separate debuginfo for /lib64/libvirt.so.0 Try: dnf --enablerepo=‘debug’ install /usr/lib/debug/.build-id/39/8e5f1c44ad85c250a0d5601039d41c770521ce.debug [Thread debugging using libthread_db enabled] Using host libthread_db library “/lib64/libthread_db.so.1”. Missing separate debuginfo for /lib64/libblkid.so.1 Try: dnf --enablerepo=‘debug’ install /usr/lib/debug/.build-id/14/8e0966aabbf339cb04c5bd4744d4cea2914fbd.debug Core was generated by `/usr/sbin/libvirtd’. Program terminated with signal SIGSEGV, Segmentation fault. #0 0x00007efecc4e8950 in qemuProcessHandleShutdown (mon=<optimized out>, vm=0x7efec8021950, guest_initiated=VIR_TRISTATE_BOOL_NO, opaque=0x7efe7c00a080) at …/…/src/qemu/qemu_process.c:623 623 event = virDomainEventLifecycleNewFromObj(vm, [Current thread is 1 (Thread 0x7efebeffd700 (LWP 119))] Missing separate debuginfos, use: dnf debuginfo-install glibc-2.31-4.fc32.x86_64 krb5-libs-1.18.2-27.fc32.x86_64 libcurl-7.69.1-6.fc32.x86_64 libgcc-10.2.1-6.fc32.x86_64 libxcrypt-4.4.17-1.fc32.x86_64 libxml2-2.9.10-7.fc32.x86_64 openssl-libs-1.1.1g-1.fc32.x86_64 p11-kit-0.23.21-2.fc32.x86_64 systemd-libs-245.8-2.fc32.x86_64 (gdb) bt #0 0x00007efecc4e8950 in qemuProcessHandleShutdown (mon=<optimized out>, vm=0x7efec8021950, guest_initiated=VIR_TRISTATE_BOOL_NO, opaque=0x7efe7c00a080) at …/…/src/qemu/qemu_process.c:623 #1 0x00007efecc505113 in qemuMonitorEmitShutdown (mon=0x7efec8023610, guest=VIR_TRISTATE_BOOL_NO) at …/…/src/qemu/qemu_monitor.c:1163 #2 0x00007efecc511b38 in qemuMonitorJSONHandleShutdown (mon=0x7efec8023610, data=<optimized out>) at …/…/src/qemu/qemu_monitor_json.c:743 #3 0x00007efecc51591c in qemuMonitorJSONIOProcessEvent (obj=0x7efe7c001a30, mon=0x7efec8023610) at …/…/src/qemu/qemu_monitor_json.c:207 #4 qemuMonitorJSONIOProcessLine (mon=mon@entry=0x7efec8023610, line=line@entry=0x7efe7c019da0 “{"timestamp": {"seconds": 1636708425, "microseconds": 543945}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-signal"}}”, msg=msg@entry=0x0) at …/…/src/qemu/qemu_monitor_json.c:236 #5 0x00007efecc515af6 in qemuMonitorJSONIOProcess (mon=0x7efec8023610, data=0x7efe7c00abd0 “{"timestamp": {"seconds": 1636708425, "microseconds": 543945}, "event": "SHUTDOWN", "data": {"guest": false, "reason": "host-signal"}}\r\n”, len=136, msg=0x0) at …/…/src/qemu/qemu_monitor_json.c:277 #6 0x00007efecc504462 in qemuMonitorIOProcess (mon=0x7efec8023610) at …/…/src/qemu/qemu_monitor.c:338 #7 qemuMonitorIO (socket=<optimized out>, cond=(unknown: 0), opaque=0x7efec8023610) at …/…/src/qemu/qemu_monitor.c:570 #8 0x00007efed11480ab in socket_source_dispatch (source=source@entry=0x7efe7c01f200, callback=0x7efecc504050 <qemuMonitorIO>, user_data=0x7efec8023610) at …/gio/gsocket.c:4009 #9 0x00007efed18fd78f in g_main_dispatch (context=0x7efeb8031000) at …/glib/gmain.c:3309 #10 g_main_context_dispatch (context=0x7efeb8031000) at …/glib/gmain.c:3974 #11 0x00007efed18fdb18 in g_main_context_iterate (context=0x7efeb8031000, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at …/glib/gmain.c:4047 #12 0x00007efed18fde33 in g_main_loop_run (loop=0x7efeb804f220) at …/glib/gmain.c:4241 #13 0x00007efed1bb46ef in virEventThreadWorker () from /lib64/libvirt.so.0 #14 0x00007efed19277f2 in g_thread_proxy (data=0x7efeb8003c00) at …/glib/gthread.c:807 #15 0x00007efed1892432 in start_thread () from /lib64/libpthread.so.0 #16 0x00007efed17a5913 in clone () from /lib64/libc.so.6

Environment:

• KubeVirt version (use virtctl version): [root@appliance-1 ~]# virtctl version Starting to serve on 127.0.0.1:8080 Client Version: version.Info{GitVersion:“”, GitCommit:“8655f4839a48175ad8580d9bcc29180ded9e3957”, GitTreeState:“dirty”, BuildDate:“2021-11-09T06:47:55Z”, GoVersion:“go1.13.14”, Compiler:“gc”, Platf orm:“linux/amd64”} Server Version: version.Info{GitVersion:“”, GitCommit:“8655f4839a48175ad8580d9bcc29180ded9e3957”, GitTreeState:“dirty”, BuildDate:“2021-11-09T06:47:55Z”, GoVersion:“go1.13.14”, Compiler:“gc”, Platf orm:“linux/amd64”}

• Kubernetes version (use kubectl version): [root@appliance-1 ~]# kubectl version Client Version: version.Info{Major:“1”, Minor:“21”, GitVersion:“v1.21.1+k3s-89c5cb30-dirty”, GitCommit:“89c5cb305abf4e34fe70cad85ac5371685e76924”, GitTreeState:“dirty”, BuildDate:“2021-10-22T06:37:02Z”, GoVersion:“go1.15.2”, Compiler:“gc”, Platform:“linux/amd64”} Server Version: version.Info{Major:“1”, Minor:“21”, GitVersion:“v1.21.1+k3s-89c5cb30-dirty”, GitCommit:“89c5cb305abf4e34fe70cad85ac5371685e76924”, GitTreeState:“dirty”, BuildDate:“2021-10-22T06:37:02Z”, GoVersion:“go1.15.2”, Compiler:“gc”, Platform:“linux/amd64”}

• VM or VMI specifications:

• Cloud provider or hardware configuration:

• OS (e.g. from /etc/os-release): [root@appliance-1 ~]# cat /etc/redhat-release CentOS Linux release 7.8.2003 (Core)

• Kernel (e.g. uname -a): [root@appliance-1 ~]# uname -a Linux appliance-1.chassis.local 3.10.0-1160.25.1.F5.1.el7_8.x86_64 #1 SMP Thu Jul 8 01:59:57 UTC 2021 x86_64 x86_64 x86_64 GNU/Linu

• Install tools:

• Others: