cri-o: panic: interface conversion: types.Result is *types100.Result, not *types040.Result

What happened?

restart CRIO panic

journalctl -xeu crio --no-page in master nodes

May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.718171257+08:00" level=info msg="Starting CRI-O, version: 1.24.0, git: unknown(clean)"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.719979327+08:00" level=info msg="Node configuration value for hugetlb cgroup is true"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.720140167+08:00" level=info msg="Node configuration value for pid cgroup is true"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.720342868+08:00" level=info msg="Node configuration value for memoryswap cgroup is true"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.720502288+08:00" level=info msg="Node configuration value for cgroup v2 is true"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.727533929+08:00" level=info msg="Node configuration value for systemd CollectMode is true"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.734021114+08:00" level=info msg="Node configuration value for systemd AllowedCPUs is true"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.734934761+08:00" level=info msg="Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.736520855+08:00" level=info msg="Using default capabilities: CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_FSETID, CAP_FOWNER, CAP_SETGID, CAP_SETUID, CAP_SETPCAP, CAP_NET_BIND_SERVICE, CAP_KILL"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.739185116+08:00" level=info msg="Conmon does support the --sync option"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.740892829+08:00" level=info msg="Using seccomp default profile when unspecified: true"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.741050040+08:00" level=info msg="No seccomp profile specified, using the internal default"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.741223400+08:00" level=info msg="Installing default AppArmor profile: crio-default"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.782624415+08:00" level=info msg="No blockio config file specified, blockio not configured"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.782698685+08:00" level=info msg="RDT not available in the host system"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.797034208+08:00" level=info msg="Found CNI network multus-cni-network (type=multus) at /etc/cni/net.d/00-multus.conf"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.827171787+08:00" level=info msg="Found CNI network k8s-pod-network (type=calico) at /etc/cni/net.d/10-calico.conflist"
May 17 11:45:58 node1 crio[26471]: time="2022-05-17 11:45:58.827218327+08:00" level=info msg="Updated default CNI network name to multus-cni-network"
May 17 11:45:59 node1 crio[26471]: time="2022-05-17 11:45:59.129960599+08:00" level=info msg="Got pod network &{Name:coredns-f657fccfd-vjj5t Namespace:kube-system ID:9c328c905bc02d37eca59f66c9d88f7958f96ac598048d28070e7b409dd1ab6b UID:4690df4b-7ad5-40f1-8f93-9f1540bf1cb5 NetNS:/var/run/netns/dc554db0-9479-4eae-a71f-0b13785f8523 Networks:[] RuntimeConfig:map[multus-cni-network:{IP: MAC: PortMappings:[] Bandwidth:<nil> IpRanges:[]}] Aliases:map[]}"
May 17 11:45:59 node1 crio[26471]: time="2022-05-17 11:45:59.130041899+08:00" level=error msg="error loading cached network config: network \"multus-cni-network\" not found in CNI cache"
May 17 11:45:59 node1 crio[26471]: time="2022-05-17 11:45:59.130054169+08:00" level=warning msg="falling back to loading from existing plugins on disk"
May 17 11:45:59 node1 crio[26471]: time="2022-05-17 11:45:59.130062689+08:00" level=info msg="Checking pod kube-system_coredns-f657fccfd-vjj5t for CNI network multus-cni-network (type=multus)"
May 17 11:45:59 node1 crio[26471]: time="2022-05-17 11:45:59.130078469+08:00" level=info msg="Checking CNI network multus-cni-network (config version=0.3.1) nsManager=&{/usr/bin/nsenter}"
May 17 11:45:59 node1 crio[26471]: panic: interface conversion: types.Result is *types100.Result, not *types040.Result
May 17 11:45:59 node1 crio[26471]: goroutine 1 [running]:
May 17 11:45:59 node1 crio[26471]: panic({0x1d69d60, 0xc000480630})
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/panic.go:941 +0x397 fp=0xc0008f6640 sp=0xc0008f6580 pc=0x441137
May 17 11:45:59 node1 crio[26471]: runtime.panicdottypeE(...)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/iface.go:262
May 17 11:45:59 node1 crio[26471]: runtime.panicdottypeI(0x0?, 0x1dbf180, 0x1dac900)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/iface.go:272 +0x7c fp=0xc0008f6668 sp=0xc0008f6640 pc=0x41649c
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/github.com/containernetworking/cni/pkg/types/100.convertFrom04x({0x238ab10?, 0xc000445e40?}, {0x200e65a?, 0xc0006683f0?})
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/containernetworking/cni/pkg/types/100/types.go:126 +0x9c5 fp=0xc0008f67b8 sp=0xc0008f6668 pc=0xf92745
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/github.com/containernetworking/cni/pkg/types/internal.Convert({0x238ab10, 0xc000445e40}, {0x200e65a, 0x5})
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/containernetworking/cni/pkg/types/internal/convert.go:73 +0x1a5 fp=0xc0008f6868 sp=0xc0008f67b8 pc=0xf8b825
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/github.com/containernetworking/cni/pkg/types/100.(*Result).GetAsVersion(0xc000481650?, {0x200e65a?, 0xc0003e4c00?})
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/containernetworking/cni/pkg/types/100/types.go:211 +0x65 fp=0xc0008f6898 sp=0xc0008f6868 pc=0xf93385
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/github.com/containernetworking/cni/pkg/types/100.GetResult({0x238ab10?, 0xc000445e40?})
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/containernetworking/cni/pkg/types/100/types.go:65 +0x37 fp=0xc0008f68d0 sp=0xc0008f6898 pc=0xf91c37
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/server.(*Server).getSandboxIPs(0xc000246e00, 0xc000842000)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/server/sandbox_network.go:146 +0x178 fp=0xc0008f6b00 sp=0xc0008f68d0 pc=0x1b51058
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/server.(*Server).restore(0xc000246e00, {0x238a480?, 0xc0003c4840})
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/server/server.go:280 +0xe6d fp=0xc0008f7558 sp=0xc0008f6b00 pc=0x1b64e0d
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/server.New({0x238a480?, 0xc0003c4840}, {0x23723f8, 0xc0004fa000})
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/server/server.go:442 +0x598 fp=0xc0008f77f8 sp=0xc0008f7558 pc=0x1b66278
May 17 11:45:59 node1 crio[26471]: main.main.func2(0xc0005aff80)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/cmd/crio/main.go:278 +0xab2 fp=0xc0008f7d28 sp=0xc0008f77f8 pc=0x1b74292
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/github.com/urfave/cli/v2.(*App).RunContext(0xc0001fad00, {0x238a4b8?, 0xc000052020}, {0xc00004e230, 0x1, 0x1})
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/urfave/cli/v2/app.go:322 +0x953 fp=0xc0008f7eb8 sp=0xc0008f7d28 pc=0x1250253
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/github.com/urfave/cli/v2.(*App).Run(...)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/urfave/cli/v2/app.go:224
May 17 11:45:59 node1 crio[26471]: main.main()
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/cmd/crio/main.go:419 +0x5ba fp=0xc0008f7f80 sp=0xc0008f7eb8 pc=0x1b7375a
May 17 11:45:59 node1 crio[26471]: runtime.main()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:250 +0x212 fp=0xc0008f7fe0 sp=0xc0008f7f80 pc=0x443cb2
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc0008f7fe8 sp=0xc0008f7fe0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: goroutine 2 [force gc (idle)]:
May 17 11:45:59 node1 crio[26471]: runtime.gopark(0x0?, 0x0?, 0x0?, 0x0?, 0x0?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:361 +0xd6 fp=0xc00006cfb0 sp=0xc00006cf90 pc=0x444076
May 17 11:45:59 node1 crio[26471]: runtime.goparkunlock(...)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:367
May 17 11:45:59 node1 crio[26471]: runtime.forcegchelper()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:301 +0xad fp=0xc00006cfe0 sp=0xc00006cfb0 pc=0x443f0d
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc00006cfe8 sp=0xc00006cfe0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: created by runtime.init.6
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:289 +0x25
May 17 11:45:59 node1 crio[26471]: goroutine 3 [GC sweep wait]:
May 17 11:45:59 node1 crio[26471]: runtime.gopark(0x1?, 0x0?, 0x0?, 0x0?, 0x0?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:361 +0xd6 fp=0xc00006d790 sp=0xc00006d770 pc=0x444076
May 17 11:45:59 node1 crio[26471]: runtime.goparkunlock(...)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:367
May 17 11:45:59 node1 crio[26471]: runtime.bgsweep(0x0?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/mgcsweep.go:297 +0xd7 fp=0xc00006d7c8 sp=0xc00006d790 pc=0x42f257
May 17 11:45:59 node1 crio[26471]: runtime.gcenable.func1()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/mgc.go:177 +0x26 fp=0xc00006d7e0 sp=0xc00006d7c8 pc=0x424e06
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc00006d7e8 sp=0xc00006d7e0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: created by runtime.gcenable
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/mgc.go:177 +0x6b
May 17 11:45:59 node1 crio[26471]: goroutine 4 [GC scavenge wait]:
May 17 11:45:59 node1 crio[26471]: runtime.gopark(0x2a745af6eaa2?, 0x10000?, 0x0?, 0x0?, 0x0?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:361 +0xd6 fp=0xc00006df20 sp=0xc00006df00 pc=0x444076
May 17 11:45:59 node1 crio[26471]: runtime.goparkunlock(...)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:367
May 17 11:45:59 node1 crio[26471]: runtime.bgscavenge(0x0?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/mgcscavenge.go:364 +0x2a5 fp=0xc00006dfc8 sp=0xc00006df20 pc=0x42d065
May 17 11:45:59 node1 crio[26471]: runtime.gcenable.func2()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/mgc.go:178 +0x26 fp=0xc00006dfe0 sp=0xc00006dfc8 pc=0x424da6
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc00006dfe8 sp=0xc00006dfe0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: created by runtime.gcenable
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/mgc.go:178 +0xaa
May 17 11:45:59 node1 crio[26471]: goroutine 5 [finalizer wait]:
May 17 11:45:59 node1 crio[26471]: runtime.gopark(0x0?, 0x214c3f8?, 0x0?, 0x0?, 0x2000000020?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:361 +0xd6 fp=0xc00006c630 sp=0xc00006c610 pc=0x444076
May 17 11:45:59 node1 crio[26471]: runtime.goparkunlock(...)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:367
May 17 11:45:59 node1 crio[26471]: runtime.runfinq()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/mfinal.go:177 +0xb3 fp=0xc00006c7e0 sp=0xc00006c630 pc=0x423eb3
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc00006c7e8 sp=0xc00006c7e0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: created by runtime.createfing
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/mfinal.go:157 +0x45
May 17 11:45:59 node1 crio[26471]: goroutine 6 [chan receive]:
May 17 11:45:59 node1 crio[26471]: runtime.gopark(0xc00006e6d8?, 0x449c1b?, 0x20?, 0xe7?, 0x4608e5?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:361 +0xd6 fp=0xc00006e6c8 sp=0xc00006e6a8 pc=0x444076
May 17 11:45:59 node1 crio[26471]: runtime.chanrecv(0xc0001ba000, 0xc00006e7a0, 0x1)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/chan.go:577 +0x56c fp=0xc00006e758 sp=0xc00006e6c8 pc=0x41208c
May 17 11:45:59 node1 crio[26471]: runtime.chanrecv2(0x12a05f200?, 0x0?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/chan.go:445 +0x18 fp=0xc00006e780 sp=0xc00006e758 pc=0x411af8
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/k8s.io/klog/v2.(*loggingT).flushDaemon(0x0?)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/k8s.io/klog/v2/klog.go:1181 +0x6a fp=0xc00006e7c8 sp=0xc00006e780 pc=0x6b352a
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/k8s.io/klog/v2.init.0.func1()
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/k8s.io/klog/v2/klog.go:420 +0x26 fp=0xc00006e7e0 sp=0xc00006e7c8 pc=0x6b0626
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc00006e7e8 sp=0xc00006e7e0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: created by github.com/cri-o/cri-o/vendor/k8s.io/klog/v2.init.0
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/k8s.io/klog/v2/klog.go:420 +0xf6
May 17 11:45:59 node1 crio[26471]: goroutine 7 [GC worker (idle)]:
May 17 11:45:59 node1 crio[26471]: runtime.gopark(0x2a7475afab0e?, 0x0?, 0x0?, 0x0?, 0x0?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:361 +0xd6 fp=0xc00006ef58 sp=0xc00006ef38 pc=0x444076
May 17 11:45:59 node1 crio[26471]: runtime.gcBgMarkWorker()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/mgc.go:1207 +0xe5 fp=0xc00006efe0 sp=0xc00006ef58 pc=0x426ee5
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc00006efe8 sp=0xc00006efe0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: created by runtime.gcBgMarkStartWorkers
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/mgc.go:1131 +0x25
May 17 11:45:59 node1 crio[26471]: goroutine 18 [GC worker (idle)]:
May 17 11:45:59 node1 crio[26471]: runtime.gopark(0x2a7475afb947?, 0x0?, 0x0?, 0x0?, 0x0?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:361 +0xd6 fp=0xc000068758 sp=0xc000068738 pc=0x444076
May 17 11:45:59 node1 crio[26471]: runtime.gcBgMarkWorker()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/mgc.go:1207 +0xe5 fp=0xc0000687e0 sp=0xc000068758 pc=0x426ee5
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc0000687e8 sp=0xc0000687e0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: created by runtime.gcBgMarkStartWorkers
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/mgc.go:1131 +0x25
May 17 11:45:59 node1 crio[26471]: goroutine 39 [select]:
May 17 11:45:59 node1 crio[26471]: runtime.gopark(0xc00069ef88?, 0x3?, 0x0?, 0x30?, 0xc00069ef2a?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:361 +0xd6 fp=0xc00069ed98 sp=0xc00069ed78 pc=0x444076
May 17 11:45:59 node1 crio[26471]: runtime.selectgo(0xc00069ef88, 0xc00069ef24, 0xc0000bd380?, 0x0, 0xc000704540?, 0x1)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/select.go:328 +0x772 fp=0xc00069eed8 sp=0xc00069ed98 pc=0x453d52
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/k8s.io/apimachinery/pkg/util/wait.poller.func1.1()
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:708 +0x1c5 fp=0xc00069efe0 sp=0xc00069eed8 pc=0xfb9c25
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc00069efe8 sp=0xc00069efe0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: created by github.com/cri-o/cri-o/vendor/k8s.io/apimachinery/pkg/util/wait.poller.func1
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:691 +0xca
May 17 11:45:59 node1 crio[26471]: goroutine 33 [syscall]:
May 17 11:45:59 node1 crio[26471]: syscall.Syscall6(0xe8, 0x8, 0xc000637c14, 0x7, 0xffffffffffffffff, 0x0, 0x0)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/syscall/asm_linux_amd64.s:43 +0x5 fp=0xc000637b60 sp=0xc000637b58 pc=0x4c42a5
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/golang.org/x/sys/unix.EpollWait(0x756d206c6c757020?, {0xc000637c14?, 0x2064656d726f6672?, 0x7065732061206e69?}, 0x6763206574617261?)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/golang.org/x/sys/unix/zsyscall_linux_amd64.go:56 +0x58 fp=0xc000637bd0 sp=0xc000637b60 pc=0x57b438
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/github.com/fsnotify/fsnotify.(*fdPoller).wait(0xc000722000)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/fsnotify/fsnotify/inotify_poller.go:86 +0x7d fp=0xc000637c80 sp=0xc000637bd0 pc=0xbb6d7d
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/github.com/fsnotify/fsnotify.(*Watcher).readEvents(0xc0000a2320)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/fsnotify/fsnotify/inotify.go:206 +0x26e fp=0xc000647fc8 sp=0xc000637c80 pc=0xbb602e
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/github.com/fsnotify/fsnotify.NewWatcher.func1()
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/fsnotify/fsnotify/inotify.go:60 +0x26 fp=0xc000647fe0 sp=0xc000647fc8 pc=0xbb5326
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc000647fe8 sp=0xc000647fe0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: created by github.com/cri-o/cri-o/vendor/github.com/fsnotify/fsnotify.NewWatcher
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/fsnotify/fsnotify/inotify.go:60 +0x1c5
May 17 11:45:59 node1 crio[26471]: goroutine 50 [select]:
May 17 11:45:59 node1 crio[26471]: runtime.gopark(0xc0006a0f78?, 0x3?, 0x0?, 0x0?, 0xc0006a0eea?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:361 +0xd6 fp=0xc0006a0d58 sp=0xc0006a0d38 pc=0x444076
May 17 11:45:59 node1 crio[26471]: runtime.selectgo(0xc0006a0f78, 0xc0006a0ee4, 0x0?, 0x0, 0x0?, 0x1)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/select.go:328 +0x772 fp=0xc0006a0e98 sp=0xc0006a0d58 pc=0x453d52
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/github.com/cri-o/ocicni/pkg/ocicni.(*cniNetworkPlugin).monitorConfDir(0xc0003e4c00, 0x0?)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/cri-o/ocicni/pkg/ocicni/ocicni.go:150 +0x165 fp=0xc0006a0fc0 sp=0xc0006a0e98 pc=0xfb0ac5
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/github.com/cri-o/ocicni/pkg/ocicni.initCNI.func1()
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/cri-o/ocicni/pkg/ocicni/ocicni.go:261 +0x2a fp=0xc0006a0fe0 sp=0xc0006a0fc0 pc=0xfb136a
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc0006a0fe8 sp=0xc0006a0fe0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: created by github.com/cri-o/cri-o/vendor/github.com/cri-o/ocicni/pkg/ocicni.initCNI
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/github.com/cri-o/ocicni/pkg/ocicni/ocicni.go:261 +0x4dd
May 17 11:45:59 node1 crio[26471]: goroutine 51 [select]:
May 17 11:45:59 node1 crio[26471]: runtime.gopark(0xc0006a16b0?, 0x2?, 0x0?, 0x30?, 0xc0006a168c?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:361 +0xd6 fp=0xc0006a1510 sp=0xc0006a14f0 pc=0x444076
May 17 11:45:59 node1 crio[26471]: runtime.selectgo(0xc0006a16b0, 0xc0006a1688, 0x418b65?, 0x0, 0x0?, 0x1)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/select.go:328 +0x772 fp=0xc0006a1650 sp=0xc0006a1510 pc=0x453d52
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x238a4b8, 0xc000052020}, 0xc0006941b0, 0xfb990a?)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:658 +0xe5 fp=0xc0006a16e8 sp=0xc0006a1650 pc=0xfb9745
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/k8s.io/apimachinery/pkg/util/wait.poll({0x238a4b8, 0xc000052020}, 0x58?, 0xfb8a45?, 0x10?)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:594 +0x9a fp=0xc0006a1728 sp=0xc0006a16e8 pc=0xfb95fa
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/k8s.io/apimachinery/pkg/util/wait.PollInfiniteWithContext({0x238a4b8, 0xc000052020}, 0x418ec7?, 0x10?)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:498 +0x46 fp=0xc0006a1768 sp=0xc0006a1728 pc=0xfb9266
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/vendor/k8s.io/apimachinery/pkg/util/wait.PollInfinite(0x0?, 0x0?)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:488 +0x46 fp=0xc0006a17a8 sp=0xc0006a1768 pc=0xfb91e6
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/internal/config/cnimgr.(*CNIManager).pollUntilReady(0xc0002bf140)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/internal/config/cnimgr/cnimgr.go:38 +0x5e fp=0xc0006a17c8 sp=0xc0006a17a8 pc=0xfba1fe
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/internal/config/cnimgr.New.func1()
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/internal/config/cnimgr/cnimgr.go:32 +0x26 fp=0xc0006a17e0 sp=0xc0006a17c8 pc=0xfba166
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc0006a17e8 sp=0xc0006a17e0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: created by github.com/cri-o/cri-o/internal/config/cnimgr.New
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/internal/config/cnimgr/cnimgr.go:32 +0xed
May 17 11:45:59 node1 crio[26471]: goroutine 59 [select]:
May 17 11:45:59 node1 crio[26471]: runtime.gopark(0xc00069c738?, 0x2?, 0x0?, 0x30?, 0xc00069c6d4?)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/proc.go:361 +0xd6 fp=0xc000083d20 sp=0xc000083d00 pc=0x444076
May 17 11:45:59 node1 crio[26471]: runtime.selectgo(0xc000083f38, 0xc00069c6d0, 0x0?, 0x0, 0x0?, 0x1)
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/select.go:328 +0x772 fp=0xc000083e60 sp=0xc000083d20 pc=0x453d52
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/internal/resourcestore.(*ResourceStore).cleanupStaleResources(0xc0007611d0)
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/internal/resourcestore/resourcestore.go:89 +0x9c fp=0xc000083fc8 sp=0xc000083e60 pc=0x1aec49c
May 17 11:45:59 node1 crio[26471]: github.com/cri-o/cri-o/internal/resourcestore.NewWithTimeout.func1()
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/internal/resourcestore/resourcestore.go:67 +0x26 fp=0xc000083fe0 sp=0xc000083fc8 pc=0x1aec266
May 17 11:45:59 node1 crio[26471]: runtime.goexit()
May 17 11:45:59 node1 crio[26471]:         /usr/lib/go-1.18/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc000083fe8 sp=0xc000083fe0 pc=0x474fe1
May 17 11:45:59 node1 crio[26471]: created by github.com/cri-o/cri-o/internal/resourcestore.NewWithTimeout
May 17 11:45:59 node1 crio[26471]:         /usr/src/packages/BUILD/src/github.com/cri-o/cri-o/internal/resourcestore/resourcestore.go:67 +0xd6
May 17 11:45:59 node1 systemd[1]: crio.service: Main process exited, code=dumped, status=6/ABRT
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ An ExecStart= process belonging to unit crio.service has exited.
░░ 
░░ The process' exit code is 'dumped' and its exit status is 6.
May 17 11:45:59 node1 systemd[1]: crio.service: Failed with result 'core-dump'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ The unit crio.service has entered the 'failed' state with result 'core-dump'.
May 17 11:45:59 node1 systemd[1]: Failed to start Container Runtime Interface for OCI (CRI-O).
░░ Subject: A start job for unit crio.service has failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A start job for unit crio.service has finished with a failure.
░░ 
░░ The job identifier is 4500 and the job result is failed.
May 17 11:45:59 node1 systemd[1]: crio.service: Scheduled restart job, restart counter is at 5.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ Automatic restarting of the unit crio.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
May 17 11:45:59 node1 systemd[1]: Stopped Container Runtime Interface for OCI (CRI-O).
░░ Subject: A stop job for unit crio.service has finished
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A stop job for unit crio.service has finished.
░░ 
░░ The job identifier is 4578 and the job result is done.
May 17 11:45:59 node1 systemd[1]: crio.service: Start request repeated too quickly.
May 17 11:45:59 node1 systemd[1]: crio.service: Failed with result 'core-dump'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ The unit crio.service has entered the 'failed' state with result 'core-dump'.
May 17 11:45:59 node1 systemd[1]: Failed to start Container Runtime Interface for OCI (CRI-O).
░░ Subject: A start job for unit crio.service has failed
░░ Defined-By: systemd
░░ Support: http://www.ubuntu.com/support
░░ 
░░ A start job for unit crio.service has finished with a failure.
░░ 
░░ The job identifier is 4578 and the job result is failed.
root@ubuntu:~# 

What did you expect to happen?

restart crio running success

How can we reproduce it (as minimally and precisely as possible)?

kubeadm install kubernetes v1.24.0 with multus，then run systemctl restart crio in node1 , crio will failed ,and node1 is master node.

node2 and node3 is worker node

root@ubuntu:~# kubectl get pods -A -o wide
NAMESPACE     NAME                                       READY   STATUS        RESTARTS   AGE     IP              NODE    NOMINATED NODE   READINESS GATES
default       virt-launcher-testvm-hk44t                 2/2     Running       0          3h21m   10.233.92.8     node3   <none>           1/1
default       virt-launcher-testvmi-nocloud-mm8rf        0/2     Init:1/2      0          3h7m    10.233.96.9     node2   <none>           1/1
kube-system   calico-kube-controllers-544df655bd-cdpc2   1/1     Running       0          4h12m   10.233.96.2     node2   <none>           <none>
kube-system   calico-node-8vzss                          1/1     Running       0          4h12m   192.168.93.22   node3   <none>           <none>
kube-system   calico-node-kjnmf                          1/1     Running       0          4h12m   192.168.93.20   node1   <none>           <none>
kube-system   calico-node-mzgwv                          1/1     Running       0          4h12m   192.168.93.21   node2   <none>           <none>
kube-system   coredns-f657fccfd-5zq5n                    1/1     Running       0          142m    10.233.96.10    node2   <none>           <none>
kube-system   coredns-f657fccfd-b479q                    1/1     Running       0          142m    10.233.92.9     node3   <none>           <none>
kube-system   coredns-f657fccfd-ksds6                    1/1     Terminating   0          4h25m   10.233.90.2     node1   <none>           <none>
kube-system   coredns-f657fccfd-vjj5t                    1/1     Terminating   0          4h25m   10.233.90.1     node1   <none>           <none>
kube-system   kube-apiserver-node1                       1/1     Running       0          4h26m   192.168.93.20   node1   <none>           <none>
kube-system   kube-controller-manager-node1              1/1     Running       0          4h26m   192.168.93.20   node1   <none>           <none>
kube-system   kube-multus-ds-4df99                       1/1     Running       0          4h25m   192.168.93.21   node2   <none>           <none>
kube-system   kube-multus-ds-ng68q                       1/1     Running       0          4h25m   192.168.93.20   node1   <none>           <none>
kube-system   kube-multus-ds-xlfgf                       1/1     Running       0          4h25m   192.168.93.22   node3   <none>           <none>
kube-system   kube-proxy-57ch9                           1/1     Running       0          4h25m   192.168.93.20   node1   <none>           <none>
kube-system   kube-proxy-bl5hg                           1/1     Running       0          4h25m   192.168.93.21   node2   <none>           <none>
kube-system   kube-proxy-xgc52                           1/1     Running       0          4h25m   192.168.93.22   node3   <none>           <none>
kube-system   kube-scheduler-node1                       1/1     Running       0          4h26m   192.168.93.20   node1   <none>           <none>
kube-system   nodelocaldns-c9rjc                         1/1     Running       0          4h25m   192.168.93.22   node3   <none>           <none>
kube-system   nodelocaldns-db5mm                         1/1     Running       0          4h25m   192.168.93.21   node2   <none>           <none>
kube-system   nodelocaldns-jzfgx                         1/1     Running       0          4h25m   192.168.93.20   node1   <none>           <none>
kubevirt      virt-api-6bff47f5d4-hn66j                  1/1     Running       0          4h8m    10.233.92.4     node3   <none>           <none>
kubevirt      virt-api-6bff47f5d4-mk24f                  1/1     Running       0          4h8m    10.233.96.4     node2   <none>           <none>
kubevirt      virt-controller-6c85d678b4-22blb           1/1     Running       0          4h7m    10.233.92.5     node3   <none>           <none>
kubevirt      virt-controller-6c85d678b4-dspj2           1/1     Running       0          3h51m   10.233.96.7     node2   <none>           <none>
kubevirt      virt-handler-bncj5                         1/1     Running       0          3h49m   10.233.96.8     node2   <none>           <none>
kubevirt      virt-handler-gjfts                         1/1     Running       0          4h7m    10.233.92.6     node3   <none>           <none>
kubevirt      virt-operator-6b8b55559f-ld7nj             1/1     Running       0          4h9m    10.233.96.3     node2   <none>           <none>
kubevirt      virt-operator-6b8b55559f-wg2gf             1/1     Running       0          4h9m    10.233.92.2     node3   <none>           <none>
root@ubuntu:~# 

Anything else we need to know?

No response

CRI-O and Kubernetes version

root@node1:~# crio --version
crio version 1.24.0
Version:        1.24.0
GitCommit:      unknown
GitCommitDate:  unknown
GitTreeState:   clean
BuildDate:      2022-05-13T15:36:29Z
GoVersion:      go1.18.2
Compiler:       gc
Platform:       linux/amd64
Linkmode:       dynamic
BuildTags:      
  apparmor
  seccomp
  containers_image_ostree_stub
  exclude_graphdriver_btrfs
  exclude_graphdriver_devicemapper
LDFlags:          -s -w -X github.com/cri-o/cri-o/internal/pkg/criocli.DefaultsPath="" -X github.com/cri-o/cri-o/internal/version.buildDate=2022-05-13T15:36:29Z 
SeccompEnabled:   true
AppArmorEnabled:  true
Dependencies:     
  

root@node1:~# kubectl version
WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.0", GitCommit:"4ce5a8954017644c5420bae81d72b09b735c21f0", GitTreeState:"clean", BuildDate:"2022-05-03T13:46:05Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.4
Server Version: version.Info{Major:"1", Minor:"24", GitVersion:"v1.24.0", GitCommit:"4ce5a8954017644c5420bae81d72b09b735c21f0", GitTreeState:"clean", BuildDate:"2022-05-03T13:38:19Z", GoVersion:"go1.18.1", Compiler:"gc", Platform:"linux/amd64"}
root@node1:~# 

OS version

root@node1:~# cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04 (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

root@node1:~# uname -a
Linux node1 5.15.0-27-generic #28-Ubuntu SMP Thu Apr 14 04:55:28 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
root@node1:~# 

Additional environment details (AWS, VirtualBox, physical, etc.)

vmware workstation

when i restart node1 with reboot ,the crio will start success

root@node1:~# systemctl status crio
● crio.service - Container Runtime Interface for OCI (CRI-O)
     Loaded: loaded (/lib/systemd/system/crio.service; enabled; vendor preset: enabled)
     Active: active (running) since Tue 2022-05-17 14:16:57 CST; 1min 20s ago
       Docs: https://github.com/cri-o/cri-o
   Main PID: 802 (crio)
      Tasks: 15
     Memory: 154.8M
        CPU: 2.240s
     CGroup: /system.slice/crio.service
             └─802 /usr/bin/crio

May 17 14:17:28 node1 crio[802]: 2022-05-17 14:17:27.961 [INFO][2264] utils.go 196: Calico CNI releasing IP address ContainerID="9c328c905bc02d37eca59f66c9d88f7958f96ac598048d28070e7b409dd1ab6b"
May 17 14:17:28 node1 crio[802]: 2022-05-17 14:17:28.017 [INFO][2271] ipam_plugin.go 411: Releasing address using handleID ContainerID="9c328c905bc02d37eca59f66c9d88f7958f96ac598048d28070e7b409dd1ab>
May 17 14:17:28 node1 crio[802]: time="2022-05-17T14:17:28+08:00" level=info msg="About to acquire host-wide IPAM lock." source="ipam_plugin.go:352"
May 17 14:17:28 node1 crio[802]: time="2022-05-17T14:17:28+08:00" level=info msg="Acquired host-wide IPAM lock." source="ipam_plugin.go:367"
May 17 14:17:28 node1 crio[802]: 2022-05-17 14:17:28.058 [INFO][2271] ipam_plugin.go 430: Released address using handleID ContainerID="9c328c905bc02d37eca59f66c9d88f7958f96ac598048d28070e7b409dd1ab6>
May 17 14:17:28 node1 crio[802]: 2022-05-17 14:17:28.058 [INFO][2271] ipam_plugin.go 439: Releasing address using workloadID ContainerID="9c328c905bc02d37eca59f66c9d88f7958f96ac598048d28070e7b409dd1>
May 17 14:17:28 node1 crio[802]: time="2022-05-17T14:17:28+08:00" level=info msg="Released host-wide IPAM lock." source="ipam_plugin.go:373"
May 17 14:17:28 node1 crio[802]: 2022-05-17 14:17:28.064 [INFO][2264] k8s.go 589: Teardown processing complete. ContainerID="9c328c905bc02d37eca59f66c9d88f7958f96ac598048d28070e7b409dd1ab6b"
May 17 14:17:28 node1 crio[802]: time="2022-05-17 14:17:28.065629945+08:00" level=info msg="Successfully cleaned up network for pod 9c328c905bc02d37eca59f66c9d88f7958f96ac598048d28070e7b409dd1ab6b"
May 17 14:17:28 node1 crio[802]: time="2022-05-17 14:17:28.065659655+08:00" level=info msg="Successfully cleaned up network for pod b07acba74691cc95b43b5c9cde3ffe21e3943bfc541d5a812909bf1a3500c9f5"
lines 1-21/21 (END)
 ^C

root@node1:~# systemctl restart crio
root@node1:~#