KVM running on node but state POWEROFF in opennebula

Hi everyone,

I have a problem with my KVM VMs on one node.
After a shutdown command (that sometimes takes a long time and even requires a force shutdown). The VM will just not receive anything and will still run but OpenNebula will show it as POWEROFF. I don’t know why this VM takes a lot of time to shutdown with OpenNebula because it works fine if I do a virsh shutdown command.

Here is what i have from oned.log

Sun May 29 01:00:06 2022 [Z0][ReM][D]: Req:6832 UID:0 IP:127.0.0.1 one.vm.action invoked , "poweroff", 180
Sun May 29 01:00:07 2022 [Z0][DiM][D]: Powering off VM 180
Sun May 29 01:00:07 2022 [Z0][ReM][D]: Req:6832 UID:0 one.vm.action result SUCCESS, 180
Sun May 29 01:05:46 2022 [Z0][VMM][D]: Message received: LOG I 197 Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/shutdown '04a6cb07-9957-49ec-8ba2-6ea21cffd3dc' 'kvmhost' 180 kvmhost
Sun May 29 01:05:46 2022 [Z0][VMM][D]: Message received: LOG E 180 Timed out shutting down 04a6cb07-9957-49ec-8ba2-6ea21cffd3dc
Sun May 29 01:05:46 2022 [Z0][VMM][D]: Message received: LOG I 180 ExitCode: 255
Sun May 29 01:05:46 2022 [Z0][VMM][D]: Message received: LOG I 180 Failed to execute virtualization driver operation: shutdown.
Sun May 29 01:05:47 2022 [Z0][VMM][D]: Message received: SHUTDOWN FAILURE 180 Timed out shutting down 04a6cb07-9957-49ec-8ba2-6ea21cffd3dc

Sun May 29 01:08:30 2022 [Z0][ReM][D]: Req:9456 UID:0 IP:127.0.0.1 one.vm.action invoked , "poweroff-hard", 180
Sun May 29 01:08:31 2022 [Z0][DiM][D]: Powering off VM 180
Sun May 29 01:08:31 2022 [Z0][ReM][D]: Req:9456 UID:0 one.vm.action result SUCCESS, 180
Sun May 29 01:08:32 2022 [Z0][VMM][D]: Message received: LOG I 180 ExitCode: 0
Sun May 29 01:08:33 2022 [Z0][VMM][D]: Message received: LOG I 180 Successfully execute virtualization driver operation: cancel.
Sun May 29 01:08:33 2022 [Z0][VMM][D]: Message received: LOG I 180 ExitCode: 0
Sun May 29 01:08:33 2022 [Z0][VMM][D]: Message received: LOG I 180 Successfully execute network driver operation: clean.
Sun May 29 01:08:33 2022 [Z0][VMM][D]: Message received: CANCEL SUCCESS 180 -

Here is from the syslog of the VM at the same time

May 29 01:00:00 kvmVMname nrpe[215598]: Host 192.168.0.238 is not allowed to talk to us!
May 29 01:00:01 kvmVMname freshclam[51099]: Sun May 29 01:00:01 2022 -> Received signal: wake up
May 29 01:00:01 kvmVMname freshclam[51099]: Sun May 29 01:00:01 2022 -> ClamAV update process started at Sun May 29 01:00:01 2022
May 29 01:00:01 kvmVMname freshclam[51099]: Sun May 29 01:00:01 2022 -> daily.cld database is up-to-date (version: 26555, sigs: 1984781, f-level: 90, builder: raynman)
May 29 01:00:01 kvmVMname freshclam[51099]: Sun May 29 01:00:01 2022 -> main.cld database is up-to-date (version: 62, sigs: 6647427, f-level: 90, builder: sigmgr)
May 29 01:00:01 kvmVMname freshclam[51099]: Sun May 29 01:00:01 2022 -> bytecode.cvd database is up-to-date (version: 333, sigs: 92, f-level: 63, builder: awillia2)
May 29 01:00:04 kvmVMname postfix/submission/smtpd[215099]: disconnect from unknown[87.246.7.213] ehlo=1 auth=0/1 rset=0/1 quit=1 commands=2/4
May 29 01:00:08 kvmVMname postfix/submission/smtpd[214113]: connect from unknown[40.70.67.193]
May 29 01:00:08 kvmVMname postfix/submission/smtpd[214113]: disconnect from unknown[40.70.67.193] ehlo=1 auth=0/1 quit=1 commands=2/3
May 29 01:00:10 kvmVMname postfix/submission/smtpd[215099]: warning: hostname net6-ip213.linkbg.com does not resolve to address 87.246.7.213: No address associated with hostname
May 29 01:00:10 kvmVMname postfix/submission/smtpd[215099]: connect from unknown[87.246.7.213]
May 29 01:00:14 kvmVMname systemd[1]: bumblebeed.service: Scheduled restart job, restart counter is at 1411.
May 29 01:00:14 kvmVMname systemd[1]: Stopped Bumblebee C Daemon.
May 29 01:00:14 kvmVMname systemd[1]: Started Bumblebee C Daemon.
May 29 01:00:14 kvmVMname bumblebeed[215599]: [85081.154675] [ERROR]No integrated video card found, quitting.
May 29 01:00:14 kvmVMname systemd[1]: bumblebeed.service: Main process exited, code=exited, status=1/FAILURE
May 29 01:00:14 kvmVMname systemd[1]: bumblebeed.service: Failed with result 'exit-code'.
May 29 01:00:15 kvmVMname postfix/submission/smtpd[215099]: disconnect from unknown[87.246.7.213] ehlo=1 auth=0/1 rset=0/1 quit=1 commands=2/4
May 29 01:00:17 kvmVMname postfix/smtpd[215618]: connect from 186-45-250-248.dynamic.tstt.net.tt[186.45.250.248]
May 29 01:00:19 kvmVMname postfix/smtpd[215618]: lost connection after EHLO from 186-45-250-248.dynamic.tstt.net.tt[186.45.250.248]
May 29 01:00:19 kvmVMname postfix/smtpd[215618]: disconnect from 186-45-250-248.dynamic.tstt.net.tt[186.45.250.248] ehlo=1 commands=1
May 29 01:00:21 kvmVMname postfix/submission/smtpd[214113]: warning: hostname net6-ip213.linkbg.com does not resolve to address 87.246.7.213: No address associated with hostname
May 29 01:00:21 kvmVMname postfix/submission/smtpd[214113]: connect from unknown[87.246.7.213]
May 29 01:00:27 kvmVMname postfix/submission/smtpd[214113]: disconnect from unknown[87.246.7.213] ehlo=1 auth=0/1 rset=0/1 quit=1 commands=2/4
May 29 01:00:30 kvmVMname postfix/submission/smtpd[215099]: warning: hostname net6-ip213.linkbg.com does not resolve to address 87.246.7.213: No address associated with hostname
May 29 01:00:30 kvmVMname postfix/submission/smtpd[215099]: connect from unknown[87.246.7.213]
May 29 01:00:34 kvmVMname nrpe[215621]: Host 192.168.0.238 is not allowed to talk to us!
May 29 01:00:34 kvmVMname nrpe[215623]: Host 192.168.0.238 is not allowed to talk to us!
May 29 01:00:35 kvmVMname postfix/submission/smtpd[214113]: connect from unknown[37.186.62.81]
May 29 01:00:36 kvmVMname postfix/submission/smtpd[215099]: disconnect from unknown[87.246.7.213] ehlo=1 auth=0/1 rset=0/1 quit=1 commands=2/4
May 29 01:00:36 kvmVMname postfix/submission/smtpd[215099]: connect from unknown[40.70.67.193]
May 29 01:00:37 kvmVMname postfix/submission/smtpd[215099]: disconnect from unknown[40.70.67.193] ehlo=1 auth=0/1 quit=1 commands=2/3
May 29 01:00:43 kvmVMname postfix/submission/smtpd[215099]: warning: hostname net6-ip213.linkbg.com does not resolve to address 87.246.7.213: No address associated with hostname
May 29 01:00:43 kvmVMname postfix/submission/smtpd[215099]: connect from unknown[87.246.7.213]
May 29 01:00:43 kvmVMname systemd[1]: unattended-upgrades.service: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Stopped Unattended Upgrades Shutdown.
May 29 01:00:43 kvmVMname systemd[1]: Stopping Session c2 of user gdm.
May 29 01:00:43 kvmVMname gdm3: Tried to look up non-existent conversation gdm-launch-environment
May 29 01:00:43 kvmVMname systemd[1]: Removed slice system-getty.slice.
May 29 01:00:43 kvmVMname gdm3: Freeing conversation 'gdm-launch-environment' with active job
May 29 01:00:43 kvmVMname systemd[1]: Removed slice system-modprobe.slice.
May 29 01:00:43 kvmVMname systemd[1]: Stopped target Graphical Interface.
May 29 01:00:43 kvmVMname systemd[1]: Stopped target Remote File Systems (Pre).
May 29 01:00:43 kvmVMname kernel: [85106.808330] rfkill: input handler enabled
May 29 01:00:43 kvmVMname systemd[1]: Stopped target Timers.
May 29 01:00:43 kvmVMname systemd[1]: apt-daily-upgrade.timer: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Stopped Daily apt upgrade and clean activities.
May 29 01:00:43 kvmVMname systemd[1]: apt-daily.timer: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Stopped Daily apt download activities.
May 29 01:00:43 kvmVMname systemd[1]: e2scrub_all.timer: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Stopped Periodic ext4 Online Metadata Check for All Filesystems.
May 29 01:00:43 kvmVMname systemd[1]: fstrim.timer: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Stopped Discard unused blocks once a week.
May 29 01:00:43 kvmVMname systemd[1]: fwupd-refresh.timer: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Stopped Refresh fwupd metadata regularly.
May 29 01:00:43 kvmVMname systemd[1]: logrotate.timer: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Stopped Daily rotation of log files.
May 29 01:00:43 kvmVMname systemd[1]: man-db.timer: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Stopped Daily man-db regeneration.
May 29 01:00:43 kvmVMname systemd[1]: motd-news.timer: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Stopped Message of the Day.
May 29 01:00:43 kvmVMname systemd[1]: systemd-tmpfiles-clean.timer: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Stopped Daily Cleanup of Temporary Directories.
May 29 01:00:43 kvmVMname systemd[1]: ua-messaging.timer: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Stopped Ubuntu Advantage update messaging.
May 29 01:00:43 kvmVMname systemd[1]: lvm2-lvmpolld.socket: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Closed LVM2 poll daemon socket.
May 29 01:00:43 kvmVMname systemd[1]: systemd-rfkill.socket: Succeeded.
May 29 01:00:43 kvmVMname systemd[1]: Closed Load/Save RF Kill Switch Status /dev/rfkill Watch.
May 29 01:00:43 kvmVMname systemd[1]: Unmounting RPC Pipe File System...
May 29 03:17:47 kvmVMname systemd[1]: Mounted RPC Pipe File System.
May 29 03:17:47 kvmVMname systemd[1]: Starting Flush Journal to Persistent Storage...
May 29 03:17:47 kvmVMname systemd-modules-load[355]: Inserted module 'msr'
May 29 03:17:47 kvmVMname systemd[1]: Finished Load Kernel Modules.
May 29 03:17:47 kvmVMname systemd[1]: Mounting FUSE Control File System...
May 29 03:17:47 kvmVMname systemd[1]: Mounting Kernel Configuration File System...
May 29 03:17:47 kvmVMname systemd[1]: Starting Apply Kernel Variables...
May 29 03:17:47 kvmVMname systemd[1]: Mounted Kernel Configuration File System.
May 29 03:17:47 kvmVMname udevadm[379]: systemd-udev-settle.service is deprecated.
May 29 03:17:47 kvmVMname systemd[1]: Mounted FUSE Control File System.
May 29 03:17:47 kvmVMname systemd[1]: Finished Set the console keyboard layout.
May 29 03:17:47 kvmVMname systemd[1]: Finished udev Coldplug all Devices.
May 29 03:17:47 kvmVMname systemd[1]: Starting Helper to synchronize boot up for ifupdown...
May 29 03:17:47 kvmVMname systemd[1]: Starting udev Wait for Complete Device Initialization...
May 29 03:17:47 kvmVMname systemd[1]: Finished Load/Save Random Seed.
May 29 03:17:47 kvmVMname multipathd[610]: --------start up--------
May 29 03:17:47 kvmVMname multipathd[610]: read /etc/multipath.conf
May 29 03:17:47 kvmVMname multipathd[610]: path checkers start up
May 29 03:17:47 kvmVMname systemd-sysusers[367]: Failed to check if group sync already exists: Invalid argument
May 29 03:17:47 kvmVMname zpool[619]: cannot import 'zpoolname': no such pool or dataset
May 29 03:17:47 kvmVMname zpool[619]: #011Destroy and re-create the pool from
May 29 03:17:47 kvmVMname zpool[619]: #011a backup source.
May 29 03:17:47 kvmVMname systemd-sysusers[367]: Failed to check if group nobody already exists: Invalid argument
May 29 03:17:47 kvmVMname systemd[1]: Finished Create System Users.
May 29 03:17:47 kvmVMname systemd[1]: Starting Create Static Device Nodes in /dev...
May 29 03:17:47 kvmVMname systemd-sysctl[375]: Not setting net/ipv4/conf/all/promote_secondaries (explicit setting exists).
May 29 03:17:47 kvmVMname zvol_wait[720]: No zvols found, nothing to do.
May 29 03:17:47 kvmVMname systemd-sysctl[375]: Not setting net/ipv4/conf/default/promote_secondaries (explicit setting exists).
May 29 03:17:47 kvmVMname apparmor.systemd[725]: Restarting AppArmor
May 29 03:17:47 kvmVMname apparmor.systemd[725]: Reloading AppArmor profiles
May 29 03:17:47 kvmVMname systemd[1]: Finished Apply Kernel Variables.
May 29 03:17:47 kvmVMname apparmor.systemd[769]: Skipping profile in /etc/apparmor.d/disable: usr.sbin.rsyslogd
May 29 03:17:47 kvmVMname systemd[1]: Finished Create Static Device Nodes in /dev.
May 29 03:17:47 kvmVMname systemd[1]: Starting udev Kernel Device Manager...
May 29 03:17:47 kvmVMname systemd-udevd[417]: Network interface NamePolicy= disabled on kernel command line, ignoring.
May 29 03:17:47 kvmVMname systemd[1]: Finished Uncomplicated firewall.
May 29 03:17:47 kvmVMname systemd[1]: Started udev Kernel Device Manager.
May 29 03:17:47 kvmVMname one-contextd[743]: Started for type local
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Show Plymouth Boot Screen being skipped.
May 29 03:17:47 kvmVMname one-contextd[744]: Acquiring lock /var/run/one-context/one-context.lock
May 29 03:17:47 kvmVMname systemd[1]: Started Dispatch Password Requests to Console Directory Watch.
May 29 03:17:47 kvmVMname one-contextd[746]: Acquired lock /var/run/one-context/one-context.lock
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Forward Password Requests to Plymouth Directory Watch being skipped.
May 29 03:17:47 kvmVMname one-contextd[762]: Mounting CD-ROM /dev/sr0 on /var/run/one-context/mount.SnMthj
May 29 03:17:47 kvmVMname systemd[1]: Reached target Local Encrypted Volumes.
May 29 03:17:47 kvmVMname one-contextd[767]: Found context /var/run/one-context/mount.SnMthj/context.sh
May 29 03:17:47 kvmVMname systemd-udevd[466]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
May 29 03:17:47 kvmVMname one-contextd[775]: Comparing /var/run/one-context/context.sh.vOfLaC and /var/run/one-context/context.sh.local for changes
May 29 03:17:47 kvmVMname systemd-udevd[461]: Using default interface naming scheme 'v245'.
May 29 03:17:47 kvmVMname one-contextd[777]: New context with changes
May 29 03:17:47 kvmVMname systemd-udevd[469]: Using default interface naming scheme 'v245'.
May 29 03:17:47 kvmVMname one-contextd[785]: Processing local scripts
May 29 03:17:47 kvmVMname systemd-udevd[456]: Using default interface naming scheme 'v245'.
May 29 03:17:47 kvmVMname one-contextd[787]: Script loc-05-grow-rootfs: Starting ...
May 29 03:17:47 kvmVMname systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.
May 29 03:17:47 kvmVMname one-contextd[844]: Script loc-05-grow-rootfs output: NOCHANGE: partition 1 is size 209487839. it cannot be grown
May 29 03:17:47 kvmVMname one-contextd[844]: resize2fs 1.45.5 (07-Jan-2020)
May 29 03:17:47 kvmVMname one-contextd[844]: The filesystem is already 26185979 (4k) blocks long.  Nothing to do!
May 29 03:17:47 kvmVMname systemd-udevd[469]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
May 29 03:17:47 kvmVMname one-contextd[845]: Script loc-05-grow-rootfs: Finished with exit code 0
May 29 03:17:47 kvmVMname systemd-udevd[461]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
May 29 03:17:47 kvmVMname one-contextd[847]: Script loc-09-timezone: Starting ...
May 29 03:17:47 kvmVMname systemd-udevd[456]: ethtool: autonegotiation is unset or enabled, the speed and duplex are not writable.
May 29 03:17:47 kvmVMname one-contextd[850]: Script loc-09-timezone: Finished with exit code 0
May 29 03:17:47 kvmVMname systemd[1]: Found device /dev/disk/by-label/UEFI.
May 29 03:17:47 kvmVMname one-contextd[852]: Script loc-10-network: Starting ...
May 29 03:17:47 kvmVMname systemd-udevd[461]: eth1: Process '/usr/bin/timeout 5 /usr/sbin/service one-context-reconfigure-delayed start' failed with exit code 124.
May 29 03:17:47 kvmVMname one-contextd[1162]: Script loc-10-network output: ifdown: interface eth0 not configured
May 29 03:17:47 kvmVMname one-contextd[1162]: ifdown: interface eth1 not configured
May 29 03:17:47 kvmVMname one-contextd[1162]: ifdown: interface eth2 not configured
May 29 03:17:47 kvmVMname one-contextd[1162]: invoke-rc.d: could not determine current runlevel
May 29 03:17:47 kvmVMname one-contextd[1162]: message repeated 2 times: [ invoke-rc.d: could not determine current runlevel]
May 29 03:17:47 kvmVMname systemd-udevd[456]: eth0: Process '/usr/bin/timeout 5 /usr/sbin/service one-context-reconfigure-delayed start' failed with exit code 124.
May 29 03:17:47 kvmVMname one-contextd[1163]: Script loc-10-network: Finished with exit code 0
May 29 03:17:47 kvmVMname avahi-daemon[1520]: Found user 'avahi' (UID 132) and group 'avahi' (GID 139).
May 29 03:17:47 kvmVMname systemd-udevd[466]: lo: Process '/usr/bin/timeout 5 /usr/sbin/service one-context-reconfigure-delayed start' failed with exit code 124.
May 29 03:17:47 kvmVMname one-contextd[1165]: Script loc-10-network-pci: Starting ...
May 29 03:17:47 kvmVMname avahi-daemon[1520]: Successfully dropped root privileges.
May 29 03:17:47 kvmVMname systemd-udevd[469]: eth2: Process '/usr/bin/timeout 5 /usr/sbin/service one-context-reconfigure-delayed start' failed with exit code 124.
May 29 03:17:47 kvmVMname one-contextd[1173]: Script loc-10-network-pci: Finished with exit code 0
May 29 03:17:47 kvmVMname avahi-daemon[1520]: avahi-daemon 0.7 starting up.
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped.
May 29 03:17:47 kvmVMname one-contextd[1175]: Script loc-11-dns: Starting ...
May 29 03:17:47 kvmVMname zed: eid=1 class=vdev.no_replicas pool_guid=0x6BA0CFD4A6BCE93A  
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Show Plymouth Boot Screen being skipped.
May 29 03:17:47 kvmVMname one-contextd[1198]: Script loc-11-dns: Finished with exit code 0
May 29 03:17:47 kvmVMname zed: eid=2 class=zpool pool_guid=0x6BA0CFD4A6BCE93A  
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Forward Password Requests to Plymouth Directory Watch being skipped.
May 29 03:17:47 kvmVMname one-contextd[1200]: Script loc-14-mount-swap: Starting ...
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Set Up Additional Binary Formats being skipped.
May 29 03:17:47 kvmVMname one-contextd[1205]: Script loc-14-mount-swap: Finished with exit code 0
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Rebuild Hardware Database being skipped.
May 29 03:17:47 kvmVMname one-contextd[1207]: Script loc-16-gen-env: Starting ...
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Platform Persistent Storage Archival being skipped.
May 29 03:17:47 kvmVMname one-contextd[1216]: Script loc-16-gen-env: Finished with exit code 0
May 29 03:17:47 kvmVMname systemd[1]: Found device RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (QEMU Virtual Machine).
May 29 03:17:47 kvmVMname one-contextd[1218]: Script loc-20-set-username-password: Starting ...
May 29 03:17:47 kvmVMname systemd[1]: Found device RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (QEMU Virtual Machine).
May 29 03:17:47 kvmVMname one-contextd[1243]: Script loc-20-set-username-password: Finished with exit code 0
May 29 03:17:47 kvmVMname systemd[1]: Found device RTL-8100/8101L/8139 PCI Fast Ethernet Adapter (QEMU Virtual Machine).
May 29 03:17:47 kvmVMname one-contextd[1245]: Script loc-22-ssh_public_key: Starting ...
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in OpenVSwitch configuration for cleanup being skipped.
May 29 03:17:47 kvmVMname one-contextd[1259]: Script loc-22-ssh_public_key: Finished with exit code 0
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Show Plymouth Boot Screen being skipped.
May 29 03:17:47 kvmVMname one-contextd[1261]: Script loc-24-ssh_host_key: Starting ...
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Forward Password Requests to Plymouth Directory Watch being skipped.
May 29 03:17:47 kvmVMname one-contextd[1264]: Script loc-24-ssh_host_key: Finished with exit code 0
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Set Up Additional Binary Formats being skipped.
May 29 03:17:47 kvmVMname one-contextd[1266]: Script loc-30-console: Starting ...
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Rebuild Hardware Database being skipped.
May 29 03:17:47 kvmVMname one-contextd[1414]: Script loc-30-console: Finished with exit code 0
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Platform Persistent Storage Archival being skipped.
May 29 03:17:47 kvmVMname one-contextd[1420]: Script loc-35-securetty: Starting ...
May 29 03:17:47 kvmVMname systemd[1]: Finished Helper to synchronize boot up for ifupdown.
May 29 03:17:47 kvmVMname one-contextd[1428]: Script loc-35-securetty: Finished with exit code 0
May 29 03:17:47 kvmVMname systemd[1]: Finished udev Wait for Complete Device Initialization.
May 29 03:17:47 kvmVMname one-contextd[1429]: Finished scripts processing
May 29 03:17:47 kvmVMname systemd[1]: Starting Device-Mapper Multipath Device Controller...
May 29 03:17:47 kvmVMname one-contextd[1430]: Done
May 29 03:17:47 kvmVMname systemd[1]: Condition check resulted in Import ZFS pools by device scanning being skipped.
May 29 03:17:47 kvmVMname one-contextd[1431]: Unmounting /var/run/one-context/mount.SnMthj
May 29 03:17:47 kvmVMname systemd[1]: Starting Install ZFS kernel module...
May 29 03:17:47 kvmVMname one-contextd[1435]: Releasing lock /var/run/one-context/one-context.lock
May 29 03:17:47 kvmVMname systemd[1]: Finished Install ZFS kernel module.
May 29 03:17:47 kvmVMname systemd[1]: Starting Import ZFS pools by cache file...
May 29 03:17:47 kvmVMname sh[1450]: ifup: interface eth0 already configured
May 29 03:17:47 kvmVMname kernel: [    0.000000] Linux version 5.4.0-113-generic (buildd@lcy02-amd64-067) (gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04.1)) #127-Ubuntu SMP Wed May 18 14:30:56 UTC 2022 (Ubuntu 5.4.0-113.127-generic 5.4.181)

You can maybe find clues on the qemu log for the VM. It should be located at /var/log/libvirt/qemu/one-<vm_id>.log. You can also try to run manually the command executed to shutdown the VM and check the output when it fails. The command should be virsh destroy one-<vm_id>.

do you have the qemu-guest-agent installed ? - May be this can help.

Then - may be update your vm configuration, too:

Features:

  • ACPI : YES

  • QEMU Guest Agent: YES

and yes “virsh destroy one-<vm_id>” is the correct command on the hypervisor to do a force shutdown.