I have newly setup ON 7.2 private cloud (all hosts Debian 12), with few test VMs and Restic backup. Now I have a problem which could be a software or setup bug.
For VM#1 (on on-node11.lcl KVM host) imported from oVirt both full and increment backup works.
Another VM#2 (on on-node10.lcl KVM host) is made from backup of VM#1, with same template.
VM#2 full backup works, yet incremental backup (increment mode = snapshot, mode = increment, FS freeze = suspend) fails. VM#2 is powered down at the time of backup.
Both KVM hosts are identical in terms of installed software packages and versions.
Log files cuts below.
One suggestion for VM log display in web front end - right now most recent messages are on the bottom, what is not convenient, but should be on the top! If no live scrolling, then put refresh button.
Tried incremental backup of another running VM (this one is Linux, not Windows). switched FS Freeze from “Suspend” to “QEMU Agent”. It fails with errors.
[root@sol]# ps aux | grep guest
root 595 0.0 0.0 44168 2068 ? Ss 13:34 0:00 /usr/bin/qemu-ga --method=virtio-serial --path=/dev/virtio-ports/org.qemu.guest_agent.0 --blacklist=guest-file-open,guest-file-close,guest-file-read,guest-file-write,guest-file-seek,guest-file-flush,guest-exec,guest-exec-status -F/etc/qemu-ga/fsfreeze-hook
…….
[root@sol]# systemctl status qemu-guest-agent
qemu-guest-agent.service - QEMU Guest Agent
Loaded: loaded (/usr/lib/systemd/system/qemu-guest-agent.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2026-04-24 13:34:55 EEST; 1h 6min ago
Main PID: 595 (qemu-ga)
CGroup: /system.slice/qemu-guest-agent.service
`-595 /usr/bin/qemu-ga --method=virtio-serial --path=/dev/virtio-ports/org.qemu.guest_agent.0 --blacklist=guest-file-open,guest-file-close,guest-file-read,guest-file-write,guest-f…
Apr 24 13:34:55 sol systemd[1]: Started QEMU Guest Agent.
from /var/log/one-30.log:
Save TPM state# --------------------------------------# --------------------------------------# Create backup live# --------------------------------------/var/tmp/one/tm/lib/backup_qcow2.rb -l -d “0” -x /var/lib/one//datastores/0/30/backup/vm.xml -p /var/lib/one//datastores/0/30Error preparing disk files: Error executing 'qemu-img create -f qcow2 -o backing_fmt=qcow2 -b /var/lib/one//datastores/0/30//var/lib/one//datastores/0/30/disk.0.snap/0 /var/lib/one//datastores/0/30//var/lib/one//datastores/0/30/disk.0.snap/1 ':qemu-img: /var/lib/one//datastores/0/30//var/lib/one//datastores/0/30/disk.0.snap/1: Could not open ‘/var/lib/one//datastores/0/30//var/lib/one//datastores/0/30/disk.0.snap/0’: No such file or directoryCould not open backing image.+ ‘[’ -d /var/lib/one//datastores/0/30/tmp ‘]’+ rm -rf /var/lib/one//datastores/0/30/tmp+ ‘[’ -d /var/lib/one//datastores/0/30/backup ‘]’+ rm -rf /var/lib/one//datastores/0/30/backup+ mkdir -p /var/lib/one//datastores/0/30/tmp+ mkdir -p /var/lib/one//datastores/0/30/backup+
Full backup works as expected. I assume incremental backup feature is somewhat buggy at the time being. If any of ON developer(s) are interested to find and fix this, please contact me.
It seems like an absolute path is being duplicated when creating the overlay, which might indicate an issue with how disk paths are reconstructed after restore.
To narrow this down, could you please provide the following for two VMs:
one where incremental backup works
one where it fails
For both VMs:
Full VM template:
onevm show -x <VM_ID>
Disk layout on datastore:
ls -la /var/lib/one/datastores/0/<VM_ID>/
ls -la /var/lib/one/datastores/0/<VM_ID>/disk.0.snap/
Backing chain (you need to power off VMs for this action):
qemu-img info --backing-chain /var/lib/one/datastores/0/<VM_ID>/disk.0
How the VM was created
Comparing a working vs failing VM should help identify what differs in the disk chain or metadata.
VM #30 (with failed incremental Restic backup) was created from the uploaded image in /var/tmp.
VM #40 (with incremental Restic going on OK) was created from the full backup of VM #30.
Template is the same for both.
Logs attached as zip file.
PS. Migrating VM to another KVM host doesn’t fix Restic incremental backup failure.
So at this point the problem seems related to the VM disk path layout itself rather than Restic or incremental backups in general.
What is still unclear is how VM#30 ended up with this absolute symlink structure. Since the VM was created from an uploaded image/import workflow, that may be related.