# Local-datastore image & system with Persistent image is not linking and corrupts image on delete **URL:** \ **Category:** Operations\ **Created:** [March 6, 2025, 11:53pm UTC](https://forum.opennebula.io/t/local-datastore-image-system-with-persistent-image-is-not-linking-and-corrupts-image-on-delete/13633 "2025-03-06T23:53:48Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![kemcfarl](https://avatars.discourse-cdn.com/v4/letter/k/d78d45/32.png) [@kemcfarl](https://forum.opennebula.io/u/kemcfarl)\ **Post date:** [March 6, 2025, 11:53pm UTC](https://forum.opennebula.io/t/local-datastore-image-system-with-persistent-image-is-not-linking-and-corrupts-image-on-delete/13633/1 "2025-03-06T23:53:48Z")
Hi, I’m been using OpenNebula successfully for a few months now with “shared” NF storage between my Front-End and all KVM Node. (Nice Product!) I have a new user that has a server with no network storage options, but has ample local disk storage from 800G SSDs. I’ve been looking at using OpenNebula local-datastore support as defined here: > **[Local Storage Datastore — OpenNebula 6.10.4 documentation](https://docs.opennebula.io/6.10/open_cluster_deployment/storage_setup/local_ds.html#local-ds)** It’s taken me a bit to get this almost right? I was able to add the system datastore and image datastore. The image data is properly showing the 736G disk with proper available storage. Output below. I was able to Import a 64G qcow2 image from the Front-end to the KVM node into datastore ID112. It performed ssh copy and dir 112 creation on the node all on its own. The image datastore is tied to this system datastore COMPATIBLE\_SYS\_DS=“109” I have bridge-list set for this single KVM host as well. I can launch the VM via VM template with the IMAGE set to persistent in the image data store just fine. The VM comes up with my Ubuntu image. However, unlike my using persistent images on NFS shared env, it made a copy of the qcow2 image to disk.0, not an symbolic link. This doesn’t seem right to me and I can’t seem to find my mistake? It gets worse, when I terminate the VM. The original image is wiped out with a tiny .snap file. Below is all the show output after launching the VM and before VM release. Here is the corrupted image after vm terminate. ```auto root@l14-kvm-tools-host:/var/lib/one/datastores# ls -lrht 112 total 4.0K drwxrwxr-x 2 oneadmin oneadmin 4.0K Mar 6 23:47 682011c2c808524b8e4e9ff189f4860a.snap lrwxrwxrwx 1 oneadmin oneadmin 38 Mar 6 23:47 682011c2c808524b8e4e9ff189f4860a -> 682011c2c808524b8e4e9ff189f4860a.snap/ root@l14-kvm-tools-host:/var/lib/one/datastores# du 4 ./107 4 ./102 4 ./112/682011c2c808524b8e4e9ff189f4860a.snap 8 ./112 8 ./109 8 ./100 36 . root@l14-kvm-tools-host:/var/lib/one/datastores# ls -lrht 109 total 0 root@l14-kvm-tools-host:/var/lib/one/datastores# ls -lrht 112 */* lrwxrwxrwx 1 oneadmin oneadmin 38 Mar 6 23:47 112/682011c2c808524b8e4e9ff189f4860a -> 682011c2c808524b8e4e9ff189f4860a.snap/ 112/682011c2c808524b8e4e9ff189f4860a.snap: total 0 lrwxrwxrwx 1 oneadmin oneadmin 1 Mar 6 23:47 682011c2c808524b8e4e9ff189f4860a.snap -> . 112: total 4.0K drwxrwxr-x 2 oneadmin oneadmin 4.0K Mar 6 23:47 682011c2c808524b8e4e9ff189f4860a.snap lrwxrwxrwx 1 oneadmin oneadmin 38 Mar 6 23:47 682011c2c808524b8e4e9ff189f4860a -> 682011c2c808524b8e4e9ff189f4860a.snap/ root@l14-kvm-tools-host:/var/lib/one/datastores# ``` Am I missing some persistent setting somewhere? I’m watching various logs with debug level turned up to max, but nothing is jumping out with error or failure. Thanks for your time/help and guidance, –Ken **BEFORE RELEASING THE VM** ```auto root@l14-kvm-tools-host:/var/lib/one/datastores# ls -lrth */* -rw-rw-r-- 1 oneadmin oneadmin 17G Mar 6 23:16 112/682011c2c808524b8e4e9ff189f4860a 109/40: total 17G -rw-r--r-- 1 oneadmin oneadmin 364K Mar 6 23:18 disk.1 -rw-rw-r-- 1 oneadmin oneadmin 4.3K Mar 6 23:18 vm.xml -rw-rw-r-- 1 oneadmin oneadmin 1.2K Mar 6 23:18 ds.xml -rw-rw-r-- 1 oneadmin oneadmin 1.9K Mar 6 23:18 deployment.0 -rw-rw-r-- 1 oneadmin oneadmin 17G Mar 6 23:40 disk.0 root@l14-kvm-tools-host:/var/lib/one/datastores# root@l14-kvm-tools-host:/var/lib/one/datastores# ls -lrht 112 total 17G -rw-rw-r-- 1 oneadmin oneadmin 17G Mar 6 23:16 682011c2c808524b8e4e9ff189f4860a root@l14-kvm-tools-host:/var/lib/one/datastores# ``` ```auto oneadmin@lab-openneb-1:/var/log/one$ onedatastore show 10.84.107.94-image DATASTORE 112 INFORMATION ID : 112 NAME : 10.84.107.94-image USER : oneadmin GROUP : oneadmin CLUSTERS : 100 TYPE : IMAGE DS_MAD : fs TM_MAD : shared BASE PATH : /var/lib/one//datastores/112 DISK_TYPE : FILE STATE : READY DATASTORE CAPACITY TOTAL: : 731.4G FREE: : 237.5G USED: : 456.6G LIMIT: : - PERMISSIONS OWNER : um- GROUP : u-- OTHER : --- DATASTORE TEMPLATE ALLOW_ORPHANS="FORMAT" BRIDGE_LIST="10.84.107.94" CLONE_TARGET="SYSTEM" CLONE_TARGET_SSH="SYSTEM" CLUSTERS="100" COMPATIBLE_SYS_DS="109" CONVERT="DRIVER" DATASTORE_CAPACITY_CHECK="yes" DISK_TYPE="FILE" DISK_TYPE_SSH="FILE" DS_MAD="fs" LN_TARGET="NONE" LN_TARGET_SSH="SYSTEM" RESTRICTED_DIRS="/" SAFE_DIRS="/var/tmp" TM_MAD="shared" TM_MAD_SYSTEM="ssh" TYPE="IMAGE_DS" IMAGES 16 oneadmin@lab-openneb-1:/var/log/one$ oneadmin@lab-openneb-1:/var/log/one$ onedatastore show 10.84.107.94-system DATASTORE 109 INFORMATION ID : 109 NAME : 10.84.107.94-system USER : oneadmin GROUP : oneadmin CLUSTERS : 100 TYPE : SYSTEM DS_MAD : - TM_MAD : ssh BASE PATH : /var/lib/one//datastores/109 DISK_TYPE : FILE STATE : READY DATASTORE CAPACITY TOTAL: : - FREE: : - USED: : - LIMIT: : - PERMISSIONS OWNER : um- GROUP : u-- OTHER : --- DATASTORE TEMPLATE ALLOW_ORPHANS="YES" DATASTORE_CAPACITY_CHECK="YES" DISK_TYPE="FILE" DS_MIGRATE="YES" NO_DECOMPRESS="NO" QCOW2_STANDALONE="NO" RESTIC_SPARSIFY="NO" RESTRICTED_DIRS="/" RSYNC_SPARSIFY="NO" SAFE_DIRS="/var/tmp" SHARED="NO" TM_MAD="ssh" TYPE="SYSTEM_DS" IMAGES oneadmin@lab-openneb-1:/var/log/one$ oneadmin@lab-openneb-1:/var/log/one$ onetemplate show Clone-ubuntu2204-UPF2-8vC TEMPLATE 4 INFORMATION ID : 4 NAME : Clone-ubuntu2204-UPF2-8vCPU-40G-convert_9 USER : oneadmin GROUP : oneadmin LOCK : None REGISTER TIME : 03/06 19:00:38 PERMISSIONS OWNER : um- GROUP : --- OTHER : --- TEMPLATE CONTENTS CONTEXT=[ NETWORK="NO" ] CPU="0.0833333333333333" DISK=[ IMAGE="Clone-ubuntu2204-UPF2-8vCPU-40G-convert_9", IMAGE_UNAME="oneadmin" ] GRAPHICS=[ LISTEN="0.0.0.0", TYPE="VNC" ] HYPERVISOR="kvm" MEMORY="32768" NIC=[ METHOD="skip", MODEL="vmxnet3", NETWORK="service", NETWORK_UID="0", NETWORK_UNAME="oneadmin" ] SCHED_REQUIREMENTS="(HYPERVISOR=kvm)" VCPU="8" oneadmin@lab-openneb-1:/var/log/one$ oneadmin@lab-openneb-1:/var/log/one$ oneimage show Clone-ubuntu2204-UPF2-8vCPU-4 IMAGE 16 INFORMATION ID : 16 NAME : Clone-ubuntu2204-UPF2-8vCPU-40G-convert_9 USER : oneadmin GROUP : oneadmin LOCK : None DATASTORE : 10.84.107.94-image TYPE : OS REGISTER TIME : 03/06 23:14:47 PERSISTENT : Yes SOURCE : /var/lib/one//datastores/112/682011c2c808524b8e4e9ff189f4860a PATH : /var/tmp/Clone-ubuntu2204-UPF2-8vCPU-40G-convert_9.qcow2 FORMAT : qcow2 SIZE : 64G STATE : used RUNNING_VMS : 1 PERMISSIONS OWNER : um- GROUP : --- OTHER : --- IMAGE TEMPLATE DESCRIPTION="laasdev" DEV_PREFIX="sd" VIRTUAL MACHINES ID USER GROUP NAME 40 oneadmin oneadmin kentest2 oneadmin@lab-openneb-1:/var/log/one$ oneadmin@lab-openneb-1:/var/log/one$ onevm show 40 VIRTUAL MACHINE 40 INFORMATION ID : 40 NAME : kentest2 USER : oneadmin GROUP : oneadmin STATE : ACTIVE LCM_STATE : RUNNING LOCK : None RESCHED : No HOST : 10.84.107.94 CLUSTER ID : 100 CLUSTER : omega START TIME : 03/06 23:18:09 END TIME : - DEPLOY ID : 58444f4c-50b4-40cb-9e1d-108513d76455 VIRTUAL MACHINE MONITORING CPU : 2.0 MEMORY : 18.2G NETTX : 0K NETRX : 0K DISKRDBYTES : 527782590 DISKRDIOPS : 26396 DISKWRBYTES : 32900096 DISKWRIOPS : 1670 ID : 40 TIMESTAMP : 1741303426 VM_QEMU_PING : QEMU Guest Agent monitoring disabled PERMISSIONS OWNER : um- GROUP : --- OTHER : --- VM DISKS ID DATASTORE TARGET IMAGE SIZE TYPE SAVE 0 10.84.107. sda Clone-ubuntu2204-UPF2-8vCPU-40G-con 16.5G/64G file YES 1 - hda CONTEXT 1M/- - - VM NICS ID NETWORK BRIDGE IP MAC PCI_ID 0 service onebr.111 - 02:00:84:8e:60:c8 SECURITY NIC_ID NETWORK SECURITY_GROUPS 0 service 0 SECURITY GROUP TYPE PROTOCOL NETWORK RANGE ID NAME VNET START SIZE 0 default OUTBOUND ALL 0 default INBOUND ALL VIRTUAL MACHINE HISTORY SEQ UID REQ HOST ACTION DS START TIME PROL 0 - - 10.84.107.94 none 109 03/06 23:18:15 0d 00h05m 0h00m1 USER TEMPLATE HYPERVISOR="kvm" SCHED_REQUIREMENTS="(HYPERVISOR=kvm)" VIRTUAL MACHINE TEMPLATE AUTOMATIC_DS_REQUIREMENTS="(\"CLUSTERS/ID\" @> 100) & (\"ID\" @> 109)" AUTOMATIC_NIC_REQUIREMENTS="(\"CLUSTERS/ID\" @> 100)" AUTOMATIC_REQUIREMENTS="(CLUSTER_ID = 100) & !(PUBLIC_CLOUD = YES) & !(PIN_POLIC CONTEXT=[ DISK_ID="1", NETWORK="NO", TARGET="hda" ] CPU="0.0833333" GRAPHICS=[ LISTEN="0.0.0.0", PORT="5940", TYPE="VNC" ] MEMORY="32768" OS=[ UUID="58444f4c-50b4-40cb-9e1d-108513d76455" ] TEMPLATE_ID="4" TM_MAD_SYSTEM="ssh" VCPU="8" VMID="40" oneadmin@lab-openneb-1:/var/log/one$ ``` ---
**Author:** ![kemcfarl](https://avatars.discourse-cdn.com/v4/letter/k/d78d45/32.png) [@kemcfarl](https://forum.opennebula.io/u/kemcfarl)\ **Post date:** [March 8, 2025, 8:10pm UTC](https://forum.opennebula.io/t/local-datastore-image-system-with-persistent-image-is-not-linking-and-corrupts-image-on-delete/13633/2 "2025-03-08T20:10:01Z")
BTW this is with one-deploy fresh install from v6.10.0.1 I’ve been looking at this for another full day. I believe I have a better understanding of the different TM\_MAD\_CONF types in oned.conf and local-datastores. In my above case I had added the local-data store with the following settings with similar CLI command: ```auto oneadmin@lab-openneb-1:~/ken$ cat datastore-local.conf NAME = "10.84.107.94-image" TYPE = "IMAGE_DS" DS_MAD = "fs" TM_MAD = "shared" BRIDGE_LIST = "10.84.107.94" COMPATIBLE_SYS_DS = "109" DATASTORE_CAPACITY_CHECK = "yes" CLUSTERS = "100" CONVERT = "DRIVER" oneadmin@lab-openneb-1:~/ken$ onedatastore create ./datastore-local.conf ``` Note I have BRIDGE\_LIST set to the 1 host that will use this Image and system datastore with no network storage available. When I created the above datastore it created the 112 directory only on kvm host 10.84.107.94 … No directory 112 on the front-end datastore directory. When I imported the image with below CLI command it copied the qcow2 image to dirctory 112 on the kvm host 10.84.107.94 only. This all lead me to believe it could run everything locally on kvm node 10.84.107.94, without needing a copy of the image on the front-end too? ```auto oneadmin@lab-openneb-1:~/ken$ cat image.tmpl NAME=Clone-ubuntu2204-UPF2-8vCPU-40G-convert_9 PATH=/var/tmp/Clone-ubuntu2204-UPF2-8vCPU-40G-convert_9.qcow2 #SOURCE=/var/tmp/Clone-ubuntu2204-UPF2-8vCPU-40G-convert_9.qcow2 TYPE=OS SIZE=65536 PERSISTENT=YES oneadmin@lab-openneb-1:~/ken$ oneimage create -d 114 image.tmpl ``` A couple questions 1. is above a supported configuration without the image on Front-end? Is image datastore create/import w/single KVM BRIDGE\_LIST set expected to perform this way? 2. I’ve changed to creating the datastore with TM\_MAD = “ssh”. vs "shared’ originally. If the BRIDGE\_LIST var is set it behaves the same way putting the Image datastore on the KVM host only. Nothing on the Front-end. Same behavior on qcow2 import too. The main difference here is at instantiation the VM template will fail in ssh mode as the image is not found on the Front-end to tar/copy. 3. It seems that with any image datastore configuration the original image must be accessible on the datastore via the front-end. Is that correct? I have local ssh copy working as long as I don’t create the image datastore with the BRIDGE\_LIST setting. Then everything goes to the Front-end on import/setup. TM\_MAD = “ssh” works with copy or persistent image copy back to front-end on VM delete. Thanks again for you time and guidance, –Ken ---
**Author:** ![kemcfarl](https://avatars.discourse-cdn.com/v4/letter/k/d78d45/32.png) [@kemcfarl](https://forum.opennebula.io/u/kemcfarl)\ **Post date:** [March 13, 2025, 10:33pm UTC](https://forum.opennebula.io/t/local-datastore-image-system-with-persistent-image-is-not-linking-and-corrupts-image-on-delete/13633/3 "2025-03-13T22:33:09Z")
Hi Maybe my post is too messy or missing the mark. Can anyone confirm that setting a Bridge\_list on IMAGE datastore “create” is support for SSH mode? ```auto oneadmin@lab-openneb-1:~/ken$ cat datastore-local-2.conf NAME = "10.84.107.94-image-2" TYPE = "IMAGE_DS" DS_MAD = "fs" TM_MAD = "ssh" BRIDGE_LIST = "10.84.107.94" <<<<<<<<<<<<<< HERE COMPATIBLE_SYS_DS = "109" DATASTORE_CAPACITY_CHECK = "yes" #CLUSTERS = "100" #CONVERT = "DRIVER" oneadmin@lab-openneb-1:~/ken$ ``` If supported where is the image datastore expected to live on the One Front-End or the KVM node itself? Thanks for the clarification, –Ken