I have a on-premise host where I have deployed a smal number of VMs. For each VM two files are created in local images datastore: one seams to be used for the VM and the other seams to be a checksum. However, none of these files is deleted from disk when I remove the VM. Also, they cannot be seen in images list of the datastore.
Is this behavior expected? Disk space is geting shorter and shorter too fast on the on-prem host.
Best regards,
Macus Rocha
Hi @marcusrocha,
Which storage driver are you using? Note that files created in the image datastore folder (i.e /var/lib/one/datastores/<img_ds_id>) are images which later will be used as a VM disks. If you run the oneimage show <img_id> command and check the SOURCE attribute you can see the corresponding file in the image datastore for the image.
Hi @cgonzalez, thanks for your answer!
I’m using DS_MAD=fs and TM_MAD=ssh
Sure these are images to be used as VM disks. Copying and keeping images on the onprem host is very usefull if you are using non-presistent VMs, as the image may be used for successive VM activations. It does greatly reduces the time needed to start a new VM in the host.
However, if you are using permatent images and permanent VMs, as the image is not removed after the machine(s) are terminated, the copy of the image datastore gets crowded with these orphaned image copies.
So, is there a particular configurations of datastores that would remove these orphaned copies?
Or, maybe, permanent VMs are not as usefull as I imagined?
Regards,
Marcus
Hi @marcusrocha,
From your answer I assume you’re using a REPLICA_HOST, right? If that the case I think it might be useful to customize the REPLICA_MAX_SIZE_MB attribute [1].
Hi @cgonzalez ,
Actually, I’m not using a REPLICA_HOST, it is an onprem cluster with a single host. What happens is that a copy of the image of the VM is copied to the host and placed in a local copy of the Images Datastore. This local copy is not removed when the VM is removed.
Regards,
Marcus
Hi @marcusrocha,
In that case I’m not sure which temporal copy your talking about, probably I’m missing something. Could you share the exact path of this temporal copy that you’re mentioning and the log file of one of the affected VM (e.g /var/log/one/<vm_id>.log) to see if that trough some light on it?
Hi @cgonzalez !
The path is “//var/lib/one/datastores/305/4adf16848e3eccae1c6a72436bf2d635”
VM log and other logs follow attached. Please let me know if you need any other information.
Regards, Marcus
datastore-305.txt (1.2 KB)
host.txt (5.3 KB)
vm.txt (3.5 KB)
925.log (2.4 KB)
Hi @marcusrocha,
I think that you’re indeed using a REPLICA_HOST. I think that’s automatically enabled when deploying an onprem cluster. Can you double check that your system Datastore doesn’t have the REPLICA_HOST attribute defined?
When REPLICA_HOST is enabled, a copy of the original images is transferred to the hypervisor node set as REPLICA_HOST to act as an image repository cache. This is intended to improve the performance when the frontend node and the hypervisor nodes are far way. This temporal copy as you called will remains there in case you deploy a new VM from that image, and OpenNebula will automatically discard it when the cache reach its maximum size.
If you want to disable this behavior you just need to remove the REPLICA_HOST attribute from the system Datastore.
NOTE that the Datastore that you shared is the image Datastore.
Hi @cgonzalez ,
Thanks a lot for your helpful information. Actually, my datastore/image/cluster configuration was a bit weird. After fixing it, things worked fine. In fact, REPLICA_HOST of the system datastore was already on.
One last question: to control cache size in the host, I should set REPLICA_MAX_SIZE_MB for the system datastore, am I rigth?
Regards,
Marcus Rocha
Hi @marcusrocha,
I guess that depends on your requirements, if REPLICA_MAX_SIZE_MB is not set, REPLICA_MAX_USED_PERC will be used, which by default will use the 90% of the capacity of the file system used by the replica.
# Maximum size of cached images on replica in MB, if not specified
# REPLICA_MAX_USED_PERC is used
REPLICA_MAX_SIZE_MB=""
# Maximum usage in % of the replica filesystem
REPLICA_MAX_USED_PERC=90
For your reference: OneStor Datastore — OpenNebula 6.4.0 documentation
Hi @cgonzalez !
Thanks a lot for your attention. However descreibed in the documentation, I was in doubt about the relation between these parameters.
Best regards,
Marcus Rocha