Reposting from reddit r/sysadmin https://www.reddit.com/r/sysadmin/comments/hoz3ur/open_nebula_on_zfs_addon_for_multiple_hosts/
Btw r/opennebula seems dead - no posts for one year…
I am considering to run our OpenNebula so that all images are stored in ZVOLs on ZFS. Currently our OpenNebula runs on 70 hosts and we will be upgrading it, as it runs a very outdated version already. We are using qcow2 delta images. The VMs are relatively short-lived and there is only around 50 base images in the system overall and adding more is not easily possible due to disk space limitations. When a base image is updated, it is more work to start using it than I would like to, because due to the disk space limitations older images need to be removed before new are added. I believe that is also the reason why the base images smell - they do not get the fixes they should get at adequate pace.
I have found the addon for ZFS https://libraries.io/github/OpenNebula/addon-zfs and played around a bit. It seems to be implemented so that it only runs on single host. Seems to work fine but I did not find a simple way how to make it work for multiple hosts
My idea of using ZFS in ONE is as follows:
- Base images are snapshots of ZFS ZVOLs
- VM images are created as a clone of the ZFS snapshot
This would solve the problem with modifying the base images. After modifying the base image, new snapshot on the ZVOL would be created and distributed to all hosts. ZFS send|receive commands are very efficient at this. Also old images would not have to be removed when only small changes are made as the disk space utilization would only grow by the delta between the new snapshot and what already was in the image.
Using clones for the VMs as raw disks will give us features similar to qcow2 delta images, only more efficient and it should be easier to use
Most of our hosts have 2TB of storage without any redundancy and we do not intend to change that, or at least not at this moment. If the disks fail, the users understand they loose their VMs.
Our OpenNebula is split into two clusters also because of disk space limitations. Implementing ZFS properly could also allow us to merge everything in to a single cluster, reducing complexity and simplifying the deployment.
What are your thoughts about implementing this in OpenNebula? Is this doable with only an addon which would provide our way of using the ZFS as datastore and a transfer mechanism to make the images (snapshots) available across OpenNebula? Has any one done anything similar before and is willing to share insight or even code?