Q: Problem migrating large all-in-one VMs from oVirt

Product Support Operations
1

Hi !

I’m in the process migrating private cloud from oVirt to Open Nebula.

All VMs (Linux and Windows) are made as all-in-one qcow2, both OS and data live in 1 partitioned disk image. As result, imported qcow2 are hundreds of gigabytes, and even after import into ON and consequential deployment, they are being left in image data store on the Front-End despite having active copy on KVM node.

If I understood correctly, please correct me if I’m wrong here, while VMs derived from imported images are active, images themselves can’t be deleted from image data store.

So the questions are:

  1. Is there any legit way to delete data from image data store while derived VMs are present in the system? For example, some type of detach VM from parent image in data store using some command line hacks?

  2. May be import VMs on ON KVM node using OS QEMU tools bypassing ON standard workflow, and then adopt in ON? Will ON copy OS/data qcow2 image into its datastore, so I can delete redundant data?

Thanks in advance for any suggestion(s).

Andrei

2

Need to add - images are not persistent, with CLONE = “YES” option in template during VM creation.

3

Hello @andreiv,

Welcome to this forum, and I would like to appreciate the effort you are performing with the migration.

So let’s check the options you have. What you are experience is a normal (or expected) behaviour for OpenNebula. Meaning that even if the images are non-persistent and you’re using CLONE = "YES" in the template, OpenNebula still keeps a reference between the instantiated VMs and the original image record in the Image Datastore. The clone operation creates separate disks for the VMs, but the base image remains registered and marked as USED as long as there are VMs instantiated from it.

So unfortunately, this behavior is still expected and there isn’t a supported way to delete the original imported image while those VMs exist.

You can find more information on the documentation:
Virtual Machine Images
VM Instances

Let me know if this is the references you need, or if you need something more specific.

Regards,

4

Hi, Francisco,

Thanks for clarification. Is there any way to import VM without keeping those huge files in image datastore? For example, import VM directly into KVM host bypassing ON, and the adopt it? Or adaptation process also creates copy of the wild VM qcow2 in image datastore?

These are details of ON internals which are not completely clear at first sight, I used several AI and they generate a wild variety of unworkable schemes.

Thanks in advance.

Andrei

5

Hi @andreiv,

The confusion might be around the platform version you are using. I’m assuming you are using 7.0, so the references I’m using are from that documentation.

So basically, there is no supported way to import a VM into OpenNebula without having its disk registered in an Image Datastore.

Even if you:

  • Manually place the qcow2 on a KVM host
  • Define a VM XML outside OpenNebula
  • Then try to “adopt” it

OpenNebula still requires the disk to be represented as an Image object in its database. The platform is image-centric by design. If you give me some time, I can double check with engineering team in case there’s a workaround, but that’s the info I have.

Cheers,

6

Hi, Francisco,

Yes, I’m using V7.01.

OK, thanks a lot, please ask dev team !

I’m understand hat product has some design concept and objective, yet keeping huge amount of redundant data is a bad option from any point of view. Re-assembling existing VMs from scratch with separate OS and data images is enormously time consuming. Current limits, simply put, narrows ON implementation considerably if anything needs to be migrated from other systems.

May be its possible to define another dummy parent entity from image datastore using some command line hack? Its a cloned non-persistent copy and has no usable connection to parent stuff in image datastore anyway.

My solution is this:

cd /var/lib/one/datastores/1
rm -f <image_name>; qemu-img create -f qcow2 <image_name> 1K; chown oneadmin:oneadmin <image_name>

Please pass my feedback to your dev team.
Andrei

7

This is not supported, but you could

  • create a template with a dummy disk image
  • instantiate this template as persistent
  • a persistent template will be created. Stop that VM
  • overwrite the disk image (in the images datastore) of the persistent VM with the image that you have. Check the name of the persistent image with the onevm command

Note, again, that this is not supported. It is just an idea that may work for your use case.

8

Hi @andreiv,

Another alternative is to define the VMs with a dummy volatile image. Then, power off the VM and on the host where it was running, replace the volatile disk image file with the actual qcow2 file.

This way, you’ll entirely bypass the Image Datastore, which has its pros and cons, though. I’d say you should try to see if that fits your use case.

Best Regards
Anton Todorov

1 Like
9

Thank you @atodorov_storpool , that will work as well and the image will be created on the system datastore instead of the image one. The main problem here is (correct me if I’m wrong) that datastores on oVirt do not have different functionalities as in OpenNebula

10

oVirt has image (ovirt-image-repository), data and export storage domains. Concept somewhat similar to ON, but without “hard-wired“ links of disk images between domains.

1 Like
11

Indeed, that is why I mentioned that such an approach has its pros and cons:
- The VM disk is entirely detached from the image datastore, so functions like disk snapshot and save-as (another image in an image datastore) are not available.
+ It is true that there is an extra step to activate, but the VM backups could work. The disk resize (up/growth) works too.

It is up to the use case which functionality is needed :innocent:

12

Is that correct that ON doesn’t support backup of the volatile disk images to both rsync and restic datastores? Or ChatGPT is wrong here?

Thanks.

13

According to the docs, it is turned off by default and should be enabled explicitly: VM Backup Operations |

14

After some hiatus I resumed migration.
Using volatile disk images as suggested by Francisco leaves VM without snapshot feature which sometimes is essential for us, so I tried different approach.
images are not persistent, with CLONE = “YES” option in template during VM creation.
Here is what is being done as test:

  1. Created Win10 VM in a standard way (oVirt qcow image imported in data store), verified that it runs OK (VM ID = 27).
  2. Imported empty qcow in image datastore.
  3. Created VM with this empty qcow image and template “Win10-BIOS-AV2“ (VM ID = 32).
  4. Copied in /var/lib/one/datastores/0
    ../27/disk.0.snap/0 → ../32/disk.0.snap/0
    ../27/disk.1 → ../32/disk.1
    Result: System starts to boot then says its damaged. No Win OS recovery helps. Really have no clue why it doesn’t work.

My next possible approach: replace in image data store on web front end VM “template“ qcow with empty qcow, because basically in my case each “template” image in data store is used to create just a single VM.
Yet I have no idea if this could break some other functionality. Please advise.
Thanks.

15

Just to clarify - it was my suggestion; Bruno’s is to work with Image datastyre.
That said, I assume that both VMs were powered off when copying the files? Then it is difficult to say why it doesn’t work if there is no significant difference in the VM configuration. But even then, the worst that could happen is to lose the license activation. It should boot at least…