Hi friends!
In the process of setting up secondary Ceph cluster we are exploring our options for ONE VM failovers.
For this, we would like to set up RBD mirroring and have a disaster recovery scenario ready (in case primary Ceph cluster becomes unreachable).
As far as I could find out, OpenNebula doesn’t immediatelly supports any form of Ceph RDB Mirroring utilization.
To make things more difficult, ONE heavily relies on naming convention and we can never have two VMs that use image with same underlaying name.
E.g. VM with ID 1337 would be the only one using Ceph image “pool/one-100-1337-0” and we can’t have different VM defined with that image.
As Ceph doesn’t support RBD mirroring to a different name (e.g. pool/image-a => pool/image-b), this looks like a show stopper…
Is there any recommendation as to how to approach this problem?
Our current thinking involves a lot of low-level tinkering and raw libvirt manipulation to boot VMs off main site.
Kind regards,
Momo.
Hi Momo,
I am not a Ceph expert so please read the following as a shared thoughts instead of specific recommendation/solution.
First to clarify the base setup:
A. An OpenNebula controller with Mysql/MariaDB
A. Two separate CEPH clusters with RBD mirroring
B. Two separate set of Hypervisors
D. The network routing is setup/resolved in a way that both sites could export/route VM’s IPs
Following the Ceph’s definition let name the current OpenNebula setup primary and the DR one remote
What do you think of creating a frontend node on the remote site that has no running OpenNebula services, only the database is replicated with the primary site and all credentials are same. The hosts from the remote site are defined in the primary OpenNeubla but in Disabled state. The remote site is configured to work with the remote ceph where the images are mirrored(same pools, same names… only with different credentials).
So in case of DR - (assuming the entire primary site is down) you need to reconfigure the remote frontend: set the database as master, start the OpenNebula services, disable the primary hosts and enable remote hosts. My assumption is, that if the remote hosts are configured as twins of the primary ones same bridge names, etc. The OpenNebula will behave like it is run on the “primary” site. Same database, same credentials, same VNets, and same Images for same VMs. This way you could just start the VMs on the remote site…
I’d think in this direction to resolve a DR case.
Please excuse me if I omit something obvious issues but unknown to me…
Best Regards,
Anton Todorov
Hi Anton,
Thank you so much for sharing your idea.
It looks very promising at a first glance.
I’ll discuss internally and provide more info as it comes by.
Again, thank you, highly appreciated 
Kind regards,
Momo.