# Migrate from Gluster to another NFS Datastore

**URL:** <https://forum.opennebula.io/t/migrate-from-gluster-to-another-nfs-datastore/12284>\
**Category:** Operations\
**Created:** [July 18, 2024, 3:31pm UTC](https://forum.opennebula.io/t/migrate-from-gluster-to-another-nfs-datastore/12284 "2024-07-18T15:31:34Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![alex.shulta](https://avatars.discourse-cdn.com/v4/letter/a/94ad74/32.png) [@alex.shulta](https://forum.opennebula.io/u/alex.shulta)\
**Post date:** [July 18, 2024, 3:31pm UTC](https://forum.opennebula.io/t/migrate-from-gluster-to-another-nfs-datastore/12284/1 "2024-07-18T15:31:34Z")

</div>

I am using Gluster as the back-end storage for my image and system datastores which consists of 9, RHEL 7 servers, replica 3. Our goal is to take out 3 Gluster servers, rebuild as RHEL 8 using CEPH and then migrate the data, VMs, etc. from the Gluster datastores to the new CEPH datastores.

I am wondering if this is even possible, first off. And if it is possible, what is the best course of action to do so (ex. re-build the servers, create new datastores attached as CEPH, then use something like ‘rsync’ to sync the old Gluster datastore with the new CEPH datastore).

Any information would be helpful!

---

<div class="post-metadata">

**Author:** ![apicon](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/apicon/32/7947_2.png) [@apicon](https://forum.opennebula.io/u/apicon)\
**Post date:** [July 22, 2024, 10:41am UTC](https://forum.opennebula.io/t/migrate-from-gluster-to-another-nfs-datastore/12284/2 "2024-07-22T10:41:47Z")

</div>

Hi, Alex.

It is certainly possible to migrate disk images from GlusterFS to CEPH.

You only need to import them by registering the disk image by using the console too (`oneimage create`) or using SunStone Disk Create Menu.

OpenNebula will detect the disk image file format (for example, `qcow2`) and register it in the CEPH Pool you desire.

Best Regards,

---

<div class="post-metadata">

**Author:** ![alex.shulta](https://avatars.discourse-cdn.com/v4/letter/a/94ad74/32.png) [@alex.shulta](https://forum.opennebula.io/u/alex.shulta)\
**Post date:** [July 22, 2024, 5:25pm UTC](https://forum.opennebula.io/t/migrate-from-gluster-to-another-nfs-datastore/12284/3 "2024-07-22T17:25:06Z")

</div>

So I essentially have to import every single disk image in the new CEPH filesystem? And then what about the existing VMs? Is it a similar situation?

---

<div class="post-metadata">

**Author:** ![apicon](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/apicon/32/7947_2.png) [@apicon](https://forum.opennebula.io/u/apicon)\
**Post date:** [July 23, 2024, 7:10am UTC](https://forum.opennebula.io/t/migrate-from-gluster-to-another-nfs-datastore/12284/4 "2024-07-23T07:10:49Z")

</div>

For an existing VM, you can import each one of its disk images into the CEPH datastore and modify the template to point to the new imported disks. Another option is cloning the existing VM template without cloning the disks and modify the cloned VM template to point the new imported disks images into CEPH

---

<div class="post-metadata">

**Author:** ![alex.shulta](https://avatars.discourse-cdn.com/v4/letter/a/94ad74/32.png) [@alex.shulta](https://forum.opennebula.io/u/alex.shulta)\
**Post date:** [July 24, 2024, 12:31pm UTC](https://forum.opennebula.io/t/migrate-from-gluster-to-another-nfs-datastore/12284/5 "2024-07-24T12:31:16Z")

</div>

Is it possible to sync the Gluster filesystem with the Ceph filesystem and then just re-link the datastore directories?

i.e., I stop all OpenNebula services and disable the hosts and power off all VMs. I then sync Gluster with Ceph. I remove the soft-links from Gluster to /var/lib/one/datastores/{x…z} and re-create the soft-links with Ceph. Having all of the same data, is this technically doable?

We have about 70 VMs and 30 images, so I’m not sure the method you gave would be a quick or easy turnaround.

---

<div class="post-metadata">

**Author:** ![apicon](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/apicon/32/7947_2.png) [@apicon](https://forum.opennebula.io/u/apicon)\
**Post date:** [July 24, 2024, 12:50pm UTC](https://forum.opennebula.io/t/migrate-from-gluster-to-another-nfs-datastore/12284/6 "2024-07-24T12:50:53Z")

</div>

Unfortunately, the sync process is not simple, as the GlusterFS and the CEPH internals are very different. VMs that contain CEPH disks do not access files contained in a POSIX location (as GlusterFS does). They use libRADOS CEPH library to access the image object directly using block mode.  
Part of the process can be automated by scripting the registration of the VM disks in CEPH and creating customized templates based on the source VMs running in GlusterFS datastore

---

<div class="post-metadata">

**Author:** ![alex.shulta](https://avatars.discourse-cdn.com/v4/letter/a/94ad74/32.png) [@alex.shulta](https://forum.opennebula.io/u/alex.shulta)\
**Post date:** [July 24, 2024, 1:09pm UTC](https://forum.opennebula.io/t/migrate-from-gluster-to-another-nfs-datastore/12284/7 "2024-07-24T13:09:50Z")

</div>

What if both filesystems are mounted via NFS and synced from one NFS mount (as Gluster) to the other NFS mount (as Ceph)? They both should appear to be the same, would they not?

---

<div class="post-metadata">

**Author:** ![apicon](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/apicon/32/7947_2.png) [@apicon](https://forum.opennebula.io/u/apicon)\
**Post date:** [July 25, 2024, 7:36am UTC](https://forum.opennebula.io/t/migrate-from-gluster-to-another-nfs-datastore/12284/8 "2024-07-25T07:36:42Z")

</div>

Unfortunately, using CEPHfs as a NFS mountpoint to serve disk image files for VMs is not recommended and it has a huge performance penalty on VM block I/O operation.

---

<div class="post-metadata">

**Author:** ![alex.shulta](https://avatars.discourse-cdn.com/v4/letter/a/94ad74/32.png) [@alex.shulta](https://forum.opennebula.io/u/alex.shulta)\
**Post date:** [July 29, 2024, 9:47pm UTC](https://forum.opennebula.io/t/migrate-from-gluster-to-another-nfs-datastore/12284/9 "2024-07-29T21:47:13Z")

</div>

So its not recommended, but would the method of what I suggested work?

---

<div class="post-metadata">

**Author:** ![apicon](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/apicon/32/7947_2.png) [@apicon](https://forum.opennebula.io/u/apicon)\
**Post date:** [August 5, 2024, 7:18am UTC](https://forum.opennebula.io/t/migrate-from-gluster-to-another-nfs-datastore/12284/10 "2024-08-05T07:18:54Z")

</div>

As we stated, disks should be imported one by one (by scripting or manually) to the CEPH pool in order to migrate from GlusterFS to CEPH RBD. Using CEPHfs (NFS client) to store VM disks is not recommended.
