# Failed to migrate VMs on shared storage(moosefs) **URL:** \ **Category:** Product Support\ **Created:** [February 13, 2019, 10:06am UTC](https://forum.opennebula.io/t/failed-to-migrate-vms-on-shared-storage-moosefs/6904 "2019-02-13T10:06:31Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![ioiioi](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ioiioi/32/4479_2.png) [@ioiioi](https://forum.opennebula.io/u/ioiioi)\ **Post date:** [February 13, 2019, 10:06am UTC](https://forum.opennebula.io/t/failed-to-migrate-vms-on-shared-storage-moosefs/6904/1 "2019-02-13T10:06:31Z")
I am using moosefs + opennebula to construct a hci system. here is my setup: - 1\*moosefs-master, 192.168.250.1 - 3\*moosefs-chunkserver+one-kvm-node(192.168.250.4-6) - 1\*one-frontend VM(192.168.250.100) ```auto onedatastores list 117 mfs-img-qcow2 5.5T 99% 0 0 img fs qcow2 on 118 mfs-sys-qcow2 5.5T 99% 0 0 sys - qcow2 on 119 mfs-img-share 5.5T 99% 0 1 img fs shared on 120 mfs-sys-share 5.5T 99% 0 0 sys - shared on ``` ```auto mount /mnt/mfs/one su - oneadmin ln -s /mnt/mfs/one/datastores/mfs-img-qcow2 /var/lib/one/datastores/117 ln -s /mnt/mfs/one/datastores/mfs-sys-qcow2 /var/lib/one/datastores/118 ln -s /mnt/mfs/one/datastores/mfs-img-shared /var/lib/one/datastores/119 ln -s /mnt/mfs/one/datastores/mfs-sys-shared /var/lib/one/datastores/120 ``` ```auto ll /var/lib/one/datastores lrwxrwxrwx 1 oneadmin oneadmin 37 Feb 13 16:22 117 -> /mnt/mfs/one/datastores/mfs-img-qcow2 lrwxrwxrwx 1 oneadmin oneadmin 37 Feb 13 16:22 118 -> /mnt/mfs/one/datastores/mfs-sys-qcow2 lrwxrwxrwx 1 oneadmin oneadmin 38 Feb 13 16:22 119 -> /mnt/mfs/one/datastores/mfs-img-shared lrwxrwxrwx 1 oneadmin oneadmin 38 Feb 13 16:26 120 -> /mnt/mfs/one/datastores/mfs-sys-shared ``` ```auto onevm list ID USER GROUP NAME STAT UCPU UMEM HOST TIME 61 oneadmin oneadmin nfs-q-d9-test1 runn 0.0 1024M node-5 0d 06h47 65 oneadmin oneadmin mfs-s-d9-test1 runn 0.0 1024M node-4 0d 00h54 ``` the vm is running well on the moosefs system datastore. there is one problem I can’t fixed. that is the migrate live. ```auto onevm migrate mfs-q-d9-test1 node-6 --live ``` It failed, the virsh complaint it’s unsafe to live migrate without share storage ```auto tail -f /var/log/one/65.log ... 18 Wed Feb 13 17:02:46 2019 [Z0][VMM][E]: migrate: Command "virsh --connect qemu+tcp://localhost/system migrate --live on e-65 qemu+ssh://node-6/system" failed: error: Unsafe migration: Migration without shared storage is unsafe ... ``` any idea how to fixed it? it seem likely related to virsh. ---
**Author:** ![ioiioi](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ioiioi/32/4479_2.png) [@ioiioi](https://forum.opennebula.io/u/ioiioi)\ **Post date:** [February 14, 2019, 2:29am UTC](https://forum.opennebula.io/t/failed-to-migrate-vms-on-shared-storage-moosefs/6904/2 "2019-02-14T02:29:57Z")
I found out that it is a bug of libvirt/4.5(I am running qemu-kvm-ev 2.12), opennebula utilize virsh to migrate vm. before migration, libvirt first detect if the vm is running under shared storage, but moosefs/lizardfs was mounted with fuse, so It’s not easy to identify the type of fs(shared or non-shared). a workaround is add “–unsafe” option to virsh command. but finally it should be fixed on libvirt. ```auto /var/lib/one/remotes/etc/vmm/kvm/kvmrc MIGRATE_OPTIONS=--unsafe ``` and ```auto onehost sync --force ``` I am not a developer, have not idea how to report bug.any one might kindly to helping report this bug to redhat? here is the bug report of redhat/ovirt [https://bugzilla.redhat.com/show\_bug.cgi?id=1632711](https://bugzilla.redhat.com/show_bug.cgi?id=1632711) [https://bugzilla.redhat.com/show\_bug.cgi?id=1644679](https://bugzilla.redhat.com/show_bug.cgi?id=1644679) ---
**Author:** ![ruben](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ruben/32/19_2.png) [@ruben](https://forum.opennebula.io/u/ruben)\ **Post date:** [February 14, 2019, 10:55am UTC](https://forum.opennebula.io/t/failed-to-migrate-vms-on-shared-storage-moosefs/6904/3 "2019-02-14T10:55:58Z")
Hi, Thanks for the update. There is a note in the documentation about this issue: [http://docs.opennebula.org/5.6/deployment/open\_cloud\_host\_setup/kvm\_driver.html#live-migration-for-other-cache-settings](http://docs.opennebula.org/5.6/deployment/open_cloud_host_setup/kvm_driver.html#live-migration-for-other-cache-settings) ---
**Author:** ![ioiioi](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ioiioi/32/4479_2.png) [@ioiioi](https://forum.opennebula.io/u/ioiioi)\ **Post date:** [February 14, 2019, 11:56am UTC](https://forum.opennebula.io/t/failed-to-migrate-vms-on-shared-storage-moosefs/6904/4 "2019-02-14T11:56:23Z")
hmm, this is another issue rather than cache, others might not dig out the /var/log/one/$vm-id.log. so to solve it with libvirt should be the ultimate solution.