# VM fails to (HA) reschedule on host error - OpenNebula 5.0.1 **URL:** \ **Category:** Product Support\ **Created:** [July 13, 2016, 1:37pm UTC](https://forum.opennebula.io/t/vm-fails-to-ha-reschedule-on-host-error-opennebula-5-0-1/2566 "2016-07-13T13:37:01Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![heathen](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/heathen/32/4720_2.png) [@heathen](https://forum.opennebula.io/u/heathen)\ **Post date:** [July 13, 2016, 1:37pm UTC](https://forum.opennebula.io/t/vm-fails-to-ha-reschedule-on-host-error-opennebula-5-0-1/2566/1 "2016-07-13T13:37:02Z")
Hello! I’m using ceph storage (both system and image storages points to the same ceph pool) and trying to set VM HA as noted in docs (with HOST\_HOOK). But VM doesn’t start on another host in case of original host failure: /var/log/one/oned.log > Wed Jul 13 09:03:09 2016 [Z0][InM][I]: Command execution fail: ‘if [-x “/var/tmp/one/im/run\_probes”]; then /var/tmp/one/im/run\_probes kvm /var/lib/one//datastores 4124 > 5 2 gamma; else exit 42; fi’ > Wed Jul 13 09:03:09 2016 [Z0][InM][I]: ssh: connect to host gamma port 22: Connection timed out > Wed Jul 13 09:03:09 2016 [Z0][InM][I]: ExitCode: 255 > Wed Jul 13 09:03:09 2016 [Z0][ONE][E]: Error monitoring Host gamma (2): - > Wed Jul 13 09:03:09 2016 [Z0][ReM][D]: Req:7520 UID:0 VirtualMachinePoolInfo invoked , -2, -1, -1, -1 > Wed Jul 13 09:03:09 2016 [Z0][ReM][D]: Req:7520 UID:0 VirtualMachinePoolInfo result SUCCESS, “\12\<…” > Wed Jul 13 09:03:09 2016 [Z0][ReM][D]: Req:9984 UID:0 VirtualMachineInfo invoked , 12 > Wed Jul 13 09:03:09 2016 [Z0][ReM][D]: Req:9984 UID:0 VirtualMachineInfo result SUCCESS, “12…” > Wed Jul 13 09:03:09 2016 [Z0][ReM][D]: Req:1136 UID:0 VirtualMachineAction invoked , “resched”, 12 > Wed Jul 13 09:03:09 2016 [Z0][DiM][D]: Setting rescheduling flag on VM 12 > Wed Jul 13 09:03:09 2016 [Z0][ReM][D]: Req:1136 UID:0 VirtualMachineAction result SUCCESS, 12 > Wed Jul 13 09:03:09 2016 [Z0][HKM][D]: Message received: EXECUTE SUCCESS 2 error: /var/log/one/12.log > Wed Jul 13 09:03:23 2016 [Z0][VM][I]: New LCM state is PROLOG\_MIGRATE\_UNKNOWN > Wed Jul 13 09:03:23 2016 [Z0][VM][I]: New state is ACTIVE > Wed Jul 13 09:03:26 2016 [Z0][TM][I]: Command execution fail: /var/lib/one/remotes/tm/ceph/mv gamma:/var/lib/one//datastores/101/12 beta:/var/lib/one//datastores/101/12 12 101 > Wed Jul 13 09:03:26 2016 [Z0][TM][I]: mv: Moving gamma:/var/lib/one/datastores/101/12 to beta:/var/lib/one/datastores/101/12 > Wed Jul 13 09:03:26 2016 [Z0][TM][E]: mv: Command “eval ssh gamma ‘tar -C /var/lib/one/datastores/101 --sparse -cf - 12’ | ssh beta ‘tar -C /var/lib/one/datastores/101 --sparse -xf -’” failed: ssh: connect to host gamma port 22: Connection timed out > Wed Jul 13 09:03:26 2016 [Z0][TM][I]: tar: This does not look like a tar archive > Wed Jul 13 09:03:26 2016 [Z0][TM][I]: tar: Exiting with failure status due to previous errors > Wed Jul 13 09:03:26 2016 [Z0][TM][E]: Error copying disk directory to target host > Wed Jul 13 09:03:26 2016 [Z0][TM][I]: ExitCode: 2 > Wed Jul 13 09:03:26 2016 [Z0][TM][E]: Error executing image transfer script: Error copying disk directory to target host > Wed Jul 13 09:03:26 2016 [Z0][VM][I]: New LCM state is PROLOG\_MIGRATE\_UNKNOWN\_FAILURE So the VM ends with the FAILURE state and it’s impossible to recover it before original host went up. While hosts are alive both live and offline migrations work well. Could anyone please suggest me the way to debug it futher? Thanks! ---
**Author:** ![atodorov\_storpool](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/atodorov_storpool/32/5327_2.png) [@atodorov\_storpool](https://forum.opennebula.io/u/atodorov_storpool)\ **Post date:** [July 13, 2016, 2:28pm UTC](https://forum.opennebula.io/t/vm-fails-to-ha-reschedule-on-host-error-opennebula-5-0-1/2566/2 "2016-07-13T14:28:28Z")
Hi, It looks like that the CEPH TM\_MAD is using the ‘mv’ script from the SSH TM\_MAD. The script is trying to move files from the dead host to the other. Obviously it is not possible 🙂 Can you try edit /var/lib/one/remotes/tm/ceph/mv and add > [`lcm_state` -eq 60] && exit 0 just before (~ line 69 ) > log “Moving $SRC to $DST” Kind Regards, Anton Todorov ---
**Author:** ![heathen](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/heathen/32/4720_2.png) [@heathen](https://forum.opennebula.io/u/heathen)\ **Post date:** [July 14, 2016, 4:52am UTC](https://forum.opennebula.io/t/vm-fails-to-ha-reschedule-on-host-error-opennebula-5-0-1/2566/3 "2016-07-14T04:52:49Z")
Anton, thank you very much for your suggestion! Now it works! The only thing is that it should looks like this (I believe the parser ate grave accent marks): > [`lcm_state` -eq 60] && exit 0 Best regards, Vladimir ---
**Author:** ![atodorov\_storpool](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/atodorov_storpool/32/5327_2.png) [@atodorov\_storpool](https://forum.opennebula.io/u/atodorov_storpool)\ **Post date:** [July 14, 2016, 6:58am UTC](https://forum.opennebula.io/t/vm-fails-to-ha-reschedule-on-host-error-opennebula-5-0-1/2566/4 "2016-07-14T06:58:39Z")
Hi @heathen, I’ve made a pull request for the change. It is adding log message and is moved after the cleanup code for the _$DST\_PATH_ on the destination host.(few lines below my first suggestion) [https://github.com/OpenNebula/one/pull/106](https://github.com/OpenNebula/one/pull/106) Kind Regards, Anton Todorov