Migration fails: could not find capabilities

Community Support
1

Hello,

after onehost flush, some VMs fail to migrate away. The log file is as follows:

Mon May 28 10:19:38 2018 [Z0][VM][I]: New LCM state is MIGRATE
Mon May 28 10:23:06 2018 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_premigrate.
Mon May 28 10:23:06 2018 [Z0][VMM][I]: ExitCode: 0
Mon May 28 10:23:06 2018 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Mon May 28 10:23:08 2018 [Z0][VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/migrate 'one-1838' 'stratus6' 'stratus16' 1838 stratus16
Mon May 28 10:23:08 2018 [Z0][VMM][E]: migrate: Command "virsh --connect qemu:///system migrate --live  one-1838 qemu+ssh://stratus6/system" failed: error: invalid argument: could not find capabilities for arch=x86_64
Mon May 28 10:23:08 2018 [Z0][VMM][E]: Could not migrate one-1838 to stratus6
Mon May 28 10:23:08 2018 [Z0][VMM][I]: ExitCode: 1
Mon May 28 10:23:08 2018 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_failmigrate.
Mon May 28 10:23:08 2018 [Z0][VMM][I]: Failed to execute virtualization driver operation: migrate.
Mon May 28 10:23:08 2018 [Z0][VMM][E]: Error live migrating VM: Could not migrate one-1838 to stratus6
Mon May 28 10:23:08 2018 [Z0][VM][I]: New LCM state is RUNNING
Mon May 28 10:23:08 2018 [Z0][LCM][I]: Fail to live migrate VM. Assuming that the VM is still RUNNING (will poll VM).

However, when I manually run onevm migrate --live 1836 stratus6 afterwards (note the same destination host as the one onevm flush itself selected on its own), the migration succeeds.

What can be different between onehost flush and onevm migrate, that the former fails, but the later succeeds?

Thanks!

-Yenya

Versions of the related components and OS (frontend, hypervisors, VMs):
I run CentOS 7 with KVM, qemu-kvm-ev-2.10.0-21.el7_5.2.1.x86_64, ONe 5.4.

2

I have discovered that apart from the frequent “could not find capabilities” errors, I occasionally also get this one:

Mon May 28 11:20:44 2018 [Z0][VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/migrate 'one-1851' 'stratus11' 'stratus23' 1851 stratus23
Mon May 28 11:20:44 2018 [Z0][VMM][E]: migrate: Command "virsh --connect qemu:///system migrate --live  one-1851 qemu+ssh://stratus11/system" failed: error: internal error: QEMU / QMP failed: qemu-kvm: could not acquire pid file: Permission denied
Mon May 28 11:20:44 2018 [Z0][VMM][E]: Could not migrate one-1851 to stratus11
Mon May 28 11:20:44 2018 [Z0][VMM][I]: ExitCode: 1

As in the previous case, trying to migrate the same VM to the same destination host using onevm migrate succeeds.

-Yenya