Libvirt live migration method used in OpenNebula

Community Support
1

Which type of libvirt live migration does OpenNebula use? According to https://wiki.libvirt.org/page/FAQ#Migration there are two methods: plain (unecrypted) and tunnelled. On our OpenNebula cloud live migration doesn’t work:
Aug 13 15:54:29 2018 [Z0][VMM][E]: migrate: Command "virsh --connect qemu:///system migrate --live one-25 qemu+ssh://nodexx.aa.bb/system" failed: error: unable to connect to server at '[nodexx.aa.bb:49152](http://nodexx.aa.bb:49152/)': No route to host
Thats probably because we didn’t open TCP 49152-49215 yet on hosts as libvirt FAQ suggest ( and there is no mention about that in OpenNebula install guide/documentation by the way!). From this I assume OpenNebula uses plain (unencrypted) live migration method. Is is possible to change this to tunnelled method?

Thanks!

2

Hi @childintime ,

You should take a look at /var/lib/one/remotes/etc/vmm/kvm/kvmrc where you could play with MIGRATE_OPTIONS, LIBVIRT_URI, etc. Some details you could find in OpenNebula’s KVM Driver doc.

Also keep in mind that you need to sync after editing the kvmrc file. I am using the following one-line comand as root for this:

su - oneadmin -c 'onehost sync --force'

Hope this helps.

Best Regards,
Anton Todorov

2 Likes
3

I had to modify the kvm migrate script to make this work. On your OpenNebula host replace /var/lib/one/remotes/vmm/kvm/migrate with:

#!/bin/bash

# -------------------------------------------------------------------------- #
# Copyright 2002-2018, OpenNebula Project, OpenNebula Systems                #
#                                                                            #
# Licensed under the Apache License, Version 2.0 (the "License"); you may    #
# not use this file except in compliance with the License. You may obtain    #
# a copy of the License at                                                   #
#                                                                            #
# http://www.apache.org/licenses/LICENSE-2.0                                 #
#                                                                            #
# Unless required by applicable law or agreed to in writing, software        #
# distributed under the License is distributed on an "AS IS" BASIS,          #
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.   #
# See the License for the specific language governing permissions and        #
# limitations under the License.                                             #
#--------------------------------------------------------------------------- #

source $(dirname $0)/../../etc/vmm/kvm/kvmrc
source $(dirname $0)/../../scripts_common.sh

deploy_id=$1
dest_host=$2

# migration can't be done with domain snapshots, drop them first
snaps=$(monitor_and_log \
   "virsh --connect $LIBVIRT_URI snapshot-list $deploy_id --name 2>/dev/null" \
   "Failed to get snapshots for $deploy_id")

for snap in $snaps; do
    exec_and_log \
        "virsh --connect $LIBVIRT_URI snapshot-delete $deploy_id --snapshotname $snap --metadata" \
        "Failed to delete snapshot $snap from $deploy_id"
done

# get a list of disks in use by the VM we're migrating
disks=$(monitor_and_log "virsh --quiet --connect $LIBVIRT_URI domblklist $deploy_id"|$AWK '{print $2}')
filenames=()

# for each disk, create a sparse image on the destination host so the
# --copy-storage-all flag works correctly
for disk in $disks; do
    json=$(monitor_and_log "qemu-img info --force-share --output=json ${disk}")
    virtual_size=$(grep '"virtual-size": ' <<< "${json}"|grep -oE [0-9]+)
    filename=$(grep '"filename": "' <<< "${json}"|cut -d'"' -f4)
    format=$(grep '"format": "' <<< "${json}"|cut -d'"' -f4)
    dir=$(dirname ${filename})
    filenames+=(${filename})

    ssh_exec_and_log "${dest_host}" "mkdir -p ${dir}"
    ssh_exec_and_log "${dest_host}" "qemu-img create -f ${format} ${filename} ${virtual_size}"
done

exec_and_log "virsh --connect $LIBVIRT_URI migrate --live --copy-storage-all $MIGRATE_OPTIONS $deploy_id $QEMU_PROTOCOL://$dest_host/system" \
    "Could not migrate $deploy_id to $dest_host"

# cleanup the old disks from the source host (ignore errors)
rm -f ${filenames[@]} || exit 0

and run

su - oneadmin -c 'onehost sync --force'

Tested with qcow2 and raw disk formats. Not sure how it works with others.