# KVM and LXD node simultaneously

**URL:** <https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342>\
**Category:** Installation & Configuration\
**Created:** [June 14, 2019, 9:54am UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342 "2019-06-14T09:54:54Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![Daniel\_Ruiz\_Molina](https://avatars.discourse-cdn.com/v4/letter/d/f08c70/32.png) [@Daniel\_Ruiz\_Molina](https://forum.opennebula.io/u/Daniel_Ruiz_Molina)\
**Post date:** [June 14, 2019, 9:54am UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/1 "2019-06-14T09:54:55Z")

</div>

Hello,

My Opennebula front-end acts as KVM-node (there is not any more KVM nodes). System is running ONE-5.8.1 in a CentOS-7. Now, I would like to add LXD functionability, but I have read LXD is ONLY for Ubuntu systems… so it seems my front-end&KVM node won’t be able to act as “LXD node”.

Anyone has installed LXD in a CentOS system? Could I have my front-end as KVM role and, also, LXD role?

Thanks.

---

<div class="post-metadata">

**Author:** ![atodorov\_storpool](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/atodorov_storpool/32/5327_2.png) [@atodorov\_storpool](https://forum.opennebula.io/u/atodorov_storpool)\
**Post date:** [June 14, 2019, 10:24am UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/2 "2019-06-14T10:24:50Z")

</div>

I am running an Ubuntu VM with LXD which is registered as a LXD host in the front-end…

Usually works, still sometimes the LXD host get contextualized with the context intended for a LXD container, though. So make sure to have the latest contextualization installed on the Ubuntu VM that host LXD.

Hope this helps.

Best Regards,  
Anton Todorov

---

<div class="post-metadata">

**Author:** ![Daniel\_Ruiz\_Molina](https://avatars.discourse-cdn.com/v4/letter/d/f08c70/32.png) [@Daniel\_Ruiz\_Molina](https://forum.opennebula.io/u/Daniel_Ruiz_Molina)\
**Post date:** [June 14, 2019, 10:30am UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/3 "2019-06-14T10:30:09Z")

</div>

But I suppose your Ubuntu VM with LXD is NOT running as a Opennebula instance, isn’t it?

---

<div class="post-metadata">

**Author:** ![atodorov\_storpool](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/atodorov_storpool/32/5327_2.png) [@atodorov\_storpool](https://forum.opennebula.io/u/atodorov_storpool)\
**Post date:** [June 14, 2019, 10:35am UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/4 "2019-06-14T10:35:43Z")

</div>

It is 🙂

Just installed Ubuntu 18.04 from the Marketplace, then followed the guide to install LXD and finally added the VM as LXD host in same front-end 😉

---

<div class="post-metadata">

**Author:** ![Daniel\_Ruiz\_Molina](https://avatars.discourse-cdn.com/v4/letter/d/f08c70/32.png) [@Daniel\_Ruiz\_Molina](https://forum.opennebula.io/u/Daniel_Ruiz_Molina)\
**Post date:** [June 14, 2019, 10:54am UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/5 "2019-06-14T10:54:48Z")

</div>

LXD VM is running as a ONE instance in the ONE front-end node? When I create an Ubuntu instance and I run “Infrastructure -\> Hosts -\> Add” I select “LXD” as type, but what is the hostname ? instance IP? instance name?  
I have tried both values but in both cases, system returns me this error: “Host key verification failed”. I have added my public key (I have checked it in /root/.ssh/authorized\_keys inside the instance), but monitoring process is failing… so ONE can’t add that instance as a Host.

---

<div class="post-metadata">

**Author:** ![Daniel\_Ruiz\_Molina](https://avatars.discourse-cdn.com/v4/letter/d/f08c70/32.png) [@Daniel\_Ruiz\_Molina](https://forum.opennebula.io/u/Daniel_Ruiz_Molina)\
**Post date:** [June 14, 2019, 11:02am UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/6 "2019-06-14T11:02:18Z")

</div>

Error I’m getting is this:  
[…]  
Fri Jun 14 12:58:33 2019 [Z0][InM][I]: Command execution failed (exit code: 255): ‘if [-x “/var/tmp/one/im/stop\_probes”]; then /var/tmp/one/im/stop\_probes lxd 7 LXD-Node; else exit 42; fi’  
Fri Jun 14 12:58:33 2019 [Z0][InM][I]: Permission denied (publickey).  
Fri Jun 14 12:58:33 2019 [Z0][InM][E]: Could not stop monitor on host 7 LQ==  
[…]

As “oneadmin”, I can run “ssh root@LXD-Node” with no password perfectly. I suppose monitoring process runs as “oneadmin”… and needs passwordless connection

---

<div class="post-metadata">

**Author:** ![Daniel\_Ruiz\_Molina](https://avatars.discourse-cdn.com/v4/letter/d/f08c70/32.png) [@Daniel\_Ruiz\_Molina](https://forum.opennebula.io/u/Daniel_Ruiz_Molina)\
**Post date:** [June 14, 2019, 11:32am UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/7 "2019-06-14T11:32:45Z")

</div>

Sorry… I have not yet install “LXD” in my Ubuntu instance… Sure adding process fails because of it. 😉

---

<div class="post-metadata">

**Author:** ![atodorov\_storpool](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/atodorov_storpool/32/5327_2.png) [@atodorov\_storpool](https://forum.opennebula.io/u/atodorov_storpool)\
**Post date:** [June 14, 2019, 11:57am UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/8 "2019-06-14T11:57:02Z")

</div>

You should add network to the Ubuntu VM, OpenNebula will assign an IP from the given network. Use that IP to access the Ubuntu VM, or better create static name in /etc/hosts (usually i do static mapping on both the FE and Host) and register the VM as a LXD host on the fe, and follow the installation guide keeping in mind that the host is virtualized so some details could need some tweaking.

---

<div class="post-metadata">

**Author:** ![Daniel\_Ruiz\_Molina](https://avatars.discourse-cdn.com/v4/letter/d/f08c70/32.png) [@Daniel\_Ruiz\_Molina](https://forum.opennebula.io/u/Daniel_Ruiz_Molina)\
**Post date:** [June 18, 2019, 10:04am UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/9 "2019-06-18T10:04:04Z")

</div>

Hi, I have executed all steps:

1. Downloaded from Marketplace Ubuntu 18.10

2. Installed “LXD” packages from new created repo

3. …but I can’t add host as a “LXD host”… I continue getting this error:  
Tue Jun 18 12:00:30 2019 [Z0][InM][D]: Monitoring host Test-LXD (11)  
Tue Jun 18 12:00:30 2019 [Z0][InM][I]: Command execution failed (exit code: 255): ‘if [-x “/var/tmp/one/im/run\_probes”]; then /var/tmp/one/im/run\_probes lxd /var/lib/one//datastores 4124 60 11 Test-LXD; else exit 42; fi’  
Tue Jun 18 12:00:30 2019 [Z0][InM][I]: Host key verification failed.  
Tue Jun 18 12:00:31 2019 [Z0][InM][I]: Command execution failed (exit code: 255): ‘if [-x “/var/tmp/one/im/run\_probes”]; then /var/tmp/one/im/run\_probes lxd /var/lib/one//datastores 4124 60 11 Test-LXD; else exit 42; fi’  
Tue Jun 18 12:00:31 2019 [Z0][InM][I]: Host key verification failed.  
Tue Jun 18 12:00:32 2019 [Z0][InM][I]: Command execution failed (exit code: 255): ‘if [-x “/var/tmp/one/im/run\_probes”]; then /var/tmp/one/im/run\_probes lxd /var/lib/one//datastores 4124 60 11 Test-LXD; else exit 42; fi’  
Tue Jun 18 12:00:32 2019 [Z0][InM][I]: Host key verification failed.  
Tue Jun 18 12:00:33 2019 [Z0][InM][I]: Command execution failed (exit code: 255): ‘if [-x “/var/tmp/one/im/run\_probes”]; then /var/tmp/one/im/run\_probes lxd /var/lib/one//datastores 4124 60 11 Test-LXD; else exit 42; fi’  
Tue Jun 18 12:00:33 2019 [Z0][InM][I]: Host key verification failed.  
Tue Jun 18 12:00:33 2019 [Z0][ONE][E]: Error monitoring Host Test-LXD (11): -

So “Test-LXD” is not added as a “LXD” host. However, I can run a passwordless connection between frontend as “oneadmin” to “Test-LXD” as “root” user, so I can’t understand error “Host key verification failed”.

Could anyone help me?

Thanks.

---

<div class="post-metadata">

**Author:** ![Daniel\_Ruiz\_Molina](https://avatars.discourse-cdn.com/v4/letter/d/f08c70/32.png) [@Daniel\_Ruiz\_Molina](https://forum.opennebula.io/u/Daniel_Ruiz_Molina)\
**Post date:** [June 19, 2019, 8:16am UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/10 "2019-06-19T08:16:15Z")

</div>

Hi again,

after some reconfiguration files, I have got my Ubuntu instance that acts as “LXD Host Node” monitored as the second host where users can deploy their LXD containers. However, I can’t deploy a container because I’m getting these errors during container creation (logs from frontend):

> ```
> ==> oned.log <==
> Wed Jun 19 10:06:37 2019 [Z0][ReM][D]: Req:7536 UID:0 IP:127.0.0.1 one.vm.deploy invoked , 2560, 12, false, 0, ""
> Wed Jun 19 10:06:37 2019 [Z0][DiM][D]: Deploying VM 2560
> Wed Jun 19 10:06:37 2019 [Z0][ReM][D]: Req:7536 UID:0 one.vm.deploy result SUCCESS, 2560
> 
> ```

```
==> sched.log <==
Wed Jun 19 10:06:37 2019 [Z0][SCHED][D]: Dispatching VMs to hosts:
        VMID Priority Host System DS
        --------------------------------------------------------------
        2560 0 12 0

==> oned.log <==
Wed Jun 19 10:06:38 2019 [Z0][ReM][D]: Req:4080 UID:0 IP:127.0.0.1 one.vm.info invoked , 2560
Wed Jun 19 10:06:38 2019 [Z0][ReM][D]: Req:4080 UID:0 one.vm.info result SUCCESS, "<VM><ID>2560</ID><UI..."
Wed Jun 19 10:06:39 2019 [Z0][AuM][D]: Message received: AUTHENTICATE SUCCESS 217 -

Wed Jun 19 10:06:39 2019 [Z0][ReM][D]: Req:1488 UID:0 IP:127.0.0.1 one.vmpool.info invoked , -2, 0, -200, -1
Wed Jun 19 10:06:39 2019 [Z0][ReM][D]: Req:1488 UID:0 one.vmpool.info result SUCCESS, "<VM_POOL><VM><ID>256..."
Wed Jun 19 10:06:39 2019 [Z0][ReM][D]: Req:4064 UID:0 IP:127.0.0.1 one.user.info invoked , 0
Wed Jun 19 10:06:39 2019 [Z0][ReM][D]: Req:4064 UID:0 one.user.info result SUCCESS, "<USER><ID>0</ID><GID..."
Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 Command execution failed (exit code: 1): /var/lib/one/remotes/tm/qcow2/clone frontend:/var/lib/one//datastores/1/23521bbd4a8bff724d4988d168b362b9 Host-for-LXD:/var/lib/one//datastores/0/2560/disk.0 2560 1

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 clone: Cloning /var/lib/one/datastores/1/23521bbd4a8bff724d4988d168b362b9 in Host-for-LXD:/var/lib/one//datastores/0/2560/disk.0

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG E 2560 clone: Command "set -e -o pipefail

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 cd /var/lib/one/datastores/0/2560

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 rm -rf "/var/lib/one/datastores/0/2560/disk.0.sn ap"

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 mkdir -p "/var/lib/one/datastores/0/2560/disk.0.snap"

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 B_FORMAT=$(qemu-img info /var/lib/one/datastores/1/23521bbd4a8bff724d4988d168b362b9 | grep "^file format:" | awk '{print $3}' || :)

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 qemu-img create -o backing_fmt=${B_FORMAT:-raw} -b /var/lib/one/datastores/1/23521bbd4a8bff724d4988d168b362b9 -f qcow2 /var/lib/one/datastores/0/2560/disk.0.snap/0

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 rm -f "/var/lib/one/datastores/0/2560/disk.0"

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 ln -s disk.0.snap/0 /var/lib/one/datastores/0/2560/disk.0

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 cd /var/lib/one/datastores/0/2560/disk.0.snap

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 ln -s . /var/lib/one/datastores/0/2560/disk.0.snap/disk.0.snap" failed: qemu-img: Could not open '/var/lib/one/datastores/1/23521bbd4a8bff724d4988d168b362b9': Could not open '/var/lib/one/datastores/1/23521bbd4a8bff724d4988d168b362b9': No such file or directory

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 qemu-img: /var/lib/one/datastores/0/2560/disk.0.snap/0: Could not open '/var/lib/one/datastores/1/23521bbd4a8bff724d4988d168b362b9': No such file or directory

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG I 2560 Could not open backing image to determine size.

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: LOG E 2560 Error copying frontend:/var/lib/one//datastores/1/23521bbd4a8bff724d4988d168b362b9 to Host-for-LXD:/var/lib/one//datastores/0/2560/disk.0

Wed Jun 19 10:06:40 2019 [Z0][TM][D]: Message received: TRANSFER FAILURE 2560 Error copying frontend:/var/lib/one//datastores/1/23521bbd4a8bff724d4988d168b362b9 to Host-for-LXD:/var/lib/one//datastores/0/2560/disk.0

Wed Jun 19 10:06:43 2019 [Z0][ReM][D]: Req:6672 UID:0 IP:127.0.0.1 one.vmpool.info invoked , -2, 0, -200, -1
Wed Jun 19 10:06:43 2019 [Z0][ReM][D]: Req:6672 UID:0 one.vmpool.info result SUCCESS, "<VM_POOL><VM><ID>256..."
Wed Jun 19 10:06:43 2019 [Z0][ReM][D]: Req:7440 UID:0 IP:127.0.0.1 one.user.info invoked , 0
Wed Jun 19 10:06:43 2019 [Z0][ReM][D]: Req:7440 UID:0 one.user.info result SUCCESS, "<USER><ID>0</ID><GID..."

```

Help, please!!!

Thanks a lot!

---

<div class="post-metadata">

**Author:** ![atodorov\_storpool](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/atodorov_storpool/32/5327_2.png) [@atodorov\_storpool](https://forum.opennebula.io/u/atodorov_storpool)\
**Post date:** [June 19, 2019, 12:29pm UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/11 "2019-06-19T12:29:33Z")

</div>

What are the TM\_MAD of the datastores (Image and System)

---

<div class="post-metadata">

**Author:** ![Daniel\_Ruiz\_Molina](https://avatars.discourse-cdn.com/v4/letter/d/f08c70/32.png) [@Daniel\_Ruiz\_Molina](https://forum.opennebula.io/u/Daniel_Ruiz_Molina)\
**Post date:** [June 19, 2019, 12:58pm UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/12 "2019-06-19T12:58:00Z")

</div>

Both are qcow2

---

<div class="post-metadata">

**Author:** ![atodorov\_storpool](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/atodorov_storpool/32/5327_2.png) [@atodorov\_storpool](https://forum.opennebula.io/u/atodorov_storpool)\
**Post date:** [June 19, 2019, 1:15pm UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/13 "2019-06-19T13:15:59Z")

</div>

Reading the TM\_MAD code I am not sure how it is supposed to work… The error is caused because the file in question is missing from the host.

```auto
qemu-img: Could not open '/var/lib/one/datastores/1/23521bbd4a8bff724d4988d168b362b9': Could not open '/var/lib/one/datastores/1/23521bbd4a8bff724d4988d168b362b9': No such file or directory

```

And this file is not copied from the front-end to the host, before trying to create a snapshot of it. It is like the code expect shared Image datastore.

Sorry, but I could’nt help anymore. Just could guide you where it looks like the issue is, decoding the error messages ☹

Best,  
Anton Todorov

---

<div class="post-metadata">

**Author:** ![Daniel\_Ruiz\_Molina](https://avatars.discourse-cdn.com/v4/letter/d/f08c70/32.png) [@Daniel\_Ruiz\_Molina](https://forum.opennebula.io/u/Daniel_Ruiz_Molina)\
**Post date:** [June 20, 2019, 1:38pm UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/14 "2019-06-20T13:38:47Z")

</div>

I have debug more information and I have seen that error is in the execution of /var/lib/one/remotes/tm/qcow2/clone script. After adding “set -x” and “trap read debug”, I have run command that fails:  
`/var/lib/one/remotes/tm/qcow2/clone frontend:/var/lib/one//datastores/1/23521bbd4a8bff724d4988d168b362b9 Host-for-LXD:/var/lib/one//datastores/0/2561/disk.0 2561 1`

exactly at line #105:  
`if [-n "$ORIGINAL_SIZE" -a "$SIZE" -gt "$ORIGINAL_SIZE"]; then`

$ORIGINAL\_SIZE and $SIZE have null (empty) value because of at line #73, in the “while”, these variables haven’t got any value…

```
XPATH="${DRIVER_PATH}/../../datastore/xpath.rb --stdin"

unset i j XPATH_ELEMENTS

while IFS= read -r -d '' element; do
    XPATH_ELEMENTS[i++]="$element"
done < <(onevm show -x $VMID| $XPATH \
                    /VM/TEMPLATE/DISK[DISK_ID=$DISK_ID]/SIZE \
                    /VM/TEMPLATE/DISK[DISK_ID=$DISK_ID]/ORIGINAL_SIZE \
                    /VM/HISTORY_RECORDS/HISTORY[last\(\)]/TM_MAD)

SIZE="${XPATH_ELEMENTS[j++]}"
ORIGINAL_SIZE="${XPATH_ELEMENTS[j++]}"
TM_MAD="${XPATH_ELEMENTS[j++]}"

```

However, I have got these values “by hand” and added them in the script manually and script continues failing at line #134, after (trying) creating a soft-link…

uuufffff… More help!!!

---

<div class="post-metadata">

**Author:** ![atodorov\_storpool](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/atodorov_storpool/32/5327_2.png) [@atodorov\_storpool](https://forum.opennebula.io/u/atodorov_storpool)\
**Post date:** [June 21, 2019, 11:02am UTC](https://forum.opennebula.io/t/kvm-and-lxd-node-simultaneously/7342/15 "2019-06-21T11:02:34Z")

</div>

Weird. my copy of tm/qcow2/clone is 133 lines…