LXD - Daemon failed to setup shared mounts

Hello Guys,

We have an issue started recently with 2 nodes not able to deploy/boot new LXD containers and failing with:

Mon Sep 20 13:46:04 2021 [Z0][VM][I]: New LCM state is LCM_INIT
Mon Sep 20 13:46:04 2021 [Z0][VM][I]: New state is PENDING
Mon Sep 20 13:46:16 2021 [Z0][VM][I]: New state is ACTIVE
Mon Sep 20 13:46:16 2021 [Z0][VM][I]: New LCM state is PROLOG
Mon Sep 20 13:46:31 2021 [Z0][VM][I]: New LCM state is BOOT
Mon Sep 20 13:46:31 2021 [Z0][VMM][I]: Generating deployment file: /var/lib/one/vms/2572/deployment.1
Mon Sep 20 13:46:34 2021 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_context.
Mon Sep 20 13:46:36 2021 [Z0][VMM][I]: ExitCode: 0
Mon Sep 20 13:46:36 2021 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/lxd/deploy '/var/lib/one//datastores/107/2572/deployment.1' 'zdh-004' 2572 zdh-004
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: deploy: Overriding container
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: deploy: Processing disk 0
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: deploy: Using qcow2 mapper for /var/lib/one/datastores/107/2572/disk.0
Mon Sep 20 13:46:37 2021 [Z0][VMM][E]: deploy: mount_on?: Mount detected in /var/snap/lxd/common/lxd/storage-pools/default/containers/one-2572/rootfs
Mon Sep 20 13:46:37 2021 [Z0][VMM][E]: deploy: mount_on?: Mount detected in /var/lib/one/datastores/107/2572/mapper/disk.1
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: deploy: --- Starting container ---
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: deploy: Name: one-2572
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: Remote: unix://
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: Architecture: x86_64
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: Created: 2021/09/17 15:57 UTC
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: Status: Stopped
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: Type: persistent
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: Profiles: default
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]:
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: Log:
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]:
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]:
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: /var/tmp/one/vmm/lxd/deploy:72:in `rescue in <main>': undefined local variable or method `e' for main:Object (NameError)
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/deploy:69:in `<main>'
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: /var/tmp/one/vmm/lxd/client.rb:102:in `wait': {"type"=>"sync", "status"=>"Success", "status_code"=>200, "operation"=>"", "error_code"=>0, "error"=>"", "metadata"=>{"id"=>"58e18cab-fe1b-4ec1-a051-f5662739fd0c", "class"=>"task", "description"=>"Starting container", "created_at"=>"2021-09-20T13:46:36.661440119+02:00", "updated_at"=>"2021-09-20T13:46:36.661440119+02:00", "status"=>"Failure", "status_code"=>400, "resources"=>{"containers"=>["/1.0/containers/one-2572"]}, "metadata"=>nil, "may_cancel"=>false, "err"=>"Daemon failed to setup shared mounts base: no such file or directory.\\nDoes security.nesting need to be turned on?", "location"=>"none"}} (LXDError)
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/container.rb:517:in `wait?'
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/container.rb:529:in `change_state'
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/container.rb:211:in `start'
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/deploy:70:in `<main>'
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: ExitCode: 1
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: ExitCode: 0
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: Successfully execute network driver operation: clean.
Mon Sep 20 13:46:37 2021 [Z0][VMM][I]: Failed to execute virtualization driver operation: deploy.
Mon Sep 20 13:46:37 2021 [Z0][VMM][E]: Error deploying virtual machine
Mon Sep 20 13:46:37 2021 [Z0][VM][I]: New LCM state is BOOT_FAILURE

Existing ones seems to be fine. I’ve searched for the error, however haven’t found any solution. I tried with:
systemctl reload snap.lxd.daemon

Could someone suggest what could be the case? The issue occurs on 2 of 3 nodes connected to cluster

Thanks.

Versions of the related components and OS (frontend, hypervisors, VMs):

OpenNebula 5.12.0.3
LXD 3.0.4

This suggests that prior to the container attempting to start, there was already storage setup in the container mountpoints. So, you would have to manually unmount the directories mentioned there and also unmap the block devices that are mounted in those directories.

In the case of qcow2 the block devices would be something like /dev/nbdX. You can access the node and list the block devices and their mountpoints with lsblk to confirm this.

Hey @dclavijo . Sorry about that, this particular instance I was trying to delete-recreate. However, below you can find the log with the fresh one:

Mon Sep 20 16:39:01 2021 [Z0][VM][I]: New state is ACTIVE
Mon Sep 20 16:39:01 2021 [Z0][VM][I]: New LCM state is PROLOG
Mon Sep 20 16:39:15 2021 [Z0][VM][I]: New LCM state is BOOT
Mon Sep 20 16:39:15 2021 [Z0][VMM][I]: Generating deployment file: /var/lib/one/vms/2582/deployment.0
Mon Sep 20 16:39:18 2021 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_context.
Mon Sep 20 16:39:19 2021 [Z0][VMM][I]: ExitCode: 0
Mon Sep 20 16:39:19 2021 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/lxd/deploy '/var/lib/one//datastores/107/2582/deployment.0' 'zdh-004' 2582 zdh-004
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: deploy: Processing disk 0
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: deploy: Using raw filesystem mapper for /var/lib/one/datastores/107/2582/disk.0
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: deploy: Mapping disk at /var/snap/lxd/common/lxd/storage-pools/default/containers/one-2582/rootfs using device /dev/loop49
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: deploy: Resizing filesystem ext4 on /dev/loop49
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: deploy: Mounting /dev/loop49 at /var/snap/lxd/common/lxd/storage-pools/default/containers/one-2582/rootfs
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: deploy: Mapping disk at /var/lib/one/datastores/107/2582/mapper/disk.1 using device /dev/loop50
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: deploy: Mounting /dev/loop50 at /var/lib/one/datastores/107/2582/mapper/disk.1
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: deploy: --- Starting container ---
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: deploy: Name: one-2582
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: Remote: unix://
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: Architecture: x86_64
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: Created: 2021/09/20 14:39 UTC
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: Status: Stopped
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: Type: persistent
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: Profiles: default
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]:
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: Log:
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]:
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]:
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: /var/tmp/one/vmm/lxd/deploy:72:in `rescue in <main>': undefined local variable or method `e' for main:Object (NameError)
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/deploy:69:in `<main>'
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: /var/tmp/one/vmm/lxd/client.rb:102:in `wait': {"type"=>"sync", "status"=>"Success", "status_code"=>200, "operation"=>"", "error_code"=>0, "error"=>"", "metadata"=>{"id"=>"87f7bf2e-a9c4-4778-8ba1-1e88e72c1701", "class"=>"task", "description"=>"Starting container", "created_at"=>"2021-09-20T16:39:22.067792696+02:00", "updated_at"=>"2021-09-20T16:39:22.067792696+02:00", "status"=>"Failure", "status_code"=>400, "resources"=>{"containers"=>["/1.0/containers/one-2582"]}, "metadata"=>nil, "may_cancel"=>false, "err"=>"Daemon failed to setup shared mounts base: no such file or directory.\\nDoes security.nesting need to be turned on?", "location"=>"none"}} (LXDError)
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/container.rb:517:in `wait?'
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/container.rb:529:in `change_state'
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/container.rb:211:in `start'
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/deploy:70:in `<main>'
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: ExitCode: 1
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: ExitCode: 0
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: Successfully execute network driver operation: clean.
Mon Sep 20 16:39:22 2021 [Z0][VMM][I]: Failed to execute virtualization driver operation: deploy.
Mon Sep 20 16:39:22 2021 [Z0][VMM][E]: Error deploying virtual machine
Mon Sep 20 16:39:22 2021 [Z0][VM][I]: New LCM state is BOOT_FAILURE

And for example identical instance from the same template on another node:

Mon Sep 20 16:42:02 2021 [Z0][VM][I]: New state is ACTIVE
Mon Sep 20 16:42:02 2021 [Z0][VM][I]: New LCM state is PROLOG
Mon Sep 20 16:42:14 2021 [Z0][VM][I]: New LCM state is BOOT
Mon Sep 20 16:42:14 2021 [Z0][VMM][I]: Generating deployment file: /var/lib/one/vms/2583/deployment.0
Mon Sep 20 16:42:17 2021 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_context.
Mon Sep 20 16:42:19 2021 [Z0][VMM][I]: ExitCode: 0
Mon Sep 20 16:42:19 2021 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Mon Sep 20 16:42:26 2021 [Z0][VMM][I]: deploy: Processing disk 0
Mon Sep 20 16:42:26 2021 [Z0][VMM][I]: deploy: Using raw filesystem mapper for /var/lib/one/datastores/107/2583/disk.0
Mon Sep 20 16:42:26 2021 [Z0][VMM][I]: deploy: Mapping disk at /var/snap/lxd/common/lxd/storage-pools/default/containers/one-2583/rootfs using device /dev/loop52
Mon Sep 20 16:42:26 2021 [Z0][VMM][I]: deploy: Resizing filesystem ext4 on /dev/loop52
Mon Sep 20 16:42:26 2021 [Z0][VMM][I]: deploy: Mounting /dev/loop52 at /var/snap/lxd/common/lxd/storage-pools/default/containers/one-2583/rootfs
Mon Sep 20 16:42:26 2021 [Z0][VMM][I]: deploy: Mapping disk at /var/lib/one/datastores/107/2583/mapper/disk.1 using device /dev/loop53
Mon Sep 20 16:42:26 2021 [Z0][VMM][I]: deploy: Mounting /dev/loop53 at /var/lib/one/datastores/107/2583/mapper/disk.1
Mon Sep 20 16:42:26 2021 [Z0][VMM][I]: deploy: --- Starting container ---
Mon Sep 20 16:42:26 2021 [Z0][VMM][I]: ExitCode: 0
Mon Sep 20 16:42:26 2021 [Z0][VMM][I]: Successfully execute virtualization driver operation: deploy.
Mon Sep 20 16:42:28 2021 [Z0][VMM][I]: ExitCode: 0
Mon Sep 20 16:42:28 2021 [Z0][VMM][I]: Successfully execute network driver operation: post.
Mon Sep 20 16:42:28 2021 [Z0][VM][I]: New LCM state is RUNNING

Can you post the VM Template ? onevm show -x 2582

@dclavijo new instance 2586:

Mon Sep 20 18:07:08 2021 [Z0][VM][I]: New state is ACTIVE
Mon Sep 20 18:07:08 2021 [Z0][VM][I]: New LCM state is PROLOG
Mon Sep 20 18:07:19 2021 [Z0][VM][I]: New LCM state is BOOT
Mon Sep 20 18:07:19 2021 [Z0][VMM][I]: Generating deployment file: /var/lib/one/vms/2586/deployment.0
Mon Sep 20 18:07:21 2021 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_context.
Mon Sep 20 18:07:23 2021 [Z0][VMM][I]: ExitCode: 0
Mon Sep 20 18:07:23 2021 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/lxd/deploy '/var/lib/one//datastores/107/2586/deployment.0' 'zdh-006' 2586 zdh-006
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Processing disk 0
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Using raw filesystem mapper for /var/lib/one/datastores/107/2586/disk.0
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Mapping disk at /var/snap/lxd/common/lxd/storage-pools/default/containers/one-2586/rootfs using device /dev/loop17
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Resizing filesystem ext4 on /dev/loop17
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Mounting /dev/loop17 at /var/snap/lxd/common/lxd/storage-pools/default/containers/one-2586/rootfs
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Mapping disk at /var/lib/one/datastores/107/2586/mapper/disk.1 using device /dev/loop18
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Mounting /dev/loop18 at /var/lib/one/datastores/107/2586/mapper/disk.1
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: --- Starting container ---
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Name: one-2586
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: Remote: unix://
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: Architecture: x86_64
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: Created: 2021/09/20 16:07 UTC
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: Status: Stopped
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: Type: persistent
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: Profiles: default
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]:
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: Log:
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]:
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]:
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Processing disk 0
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Using raw filesystem mapper for /var/lib/one/datastores/107/2586/disk.0
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Unmapping disk at /var/snap/lxd/common/lxd/storage-pools/default/containers/one-2586/rootfs
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Umounting disk mapped at /dev/loop17
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Unmapping disk at /var/lib/one/datastores/107/2586/mapper/disk.1
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: deploy: Umounting disk mapped at /dev/loop18
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: /var/tmp/one/vmm/lxd/client.rb:102:in `wait': {"type"=>"sync", "status"=>"Success", "status_code"=>200, "operation"=>"", "error_code"=>0, "error"=>"", "metadata"=>{"id"=>"ddf03c1d-a5fd-4ad8-8711-04c6023618b7", "class"=>"task", "description"=>"Starting container", "created_at"=>"2021-09-20T18:07:25.185263465+02:00", "updated_at"=>"2021-09-20T18:07:25.185263465+02:00", "status"=>"Failure", "status_code"=>400, "resources"=>{"containers"=>["/1.0/containers/one-2586"]}, "metadata"=>nil, "may_cancel"=>false, "err"=>"Daemon failed to setup shared mounts base: no such file or directory.\\nDoes security.nesting need to be turned on?", "location"=>"none"}} (LXDError)
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/container.rb:517:in `wait?'
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/container.rb:529:in `change_state'
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/container.rb:211:in `start'
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: from /var/tmp/one/vmm/lxd/deploy:70:in `<main>'
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: ExitCode: 1
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: ExitCode: 0
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: Successfully execute network driver operation: clean.
Mon Sep 20 18:07:31 2021 [Z0][VMM][I]: Failed to execute virtualization driver operation: deploy.
Mon Sep 20 18:07:31 2021 [Z0][VMM][E]: Error deploying virtual machine
Mon Sep 20 18:07:31 2021 [Z0][VM][I]: New LCM state is BOOT_FAILURE
[root@zdh-001.XXX.com][~]: onevm show -x 2586
<VM>
  <ID>2586</ID>
  <UID>0</UID>
  <GID>0</GID>
  <UNAME>oneadmin</UNAME>
  <GNAME>oneadmin</GNAME>
  <NAME>test3.arthur.com</NAME>
  <PERMISSIONS>
    <OWNER_U>1</OWNER_U>
    <OWNER_M>1</OWNER_M>
    <OWNER_A>0</OWNER_A>
    <GROUP_U>0</GROUP_U>
    <GROUP_M>0</GROUP_M>
    <GROUP_A>0</GROUP_A>
    <OTHER_U>0</OTHER_U>
    <OTHER_M>0</OTHER_M>
    <OTHER_A>0</OTHER_A>
  </PERMISSIONS>
  <LAST_POLL>0</LAST_POLL>
  <STATE>3</STATE>
  <LCM_STATE>36</LCM_STATE>
  <PREV_STATE>3</PREV_STATE>
  <PREV_LCM_STATE>36</PREV_LCM_STATE>
  <RESCHED>0</RESCHED>
  <STIME>1632154004</STIME>
  <ETIME>0</ETIME>
  <DEPLOY_ID/>
  <MONITORING/>
  <TEMPLATE>
    <AUTOMATIC_DS_REQUIREMENTS><![CDATA[("CLUSTERS/ID" @> 0 | "CLUSTERS/ID" @> 101)]]></AUTOMATIC_DS_REQUIREMENTS>
    <AUTOMATIC_NIC_REQUIREMENTS><![CDATA[("CLUSTERS/ID" @> 0 | "CLUSTERS/ID" @> 101)]]></AUTOMATIC_NIC_REQUIREMENTS>
    <AUTOMATIC_REQUIREMENTS><![CDATA[(CLUSTER_ID = 0 | CLUSTER_ID = 101) & !(PUBLIC_CLOUD = YES) & !(PIN_POLICY = PINNED)]]></AUTOMATIC_REQUIREMENTS>
    <CONTEXT>
      <DISK_ID><![CDATA[1]]></DISK_ID>
      <ETH0_CONTEXT_FORCE_IPV4><![CDATA[]]></ETH0_CONTEXT_FORCE_IPV4>
      <ETH0_DNS><![CDATA[X.X.X.200 8.8.8.8]]></ETH0_DNS>
      <ETH0_EXTERNAL><![CDATA[]]></ETH0_EXTERNAL>
      <ETH0_GATEWAY><![CDATA[X.X.X.1]]></ETH0_GATEWAY>
      <ETH0_GATEWAY6><![CDATA[]]></ETH0_GATEWAY6>
      <ETH0_IP><![CDATA[X.X.X.25]]></ETH0_IP>
      <ETH0_IP6><![CDATA[]]></ETH0_IP6>
      <ETH0_IP6_PREFIX_LENGTH><![CDATA[]]></ETH0_IP6_PREFIX_LENGTH>
      <ETH0_IP6_ULA><![CDATA[]]></ETH0_IP6_ULA>
      <ETH0_MAC><![CDATA[02:00:c2:93:5d:19]]></ETH0_MAC>
      <ETH0_MASK><![CDATA[255.255.255.0]]></ETH0_MASK>
      <ETH0_METRIC><![CDATA[]]></ETH0_METRIC>
      <ETH0_METRIC6><![CDATA[]]></ETH0_METRIC6>
      <ETH0_MTU><![CDATA[1500]]></ETH0_MTU>
      <ETH0_NETWORK><![CDATA[X.X.X.0]]></ETH0_NETWORK>
      <ETH0_SEARCH_DOMAIN><![CDATA[]]></ETH0_SEARCH_DOMAIN>
      <ETH0_VLAN_ID><![CDATA[]]></ETH0_VLAN_ID>
      <ETH0_VROUTER_IP><![CDATA[]]></ETH0_VROUTER_IP>
      <ETH0_VROUTER_IP6><![CDATA[]]></ETH0_VROUTER_IP6>
      <ETH0_VROUTER_MANAGEMENT><![CDATA[]]></ETH0_VROUTER_MANAGEMENT>
      <NETWORK><![CDATA[YES]]></NETWORK>
      <SET_HOSTNAME><![CDATA[test3.arthur.com]]></SET_HOSTNAME>
      <SSH_PUBLIC_KEY><![CDATA[ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAB3mRO+Ll29a7bVAn1T21oJNSdvMIOOAj5gja8ZNnO91QesUHZm5iwaAg2SUuX3Qu8jakMgUgrmYntIC6OdFv/fEAAo3lni107DZv49QmauJsUbb/lfHF6r/ApI/6jLEbhYzUaMfZwC2kTOIdDcNbKUFA/iKsU/QySqgjSTIUuHXYnQRQ== den@parrot]]></SSH_PUBLIC_KEY>
      <START_SCRIPT_BASE64><![CDATA[ZWNobyAtZSAiVXMzcmFiYzEyM1xuVXMzcmFiYzEyMyIgfCBwYXNzd2Qgcm9vdA==]]></START_SCRIPT_BASE64>
      <TARGET><![CDATA[hda]]></TARGET>
    </CONTEXT>
    <CPU><![CDATA[2]]></CPU>
    <DISK>
      <ALLOW_ORPHANS><![CDATA[NO]]></ALLOW_ORPHANS>
      <CLONE><![CDATA[YES]]></CLONE>
      <CLONE_TARGET><![CDATA[SYSTEM]]></CLONE_TARGET>
      <CLUSTER_ID><![CDATA[0,101]]></CLUSTER_ID>
      <DATASTORE><![CDATA[LXDimages]]></DATASTORE>
      <DATASTORE_ID><![CDATA[106]]></DATASTORE_ID>
      <DEV_PREFIX><![CDATA[sd]]></DEV_PREFIX>
      <DISK_ID><![CDATA[0]]></DISK_ID>
      <DISK_SNAPSHOT_TOTAL_SIZE><![CDATA[0]]></DISK_SNAPSHOT_TOTAL_SIZE>
      <DISK_TYPE><![CDATA[FILE]]></DISK_TYPE>
      <DRIVER><![CDATA[raw]]></DRIVER>
      <IMAGE><![CDATA[centos_7 - LXD]]></IMAGE>
      <IMAGE_ID><![CDATA[134]]></IMAGE_ID>
      <IMAGE_STATE><![CDATA[2]]></IMAGE_STATE>
      <LN_TARGET><![CDATA[NONE]]></LN_TARGET>
      <ORIGINAL_SIZE><![CDATA[1024]]></ORIGINAL_SIZE>
      <READONLY><![CDATA[NO]]></READONLY>
      <SAVE><![CDATA[NO]]></SAVE>
      <SIZE><![CDATA[10240]]></SIZE>
      <SOURCE><![CDATA[/var/lib/one//datastores/106/4f05727e493889ba863a2ca3b894d0dc]]></SOURCE>
      <TARGET><![CDATA[sda]]></TARGET>
      <TM_MAD><![CDATA[shared]]></TM_MAD>
      <TYPE><![CDATA[FILE]]></TYPE>
    </DISK>
    <GRAPHICS>
      <LISTEN><![CDATA[0.0.0.0]]></LISTEN>
      <PORT><![CDATA[8486]]></PORT>
      <TYPE><![CDATA[VNC]]></TYPE>
    </GRAPHICS>
    <MEMORY><![CDATA[2048]]></MEMORY>
    <NIC>
      <AR_ID><![CDATA[0]]></AR_ID>
      <BRIDGE><![CDATA[br0]]></BRIDGE>
      <BRIDGE_TYPE><![CDATA[linux]]></BRIDGE_TYPE>
      <CLUSTER_ID><![CDATA[0,101]]></CLUSTER_ID>
      <IP><![CDATA[X.X.X.25]]></IP>
      <MAC><![CDATA[02:00:c2:93:5d:19]]></MAC>
      <NAME><![CDATA[NIC0]]></NAME>
      <NETWORK><![CDATA[X.X.X.0]]></NETWORK>
      <NETWORK_ID><![CDATA[3]]></NETWORK_ID>
      <NETWORK_UNAME><![CDATA[oneadmin]]></NETWORK_UNAME>
      <NIC_ID><![CDATA[0]]></NIC_ID>
      <SECURITY_GROUPS><![CDATA[0]]></SECURITY_GROUPS>
      <TARGET><![CDATA[one-2586-0]]></TARGET>
      <VN_MAD><![CDATA[bridge]]></VN_MAD>
    </NIC>
    <OS>
      <BOOT><![CDATA[]]></BOOT>
    </OS>
    <SECURITY_GROUP_RULE>
      <PROTOCOL><![CDATA[ALL]]></PROTOCOL>
      <RULE_TYPE><![CDATA[OUTBOUND]]></RULE_TYPE>
      <SECURITY_GROUP_ID><![CDATA[0]]></SECURITY_GROUP_ID>
      <SECURITY_GROUP_NAME><![CDATA[default]]></SECURITY_GROUP_NAME>
    </SECURITY_GROUP_RULE>
    <SECURITY_GROUP_RULE>
      <PROTOCOL><![CDATA[ALL]]></PROTOCOL>
      <RULE_TYPE><![CDATA[INBOUND]]></RULE_TYPE>
      <SECURITY_GROUP_ID><![CDATA[0]]></SECURITY_GROUP_ID>
      <SECURITY_GROUP_NAME><![CDATA[default]]></SECURITY_GROUP_NAME>
    </SECURITY_GROUP_RULE>
    <TEMPLATE_ID><![CDATA[48]]></TEMPLATE_ID>
    <TM_MAD_SYSTEM><![CDATA[shared]]></TM_MAD_SYSTEM>
    <VMID><![CDATA[2586]]></VMID>
  </TEMPLATE>
  <USER_TEMPLATE>
    <HYPERVISOR><![CDATA[lxd]]></HYPERVISOR>
    <INPUTS_ORDER><![CDATA[]]></INPUTS_ORDER>
    <LXD_PROFILE><![CDATA[]]></LXD_PROFILE>
    <LXD_SECURITY_NESTING><![CDATA[yes]]></LXD_SECURITY_NESTING>
    <LXD_SECURITY_PRIVILEGED><![CDATA[yes]]></LXD_SECURITY_PRIVILEGED>
    <MEMORY_UNIT_COST><![CDATA[MB]]></MEMORY_UNIT_COST>
    <SCHED_REQUIREMENTS><![CDATA[ID="7"]]></SCHED_REQUIREMENTS>
  </USER_TEMPLATE>
  <HISTORY_RECORDS>
    <HISTORY>
      <OID>2586</OID>
      <SEQ>0</SEQ>
      <HOSTNAME>zdh-006</HOSTNAME>
      <HID>7</HID>
      <CID>101</CID>
      <STIME>1632154028</STIME>
      <ETIME>0</ETIME>
      <VM_MAD><![CDATA[lxd]]></VM_MAD>
      <TM_MAD><![CDATA[shared]]></TM_MAD>
      <DS_ID>107</DS_ID>
      <PSTIME>1632154028</PSTIME>
      <PETIME>1632154039</PETIME>
      <RSTIME>1632154039</RSTIME>
      <RETIME>0</RETIME>
      <ESTIME>0</ESTIME>
      <EETIME>0</EETIME>
      <ACTION>0</ACTION>
      <UID>-1</UID>
      <GID>-1</GID>
      <REQUEST_ID>-1</REQUEST_ID>
    </HISTORY>
  </HISTORY_RECORDS>
</VM>

P.S. IPs are replaced with X.X.X