VM creation failed. XPath empty

Community Support
1

Dear All,
I wish that somebody can help me.

We’ve just received my servers to install Opennebula 6.8 CE.
Storage with Ceph (reef version)
Node : HP Proliant DL385. Ubuntu 22.04 LTS. KVM node
Front-end opennebula is installed for the moment in one our ceph server (We are waiting more servers).
We have our own dedicated switch (10/25Gbps) for ceph/node connection.
Each node will be connected to isolated switches and external switch.

I’ve successfully created Datastore with ceph
I’ve successfully created networks (openvswitch with vxlan).
I could download alpine and ubuntu image from the market.

But now I try to create a simple VM.

I have an error that I can’t resolve myself:

Mon Dec 18 15:17:25 2023 [Z0][VM][I]: New state is ACTIVE
Mon Dec 18 15:17:25 2023 [Z0][VM][I]: New LCM state is PROLOG
Mon Dec 18 15:17:27 2023 [Z0][VM][I]: New LCM state is BOOT
Mon Dec 18 15:17:27 2023 [Z0][VMM][I]: Generating deployment file: /var/lib/one/vms/6/deployment.0
Mon Dec 18 15:17:28 2023 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_context.
Mon Dec 18 15:17:29 2023 [Z0][VMM][I]: ExitCode: 0
Mon Dec 18 15:17:29 2023 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Mon Dec 18 15:17:29 2023 [Z0][VMM][I]: ExitCode: 0
Mon Dec 18 15:17:29 2023 [Z0][VMM][I]: Successfully execute virtualization driver operation: /bin/mkdir -p.
Mon Dec 18 15:17:29 2023 [Z0][VMM][I]: ExitCode: 0
Mon Dec 18 15:17:29 2023 [Z0][VMM][I]: Successfully execute virtualization driver operation: /bin/cat - >/var/lib/one//datastores/100/6/vm.xml.
Mon Dec 18 15:17:29 2023 [Z0][VMM][I]: ExitCode: 0
Mon Dec 18 15:17:29 2023 [Z0][VMM][I]: Successfully execute virtualization driver operation: /bin/cat - >/var/lib/one//datastores/100/6/ds.xml.
Mon Dec 18 15:17:30 2023 [Z0][VMM][I]: Command execution fail (exit code: 255): cat << 'EOT' | /var/tmp/one/vmm/kvm/deploy '/var/lib/one//datastores/100/6/deployment.0' 'onenode01' 6 onenode01
Mon Dec 18 15:17:30 2023 [Z0][VMM][I]: XPath set is empty
Mon Dec 18 15:17:30 2023 [Z0][VMM][I]: error: Failed to create domain from /var/lib/one//datastores/100/6/deployment.0
Mon Dec 18 15:17:30 2023 [Z0][VMM][I]: error: internal error: process exited while connecting to monitor: 2023-12-18T14:17:30.413410Z qemu-kvm-one: -blockdev {"driver":"rbd","pool":"one","image":"one-1-6-0","server":[{"host":"172.16.5.1","port":"6789"},{"host":"172.16.5.2","port":"6789"},{"host":"172.16.5.3","port":"6789"}],"node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: error connecting: Operation not supported
Mon Dec 18 15:17:30 2023 [Z0][VMM][I]: Could not create domain from /var/lib/one//datastores/100/6/deployment.0
Mon Dec 18 15:17:30 2023 [Z0][VMM][I]: ExitCode: 255
Mon Dec 18 15:17:30 2023 [Z0][VMM][I]: ExitCode: 0
Mon Dec 18 15:17:30 2023 [Z0][VMM][I]: Successfully execute network driver operation: clean.
Mon Dec 18 15:17:30 2023 [Z0][VMM][I]: Failed to execute virtualization driver operation: deploy.
Mon Dec 18 15:17:30 2023 [Z0][VMM][E]: DEPLOY: XPath set is empty error: Failed to create domain from /var/lib/one//datastores/100/6/deployment.0 error: internal error: process exited while connecting to monitor: 2023-12-18T14:17:30.413410Z qemu-kvm-one: -blockdev {"driver":"rbd","pool":"one","image":"one-1-6-0","server":[{"host":"172.16.5.1","port":"6789"},{"host":"172.16.5.2","port":"6789"},{"host":"172.16.5.3","port":"6789"}],"node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: error connecting: Operation not supported Could not create domain from /var/lib/one//datastores/100/6/deployment.0 ExitCode: 255
Mon Dec 18 15:17:30 2023 [Z0][VM][I]: New LCM state is BOOT_FAILURE

The lonely comprehensible message is : DEPLOY: XPath set is empty error: Failed to create domain.
But it can help me.

Do you have an idea ?

Best regards
Olivier

2

Are you able to connect as oneadmin user to libvirtd running on the hypervisor host?

[oneadmin@front-end]$ ssh <CN> virsh -c qemu:///system list --all

Please, ensure you have on CN as below:

[root@CN]$ grep -i oneadmin -r /etc/libvirt/*
/etc/libvirt/qemu.conf:user = "oneadmin"
/etc/libvirt/qemu.conf:group = "oneadmin"
3

Thanks for your reply,

everything is ok on my side.
virsh -c qemu:///system list --all
show only our test vm not opennebula vm.

and as root :

grep -i oneadmin -r /etc/libvirt/*
/etc/libvirt/qemu.conf:user = "oneadmin"
/etc/libvirt/qemu.conf:group = "oneadmin"

I’ve checked the deployment.x file in /var/lib/one/datastore/100/6
I see that no information about ceph auth like:

<auth username='libvirt'>
        <secret type='ceph' uuid='47961330-cd81-458e-a426-c6dec192f0f1'/>
</auth>

I had this information on the datastore ceph, and it’s present on the file ds.xml.
I’ve always the same message:

Tue Dec 19 09:20:34 2023 [Z0][VMM][I]: Command execution fail (exit code: 255): cat << 'EOT' | /var/tmp/one/vmm/kvm/deploy '/var/lib/one//datastores/100/6/deployment.4' 'onenode01' 6 onenode01
Tue Dec 19 09:20:34 2023 [Z0][VMM][I]: XPath set is empty
Tue Dec 19 09:20:34 2023 [Z0][VMM][I]: error: Failed to create domain from /var/lib/one//datastores/100/6/deployment.4
Tue Dec 19 09:20:34 2023 [Z0][VMM][I]: error: internal error: process exited while connecting to monitor: 2023-12-19T08:20:34.189974Z qemu-kvm-one: -blockdev {"driver":"rbd","pool":"one","image":"one-1-6-0","server":[{"host":"172.16.5.1","port":"6789"},{"host":"172.16.5.2","port":"6789"},{"host":"172.16.5.3","port":"6789"}],"node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: error connecting: Operation not supported
Tue Dec 19 09:20:34 2023 [Z0][VMM][I]: Could not create domain from /var/lib/one//datastores/100/6/deployment.4
Tue Dec 19 09:20:34 2023 [Z0][VMM][I]: ExitCode: 255
Tue Dec 19 09:20:34 2023 [Z0][VMM][I]: ExitCode: 0
Tue Dec 19 09:20:34 2023 [Z0][VMM][I]: Successfully execute network driver operation: clean.
Tue Dec 19 09:20:34 2023 [Z0][VMM][I]: Failed to execute virtualization driver operation: deploy.
Tue Dec 19 09:20:34 2023 [Z0][VMM][E]: DEPLOY: XPath set is empty error: Failed to create domain from /var/lib/one//datastores/100/6/deployment.4 error: internal error: process exited while connecting to monitor: 2023-12-19T08:20:34.189974Z qemu-kvm-one: -blockdev {"driver":"rbd","pool":"one","image":"one-1-6-0","server":[{"host":"172.16.5.1","port":"6789"},{"host":"172.16.5.2","port":"6789"},{"host":"172.16.5.3","port":"6789"}],"node-name":"libvirt-2-storage","cache":{"direct":true,"no-flush":false},"auto-read-only":true,"discard":"unmap"}: error connecting: Operation not supported Could not create domain from /var/lib/one//datastores/100/6/deployment.4 ExitCode: 255
Tue Dec 19 09:20:34 2023 [Z0][VM][I]: New LCM state is BOOT_FAILURE

I don’t understand the message: XPath set is empty

We’ve succeeded to create vm manually using ceph disk on the host.

Thanks for your help
Olivier

4

Update:

I reinstall completely the node server and nos it seems working.

I don’t understand what’s happened.
But for the moment, it’s okay.

Thanks for have read this post.
Best regards
Olivier