Hello community
I hope I can find some help here.
I am pretty new to opennebula and ceph and there are no colleagues who could assist me
Our solution consists of a ceph cluster (14.2.12), 3 sunstone server (5.10.1) and 5 hypervisors. All hosts running CentOS 7. Everything worked properly untill I updated the ceph mons/osds and the hypervisors from 14.2.11 to 14.2.12 yesterday.
When creating a new vm I get the following error:
Tue Oct 27 11:34:07 2020 [Z0][VM][I]: New state is ACTIVE
Tue Oct 27 11:34:07 2020 [Z0][VM][I]: New LCM state is PROLOG
Tue Oct 27 11:34:09 2020 [Z0][VM][I]: New LCM state is BOOT
Tue Oct 27 11:34:09 2020 [Z0][VMM][I]: Generating deployment file: /var/lib/one/vms/1085/deployment.0
Tue Oct 27 11:34:10 2020 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_context.
Tue Oct 27 11:34:10 2020 [Z0][VMM][I]: ExitCode: 0
Tue Oct 27 11:34:10 2020 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Tue Oct 27 11:34:11 2020 [Z0][VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/deploy '/var/lib/one//datastores/103/1085/deployment.0' 'virt01' 1085 virt01
Tue Oct 27 11:34:11 2020 [Z0][VMM][I]: error: Failed to create domain from /var/lib/one//datastores/103/1085/deployment.0
Tue Oct 27 11:34:11 2020 [Z0][VMM][I]: error: internal error: qemu unexpectedly closed the monitor: unable to parse addrs in 'mon01:6789;mon02:6789;mon03:6789'
Tue Oct 27 11:34:11 2020 [Z0][VMM][I]: 2020-10-27T10:34:10.804685Z qemu-kvm: -drive file=rbd:one/one-456-1085-0:id=libvirt:auth_supported=cephx\;none:mon_host=mon01\:6789\;mon02\:6789\;mon03\:6789,file.password-secret=virtio-disk0-secret0,format=raw,if=none,id=drive-virtio-disk0,cache=none: error connecting: Invalid argument
Tue Oct 27 11:34:11 2020 [Z0][VMM][E]: Could not create domain from /var/lib/one//datastores/103/1085/deployment.0
Tue Oct 27 11:34:11 2020 [Z0][VMM][I]: ExitCode: 255
Tue Oct 27 11:34:11 2020 [Z0][VMM][I]: Failed to execute virtualization driver operation: deploy.
Tue Oct 27 11:34:11 2020 [Z0][VMM][E]: Error deploying virtual machine: Could not create domain from /var/lib/one//datastores/103/1085/deployment.0
Tue Oct 27 11:34:11 2020 [Z0][VM][I]: New LCM state is BOOT_FAILURE
And here the qemu log:
2020-10-27 13:48:11.540+0000: starting up libvirt version: 4.5.0, package: 33.el7_8.1 (CentOS BuildSystem <http://bugs.centos.org>, 2020-05-12-16:25:35, x86-01.bsys.centos.org), qemu version: 2.12.0qemu-kvm-ev-2.12.0-44.1.el7_8.1, kernel: 3.10.0-1127.el7.x86_64, hostname: virt05.23media.de
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin \
QEMU_AUDIO_DRV=none \
/usr/libexec/qemu-kvm \
-name guest=one-1087,debug-threads=on \
-S \
-object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-67-one-1087/master-key.aes \
-machine pc-i440fx-rhel7.6.0,accel=kvm,usb=off,dump-guest-core=off \
-cpu host \
-m 2048 \
-realtime mlock=off \
-smp 1,sockets=1,cores=1,threads=1 \
-uuid 9c568743-0eba-41d9-aaab-6a9c21e515a4 \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,fd=33,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=localtime \
-no-shutdown \
-boot strict=on \
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 \
-object secret,id=virtio-disk0-secret0,data=123456789,keyid=masterKey0,iv=/123456789xxx==,format=base64 \
-drive 'file=rbd:one/one-456-1087-0:id=libvirt:auth_supported=cephx\;none:mon_host=mon01\:6789\;mon02\:6789\;mon03\:6789,file.password-secret=virtio-disk0-secret0,format=raw,if=none,id=drive-virtio-disk0,cache=none' \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1,write-cache=on \
-drive file=/var/lib/one//datastores/103/1087/disk.1,format=raw,if=none,id=drive-ide0-0-0,readonly=on \
-device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 \
-netdev tap,fd=35,id=hostnet0,vhost=on,vhostfd=36 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=02:00:3e:71:c5:0b,bus=pci.0,addr=0x3 \
-vnc 0.0.0.0:1087 \
-k de \
-device cirrus-vga,id=video0,bus=pci.0,addr=0x2 \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 \
-sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \
-msg timestamp=on
2020-10-27 13:48:11.540+0000: Domain id=67 is tainted: host-cpu
unable to parse addrs in 'mon01:6789;mon02:6789;mon03:6789'
2020-10-27T13:48:11.636396Z qemu-kvm: -drive file=rbd:one/one-456-1087-0:id=libvirt:auth_supported=cephx\;none:mon_host=mon01\:6789\;mon02\:6789\;mon03\:6789,file.password-secret=virtio-disk0-secret0,format=raw,if=none,id=drive-virtio-disk0,cache=none: error connecting: Invalid argument
2020-10-27 13:48:11.648+0000: shutting down, reason=failed
Please let me know what further information you need and where I can find it
Many thank in advance!