Hello,
I try setup LVM datastore.
Hi Anton,
Usual questions 
… What TM_MAD and DS_MAD do you use for the datastores, what is their configuration in /etc/one/oned.conf, which configuration guid do you follow, etc.
Which local disk - on the front-end or on the kvm host?
Some more background is needed to figure out where is the issue.
Kind Regards,
Anton Todorov
Hello Anton,
I use this docs - http://docs.opennebula.org/5.2/deployment/open_cloud_storage_setup/lvm_drivers.html
[root@node02 one]# onedatastore list
ID NAME SIZE AVAIL CLUSTERS IMAGES TYPE DS TM STAT
2 files 167.6G 97% 0 0 fil fs ssh on
105 production 167.6G 97% 0 0 img fs fs_lvm on
106 lvm_system 500G 100% 0 0 sys - fs_lvm on
Now IMGAGE show local space.
Local Disc on node01 (Frontend + kvm node)
Hello Anton,
I could bet you mean the SYSTEM DS
The system DS is monitored with remotes/tm/fs_lvm/monitor which returns LVM sizes if LVM VG is found otherwise the local filesystem space.
The image DS is monitored with remotes/datastore/fs/monitor which looks in the local filesystem only so it is working as designed but in your case it is wrong. I think that some code from tm/fs_lvm/monitor should be merged to datastore/fs/monitor to cover this use case.
The VM deployment is baffling me because same code is used to provision both persistent and non-persistent images by symlinking to the LVM LV and there is no way to create a file. Are you sure that there is a file not symlink to /dev/$VG/$LV? Could you provide the output of ls -la /var/lib/one/datastores/$DS_ID/$VM_ID ?
Kind Regards,
Anton Todorov
Hello Anton,
[root@node02 one]# ls -la /var/lib/one/datastores/106/19
total 368
drwxrwxr-x 2 oneadmin oneadmin 51 Mar 26 17:09 .
drwxrwxr-x 3 oneadmin oneadmin 15 Mar 26 17:09 …
-rw-rw-r-- 1 oneadmin oneadmin 1035 Mar 26 17:09 deployment.0
lrwxrwxrwx 1 oneadmin oneadmin 27 Mar 26 17:09 disk.0 -> /dev/vg-one-106/lv-one-19-0
-rw-r–r-- 1 oneadmin oneadmin 372736 Mar 26 17:09 disk.1
I can’t create VMs. every time get error:
Sun Mar 26 17:09:17 2017 : Error deploying virtual machine: Could not create domain from /var/lib/one//datastores/106/19/deployment.0
There should be some more lines around this one in /var/log/one/oned.log pointing to exact reason for the deployment failure of the VM…
Anton
Hello Anton,
Try new VM from Market template Ubuntu 16.
Sun Mar 26 17:26:15 2017 [Z0][VM][I]: New state is ACTIVE
Sun Mar 26 17:26:15 2017 [Z0][VM][I]: New LCM state is PROLOG
Sun Mar 26 17:26:54 2017 [Z0][VM][I]: New LCM state is BOOT
Sun Mar 26 17:26:54 2017 [Z0][VMM][I]: Generating deployment file: /var/lib/one/vms/20/deployment.0
Sun Mar 26 17:26:54 2017 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_context.
Sun Mar 26 17:26:55 2017 [Z0][VMM][I]: ExitCode: 0
Sun Mar 26 17:26:55 2017 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Sun Mar 26 17:26:55 2017 [Z0][VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/deploy ‘/var/lib/one//datastores/106/20/deployment.0’ ‘172.16.10.212’ 20 172.16.10.212
Sun Mar 26 17:26:55 2017 [Z0][VMM][I]: error: Failed to create domain from /var/lib/one//datastores/106/20/deployment.0
Sun Mar 26 17:26:55 2017 [Z0][VMM][I]: error: internal error: process exited while connecting to monitor: 2017-03-26T14:26:54.889761Z qemu-kvm: -drive file=/var/lib/one//datastores/106/20/disk.0,format=qcow2,if=none,id=drive-virtio-disk0,cache=none: Image is not in qcow2 format
Sun Mar 26 17:26:55 2017 [Z0][VMM][E]: Could not create domain from /var/lib/one//datastores/106/20/deployment.0
Sun Mar 26 17:26:55 2017 [Z0][VMM][I]: ExitCode: 255
Sun Mar 26 17:26:55 2017 [Z0][VMM][I]: Failed to execute virtualization driver operation: deploy.
Sun Mar 26 17:26:55 2017 [Z0][VMM][E]: Error deploying virtual machine: Could not create domain from /var/lib/one//datastores/106/20/deployment.0
Sun Mar 26 17:26:55 2017 [Z0][VM][I]: New LCM state is BOOT_FAILURE
The LVM LV is raw, but somewhere in the template (or in the image) you have “image mapping driver” set to qcow2…
check the image attributes regarding the mapping driver following this order the VM template, the “master” image attributes, the default values in /etc/one/oned.conf (or /etc/one/vmm_exec/kvm not sure where was exactly)
As a workaround you can try setting the “image mapping driver” to raw in the VM template (VM template → update → storage → Advanced options)
Best,
Anton
Best,
Anton
Thanks Anton!
Works!
I go to tests!
Hello Anton,
Maybe you can help me. With LVM DS migraton not working. I try live and cold.
Thanks!
Sun Mar 26 18:39:37 2017 [Z0][VM][I]: New LCM state is MIGRATE
Sun Mar 26 18:39:37 2017 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_premigrate.
Sun Mar 26 18:39:37 2017 [Z0][VMM][I]: ExitCode: 0
Sun Mar 26 18:39:37 2017 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Sun Mar 26 18:39:38 2017 [Z0][VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/migrate ‘one-22’ ‘172.16.10.211’ ‘172.16.10.212’ 22 172.16.10.212
Sun Mar 26 18:39:38 2017 [Z0][VMM][E]: migrate: Command “virsh --connect qemu:///system migrate --live one-22 qemu+ssh://172.16.10.211/system” failed: error: Cannot access storage file ‘/var/lib/one//datastores/106/22/disk.1’ (as uid:9869, gid:9869): No such file or directory
Sun Mar 26 18:39:38 2017 [Z0][VMM][E]: Could not migrate one-22 to 172.16.10.211
Sun Mar 26 18:39:38 2017 [Z0][VMM][I]: ExitCode: 1
Sun Mar 26 18:39:38 2017 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_failmigrate.
Sun Mar 26 18:39:38 2017 [Z0][VMM][I]: Failed to execute virtualization driver operation: migrate.
Sun Mar 26 18:39:38 2017 [Z0][VMM][E]: Error live migrating VM: Could not migrate one-22 to 172.16.10.211
Sun Mar 26 18:39:38 2017 [Z0][VM][I]: New LCM state is RUNNING
Sun Mar 26 18:39:38 2017 [Z0][LCM][I]: Fail to live migrate VM. Assuming that the VM is still RUNNING (will poll VM).
Hi Anton,
This line tell us that /var/lib/one/remotes/tm/fs_lvm/premigrate completed without errors, which means that on the destiration host(172.16.10.211) you should have a symlink /var/lib/one//datastores/106/22/disk.0 to /dev/$VG/$LV and this VG/LV should be in activated state.
But the contextualization CDROM ISO disk is missing or not accessible by the oneadmin user(or the user running libvirtd/qemu-kvm). You should check that the SYSTEM DS path '/var/lib/one//datastores/106/ is shared, visible and accessible by oneadmin on both kvm hosts. Also, check is selinux/apparmor involved too.
Best,
Anton
Hello Anton,
On second node no VM forlder in datastore. I think LVM DS not work correctly.
How to symlink to lv can move to second node?
Hi Anton,
You should configure shared filesystem (NFS or other) between the nodes. The fs-lvm driver expects shared filesytem between the nodes. As alternative you could patch the premigrate script to copy the source VM folder to the destination host before activating the LVM LVs and the postmigrate script to remove the source VM directory after activating the volumes.
Best,
Anton Todorov
Hello Anton,
Thanks, now I understand how it works!