Fs_lvm working one-way

Community Support Storage
1

Hi,

I want to use fs_lvm so I created 2 datastores lvm_system (111) amd lvm_default (112) as per doc.
OS is downloaded to lvm_default (112) and /var/lib/one/datastores/112 is NFS mount available on FE (one.lan) and 2 nodes (e54.lan e55.lan)

OS deploys correctly creating link from /var/lib/one/datastores/111/XXX/disk.0 on a node to vg-one-111/lv-one-XXX-0 on the same node
It means fs_lvm driver copies qcow2 image from NFS mounted dir (111) converting it to raw and dumping into lv-one-XXX-0

Now the problem is when I want to STOP or UNDEPLOY. The driver (or maybe it is not fs_lvm that works here?) tries to move the disk to the same datastore (111) via FE (one.lan)
FE does not have vg-one-111, event it if have then mv will fail complaing about missing lv-one-XXX-0

What’s the goal behind moving data to the same system datastore on FE?

There is nothing else in the fs_lvm documentation that I could use to make it work.

I am fighting with this for a few days beacuse I really like OpenNebula over CloudStack or OpenStack. Please help.

Here is log after STOP:

[Z0][VM][I]: New LCM state is EPILOG_STOP
[Z0][TM][I]: Command execution failed (exit code: 5): /var/lib/one/remotes/tm/fs_lvm/mv e54.lan:/var/lib/one//datastores/111/71/disk.0 one.lan:/var/lib/one//datastores/111/71/disk.0 71 112
[Z0][TM][I]: mv: SRC_PATH=/var/lib/one/datastores/111/71/disk.0
[Z0][TM][I]: mv: DST_PATH=/var/lib/one/datastores/111/71/disk.0
[Z0][TM][I]: mv: SRC_HOST=e54.lan
[Z0][TM][I]: mv: DST_HOST=one.lan
[Z0][TM][I]: mv: DST_DIR=/var/lib/one/datastores/111/71
[Z0][TM][I]: mv: SRC_DS_DIR=/var/lib/one/datastores/111/71
[Z0][TM][I]: mv: SRC_VM_DIR=disk.0
[Z0][TM][I]: mv: SRC_DS_SYS_ID=111
[Z0][TM][I]: mv: DST_DS_SYS_ID=111
[Z0][TM][I]: mv: /var/lib/one/datastores/111/71/disk.0 is disk
[Z0][TM][I]: mv: DISK_ID=0
[Z0][TM][I]: mv: XPATH=/var/lib/one/remotes/tm/fs_lvm/…/…/datastore/xpath.rb --stdin
[Z0][TM][I]: mv: TYPE=BLOCK
[Z0][TM][I]: mv: LV_NAME=lv-one-71-0
[Z0][TM][I]: mv: SRC_VG_NAME=vg-one-111
[Z0][TM][I]: mv: SRC_DEV=/dev/vg-one-111/lv-one-71-0
[Z0][TM][I]: mv: DST_VG_NAME=vg-one-111
[Z0][TM][I]: mv: DST_DEV=/dev/vg-one-111/lv-one-71-0
[Z0][TM][I]: mv: activate on one.lan
[Z0][TM][I]: mv: CMD= set -ex -o pipefail
[Z0][TM][I]: sudo sync
[Z0][TM][I]: sudo lvscan
[Z0][TM][I]: sudo lvchange -ay “/dev/vg-one-111/lv-one-71-0”
[Z0][TM][I]:
[Z0][TM][I]: hostname -f >"/var/lib/one/datastores/111/71/.host" || :
[Z0][TM][E]: mv: Command " set -ex -o pipefail
[Z0][TM][I]: sudo sync
[Z0][TM][I]: sudo lvscan
[Z0][TM][I]: sudo lvchange -ay “/dev/vg-one-111/lv-one-71-0”
[Z0][TM][I]:
[Z0][TM][I]: hostname -f >"/var/lib/one/datastores/111/71/.host" || :" failed: + sudo sync
[Z0][TM][I]: + sudo lvscan
[Z0][TM][I]: + sudo lvchange -ay /dev/vg-one-111/lv-one-71-0
[Z0][TM][I]: Volume group “vg-one-111” not found
[Z0][TM][I]: Cannot process volume group vg-one-111
[Z0][TM][E]: Error activating disk /var/lib/one/datastores/111/71/disk.0
[Z0][TM][E]: Error executing image transfer script: Error activating disk /var/lib/one/datastores/111/71/disk.0
[Z0][VM][I]: New LCM state is EPILOG_STOP_FAILURE

And DS config
[oneadmin@one ~]$ onedatastore show 111
DATASTORE 111 INFORMATION
ID : 111
NAME : lvm_system
USER : oneadmin
GROUP : oneadmin
CLUSTERS : 0
TYPE : SYSTEM
DS_MAD : -
TM_MAD : fs_lvm
BASE PATH : /var/lib/one//datastores/111
DISK_TYPE : FILE
STATE : READY

…

DATASTORE TEMPLATE
ALLOW_ORPHANS=“NO”
BRIDGE_LIST=“e54.lan”
DISK_TYPE=“FILE”
DS_MIGRATE=“YES”
RESTRICTED_DIRS="/"
SAFE_DIRS="/var/tmp"
SHARED=“YES”
TM_MAD=“fs_lvm”
TYPE=“SYSTEM_DS”

IMAGES
[oneadmin@one ~]$ onedatastore show 112
DATASTORE 112 INFORMATION
ID : 112
NAME : lvm_default
USER : oneadmin
GROUP : oneadmin
CLUSTERS : 0
TYPE : IMAGE
DS_MAD : fs
TM_MAD : fs_lvm
BASE PATH : /var/lib/one//datastores/112
DISK_TYPE : BLOCK
STATE : READY

…

DATASTORE TEMPLATE
ALLOW_ORPHANS=“NO”
CLONE_TARGET=“SYSTEM”
DISK_TYPE=“BLOCK”
DRIVER=“raw”
DS_MAD=“fs”
LN_TARGET=“SYSTEM”
SAFE_DIRS="/var/tmp /tmp"
TM_MAD=“fs_lvm”
TYPE=“IMAGE_DS”

IMAGES
12

2

Hello,

not sure to have the right answer, but you should add:

BRIDGE_LIST="e54.lan e55.lan"
to datastore 111.

I am using that on my datastores, to indicate that all datastores operations are done on e54 and e55 (or equiv :wink: ), and not one Front.

Regards,
Nicolas

1 Like
3

Thank you Nicolas for the info. Not long before your post I found working attribute set by trial and error but still was not sure which attribute in which DS made it work.
The BRIDGE_LIST might be the key as you suggest.

So I am now using BRIDGE_LIST with fs_lvm SYSTEM DS. Looks like it is not needed for the other (shared/qcow2) two.

For now I have 1 IMAGES DS for each destination SYSTEM DS format (fs_lvm, shared, qcow2)
and IMAGES DSes are bound to SYSTEM ones with COMPATIBLE_SYS_DS attribute to skip choosing right destination DS when instantiating VM.

My DSes pairs are:
default => system (shared)
lvm_default => lvm_system (fs_lvm)
qcow2_default => qcow2_system (qcow2)

All 3 IMAGES DSes are served to FE and nodes via NFS.
Anyone needing more details on my setup please reply.

There are changes in release 5.10 that are supposed to allow for single IMAGES DS that will autoconvert images to multiple SYSTEM DSes. See https://github.com/OpenNebula/one/issues/3244

My fio tests show LVM is fastest (the difference is mainly in write/randwrite)
so I use this DS for production and raw/qcow2 files for temporary VMs.

Migrations still does not work for me (Error restoring VM: Could not recalculate paths in /var/lib/one//datastores/116/83/checkpoint.xml)
but this may be because SYSTEM DSes are not shared which I am fine with.