Hi.
I am trying to configure FS_LVM Datastore in an installation in Production.
In our case we have created a LUN 0f 30Tb to be used to implement the System Datastore via FS_LVM. This LUN will be shared among the ONED node and 5 KVM nodes.
What we did was:
IN ONED NODE:
Create a Physical Volume using the pvcreate Centos command, attaching this PV to the LUN discover via multipath
Create a Volume Group via “vgcreate”
Create a Logical Volume via “lvcreate” assigning it 100% of the VG space.
Format the LV via “mkfs.xfs”
Mount the Logical Volume via /etc/fstab
REST OF THE NODES (KVM NODES):
-Mount the Logical Volume via /etc/fstab
At this point I have full visibility and read /write access to the LUN in all the involved hosts
After I create the fs_lvm system datastore , with this configuration:
[root@cnsrvbld22 one]# onedatastore show 121
DATASTORE 121 INFORMATION
ID : 121
NAME : golden-system
USER : oneadmin
GROUP : oneadmin
CLUSTER : GOLDEN
TYPE : SYSTEM
DS_MAD : -
TM_MAD : fs_lvm
BASE PATH : /var/lib/one//datastores/golden/121
DISK_TYPE : FILE
STATE : READY
Note: In the procedure I create the Logical Volume… I know that the Logical Volume normally will be created by OpenNebula, but if I don’t create it, I don’t see a way to mount the LUN in the rest of the nodes (except if I just mount it as a normal xfs file system (not LVM).
LVM activated exclusively only one node, two LVs, both formatted to xfs and mounted to /etc/one and /var/lib/one
NFS Export /var/lib/one
Compute nodes 3+
LUN 7TB used for images
Configured clusterd LVM (cLVM), vg group vg-images - just VG, lvolumes are created by oned.
Mounted exported NFS share under /var/lib/one
I use NFS because its easy way to share this partition over hosts, you can also use clustered filesystem like GFS but it just complicate setup. Your VM images will run from non system datastore. In system datastore will be just symlinks to LVs
Datastores
System default datastore in /var/lib/one
Datastore for “images” with this config:
[oneadmin@engine1 ~]$ onedatastore show 104
DATASTORE 104 INFORMATION
ID : 104
NAME : images
USER : oneadmin
GROUP : oneadmin
CLUSTER : feldhost
TYPE : IMAGE
DS_MAD : lvm
TM_MAD : lvm
BASE PATH : /var/lib/one/datastores/104
DISK_TYPE : BLOCK
STATE : READY
I use LVM BLOCK, important is BRIDGE_LIST which is just IP addresses or DNS names of nodes, which can manipulate with that LVM VG. Also CLONE_TARGET=SELF is important.
It’s working good, I just have problems sometime with cloning LVs and by importing images from OneMarket to lvm images datastore. You can clone manualy with dd command and also you can easily resize VM images just by lvextend command. Disadvantage of LVs is that you have to clone whole volume bite by bite, and also in clustered env there is no thin provisioning. I also try to setup FS based datastore with clustered filesystem GFS2, but I have some problems with data corruption…