FS_LVM in OpenNebula

Community Support
2

Hello,

I have probably simillar setup. I have SAN and exported 7T LUN, which is shared across hosts. This is my setup:

Two Node SAN cluster powered by ESOS

Two Node Frontend Cluster

  • LUN 20GB used for /etc/one and /var/lib/one
  • LVM activated exclusively only one node, two LVs, both formatted to xfs and mounted to /etc/one and /var/lib/one
  • NFS Export /var/lib/one

Compute nodes 3+

  • LUN 7TB used for images
  • Configured clusterd LVM (cLVM), vg group vg-images - just VG, lvolumes are created by oned.
  • Mounted exported NFS share under /var/lib/one
  • I use NFS because its easy way to share this partition over hosts, you can also use clustered filesystem like GFS but it just complicate setup. Your VM images will run from non system datastore. In system datastore will be just symlinks to LVs

Datastores

  • System default datastore in /var/lib/one

  • Datastore for “images” with this config:

    [oneadmin@engine1 ~]$ onedatastore show 104
    DATASTORE 104 INFORMATION
    ID : 104
    NAME : images
    USER : oneadmin
    GROUP : oneadmin
    CLUSTER : feldhost
    TYPE : IMAGE
    DS_MAD : lvm
    TM_MAD : lvm
    BASE PATH : /var/lib/one/datastores/104
    DISK_TYPE : BLOCK
    STATE : READY

    DATASTORE CAPACITY
    TOTAL: : 7.3T
    FREE: : 6.5T
    USED: : 845.1G
    LIMIT: : -

    PERMISSIONS
    OWNER : um-
    GROUP : u–
    OTHER : —

    DATASTORE TEMPLATE
    BASE_PATH="/var/lib/one/datastores/"
    BRIDGE_LIST="node1 node2 node3"
    CLONE_TARGET="SELF"
    DISK_TYPE="BLOCK"
    DS_MAD="lvm"
    LN_TARGET="NONE"
    TM_MAD="lvm"
    TYPE="IMAGE_DS"
    VG_NAME=“vg-images”

I use LVM BLOCK, important is BRIDGE_LIST which is just IP addresses or DNS names of nodes, which can manipulate with that LVM VG. Also CLONE_TARGET=SELF is important.

It’s working good, I just have problems sometime with cloning LVs and by importing images from OneMarket to lvm images datastore. You can clone manualy with dd command and also you can easily resize VM images just by lvextend command. Disadvantage of LVs is that you have to clone whole volume bite by bite, and also in clustered env there is no thin provisioning. I also try to setup FS based datastore with clustered filesystem GFS2, but I have some problems with data corruption…

Hope to help you.

1 Like