LVM setup on opennebula?

I have setup open nebula with a single kvm node. The datastore is shared using NFS now. Although I want to switch datastores to LVM. All this setup is running on proxmox using nested virtualization. I am trying to replicate our production setup to test some of our custom backup scripts. Prod server uses LVM for datastores.
What I did?
I attached another disk to kvm node and created a volume groups (i.e. vg-one-110 and vg-one-111) then I added a datastore to opennebula node, 110, 111 ids matched. Then I try to deploy a vm in vg-one-110 which is the system store but in the scheduler log there is no suitable datastore.
My doubt is for NFS I shared the storage with all hosts, maybe that is causing it because opennebula node can not see or access the vg on kvm node.

Hi, for LVM datastore support you need to install this addon

Thanks for reply.
I followed this doc LVM Datastore — OpenNebula 5.12.10 documentation for LVM setup. The link has the same procedure, should I redo them?

Hi, there are two LVM datastore types.

-fs_lvm
-block LVM

The first is included in the Opennebula, the second was part of Opennebula, later split into the addon. The main difference is about images datastore handling.

Fs-lvm stores images as files on a shared filesystem.

Block-lvm stores images as LV in VG.

In case, you want to use persistent images, I think is better to use block LVM. The only drawback needs to use cLVM if you want more nodes.

Fs LVM persistent image deployment takes a long time because of copying the image from the filesystem to LVM and on VM termination it copies back LVM LV to the filesystem. The advantage is no need for cLVM.

Hi there,

I followed all the guides out there to set up but none worked.
Lastly, this seems to have worked -

  1. Created system datastore with below attributes
DATASTORE 112 INFORMATION                                                       
ID             : 112                 
NAME           : lvm-system          
USER           : oneadmin            
GROUP          : oneadmin            
CLUSTERS       : 0                   
TYPE           : SYSTEM              
DS_MAD         : -                   
TM_MAD         : fs_lvm              
BASE PATH      : /var/lib/one//datastores/112
DISK_TYPE      : FILE                
STATE          : READY               

DATASTORE CAPACITY                                                              
TOTAL:         : 10G                 
FREE:          : 9.8G                
USED:          : 200M                
LIMIT:         : -                   

PERMISSIONS                                                                     
OWNER          : um-                 
GROUP          : u--                 
OTHER          : ---                 

DATASTORE TEMPLATE                                                              
ALLOW_ORPHANS="NO"
BRIDGE_LIST="172.17.26.223"
DISK_TYPE="FILE"
DS_MIGRATE="YES"
RESTRICTED_DIRS="/"
SAFE_DIRS="/var/tmp"
SHARED="YES"
TM_MAD="fs_lvm"
TYPE="SYSTEM_DS"

IMAGES         
  1. Then created a VG in KVM Node with VG-one-112 as docs suggest
  2. Created Image datastore with below config
ID             : 113                 
NAME           : lvm-image           
USER           : oneadmin            
GROUP          : oneadmin            
CLUSTERS       : 0                   
TYPE           : IMAGE               
DS_MAD         : fs                  
TM_MAD         : fs_lvm              
BASE PATH      : /var/lib/one//datastores/113
DISK_TYPE      : FILE                
STATE          : READY               

DATASTORE CAPACITY                                                              
TOTAL:         : 31G                 
FREE:          : 26.7G               
USED:          : 4.3G                
LIMIT:         : -                   

PERMISSIONS                                                                     
OWNER          : um-                 
GROUP          : u--                 
OTHER          : ---                 

DATASTORE TEMPLATE                                                              
ALLOW_ORPHANS="NO"
CLONE_TARGET="SYSTEM"
CLONE_TARGET_SSH="SYSTEM"
DISK_TYPE="FILE"
DISK_TYPE_SSH="file"
DRIVER="raw"
DS_MAD="fs"
LN_TARGET="SYSTEM"
LN_TARGET_SSH="SYSTEM"
RESTRICTED_DIRS="/"
SAFE_DIRS="/"
TM_MAD="fs_lvm"
TM_MAD_SYSTEM="ssh"
TYPE="IMAGE_DS"

IMAGES         
11             
12             
13             
  1. Shared Image datastore to kvm node

And it is working.
Even though it is working, I copied the datastore configuration from a working production environment. And I don’t understand all of them. It failed when I created datastores from sunstone and I had to add new attributes from production env to the current one.
Can you direct me to the documentation of such configuration?

The docs is here SAN Datastore — OpenNebula 6.2.0 documentation

The most important options are, for Image DS:

DS_MAD         : fs                  
TM_MAD         : fs_lvm 

and for System DS:

TM_MAD         : fs_lvm
BRIDGE_LIST