# Best driver with iSCSI shared storage **URL:** \ **Category:** Operations\ **Created:** [November 8, 2019, 10:50pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811 "2019-11-08T22:50:30Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Morpheus](https://avatars.discourse-cdn.com/v4/letter/m/ce73a5/32.png) [@Morpheus](https://forum.opennebula.io/u/Morpheus)\ **Post date:** [November 8, 2019, 10:50pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/1 "2019-11-08T22:50:30Z")
Hi, We are working on new private OpenNebula 5.8 deployment. All our compute nodes are connected to same HighSpeed storage over iSCSI. All of our VMs are using persistent images. What would be the recommended way of Datastore deployment in this case? Thank you. ---
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [November 8, 2019, 11:04pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/2 "2019-11-08T23:04:04Z")
Hi, what storage system is it? The best way is to write own driver. For example, we have HPE 3PAR storage and we wrote own driver, which communicates with storage over API and for each OpenNebula image it creates volume on the storage system. So basically each VM has its own volume(s) on 3PAR. We also leverage 3PAR QoS functionality, deduplication, and compression. We also have separate stats for each volume. This deployment is abbreviated as “volume centric” and these days replaces old “LUN centric” approach. ---
**Author:** ![Morpheus](https://avatars.discourse-cdn.com/v4/letter/m/ce73a5/32.png) [@Morpheus](https://forum.opennebula.io/u/Morpheus)\ **Post date:** [November 8, 2019, 11:08pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/3 "2019-11-08T23:08:19Z")
It is an Dell/EqualLogic HW controller based storage in m1000E chassis. ---
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [November 8, 2019, 11:12pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/4 "2019-11-08T23:12:01Z")
this? [https://www.dell.com/support/home/si/en/sibsdt1/product-support/product/equallogic-ps-m4110/docs](https://www.dell.com/support/home/si/en/sibsdt1/product-support/product/equallogic-ps-m4110/docs) ---
**Author:** ![Morpheus](https://avatars.discourse-cdn.com/v4/letter/m/ce73a5/32.png) [@Morpheus](https://forum.opennebula.io/u/Morpheus)\ **Post date:** [November 8, 2019, 11:32pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/5 "2019-11-08T23:32:56Z")
Yes. 3 of those. ---
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [November 8, 2019, 11:37pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/6 "2019-11-08T23:37:27Z")
There is no existing driver for this. Of course, you can export one big lun to each node and use clustered FS on top of it with qcow2 images or use cLVM. Both is complicated and complex. Best way is to create driver for you storage system and automate volume creation using ssh cli. We are able to do this, if you are interested contact us via email. [https://www.feldhost.net/products/opennebula](https://www.feldhost.net/products/opennebula) ---
**Author:** ![ahuertas](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ahuertas/32/7487_2.png) [@ahuertas](https://forum.opennebula.io/u/ahuertas)\ **Post date:** [November 11, 2019, 10:39am UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/7 "2019-11-11T10:39:48Z")
Hello @Morpheus You can use the LVM driver in OpenNebula. This is the recommended driver for high-end SAN as your case. To configure the nodes, you can take a look at this [guide](http://docs.opennebula.org/5.8/deployment/open_cloud_storage_setup/lvm_drivers.html#node-setup). Then in OpenNebula you need to create the datastores with **fs\_lvm** as **TM\_MAD**. You can find more information about that [here](http://docs.opennebula.org/5.8/deployment/open_cloud_storage_setup/lvm_drivers.html#opennebula-configuration). Finally, to have a complete view of how it works, you can check the full documentation page [here](http://docs.opennebula.org/5.8/deployment/open_cloud_storage_setup/lvm_drivers.html). ---
**Author:** ![Morpheus](https://avatars.discourse-cdn.com/v4/letter/m/ce73a5/32.png) [@Morpheus](https://forum.opennebula.io/u/Morpheus)\ **Post date:** [November 12, 2019, 8:53pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/8 "2019-11-12T20:53:47Z")
Hi, I have followed to instructions, but I am getting following error oneadmin@amscn03:~ onedatastore create ./ds.cfg [one.datastore.allocate] Parse error: syntax error, unexpected VARIABLE, expecting EQUAL or EQUAL\_EMPTY at line 3, columns 88:95 oneadmin@amscn03:~ cat ./ds.cfg NAME = AmsMainDataStore TM\_MAD = fs\_lvm TYPE = SYSTEM\_DS BRIDGE\_LIST = amscn01 amscn02 amscn03 amscn04 amscn05 amscn06 amscn07 amscn08 oneadmin@amscn03:~$ ---
**Author:** ![ahuertas](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ahuertas/32/7487_2.png) [@ahuertas](https://forum.opennebula.io/u/ahuertas)\ **Post date:** [November 13, 2019, 8:51am UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/9 "2019-11-13T08:51:28Z")
Hello @Morpheus Try with the following template: ```auto NAME = AmsMainDataStore TM_MAD = fs_lvm TYPE = SYSTEM_DS BRIDGE_LIST = "amscn01,amscn02,amscn03,amscn04,amscn05,amscn06,amscn07,amscn08" ``` ---
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [November 13, 2019, 8:48pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/10 "2019-11-13T20:48:38Z")
FS\_LVM or BLOCK\_LVM uses the LUN centric approach; you export one big LUN, and VMs sit on it. You lost per VM statistic direct on the storage system, lost QoS (bandwidth and IOPS limiting) per VM, and so on… It works, but it is worse than develop a storage driver directly for your storage system. ---
**Author:** ![Morpheus](https://avatars.discourse-cdn.com/v4/letter/m/ce73a5/32.png) [@Morpheus](https://forum.opennebula.io/u/Morpheus)\ **Post date:** [November 13, 2019, 9:29pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/11 "2019-11-13T21:29:43Z")
Hi, @feldsam, Thank you for your input. Our implementation is very application specific and we are not looking for limiting the access to storage. Those are very specific VMs, which we are very much familiar with. @ahuertas, Thank you for response, I was able to figure that part out, but I am having other issues. I have shared 20 TB VG ( currently VG\_NAME = vg-one-107) storage. To build it I have followed the document you have mentioned(use\_lvmetad = 0). I am looking to have one Datastore with some amount of persistent images and VMs using those images from that storage without cloning on initiation. I have build following template for Datastore: NAME = AmsMainDataStore DISK\_TYPE = BLOCK DS\_MAD = lvm TM\_MAD = fs\_lvm TYPE = IMAGE\_DS VG\_NAME = vg-one-107 LN\_TARGET = SELF CLONE\_TARGET = SELF BRIDGE\_LIST = “amscn01 amscn02 amscn03 amscn04 amscn05 amscn06 amscn07 amscn08” I think that I need to use DS\_MAD as lvm, but I do not have lvm scripts in ~/remotes/datastore/ I have also changed in /etc/one/oned.conf TM\_MAD\_CONF = [ NAME = “fs\_lvm”, LN\_TARGET = “SELF”, CLONE\_TARGET = “SELF”, SHARED=“YES”, DRIVER = “raw” ] Any advice will be highly appreciated. Best regards Garry ---
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [November 13, 2019, 9:40pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/12 "2019-11-13T21:40:53Z")
Hi, in past I was using LVM block storage, somethign simillar to your setup and I used this block lvm storage driver. > **[OpenNebula/addon-lvm](https://github.com/OpenNebula/addon-lvm)** > > Storage drivers for block lvm. Contribute to OpenNebula/addon-lvm development by creating an account on GitHub. It was part of OpenNebula in past, but dev teams removes it and replaced by fs\_lvm. If you want use block lvm, use that driver and look on this [https://www.systutorials.com/docs/linux/man/8-lvmlockd/](https://www.systutorials.com/docs/linux/man/8-lvmlockd/) it is shared lvm with use of sanlock and you dont need to run corosync/pacemaker and cLVM. EDIT: lvmlockd also supports lvm thin provisioning and snapshots. ---
**Author:** ![Morpheus](https://avatars.discourse-cdn.com/v4/letter/m/ce73a5/32.png) [@Morpheus](https://forum.opennebula.io/u/Morpheus)\ **Post date:** [November 13, 2019, 10:03pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/13 "2019-11-13T22:03:24Z")
@feldsam Thank you. It is very “painful” topic, we are using Debian 9, for hosts. lvmlockd is not supported in “stretch” only in “buster/testing”. cman also not in “strech”. Also corosync, clvm and pcs are in conflict with opennebula 5.8 packages. I have few instances deployed with opennebula 4.14 and clvm/cman for few years now. ---
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [November 13, 2019, 10:06pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/14 "2019-11-13T22:06:53Z")
I can recommend you Oracle Linux 7, it is free and is better that CentOS 7. In past years, we use Centos, fedora, also tried debian stuff, but OL7 is best. EDIT: my offer is still valid, we can help you with your setup 😉 ---
**Author:** ![ahuertas](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ahuertas/32/7487_2.png) [@ahuertas](https://forum.opennebula.io/u/ahuertas)\ **Post date:** [November 14, 2019, 8:27am UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/15 "2019-11-14T08:27:21Z")
Hello @Morpheus Your **DS\_MAD** as you said should be inside `/var/lib/one/remotes/datastore`. There is no lvm for DS\_MAD, so you can leave it blank and it should work. ---
**Author:** ![Morpheus](https://avatars.discourse-cdn.com/v4/letter/m/ce73a5/32.png) [@Morpheus](https://forum.opennebula.io/u/Morpheus)\ **Post date:** [November 15, 2019, 12:16am UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/16 "2019-11-15T00:16:19Z")
Hi @ahuertas, Thank you for response. Unfortunately it requires the DS\_MAD. I am getting following error: [one.datastore.allocate] No DS\_MAD in template. Template: NAME = AmsMainDataStore DISK\_TYPE = BLOCK TM\_MAD = fs\_lvm TYPE = IMAGE\_DS VG\_NAME = vg-one-108 BRIDGE\_LIST = “amscn01 amscn02 amscn03 amscn04 amscn05 amscn06 amscn07 amscn08” ---
**Author:** ![ahuertas](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ahuertas/32/7487_2.png) [@ahuertas](https://forum.opennebula.io/u/ahuertas)\ **Post date:** [November 18, 2019, 8:14am UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/17 "2019-11-18T08:14:59Z")
Hello @Morpheus Sorry I meant using `-` as blank value. ---
**Author:** ![Morpheus](https://avatars.discourse-cdn.com/v4/letter/m/ce73a5/32.png) [@Morpheus](https://forum.opennebula.io/u/Morpheus)\ **Post date:** [November 18, 2019, 5:15pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/18 "2019-11-18T17:15:00Z")
@ahuertas, sorry, but it doesn’t work. I have tried different variations without success. ---
**Author:** ![ahuertas](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ahuertas/32/7487_2.png) [@ahuertas](https://forum.opennebula.io/u/ahuertas)\ **Post date:** [November 19, 2019, 8:09am UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/19 "2019-11-19T08:09:27Z")
Try using **DS\_MAD=fs** ---
**Author:** ![Morpheus](https://avatars.discourse-cdn.com/v4/letter/m/ce73a5/32.png) [@Morpheus](https://forum.opennebula.io/u/Morpheus)\ **Post date:** [November 20, 2019, 8:03pm UTC](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811/20 "2019-11-20T20:03:34Z")
Unfortunately it does not work. [Next page](https://forum.opennebula.io/t/best-driver-with-iscsi-shared-storage/7811.md?page=2)