Default Image Datastore + SAN System Datastore

I have a Dell PowerVault 3200i that I want to use as System Datastore.
I have 1 frontend server and a few nodes (all KVM) all using v6.0.0.2 release.

To get iSCSI working, I followed the instructions here.

On the frontend:
I created 2 datastores: 1 for System and 1 for Image (I don’t understand how the Image DS is used here but I just followed the doc)

$ cat iscsi_lvm_system.ds
NAME = iscsi_lvm_system
TM_MAD = fs_lvm
TYPE = SYSTEM_DS
DISK_TYPE = "BLOCK"
BRIDGE_LIST = "node1 node2 nodeN"
$ onedatastore create iscsi_lvm_system.ds
ID: 102
$ cat issci_lvm_image.ds
NAME = iscsi_lvm_image
DS_MAD = fs
TM_MAD = fs_lvm
DISK_TYPE = "BLOCK"
TYPE = IMAGE_DS
SAFE_DIRS=”/var/tmp /tmp"
BRIDGE_LIST = "node1 node2 nodeN"
$ onedatastore create iscsi_lvm_image.ds
ID: 103

On the nodes:
All nodes are able to access 2 LUNs on the iSCSI and I was able to setup the Volume Groups properly for each DS.

$ sudo vgs
  VG         #PV #LV #SN Attr   VSize     VFree    
  vg-one-102   1   0   0 wz--n-   <14.36t   <14.36t
  vg-one-103   1   0   0 wz--n- <1024.00g <1024.00g

Issues:

  • When I list the datastores on fe, the Image datastore doesn’t have the correct size.
  ID NAME                   SIZE AVA CLUSTERS IMAGES TYPE DS      TM      STAT
 103 iscsi_lvm_image        133G 86% 0             1 img  fs      fs_lvm  on  
 102 iscsi_lvm_system      14.4T 100 0             0 sys  -       fs_lvm  on  
   2 files                  1.1T 94% 0             0 fil  fs      ssh     on  
   1 default                1.1T 94% 0             2 img  fs      ssh     on  
   0 system                    - -   0             0 sys  -       ssh     on  
  • I downloaded an alpine image to the Image Datastore from the marketplace, but when I tried to deploy it to a node and specified the iscsi_lvm_system as Datastore, it fails with the ff logs:
Sun Jun 6 12:14:45 2021 [Z0][VM][I]: New state is ACTIVE
Sun Jun 6 12:14:45 2021 [Z0][VM][I]: New LCM state is PROLOG
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: Command execution failed (exit code: 2): /var/lib/one/remotes/tm/fs_lvm/clone frontend:/var/lib/one//datastores/103/2754b8bdc3e009edc02be6c240fcceea node3:/var/lib/one//datastores/102/7/disk.0 7 103
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: /var/lib/one/remotes/tm/fs_lvm/clone: line 87: [: too many arguments
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: /var/lib/one/remotes/tm/fs_lvm/clone: line 99: [: File not found: integer expression expected
Sun Jun 6 12:14:49 2021 [Z0][TrM][E]: clone: Command " set -e -o pipefail
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: mkdir -p /var/lib/one//datastores/102/7
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]:
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: hostname -f >"/var/lib/one//datastores/102/7/.host" || :
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]:
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: # zero trailing space
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: if [ "yes" = "yes" ]; then
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: LVSIZE=$(sudo -n lvs --nosuffix --noheadings --units B -o lv_size "/dev/vg-one-102/lv-one-7-0" | tr -d '[:blank:]')
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: dd if=/dev/zero of="/dev/vg-one-102/lv-one-7-0" bs=64k oflag=seek_bytes iflag=count_bytes seek="267386880" count="$(( LVSIZE - 267386880 ))"
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: fi
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]:
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: if [[ "cannot open `/var/lib/one//datastores/103/2754b8bdc3e009edc02be6c240fcceea' (No such file or directory)" =~ "LUKS encrypted file" ]]; then
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: cat "/var/lib/one//datastores/103/2754b8bdc3e009edc02be6c240fcceea" > "/dev/vg-one-102/lv-one-7-0"
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: else
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: qemu-img convert -O raw "/var/lib/one//datastores/103/2754b8bdc3e009edc02be6c240fcceea" "/dev/vg-one-102/lv-one-7-0"
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: fi
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]:
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: rm -f "/var/lib/one//datastores/102/7/disk.0"
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: ln -s "/dev/vg-one-102/lv-one-7-0" "/var/lib/one//datastores/102/7/disk.0"" failed: 16+0 records in
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: 16+0 records out
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.162278 s, 6.5 MB/s
Sun Jun 6 12:14:49 2021 [Z0][TrM][I]: bash: line 14: unexpected EOF while looking for matching ``'
Sun Jun 6 12:14:49 2021 [Z0][TrM][E]: Error cloning /var/lib/one//datastores/103/2754b8bdc3e009edc02be6c240fcceea to lv-one-7-0
Sun Jun 6 12:14:49 2021 [Z0][TrM][E]: Error executing image transfer script: Error cloning /var/lib/one//datastores/103/2754b8bdc3e009edc02be6c240fcceea to lv-one-7-0
Sun Jun 6 12:14:49 2021 [Z0][VM][I]: New LCM state is PROLOG_FAILURE

I guess this is relatetd to the Image Datastore with the storage size not reflecting correctly.

  • I am under the impression that the Local Image Datastore will work with SAN/LVM System Datastores on nodes because there’s no mention in the docs about it. NFS was mentioned but it was only for live migration (which I intend to do but not for now). But when deploying an image from the default Image datatstore to a SAN/LVM System datasotore, it throws an error on the scheduler:
Image from default (local) to LVM (SAN) System datastore will result in:
Sun Jun  6 11:09:37 2021 [Z0][VM][E]: Error deploying virtual machine 6 to HID: 3. Reason: [one.vm.deploy] Image Datastore does not support transfer mode: fs_lvm

Is NFS NECESSARY for SAN/LVM System Datastore to work?
What is the purpose of SAN/LVM Image Datatstore on the Documentation mentioned above?

If anyone can enlighten me, I would really appreciate the help. Keep building!