# Front-end cannot see system datastore mounted on KVM host **URL:** \ **Category:** Product Support\ **Created:** [October 3, 2016, 1:31pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042 "2016-10-03T13:31:23Z")\ **Posts on this page:** 16\ **Page:** 1
**Author:** ![bokus](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/bokus/32/4845_2.png) [@bokus](https://forum.opennebula.io/u/bokus)\ **Post date:** [October 3, 2016, 1:31pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/1 "2016-10-03T13:31:23Z")
First of all, I have a lot of experience, but not with the storage. This is my biggest pain point while doing OpenNebula test. It seems to me that the manual is written for the storage guys, I found it hard to just plainly make a simple setup. Any help is appreciated. The setup is simple. One front-end and one KVM host, both running centos 7. Installation went just fine until I reached the point of setting up system datastore. My kvm host has SAN based storage, connected via multipath and this is all fine. I have a “device” mpathb which is ready for mounting. OK. Currently I have three automatically created datastores. All residing on the front-end. ID=0 is for the system and it is displaying capacity of ZERO. Strange since is is on the same place as OpenNebula. We don’t need this anyway since this is a boot disk for the OS. ID=1 is for the image ds, and ID=2 for the files. OK. This is my kvm host as seen in the front-end: ID NAME CLUSTER RVM ALLOCATED\_CPU ALLOCATED\_MEM STAT 1 csesx5 default 1 10 / 3200 (0%) 128M / 755.6G (0%) on (don’t be confused with the hostname, it is our lab environment) Now I wanted to add another system ds which is connected to my kvm host: - I have mounted my 2TB disk with **mount /dev/mapper/mpathb /var/lib/one/datastores/101** - Then I have checked the situation with **/var/tmp/one/im/kvm-probes.d/monitor\_ds.sh** DS\_LOCATION\_USED\_MB=1199 DS\_LOCATION\_TOTAL\_MB=17878 DS\_LOCATION\_FREE\_MB=16680 DS = [ ID = 0, USED\_MB = 1199, TOTAL\_MB = 17878, FREE\_MB = 16680 ] DS = [ ID = 101, USED\_MB = 81, TOTAL\_MB = 2064108, FREE\_MB = 1959155 ] As you can see ID=101 is properly seen, but I am a bit confused with the first part “DS\_LOCATION” as it only references ID=0, like my newly added 101 is not parsed at all. - After that I have created a new datastore on the front-end with the same ID and executed **onedatastore list** ID NAME SIZE AVAIL CLUSTERS IMAGES TYPE DS TM STAT 0 system - - 0 0 sys - ssh on 1 default 50G 91% 0 1 img fs ssh on 2 files 50G 91% 0 0 fil fs ssh on 101 kvm-ds - - 0 0 sys - ssh on As you can see OpenNebula still cannot use it. I tried restarting the process but nothing new happened. Size/Avail is always at 0… I guess I am doing something wrong and it’s because I do not understand the whole process. How is OpenNebula supposed to notice what is going on at the kvm host? How does it see the creation or deletion of the datastores on the host? What piece of the puzzle am I missing? THANKS ---
**Author:** ![bokus](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/bokus/32/4845_2.png) [@bokus](https://forum.opennebula.io/u/bokus)\ **Post date:** [October 3, 2016, 1:52pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/2 "2016-10-03T13:52:18Z")
Replying to myself… One addition. I can see that the front-end can see the datastore through the host: **onehost show 1** HOST 1 INFORMATION ID : 1 NAME : csesx5 CLUSTER : default STATE : MONITORED IM\_MAD : kvm VM\_MAD : kvm LAST MONITORING TIME : 10/03 15:45:48 HOST SHARES TOTAL MEM : 755.6G USED MEM (REAL) : 4.9G USED MEM (ALLOCATED) : 128M TOTAL CPU : 3200 USED CPU (REAL) : 0 USED CPU (ALLOCATED) : 10 RUNNING VMS : 1 LOCAL SYSTEM DATASTORE #0 CAPACITY TOTAL: : 17.5G USED: : 1.2G FREE: : 16.3G LOCAL SYSTEM DATASTORE #101 CAPACITY TOTAL: : 2T USED: : 81M FREE: : 1.9T **But still I cannot find a way to use it as a system datastore. This is my main question now…** ---
**Author:** ![mcabrerizo](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/mcabrerizo/32/4829_2.png) [@mcabrerizo](https://forum.opennebula.io/u/mcabrerizo)\ **Post date:** [October 3, 2016, 1:57pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/3 "2016-10-03T13:57:08Z")
Hi! maybe I’m wrong but I think all is ok. My system datastore 0, in a fresh installation shows 0 as well. Have you tried to run a VM? The system datastore as you may already know it’s the place where the VM image is moved from the image datastore so the hypervisor can run the VM. Though the system datastores may look 0 or - size that would be because you haven’t still run a VM. Once a VM is instantiated in datastore 0 or datastore 101 the real SIZE is known and it will be modified so it won’t be - anymore. As the onedatastore list says both system datastore are ON in the status column so they are ready to use. Also as both system datastore are configured as TYPE SSH, scp will be used to transfer images from the image datastore to the system datastores. Again, maybe I’m wrong but I’d try to instantiate a VM and have a look again. Cheers! ---
**Author:** ![bokus](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/bokus/32/4845_2.png) [@bokus](https://forum.opennebula.io/u/bokus)\ **Post date:** [October 3, 2016, 2:06pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/4 "2016-10-03T14:06:03Z")
OK, but how can I instruct OpenNebula to use 101 datastore only? I have created a system datastore with ID=101. Is that the proper way? In other words, is there a correlation between datastore ID on the host, and the datastore ID on the front-end? ---
**Author:** ![bokus](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/bokus/32/4845_2.png) [@bokus](https://forum.opennebula.io/u/bokus)\ **Post date:** [October 3, 2016, 2:15pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/5 "2016-10-03T14:15:06Z")
Interesting, I have tried to instantiate a VM as you suggested and it did chose the 101 datastore. It failed though because of the permission problems. But I am still not at ease because I don’t see a clear mechanism on how does ONE chooses the appropriate system datastore. Error executing image transfer script: Error creating directory /var/lib/one/datastores/101/1 at csesx5: mkdir: cannot create directory ’/var/lib/one/datastores/101/1’: Permission denied ---
**Author:** ![mcabrerizo](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/mcabrerizo/32/4829_2.png) [@mcabrerizo](https://forum.opennebula.io/u/mcabrerizo)\ **Post date:** [October 3, 2016, 2:50pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/6 "2016-10-03T14:50:38Z")
Hi again. I think the answer you need is here: [http://docs.opennebula.org/5.0/operation/host\_cluster\_management/datastore\_guide.html](http://docs.opennebula.org/5.0/operation/host_cluster_management/datastore_guide.html) For example with SCHED\_DS\_REQUIREMENTS=“ID=101” declared in your VM template you can tell OpenNebula’s scheduler what datastore to use. By default all datastores you create are members of the default cluster (ID=0) Cheers ---
**Author:** ![bokus](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/bokus/32/4845_2.png) [@bokus](https://forum.opennebula.io/u/bokus)\ **Post date:** [October 3, 2016, 2:55pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/7 "2016-10-03T14:55:34Z")
Thanks a lot. Currently my template does not have this set, but the system uses 101 anyway for some reason. Anyhow, it’s great to know I can force it! BTW the permission problem was because oneadmin user was not allowed to write… But now I have transfer problems… 🙂 Error executing image transfer script: Error copying nebula.localdomain:/var/lib/one//datastores/1/eb2ea40361ad180152f884e41644b34c to csesx5:/var/lib/one//datastores/101/2/disk.0 Bit by bit I will instantiate… ---
**Author:** ![mcabrerizo](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/mcabrerizo/32/4829_2.png) [@mcabrerizo](https://forum.opennebula.io/u/mcabrerizo)\ **Post date:** [October 3, 2016, 3:02pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/8 "2016-10-03T15:02:30Z")
Hi! As I’ve no previous experience with SAN storage in my environments I apologize if my answers are not accurate. I’m sure you’ll get it running. About your copy error, as you mentioned that you’re using CentOS 7, is SELinux enabled? If not I guess you’ve checked that you can write flawlessly in the /var/lib/one/datastores/101 folder… sorry those are my usual troubleshooting steps 🙂 And I’d like to correct myself on my first answer, as Carlos answered here ([Ssh system datastore capacity](https://forum.opennebula.io/t/ssh-system-datastore-capacity/1179/2)) when using SSH datastores: “the capacity is supposed to be inside each one of the host information (host tab, or onehost show). This is because each host will have a different local storage.” So you were right using onehost to show datastores capacity. Cheers! ---
**Author:** ![bokus](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/bokus/32/4845_2.png) [@bokus](https://forum.opennebula.io/u/bokus)\ **Post date:** [October 3, 2016, 3:07pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/9 "2016-10-03T15:07:49Z")
Yep, selinux is disabled. I have also put oneadmin public key in there, as per user guide. After I have changed ownership for datastores/101 directory, I can write there no problems. So the first error dissapeared, no I have this transfer error instead. Never mind, thanks a lot for your help, you have jumpstarted me when you suggested it is fine now. 🙂 I will try to see why the transfer failed. Should be something with ssh but again I have put ssh key in there, hmmm… ---
**Author:** ![bokus](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/bokus/32/4845_2.png) [@bokus](https://forum.opennebula.io/u/bokus)\ **Post date:** [October 3, 2016, 3:13pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/10 "2016-10-03T15:13:42Z")
DNS name not resolved for scp copying… It’s a lab environment, many little things to be taken care of. 🙂 ---
**Author:** ![mcabrerizo](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/mcabrerizo/32/4829_2.png) [@mcabrerizo](https://forum.opennebula.io/u/mcabrerizo)\ **Post date:** [October 3, 2016, 3:16pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/11 "2016-10-03T15:16:03Z")
Yes I know 😄 a lab is a lab! I’m sure you’ll get it working, if not here’s a community to give you support. Cheers! ---
**Author:** ![bokus](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/bokus/32/4845_2.png) [@bokus](https://forum.opennebula.io/u/bokus)\ **Post date:** [October 3, 2016, 3:18pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/12 "2016-10-03T15:18:08Z")
Yep, sure, now I have “failed: Host key verification failed” LOL… Saga continues. 😃 ---
**Author:** ![mcabrerizo](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/mcabrerizo/32/4829_2.png) [@mcabrerizo](https://forum.opennebula.io/u/mcabrerizo)\ **Post date:** [October 3, 2016, 3:25pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/13 "2016-10-03T15:25:55Z")
Well that’s quite common, don’t worry, you have to check ssh works well without password (check [http://docs.opennebula.org/5.0/deployment/node\_installation/kvm\_node\_installation.html#step-4-configure-passwordless-ssh](http://docs.opennebula.org/5.0/deployment/node_installation/kvm_node_installation.html#step-4-configure-passwordless-ssh)). As you’ve already copied the ssh pub keys, from the opennebula frontend try to: 1. su - oneadmin 2. ssh oneadmin@name\_of\_your\_frontend, check if no password is needed and exit 3. ssh oneadmin@name\_of\_your\_kvm\_node, check if no password is needed and exit Cheers! ---
**Author:** ![bokus](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/bokus/32/4845_2.png) [@bokus](https://forum.opennebula.io/u/bokus)\ **Post date:** [October 4, 2016, 1:41pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/14 "2016-10-04T13:41:39Z")
Thanks. Continuing with another post, I have recreated known\_hosts and still the same message appears. DOH ---
**Author:** ![bokus](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/bokus/32/4845_2.png) [@bokus](https://forum.opennebula.io/u/bokus)\ **Post date:** [October 4, 2016, 2:42pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/15 "2016-10-04T14:42:56Z")
Miguel, you have solved my problem while you were working to help someone else. 🙂 Just wanted to start a new thread when I saw your answer about scp problem. When I understood that scp requires local host key as well, I did ssh to the local hostname, stored the key in the cache (known\_hosts) and from that point my VM cloning to kvm host works fine. Phew. 🙂 I know this is scp basics, but when someone is not using linux on a daily basis it’s not that obvious. Thanks man, you’ve been of great help! ---
**Author:** ![mcabrerizo](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/mcabrerizo/32/4829_2.png) [@mcabrerizo](https://forum.opennebula.io/u/mcabrerizo)\ **Post date:** [October 4, 2016, 2:49pm UTC](https://forum.opennebula.io/t/front-end-cannot-see-system-datastore-mounted-on-kvm-host/3042/16 "2016-10-04T14:49:00Z")
Awesome! I was answering you right now and I’ve seen you’ve solved it. The community is here to help so I’m really happy… Have a lot of fun with your lab!