# Possible bug using local datastores for system images **URL:** \ **Category:** Product Support\ **Created:** [December 30, 2015, 7:28pm UTC](https://forum.opennebula.io/t/possible-bug-using-local-datastores-for-system-images/1642 "2015-12-30T19:28:20Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![jcataluna](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/jcataluna/32/4535_2.png) [@jcataluna](https://forum.opennebula.io/u/jcataluna)\ **Post date:** [December 30, 2015, 7:28pm UTC](https://forum.opennebula.io/t/possible-bug-using-local-datastores-for-system-images/1642/1 "2015-12-30T19:28:20Z")
Hi all, I’m trying to setup a cluster with 2 nodes: 32GB, 2TB dedicated HDD. Node 1 its running also the frontend. The purpose of the cluster its to deploy a Hadoop cluster for a degree final project. My intention its to use the 2TB disk on each of the nodes to run the VM’s. The only way I’ve found it in the documentation to use this local store, its setting up a FileSystem/SSH datastore. Cluster Configuration: ``` oneadmin@kcldsrv1:~$ onecluster show 100 CLUSTER 100 INFORMATION ID : 100 NAME : KataClust CLUSTER TEMPLATE DATASTORE_LOCATION="/onestore/" HOSTS 0 1 DATASTORES 1 2 100 101 oneadmin@kcldsrv1:/var/log/one$ onedatastore list ID NAME SIZE AVAIL CLUSTER IMAGES TYPE DS TM STAT 0 system 196.7G 94% - 0 sys - shared off 1 default 196.7G 94% KataClust 2 img fs shared on 2 files 196.7G 94% KataClust 0 fil fs ssh on 100 local_system - - KataClust 0 sys - ssh on 101 local_image 393.6G 95% KataClust 1 img fs ssh on oneadmin@kcldsrv1:/var/log/one$ onedatastore show 100 DATASTORE 100 INFORMATION ID : 100 NAME : local_system USER : oneadmin GROUP : oneadmin CLUSTER : KataClust TYPE : SYSTEM DS_MAD : - TM_MAD : ssh BASE PATH : /onestore/100 DISK_TYPE : FILE STATE : READY DATASTORE CAPACITY TOTAL: : - FREE: : - USED: : - LIMIT: : - PERMISSIONS OWNER : um- GROUP : u-- OTHER : --- DATASTORE TEMPLATE BASE_PATH="/onestore/" BRIDGE_LIST="kcldsrv1.cutresoft.bog kcldsrv2.cutresoft.bog" SHARED="NO" TM_MAD="ssh" TYPE="SYSTEM_DS" IMAGES ``` datastore 1 its for images, (shared) Basepath: /var/lib/one/datastores datastore 101 its to mount big data file systems for Hadoop HDFS (ssh). Basepath: /onestore datastore 100 its for running vm’s located in the hosts dir /onestore (ssh) Basepath: /onestore Ok, I have imported a marketplace template with the resulting OS images on DS 1. With that configuration, the vm’s can not be copied from DS\_IMAGE 1 (default) to DS\_SYSTEM 100, as you can see on the following abstract from the log file: It starts trying to copy from the right src to the right destination through the right tm shared as it is the DS 1, I mean, a shared DS. `Wed Dec 30 18:22:35 2015 [Z0][TM][I]: Command execution fail: /var/lib/one/remotes/tm/shared/clone kcldsrv1.cutresoft.bog:/var/lib/one//datastores/1/2fcffd24b5cc9fc7d3dda2d360d1d28c kcldsrv1.cutresoft.bog:/onestore/100/7/disk.0 7 1` Later, the script interchange src and dst trying to copy a non existent file: ``` Wed Dec 30 18:22:35 2015 [Z0][TM][I]: clone: Cloning /onestore/1/2fcffd24b5cc9fc7d3dda2d360d1d28c in kcldsrv1.cutresoft.bog:/onestore/100/7/disk.0 Wed Dec 30 18:22:35 2015 [Z0][TM][E]: clone: Command "cd /onestore/100/7; cp /onestore/1/2fcffd24b5cc9fc7d3dda2d360d1d28c /onestore/100/7/disk.0 " failed: Warning: Permanently added 'kcldsrv1.cutresoft.bog,192.168.1.135' (ECDSA) to the list of known hosts. Wed Dec 30 18:22:35 2015 [Z0][TM][I]: cp: cannot stat '/onestore/1/2fcffd24b5cc9fc7d3dda2d360d1d28c': No such file or directory ``` I have modified /var/lib/one/remotes/tm/shared/clone to fix the wrong src file. From: SRC\_PATH="${DST\_DS\_PATH}${SRC\_ARG\_PATH##$SRC\_DS\_PATH}" To: SRC\_PATH=$SRC\_ARG\_PATH IMO the shared driver its not taking into account the type of the destination DS. With that fix, my VM’s are being created properly. I’m pretty new to OpenNebula so I suppose I must be wrong but the thing is, that the fix solves the problem. Could this be a bug in the tm shared code? Regards. ---
**Author:** ![ruben](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ruben/32/19_2.png) [@ruben](https://forum.opennebula.io/u/ruben)\ **Post date:** [December 30, 2015, 8:50pm UTC](https://forum.opennebula.io/t/possible-bug-using-local-datastores-for-system-images/1642/2 "2015-12-30T20:50:19Z")
Hi Jesus, There are three different parameters used to tune path generation for images: (from oned.conf) - DATASTORE\_LOCATION: **Default** Path for Datastores in the hosts. It IS the same for all the hosts in the cluster. DATASTORE\_LOCATION IS ONLY FOR THE HOSTS AND **NOT** THE FRONT-END. It defaults to /var/lib/one/datastores (or $ONE\_LOCATION/var/datastores in self-contained mode) You can define a different DATASTORE\_LOCATION in each cluster by updating its properties with onecluster update. - DATASTORE\_BASE\_PATH: This is the base path for the SOURCE attribute of the images registered in a Datastore. This is a default value, that can be changed when the datastore is created. So BASE\_PATH is for the origin of the images, i.e. where the images are stored. DATASTORE\_LOCATION is where the images are copied to. So if you want the datastores (and the images) to be copied to /onestore in the hypervisors you need to update DATASTORE\_LOCATION and not BASE\_PATH. Hope it helps ---
**Author:** ![jcataluna](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/jcataluna/32/4535_2.png) [@jcataluna](https://forum.opennebula.io/u/jcataluna)\ **Post date:** [December 31, 2015, 12:43am UTC](https://forum.opennebula.io/t/possible-bug-using-local-datastores-for-system-images/1642/3 "2015-12-31T00:43:18Z")
Hi Ruben, good night. I have defined DATASTORE\_LOCATION in the cluster to /onestore. Reading your explanation, BASE\_PATH it only affects to FrontEnd. It’s the path where FrontEnd expect the images to b. The flow its like that: 1. Original image its created on BASE\_PATH/\ on the FrontEnd 2. Image its copied to DATASTORE\_LOCATION/\/\ on Host, on boot. 3. Image its moved back to BASE\_PATH/\/\ on FrontEnd on vm stop Its that right? Even changing cluster DATASTORE\_LOCATION to a different path of DS BASE\_PATH the copy its not working without the suggested fix Regards. ---
**Author:** ![ruben](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ruben/32/19_2.png) [@ruben](https://forum.opennebula.io/u/ruben)\ **Post date:** [December 31, 2015, 1:43pm UTC](https://forum.opennebula.io/t/possible-bug-using-local-datastores-for-system-images/1642/4 "2015-12-31T13:43:48Z")
note that you need to re-register the images so the SOURCE attribute is generated with the right BASE\_PATH.