HI I’m using version 6.10.1 for a few months now. (Its a great product).
We are starting to see a datastore creation issue as we’ve scaled up in hosts/datastores.
Typically when we add a new datastore within a few seconds the new ID is at /var/lib/one/datastore.
We have NFS mounted “shared” datastores. Once the IDs is created, then I create NFS mount symbolic link to the datastore directory.
Recently we’ve started seeing a huge delay in the ID directory being created on the filesystem at “/var/lib/one/datatore”.
Some times it takes an hour and as long as 24hours+. I’ve tried restarting ‘oned’ process to kick it, but that doesn’t trigger it. I can’t find any error related logs in oned or monitor.
Any suggestion as to where to find the error or what set to trigger it to complete ID setup and monitoring?
Thanks for your time and help,
-_Ken
Here is a one from today:
created via cli as:
oneadmin@s002-openneb:~/ken$ cat system_datastore.txt
NAME = laas-m7-ssd-ds1-s002-nfs15
DISK_TYPE = "FILE"
SHARED = "YES"
TM_MAD = "shared"
TYPE = "SYSTEM_DS"
DATASTORE_CAPACITY_CHECK = "YES"
LIMIT_MB = 1500000
DS_TYPE="ZFS"
oneadmin@s002-openneb:~/ken$ onedatastore create ./system_datastore.txt
I see it quickly, but with values of zero size
ID NAME SIZE AVA CLUSTERS IMAGES TYPE DS TM STAT
179 laas-m7-ssd-ds1-s002-nfs15 0M - 0 0 sys - shared on
oneadmin@s002-openneb:~/ken$ onedatastore show 179
DATASTORE 179 INFORMATION
ID : 179
NAME : laas-m7-ssd-ds1-s002-nfs15
USER : oneadmin
GROUP : oneadmin
CLUSTERS : 0
TYPE : SYSTEM
DS_MAD : -
TM_MAD : shared
BASE PATH : /var/lib/one//datastores/179
DISK_TYPE : FILE
STATE : READY
DATASTORE CAPACITY
TOTAL: : 0M
FREE: : 0M
USED: : 0M
LIMIT: : 1.4T
PERMISSIONS
OWNER : um-
GROUP : u--
OTHER : ---
DATASTORE TEMPLATE
ALLOW_ORPHANS="FORMAT"
DATASTORE_CAPACITY_CHECK="YES"
DISK_TYPE="FILE"
DS_MIGRATE="YES"
DS_TYPE="ZFS"
LIMIT_MB="1500000"
RESTRICTED_DIRS="/"
SAFE_DIRS="/var/tmp"
SHARED="YES"
TM_MAD="shared"
TYPE="SYSTEM_DS"
IMAGES
oneadmin@s002-openneb:~/ken$
checking here the ID is not created on the filesystem.
oneadmin@s002-openneb:~/datastores$
oneadmin@s002-openneb:~/datastores$ ls
0 100 109 111 113 115 117 121 123 125 127 129 131 133 135 137 139 141 143 145 147 149 151 153 155 157 2
1 101 110 112 114 116 120 122 124 126 128 130 132 134 136 138 140 142 144 146 148 150 152 154 156 158 ISO
oneadmin@s002-openneb:~/datastores$ ls *179*
ls: cannot access '*179*': No such file or directory
oneadmin@s002-openneb:~/datastores$
This log is in oned.log
Fri May 16 16:46:13 2025 [Z0][ImM][D]: Monitoring datastore laas-m7-ssd-ds1-s002-nfs15 (179)
a normal datastore have a 2nd log like this
Fri May 16 16:45:43 2025 [Z0][ImM][D]: Monitoring datastore laas-m7-ssd-ds1-s002-nfs9 (117)
Fri May 16 16:45:54 2025 [Z0][ImM][D]: Datastore laas-m7-ssd-ds1-s002-nfs9 (117) successfully monitored.