I’ve set up OpenNebula (6) in combination with Gluster datastores. But when I download an image from the Appstore (i.e. the OpenNebula Debian 10 image) the image stays LOCKED forever. If I download it to local storage it works fine, so it feels like something to do with how OpenNebula interacts with the Gluster datastores.
To clarify: I have three datastores in addition to the default ones, all of which are TM_MAD SHARED (they are symlinks in to the gluster mount on the OneAdmin node and on the KVM node).
After setting this up I downloaded images from the Appstore. I can see the file being downloaded in the datastore directory, and after a while file seems completed: It no longer changes in size and the md5 hash from the app store is the same as md5sum on the file itself. However, it still remains LOCKED.
To check its related to the Gluster datastores I’ve also downloaded an image to the local filesystem store and that works fine (state READY).
I’ve also ruled out it is related to the Appstores download - If I create an image by copying an ISO file from the local filesystem in to the Gluster datastores the image also ends up LOCKED, even though the copy operation itself works fine and the files have matching md5sums.
OpenNebula logs are not telling me anything here, no blatant error messages or such. Hopefully someone here can either share experiences or point me to where I can further troubleshoot the issue?
the host had in the mean time been upgraded to 6.0.0.1-1.ce debian packages, which upon inspection seem to have most of the patch lines you refer to. Unfortunately, I still have no luck on the gluster volume with sharding enabled, the image remains in a LOCKED state.
To reconfirm I’ve changed the datastore to a gluster volume without sharding again and when I do the same operation there, the image download finishes properly and reaches status READY.
Since I plan to keep sharding disabled in my setup this resolves my issue, but in case someone else using gluster with sharding runs in to this: take heed
Thanks for your feedback and pointers, it’s much appreciated!
Best,
Florian
On an unrelated note, the issue with oned not starting turned out to be a known issue with authentication tokens and was easily remediated with the instructions from Known Issues — OpenNebula 6.0.1 documentation
thanks for your feedback. Before I applied your patches I also found something else that might be noteworthy and perhaps related: The gluster store had been configured tor sharding. When I tried the same with the sharding disabled it works as expected. I will try to figure out some decent ways to verify the scenario’s.
First now, for some reason oned won’t start Will research and if needed post a separate thread.
I’m not really familiar with how GlusterFS sharding works and which limitations it might have. From the OpenNebula point of view if there’s not any especial requirement for managing/accessing files when this is enabled it should works. For shared drivers we only assume that the underlying file system is POSIX compliant and that every host have access to the same storage.
Will research and if needed post a separate thread.