Linstor datastore Marketplace export requires full virtual image size despite thin provisioning


I am using OpenNebula with a LINSTOR-backed image datastore and I am trying to upload/export an image to the OpenNebula Marketplace.

The image is thin-provisioned in the LINSTOR/LVM thinpool. The OpenNebula image is registered as a 600G image, but the actual allocated data in the LVM thinpool is much smaller, around 24G.

Example from oneimage show:

IMAGE ID      : 172
NAME          : example-thin-image
DATASTORE     : LINSTOR image datastore
FORMAT        : qcow2
SIZE          : 600G
STATE         : rdy
SOURCE        : OpenNebula-Image-172

The LINSTOR/LVM thin volume shows something like:

OpenNebula-Image-172_00000  LSize <600G  Data% ~4%

So the actual consumed storage in the thinpool is much lower than the virtual image size.

However, when I try to export/upload the image to the Marketplace, the export appears to create a full-size staging file. The export failed with an error similar to:

RuntimeError: Unable to dd export image file
ERROR: bash: Command "could not dd image" failed

On the datastore bridge host, I found a partial temporary file under /var/tmp that had already grown to several hundred GB before the filesystem filled up:

/var/tmp/OpenNebula-Image-172-xxx  ~398G

The bridge host OS filesystem then reached 100% usage, causing the Marketplace export to fail.

My understanding is that the LINSTOR datastore export path is effectively doing something like:

LINSTOR/DRBD block device -> dd -> regular staging file -> Marketplace upload

Because of that, the export file grows toward the full virtual block device size, even though the backing image is thin-provisioned.

My questions are:

  1. Is this expected behavior for Marketplace export from a LINSTOR image datastore?

  2. Does the Marketplace upload/export process require staging space equal to the full virtual image size?

  3. Is there a supported way to make the export sparse-aware, for example using dd conv=sparse, or to export as a compressed/sparse qcow2 instead?

  4. Is there a recommended workflow for uploading thin-provisioned LINSTOR images to the Marketplace without consuming the full virtual image size in staging storage and object storage?

  5. Would the recommended approach be to keep/use the original qcow2 file and upload that to the Marketplace instead of exporting from the LINSTOR block device?

In short: the image is thin-provisioned and only consumes a small amount of physical storage in LINSTOR, but Marketplace export appears to require the full virtual size as staging space. I would like to know whether this is expected and what the recommended best practice is.

Thanks.

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.