Permission denied when deploying LXD on system datastore

Community Support
1

I’m seeing a confusing failure when I try to deploy a container I downloaded from the Marketplace. I have a system DS defined and ‘oneadmin’ and the nodes can write to it:

DATASTORE 105 INFORMATION
ID             : 105
NAME           : LXDds
USER           : oneadmin
GROUP          : oneadmin
CLUSTERS       : 0
TYPE           : SYSTEM
DS_MAD         : -
TM_MAD         : ssh
BASE PATH      : /var/lib/one//datastores/105
DISK_TYPE      : FILE
STATE          : READY

DATASTORE CAPACITY
TOTAL:         : -
FREE:          : -
USED:          : -
LIMIT:         : -

PERMISSIONS
OWNER          : um-
GROUP          : u--
OTHER          : ---

DATASTORE TEMPLATE
ALLOW_ORPHANS="NO"
DISK_TYPE="FILE"
DS_MIGRATE="YES"
RESTRICTED_DIRS="/"
SAFE_DIRS="/var/tmp"
SHARED="NO"
TM_MAD="ssh"
TYPE="SYSTEM_DS"

When I try to deploy and specify this DS I get this:

Wed Oct 14 12:31:30 2020 [Z0][VM][I]: New state is ACTIVE
Wed Oct 14 12:31:30 2020 [Z0][VM][I]: New LCM state is PROLOG
Wed Oct 14 12:31:52 2020 [Z0][VM][I]: New LCM state is BOOT
Wed Oct 14 12:31:52 2020 [Z0][VMM][I]: Generating deployment file: /var/lib/one/vms/146/deployment.0
Wed Oct 14 12:31:55 2020 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_context.
Wed Oct 14 12:31:55 2020 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Wed Oct 14 12:31:56 2020 [Z0][VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/lxd/deploy '/var/lib/one//datastores/105/146/deployment.0' 'hpc-onet6' 146 hpc-onet6
Wed Oct 14 12:31:56 2020 [Z0][VMM][I]: deploy: Processing disk 0
Wed Oct 14 12:31:56 2020 [Z0][VMM][I]: deploy: Using qcow2 mapper for /var/lib/one/datastores/105/146/disk.0
Wed Oct 14 12:31:56 2020 [Z0][VMM][E]: deploy: do_map: Failed to blk_new_open '/var/lib/one/datastores/105/146/disk.0': Could not open '/var/lib/one/datastores/105/146/disk.0': Permission denied
Wed Oct 14 12:31:56 2020 [Z0][VMM][I]: deploy: Mapping disk at /var/lib/lxd/storage-pools/default/containers/one-146/rootfs using device
Wed Oct 14 12:31:56 2020 [Z0][VMM][I]: deploy: Processing disk 0
Wed Oct 14 12:31:56 2020 [Z0][VMM][I]: deploy: Using qcow2 mapper for /var/lib/one/datastores/105/146/disk.0
Wed Oct 14 12:31:56 2020 [Z0][VMM][I]: deploy: Unmapping disk at /var/lib/lxd/storage-pools/default/containers/one-146/rootfs
Wed Oct 14 12:31:56 2020 [Z0][VMM][E]: deploy: Failed to detect block device from /var/lib/lxd/storage-pools/default/containers/one-146/rootfs
Wed Oct 14 12:31:56 2020 [Z0][VMM][I]: deploy: Unmapping disk at /var/lib/one/datastores/105/146/mapper/disk.1
Wed Oct 14 12:31:56 2020 [Z0][VMM][E]: deploy: Failed to detect block device from /var/lib/one/datastores/105/146/mapper/disk.1
Wed Oct 14 12:31:56 2020 [Z0][VMM][I]: /var/tmp/one/vmm/lxd/deploy:64:in `<main>': failed to setup container storage (RuntimeError)
Wed Oct 14 12:31:56 2020 [Z0][VMM][I]: ExitCode: 1
Wed Oct 14 12:31:56 2020 [Z0][VMM][I]: Failed to execute virtualization driver operation: deploy.
Wed Oct 14 12:31:56 2020 [Z0][VMM][E]: Error deploying virtual machine
Wed Oct 14 12:31:56 2020 [Z0][VM][I]: New LCM state is BOOT_FAILURE

Why is it saying deploy: Using qcow2 mapper for /var/lib/one/datastores/105/146/disk.0 when I set up the datastore using the SSH transfer manager? I didn’t change the container image, it’s set to raw. What am I doing wrong?

My LXD host seems to see the datastore fine:

$ onehost show 11
HOST 11 INFORMATION
ID                    : 11
NAME                  : hpc-onet6
CLUSTER               : default
STATE                 : MONITORED
IM_MAD                : lxd
VM_MAD                : lxd
LAST MONITORING TIME  : 10/14 12:45:42
...
...

LOCAL SYSTEM DATASTORE #105 CAPACITY
TOTAL:                : 1T
USED:                 : 58.7G
FREE:                 : 1014.3G
2

Wed Oct 14 12:31:56 2020 [Z0][VMM][E]: deploy: do_map: Failed to blk_new_open ‘/var/lib/one/datastores/105/146/disk.0’: Could not open ‘/var/lib/one/datastores/105/146/disk.0’: Permission denied

The driver runs file -L -s <image path> and depending on the output it decides which mapping system to use (qcow2, loop devices, rbd, etc.). Take

Your output should be similar to

file -L -s /var/lib/one/datastores/0/2/disk.0
/var/lib/one/datastores/0/2/disk.0: QEMU QCOW Image (v3), has backing file (path /var/lib/one/datastores/1/ff345705ee5e2e16bb97002a7fd25e7f), 1073741824 bytes

But then qemu-nbd -c <first available nbd device> <image path> --fork fails with your image.

Make sure your image is in fact raw, you can run qemu-img info <image path>

3

That’s what it was, I was downloading the image to the default image datastore, which is QCOW. The container deploys fine on a new shared image and system DS. Thanks.