Unable to create VM on NFS shared storage

Community Support Storage
1

Hi, I have a lab setup of opennebula 6.2.0 with 1 FE server, and 2 backend servers.
The underlying OS is Debian 11.2
I am testing with the Debian 11 Image from the OpenNebula Public Marketplace

Using the SSH driver, with nothing shared between the hosts, I can start VM’s

I would like to have a shared file system and use a NFS server to host vm disks and images

I have followed the instructions here

https://docs.opennebula.io/6.2/open_cluster_deployment/storage_setup/nas_ds.html

and created a system and image datastore, from the text configuration file examples listed there.
For testing I’ve exported and mounted the whole /var/lib/one/datastores on each host from the NFS server using NFSv4

my NFS uid/gid for user oneadmin are the same across the NFS clients and server. And the oneadmin user can rw files over NFS without any error from all NFS clients.

But when I create a VM I get the following error

Wed Jan 12 22:41:33 2022: DEPLOY: error: Failed to create domain from /var/lib/one//datastores/128/53/deployment.0 error: internal error: process exited while connecting to monitor: 2022-01-12T22:41:33.858325Z qemu-kvm-one: -blockdev {“driver”:“file”,“filename”:"/var/lib/one//datastores/127/9f6e59910b2f353863c7bd0ddee8cb3b",“node-name”:“libvirt-3-storage”,“cache”:{“direct”:true,“no-flush”:false},“auto-read-only”:true,“discard”:“unmap”}: Could not open ‘/var/lib/one//datastores/127/9f6e59910b2f353863c7bd0ddee8cb3b’: Permission denied Could not create domain from /var/lib/one//datastores/128/53/deployment.0 ExitCode: 255

Can anyone advise on what is going wrong here ?

Thanks in advance,
Kate

1 Like
2

RESOLVED

I’ve worked around this … I found that Debian 11 has AppArmor enforced for libvirt
I’ve disabled AppArmor following this

https://wiki.debian.org/AppArmor/HowToUse#Disable_AppArmor

and the VM will started with “shared” now for system and image datastores.
It’s not perfect, but it gets me going again.

3

Hi @kate,

Just a side note, if you want to avoid completely disabling AppArmor (as it might lead to security leaks) you can just provide permissions for the specific paths where the VMs are going to run into the /etc/apparmor.d/abstractions/libvirt-qemu profile. For example:

/var/lib/one/datastores/** rwk,
1 Like