Resize VM's volatile disk on the fly: Failed to get "write" lock

gray380 · October 11, 2019, 12:17pm

Hello,

I’ve tried to resize volatile disk while VM is running.
The task (in both cases: sunstone and cli) has failed with the following outputs:

Fri Oct 11 14:54:36 2019 [Z0][VM][I]: New LCM state is DISK_RESIZE
Fri Oct 11 14:54:38 2019 [Z0][VMM][I]: Command execution failed (exit code: 1): /var/lib/one/remotes/tm/qcow2/resize one-kvm-node-01-int:/var/lib/one//datastores/101/278/disk.2 51200 278 101
Fri Oct 11 14:54:38 2019 [Z0][VMM][I]: /var/lib/one/remotes/datastore/xpath.rb:66:in `block in <main>': undefined method `elements' for nil:NilClass (NoMethodError)
Fri Oct 11 14:54:38 2019 [Z0][VMM][I]: from /var/lib/one/remotes/datastore/xpath.rb:60:in `each'
Fri Oct 11 14:54:38 2019 [Z0][VMM][I]: from /var/lib/one/remotes/datastore/xpath.rb:60:in `<main>'
Fri Oct 11 14:54:38 2019 [Z0][VMM][I]: /var/lib/one/remotes/tm/qcow2/resize: line 39: [: -eq: unary operator expected
Fri Oct 11 14:54:38 2019 [Z0][VMM][E]: resize: Command "qemu-img resize /var/lib/one/datastores/101/278/disk.2 51200M" failed: qemu-img: Could not open '/var/lib/one/datastores/101/278/disk.2': Failed to get "write" lock
Fri Oct 11 14:54:38 2019 [Z0][VMM][I]: Is another process using the image [/var/lib/one/datastores/101/278/disk.2]?
Fri Oct 11 14:54:38 2019 [Z0][VMM][E]: Error resizing image /var/lib/one/datastores/101/278/disk.2
Fri Oct 11 14:54:38 2019 [Z0][VMM][I]: Failed to execute transfer manager driver operation: tm_resize.
Fri Oct 11 14:54:38 2019 [Z0][VMM][E]: Error resizing disk: Error resizing image /var/lib/one/datastores/101/278/disk.2
Fri Oct 11 14:54:38 2019 [Z0][VM][I]: New LCM state is RUNNING
Fri Oct 11 14:54:38 2019 [Z0][LCM][I]: VM disk resize operation completed.

System datastore (NFS share):

$ onedatastore show 101
DATASTORE 101 INFORMATION                                                       
ID             : 101                 
NAME           : emc-dd-s            
USER           : oneadmin            
GROUP          : tsu                 
CLUSTERS       : 100                 
TYPE           : SYSTEM              
DS_MAD         : -                   
TM_MAD         : qcow2               
BASE PATH      : /var/lib/one//datastores/101
DISK_TYPE      : FILE                
STATE          : READY               

DATASTORE CAPACITY                                                              
TOTAL:         : 1.5T                
FREE:          : 893.7G              
USED:          : 606G                
LIMIT:         : -                   

PERMISSIONS                                                                     
OWNER          : um-                 
GROUP          : u--                 
OTHER          : ---                 

DATASTORE TEMPLATE                                                              
ALLOW_ORPHANS="NO"
DISK_TYPE="FILE"
DS_MIGRATE="YES"
RESTRICTED_DIRS="/"
SAFE_DIRS="/var/tmp"
SHARED="YES"
TM_MAD="qcow2"
TYPE="SYSTEM_DS"

NFS mount options:

hard,intr,nolock,nfsvers=3,tcp,timeo=1200,rsize=1048600,wsize=1048600,bg

So to rezize the disk I have to power off the VM, resize the disk from webui or cli, power on the VM and after that run two commands inside the running VM:

sudo  growpart /dev/vdb 1
sudo xfs_growfs -d /mounpoint

Does opennebula support online disk resizing or I’ve faced some bug?

Versions of the related components and OS (frontend, hypervisors, VMs):
OpenNebula 5.8.5
KVM nodes: CentOS Linux release 7.7.1908 (Core)
VM: Ubuntu 19.04

Steps to reproduce:
$ onevm disk-resize 278 2 51200

atodorov_storpool · October 14, 2019, 6:14am

Hi,

For some reason the resize script could not get the VM’s LCM_STATE and the following test is failing

github.com

OpenNebula/one/blob/master/src/tm_mad/common/resize#L39


      
              TMCOMMON=$ONE_LOCATION/var/remotes/tm/tm_common.sh
              DATASTORES=$ONE_LOCATION/var/datastores
          fi
          
          . $TMCOMMON
          
          SRC_PATH=$(arg_path $SRC)
          SRC_HOST=$(arg_host $SRC)
          
          # for running VMs, leave disk resize on kvm/resize_disk
          if [ `lcm_state $VMID` -eq 62 ]; then
              log "Not resizing disk ${SRC_PATH} for running virt. machine directly"
              exit 0
          fi
          
          ssh_exec_and_log "${SRC_HOST}" "qemu-img resize ${SRC_PATH} ${SIZE}M" \
                           "Error resizing image ${SRC_PATH}"

Most probably you are trying unsupported resize operation but the script is failing to raise the reason:

Not resizing disk /var/lib/one//datastores/101/278/disk.2 for running virt. machine directly

Please check does the oneadmin user have rights to run OpenNebula commands or you are trying to resize wrong VM?

Edit: Just catch up with your other forum posts. It looks like the there si something incomplete in the reconfiguration to use non-default XMLRPC port.

Hope this helps.

Best Regards,
Anton Todorov

Topic		Replies	Views
Disk resize failure Storage	0	475	May 15, 2019
Cannot resize disk 0 Community Support	3	1576	April 24, 2018
Qemu-kvm: Failed to get "write" lock Storage	6	17341	April 5, 2020
So I resized a qcow2 disk on my vm Community Support	0	570	May 9, 2016
Non-live image resize Community Support	4	1573	August 14, 2017

Resize VM's volatile disk on the fly: Failed to get "write" lock

Related topics