Lease of deleted VM not freed properly (stuck)

Hi all,

we got to a point where we have a Vnet with one ‘active’ lease of an already deleted VM.

VIRTUAL NETWORK 14 INFORMATION
ID             : 14
NAME           : BR_3139
USER           : admin
GROUP          : admin
CLUSTERS       : 0,101
BRIDGE         : br_3139
VN_MAD         : dummy
USED LEASES    : 1

PERMISSIONS
OWNER          : um-
GROUP          : ---
OTHER          : ---

VIRTUAL NETWORK TEMPLATE
BRIDGE="br_3139"
GATEWAY6="XXX::XXX"
PHYDEV=""
SECURITY_GROUPS="0"
VLAN_ID=""
VN_MAD="dummy"

ADDRESS RANGE POOL
AR 0
SIZE           : 1
LEASES         : 1

RANGE                                   FIRST                               LAST
MAC                                         XXX                                  XXX
IP6_GLOBAL                            XXX                                  XXX


LEASES
AR  OWNER                         MAC              IP                 IP6_GLOBAL
0   V:158                                XXX               -                     XXX

VIRTUAL ROUTERS

but VM 158 is in DONE state:

VIRTUAL MACHINE 158 INFORMATION
ID                  : 158
NAME                :  XXX
USER                : admin
GROUP               : admin
STATE               : DONE
LCM_STATE           : LCM_INIT
RESCHED             : No
START TIME          : 04/20 16:04:13
END TIME            : 04/20 16:10:36
DEPLOY ID           : one-158

VIRTUAL MACHINE MONITORING
CPU                 : 0.0
MEMORY              : 0K
NETTX               : 17K
NETRX               : 213K

PERMISSIONS
OWNER               : um-
GROUP               : ---
OTHER               : ---

VM DISKS
 ID DATASTORE  TARGET IMAGE                               SIZE      TYPE SAVE
  0 default    sd     XXX HA              3G/8.3G   file   NO
  1 -          hda    CONTEXT                             3M/-      -       -

VM NICS
 ID NETWORK              BRIDGE       IP              MAC               PCI_ID
  0 MGMT             br_mgmt            XXX             XXX
  1 BR_3134          br_3134               -                XXX
  2 BR_3139_SingleIP br_3139         -                XXX

SECURITY

VIRTUAL MACHINE HISTORY
SEQ HOST            ACTION             DS           START        TIME     PROLOG
  0 olnmpep02318n00 terminate-hard      0  04/20 16:04:20   0d 00h06m   0h00m36s

I think the problem is related to a change in the name of the VNet, so, the steps to reproduce are (I did not double-check, as I did not find any way to delete the lease yet):

  1. Create a VM and attach it to some VNET
  2. Change the name of the VNET
  3. Remove the VM
  4. The lease gets stuck

The log for that VM:
Thu Apr 20 16:04:20 2017 [Z0][VM][I]: New state is ACTIVE
Thu Apr 20 16:04:20 2017 [Z0][VM][I]: New LCM state is PROLOG
Thu Apr 20 16:04:56 2017 [Z0][VM][I]: New LCM state is BOOT
Thu Apr 20 16:04:56 2017 [Z0][VMM][I]: Generating deployment file: /var/lib/one/vms/158/deployment.0
Thu Apr 20 16:04:56 2017 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_context.
Thu Apr 20 16:04:57 2017 [Z0][VMM][I]: ExitCode: 0
Thu Apr 20 16:04:57 2017 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Thu Apr 20 16:04:58 2017 [Z0][VMM][I]: ExitCode: 0
Thu Apr 20 16:04:58 2017 [Z0][VMM][I]: Successfully execute virtualization driver operation: deploy.
Thu Apr 20 16:04:58 2017 [Z0][VMM][I]: ExitCode: 0
Thu Apr 20 16:04:58 2017 [Z0][VMM][I]: Successfully execute network driver operation: post.
Thu Apr 20 16:04:58 2017 [Z0][VM][I]: New LCM state is RUNNING
Thu Apr 20 16:10:34 2017 [Z0][VM][I]: New LCM state is SHUTDOWN
Thu Apr 20 16:10:35 2017 [Z0][VMM][I]: ExitCode: 0
Thu Apr 20 16:10:35 2017 [Z0][VMM][I]: Successfully execute virtualization driver operation: cancel.
Thu Apr 20 16:10:35 2017 [Z0][VMM][I]: ExitCode: 0
Thu Apr 20 16:10:35 2017 [Z0][VMM][I]: Successfully execute network driver operation: clean.
Thu Apr 20 16:10:35 2017 [Z0][VM][I]: New LCM state is EPILOG
Thu Apr 20 16:10:36 2017 [Z0][VM][I]: New state is DONE
Thu Apr 20 16:10:36 2017 [Z0][VM][I]: New LCM state is LCM_INIT

Some questions:

  • Anyone knows how to remove the stuck lease? (I was not able to do it with the CLI or the UI).
    If someone knows the DB table where I could remove the lease, it would be great.
  • Some stuff is referenced by name of the VNET. All the references should be changed to the ID of the VM, right?
    Example: You do some VM template with some NIC to a VNET, you change the VNET name, and the Template is not able to be instantiated, because the vnet is referenced by name instead of id. ( :confused: )
  • TIP: Do NOT change the name of the VNETs

the way to fix it is by using onedb fsck.

1 Like

Good to know! Thx!

Hello @jsfrerot @apoz @ahuertas ,
Just a very basic query, Is there any provision for lease period for VM in OpenNebula.
To elaborate: I want to create a VM for a certain time i.e. for x days so is there any lease period or expiry date feature for VM in OpenNebula.
Any help or documents related to the topic is much appreciated!
Thanks in Advance!

Hello @Rahul_Sharma,

You can set actions to terminate it after x days. You can set it in VM template or while instantiating VM.

Reference: Managing Virtual Machines Instances — OpenNebula 6.0.3 documentation

Hello @Nikhil_Kumar thanks for the response!
can we create a custom action or say VM state like terminate ,poweroff?

Hello @Rahul_Sharma ,

You can perform various actions as mentioned below,

I want to create action other than these. ie for showback or email every 5 days.