Assistance with Managing vCenter Template/VMDK's - Unable to implement consistent Behaviors

I am trying to trying to fit our OS Image management process into our OpenNebula 5.8.1 Federated VMware vCenter POC environment, and seem to be having a difficult time reproducing our process making any process work at all… We haven’t even got onto with the process of replicating/importing image/templates into the Slave Nodes so we will focus on the Master…

1.) Import existing OS VMware Template into OpenNebula in our Master Node.

This step works fine. I can deploy instances from this OpenNebula Template all day long, and the behavior of the process works as expected. The VM spins up clones the template into the {ServerName} VM Folder along with it’s associated vmdk disk - one context runs and I have a working VM.

2.) Create modified/clone disk image from the originally imported Template. I’ve tried two different methods which I though might be possible solutions from the documentation - with no positive results.

2a. ) Start a new VM as persistent, run our onboarding configuration management scripts, poweroff and perform a ‘save as’ to create a new template. Then change it back to non-persistent so that it can be used to instantiate new images. The copy is then terminated.

This does create a new Template with the modification number in the name - and a new disk image with the new template name + -disk-0 suffix. However; instead of creating a new VM Folder with the instances name on the vCenter Datastore, This only stores the vmdk as /one/53/one-53.vmdk. Subsequently, when a new vm is instantiated - this behavior is mimicked and all new instantiated VM’s vmdk’s are placed within this same directory, but the rest of the vCenters VM files are placed within the folder with the VM’s name. Although this solution kind of works, the operation team is concerned about having to rewrite / modify existing operational processes. Is there a way to modify/change this behavior?

2b.) The second process I found was to create a VM Template in the vCenter to act as a seed VMWare Template - Import that Template and than modify the Associated OpenNebula Template generated by the Template Import Process to attach another Storage Image that contains the modified disk image. ( Side note - we were thinking this might be the best option for generating the updated image in the Marketplace and be able to use that vmdk image to export into the Slave Nodes controlling vCenters in other Datacenters) - But this process appears not to be working at all: Process Follows:

Create Skeleton/Seed VM and Convert to Template.

Import vCenter Template

CPU = “1”
DESCRIPTION = “vCenter Template imported by OpenNebula from Cluster LAB_HOST_CLSTR”
DISK = [
IMAGE_ID = “54”,
LISTEN = “”,
TYPE = “vnc” ]
HYPERVISOR = “vcenter”
MEMORY = “2048”
NAME = “cps-rhel7-seed-template”
NIC = [
VCENTER_CCR_REF = “domain-c137”
VCENTER_INSTANCE_ID = “7da4f44f-4092-4712-9496-beea7c4ba527”
VCPU = “1”

Instantiate a new VM with the attached the vmdk disk that has no OS installed:
System Boots to PXE Screen as anticipated as no OS is installed:


Everything is stored in proper directory on vCenter.


Now Lets attach a different vmdk to the OpenNebula Template: I will use the Image from step 2a.
It was image 54, changed template to use image 50

The Deployment Fails:

Wed May 1 12:54:02 2019 [Z0][VM][I]: New state is ACTIVE
Wed May 1 12:54:02 2019 [Z0][VM][I]: New LCM state is PROLOG
Wed May 1 12:54:02 2019 [Z0][VM][I]: New LCM state is BOOT
Wed May 1 12:54:02 2019 [Z0][VMM][I]: Generating deployment file: /var/lib/one/vms/130/deployment.0
Wed May 1 12:54:02 2019 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_context.
Wed May 1 12:54:02 2019 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Wed May 1 12:54:03 2019 [Z0][VMM][I]: Command execution failed (exit code: 255): /var/lib/one/remotes/vmm/vcenter/deploy '/var/lib/one/vms/130/deployment.0' 'LAB_HOST_CLSTR' 130 LAB_HOST_CLSTR
Wed May 1 12:54:03 2019 [Z0][VMM][E]: deploy: Deploy of VM 130 on vCenter cluster LAB_HOST_CLSTR with /var/lib/one/vms/130/deployment.0 failed due to "disk with path cps-build-rhel7/cps-build-rhel7.vmdk not found in the vCenter VM".
Wed May 1 12:54:03 2019 [Z0][VMM][I]: Failed to execute virtualization driver operation: deploy.
Wed May 1 12:54:03 2019 [Z0][VMM][E]: Error deploying virtual machine
Wed May 1 12:54:03 2019 [Z0][VM][I]: New LCM state is BOOT_FAILURE

Here is an Interesting Development

If I instantiate from the command line, this deployment does not fail. It in fact instantiates and successfully boots the VM - However, in this case the vmdk is stored in the root of the vmfs filesystem for the datastore, not in any folders. If I remember correctly if one were to deploy within the vcenter environment, there is an option that states ‘store vmdk with virtual machine’. Is there a way to specify the folder that contains the rest of the virutal machine files?

>  [oneadmin@clddvlsegonfs1 ~]$ onetemplate list
>   ID USER            GROUP           NAME                                REGTIME
>   48 oneadmin        oneadmin        cps-rhel7-seed-template      05/01 12:31:35
>   47 oneadmin        oneadmin        cps-build-rhel7-mod1         05/01 11:04:13
>   46 oneadmin        oneadmin        cps-build-rhel7-copy         05/01 10:48:30
>   44 oneadmin        oneadmin        cps-build-rhel7              04/30 12:44:03
>   39 oneadmin        oneadmin        kak-vcenter-clone-small-rhe  04/25 16:35:54
>   38 oneadmin        oneadmin        kak-vcenter-small-rhel7      04/25 16:13:36
>   31 oneadmin        oneadmin        export-mkit-newimage5-templ  04/23 19:05:03
>   22 oneadmin        oneadmin        opennebula-rhel-7.6-1902     04/22 17:53:13
>   13 oneadmin        oneadmin        one-rhel7-small-cps-start    04/22 15:03:15
>   11 oneadmin        oneadmin        small-rhel7                  04/17 20:18:38
>    4 oneadmin        oneadmin        opennebula-win-2016-1902.1   04/11 18:03:05
> [oneadmin@clddvlsegonfs1 ~]$ oneimage list
>   54 oneadmin   oneadmin   cps-rhel7-seed- CLF_SEG_LA      5G OS    No rdy     0
>   53 oneadmin   oneadmin   cps-build-rhel7 CLF_SEG_LA     50G OS    No rdy     0
>   52 oneadmin   oneadmin   cps-build-rhel7 CLF_SEG_LA     50G OS   Yes rdy     0
>   50 oneadmin   oneadmin   cps-build-rhel7 CLF_SEG_LA      5G OS    No rdy     0
>   45 oneadmin   oneadmin   kak-mkt-small-r CLF_SEG_LA     10G OS    No used    2
>   44 oneadmin   oneadmin   kak-vcenter-sma CLF_SEG_LA     10G OS    No rdy     0
>   38 oneadmin   oneadmin   export-mkit-new CLF_SEG_LA     10G OS    No rdy     0
>   33 oneadmin   oneadmin   export-mkit-new CLF_SEG_LA     10G OS    No rdy     0
>   25 oneadmin   oneadmin   opennebula-rhel CLF_SEG_LA     50G OS    No used    1
>   16 oneadmin   oneadmin   one-rhel7-small CLF_SEG_LA     10G OS    No rdy     0
>   12 oneadmin   oneadmin   small-rhel7     CLF_SEG_LA     10G OS    No rdy     0
>    8 oneadmin   oneadmin   opennebula-win- CLF_SEG_LA    100G OS    No used    1
> [oneadmin@clddvlsegonfs1 ~]$ onetemplate instantiate 48 --nic vDS_2905_10.26.48.0_24:IP= --disk 25  --name cps-rhel-seed-image25
> VM ID: 131

Expected results:

The Documentation is really fragmented on what this process is supposed to look like, I have been looking at vOneCloud documentation as well as the Fullstack. If someone can point me to some Support Queries, with working examples that would be fantastic. Or provide assistance - It would be greatly appreciated, there are two of us looking at this and have been stuck for a couple days now.