Issue with creating vm's

Strange thing happened as nothing was changed in the environment just few days passed and suddenly errors while building vm’s on vmware.
The error that I’m getting is below. Any ideas on what is happening here ? Template worked fine and is actually build with all the right configs.

Wed Feb  1 09:48:42 2017 [Z0][VMM][D]: Message received: LOG I 271 Command execution fail: /var/lib/one/remotes/vmm/vcenter/deploy '/var/lib/one/vms/271/deployment.0' 'CloudCustomers' 271 CloudCustomers

Wed Feb  1 09:48:42 2017 [Z0][VMM][D]: Message received: LOG I 271 /usr/lib/one/ruby/vendors/rbvmomi/lib/rbvmomi/type_loader.rb:66: warning: already initialized constant RbVmomi::VIM::Datastore

Wed Feb  1 09:48:42 2017 [Z0][VMM][D]: Message received: LOG I 271 /usr/lib/one/ruby/vcenter_driver.rb:50: warning: previous definition of Datastore was here

Wed Feb  1 09:48:42 2017 [Z0][VMM][D]: Message received: LOG I 271 Deploy of VM 271 on host CloudCustomers with /var/lib/one/vms/271/deployment.0 failed due to "SystemError: A general system error occurred: Connection refused"

Wed Feb  1 09:48:42 2017 [Z0][VMM][D]: Message received: LOG I 271 ExitCode: 255

Wed Feb  1 09:48:42 2017 [Z0][VMM][D]: Message received: LOG I 271 Failed to execute virtualization driver operation: deploy.

Wed Feb  1 09:48:42 2017 [Z0][VMM][D]: Message received: DEPLOY FAILURE 271 -

Must have been something weird with vcenter. Rebooted vcenter server and machines are building now fine.

Strangely this issue creep up again. This time during terminating machines. Exactly same problem, but on different action. Any ideas on how to get more information on what is happening here ?

What info is nebula using to interact with vm’s in vcenter? Meaning, if I move vm’s to different data store does it keep reference of the datastore or just vm uuid and all other info is pulled dynamically?
THanks

Hi nicpon!
the “SystemError: A general system error occurred: Connection refused” is a message returned by vSphere’s API when OpenNebula tries to perform an action so it should be a problem with vCenter. As you may have already been googling, there are many blog posts out there for that “Connection refused” stating that the vCenter Workflow Service would not be running so you may try to check the status of the vmware-vpx-workflow service.

I haven’t found any information about why that service could not be running :confused: so I would start checking logs… maybe vSphere tells you why the service has been stopped or what else can be happening.

For your second question about moving the VM to a different datatore while running in the same ESX host (I presume), if you run a Migrate operation from vSphere the VM should be monitored and accesible from OpenNebula once the relocate operation ends, as the DEPLOY_ID (UUID) hasn’t changed while the VCENTER_DATASTORE attribute is used when deploying the VM to choose what datastore will host the cloned vm from the template.

Cheers!