Cannot schedule VM, there is no suitable host

Community Support
1

Hi all,

Thank you for looking at my request. I’m trying to understand why creating a VM from a Marketplace image is not working. My setup is I have a VMware cluster with VCenter 5.5 running. I have installed Opennebula 4.14 from source and I have imported templates, VMs, networks and such without a problem. I’m able to deploy VMs with templates imported from VCenter. I have created VMFS datastores for system and images. Downloading images to the images datastore works as expected. But when I try to create a VM from a template/image from the Marketplace I get the following message in sched.log

Fri Oct 2 23:32:35 2015 [Z0][VM][D]: Found 1 pending/rescheduling VMs.
Fri Oct 2 23:32:35 2015 [Z0][HOST][D]: Discovered 1 enabled hosts.
Fri Oct 2 23:32:35 2015 [Z0][SCHED][D]: Match-making results for VM 11:
Cannot schedule VM, there is no suitable host.

Fri Oct 2 23:32:35 2015 [Z0][SCHED][D]: Dispatching VMs to hosts:
VMID Host System DS

Am I missing something? Am I suppose to associate a cluster to the datastores in order for this to work? Below is what it shows when running onedatastore commands:

[oneadmin@opennebula414 one]$ onedatastore show 0
DATASTORE 0 INFORMATION
ID : 0
NAME : system
USER : oneadmin
GROUP : oneadmin
CLUSTER : -
TYPE : SYSTEM
DS_MAD : -
TM_MAD : vmfs
BASE PATH : /vmfs/volumes/0
DISK_TYPE : FILE
STATE : READY

DATASTORE CAPACITY
TOTAL: : 0M
FREE: : 0M
USED: : 0M
LIMIT: : -

PERMISSIONS
OWNER : um-
GROUP : u–
OTHER : —

DATASTORE TEMPLATE
BASE_PATH="/vmfs/volumes/"
SHARED="YES"
TM_MAD="vmfs"
TYPE=“SYSTEM_DS”

IMAGES
[oneadmin@opennebula414 one]$ onedatastore show 1
DATASTORE 1 INFORMATION
ID : 1
NAME : default
USER : oneadmin
GROUP : oneadmin
CLUSTER : -
TYPE : IMAGE
DS_MAD : vmfs
TM_MAD : vmfs
BASE PATH : /vmfs/volumes/1
DISK_TYPE : FILE
STATE : READY

DATASTORE CAPACITY
TOTAL: : 2T
FREE: : 2T
USED: : 884M
LIMIT: : -

PERMISSIONS
OWNER : um-
GROUP : u–
OTHER : —

DATASTORE TEMPLATE
BASE_PATH="/vmfs/volumes/"
BRIDGE_LIST="172.27.17.12 172.27.17.13 172.27.17.14 172.27.17.15"
CLONE_TARGET="SYSTEM"
DISK_TYPE="FILE"
DS_MAD="vmfs"
LN_TARGET="NONE"
TM_MAD="vmfs"
TYPE=“IMAGE_DS”

IMAGES
1
3

What is odd is that unlike the images datastore, the system datastore shows no stats, yet it is a 2TB VMFS volume. I also have plenty of resources available on the cluster.

I’m so close to getting this working, but can’t understand what I’m missing. If someone can point me in the right direction I would greatly appreciated.

Thanks,
Jesus

2

I checked the sources and I think the above log line is generated by file src/scheduler/src/sched/Scheduler.cc line:859
If you look above this line thre are bunch of tests that IMO are logging rejection reasons to VM’s log. So check the VM log for additional hints. I assume the logfile name in your case is /var/log/one/11.log.

I hope this will help,
Anton Todorov

3

hi, by looking at the datastore value:
CLUSTER : -

You probably need to add the datastore and host to the cluster. You are deploying the VM to a cluster, opennebula then goes looking for suitables hosts, finds one, but it isnt available to the cluster.
The datastore probably has the same issue, because it’s not assigned to a cluster yet.

Like Anton said, check the steps in the number-of-vm.log in /var/log/one/XX.log, that should show you the exact problem.

hope this helps

4

Hi all,

Thanks for the reply. Some of the thoughts have helped, but I’m still struggling. I have done this and now I have assigned my host to cluster as well as the datastores, but still no go. I figured out with your replies to look at the logs and when the system was saying that there was no suitable hosts, I checked the requirements on the VM deployment and it states the following:

AUTOMATIC_REQUIREMENTS CLUSTER_ID = 100 & !(PUBLIC_CLOUD = YES)

Looking at my vCenter host, the attribute PUBLIC_CLOUD was set to YES. Changing this finally made the system assign the vCenter host and try to deploy, yet now I have another issue:

Sun Oct 4 02:01:10 2015 [Z0][VM][I]: New state is ACTIVE
Sun Oct 4 02:01:10 2015 [Z0][VM][I]: New LCM state is PROLOG
Sun Oct 4 02:01:10 2015 [Z0][VM][I]: New LCM state is BOOT
Sun Oct 4 02:01:10 2015 [Z0][VMM][I]: Generating deployment file: /var/lib/one/vms/2/deployment.0
Sun Oct 4 02:01:10 2015 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Sun Oct 4 02:01:10 2015 [Z0][VMM][I]: Command execution fail: /var/lib/one/remotes/vmm/vcenter/deploy ‘/var/lib/one/vms/2/deployment.0’ ‘SJCCLUSTER’ 2 SJCCLUSTER
Sun Oct 4 02:01:10 2015 [Z0][VMM][I]: Deploy of VM 2 on host SJCCLUSTER with /var/lib/one/vms/2/deployment.0 failed due to "Cannot find VM_TEMPLATE in VCenter element."
Sun Oct 4 02:01:10 2015 [Z0][VMM][I]: ExitCode: 255
Sun Oct 4 02:01:10 2015 [Z0][VMM][I]: Failed to execute virtualization driver operation: deploy.
Sun Oct 4 02:01:10 2015 [Z0][VMM][E]: Error deploying virtual machine
Sun Oct 4 02:01:10 2015 [Z0][VM][I]: New LCM state is BOOT_FAILURE

Ok so it cannot find vCenter element. So I modify the downloaded marketplace ttylinux template to state that this is a vCenter template, but since this was downloaded, it does not have a template id which seems to be required because even when I set the Hypervisor to vCenter the log still states the same error.

So my question is, only vCenter templates that have been imported by adding the vCenter host can be deployed from OpenNebula? It seems like the concept of the datastores (system/images/files) for a vCenter centric environment do not apply?

Thanks,
Jesus

5

Hi Jesús,

There seems to be a slight confusion in the functionality of each driver. In OpenNebula there are two set of drivers:

  • VMware / ESX drivers. They interact directly with ESX hypervisors, and manage storage and network resources. These drivers will be deprecated in future versions of OpenNebula.

  • vCenter drivers. Interact directly with vCenter, and delegates the management of storage and network resources to vCenter, consuming them from OpenNebula. Functionality involving advance management of this resources (like the Marketplace, that needs access to the storage Datastores) are not supported. vCenter drivers uses VM Templates defined directly in vCenter, which already aggregates capacity, network and storage resources. These drivers will be enhanced in future versions of OpenNebula.

Hope this clarifies the problem.

6

Hi Tino,

This makes sense now. I thought that I needed to setup these datastores for getting marketplace images to function, but based on what you saying this is not possible.

Anyhow, thanks for clarifying this for me as I was confused as to what needed to be done. Now I understand to only use the features within vCenter to deploy VMs. If I use a different Hypervisor like straight up ESXi 5.5 and below and or Xen then the setup of these datastores apply.

Much appreciated,
Jesus

7

Hello!

I am having a similar problem, where I get the “Cannot schedule VM, there is no suitable host” error.

I am trying to start a new VM, but it doesn’t get deployed on the host. Can you please help me figure out what is happening?
I’ve done some reading and debugging, but all I can see in the logs is the following debug info:

Fri Nov  4 12:26:17 2016 [Z0][VM][D]: Found 1 pending/rescheduling VMs.
Fri Nov  4 12:26:17 2016 [Z0][HOST][D]: Discovered 1 enabled hosts.
Fri Nov  4 12:26:17 2016 [Z0][SCHED][D]: Match-making results for VM 39:
        Cannot schedule VM, there is no suitable host.

Fri Nov  4 12:26:17 2016 [Z0][SCHED][D]: Dispatching VMs to hosts:
        VMID    Host    System DS
        -------------------------

I already checked the VM’s log, but it is empty:

-rw-r--r-- 1 oneadmin oneadmin 0 Nov  4 12:23 /var/log/one/39.log

My setup consists of a single machine that has both the frontend and a KVM backend.

Resources are available on the host:

[root] # onehost list
  ID NAME            CLUSTER   RVM      ALLOCATED_CPU      ALLOCATED_MEM STAT  
   0 siphub          default     7    534 / 800 (66%)   7G / 31.4G (22%) on  

[root] # onedatastore list
  ID NAME                SIZE AVAIL CLUSTERS     IMAGES TYPE DS      TM      STAT
   0 system                 - -     0                 0 sys  -       ssh     on  
   1 default             1.8T 87%   0                 7 img  fs      ssh     on  
   2 files               1.8T 87%   0                 1 fil  fs      ssh     on

Can you please help me debug this? Let me know if there is anything else I can provide you.

Thanks,
Răzvan Crainea

8

Actually, I figured out what was the problem - I was requesting 512 GB of memory (instead of 512 MB). I noticed that when I increased the scheduler’s debugging level to 5.

So I guess now the question is why am I not getting the logs into the VM’s log file?

Do you have any thoughts?

Thanks,
Răzvan

9

Glad you solved it.
The scheduler decides where a VM will be deployed, so I guess you won’t see any info on the VM log, as that log will be filled with info as soon as the VM is deployed.

In your case it wasn’t possible to deploy as not enough resources were found by the scheduler so the VM log wouldn’t be receiving new data though I may be wrong.

Cheers!

1 Like
10

I got same error after upgrade from 5.2 to 5.4

[oneadmin@nebula52-dev one]$ onehost list
ID NAME CLUSTER RVM ALLOCATED_CPU ALLOCATED_MEM STAT
0 CloudCustomers default 0 0 / 1601 (0%) 0K / 288G (0%) on

[oneadmin@nebula52-dev one]$ onedatastore list
ID NAME SIZE AVAIL CLUSTERS IMAGES TYPE DS TM STAT
0 system - - - 0 sys - ssh on
1 default 35.6G 93% - 1 img fs ssh on
2 files 35.6G 93% 0 0 fil fs ssh on
100 OpenStackNova 465G 100% - 0 img vcenter vcenter on
102 Sas1 465G 100% 0 1 img vcenter vcenter on
103 Sas2 465G 100% - 0 img vcenter vcenter on
107 OpenStackNova 465G 100% - 0 sys - vcenter on
108 Sas1 (SYS) 465G 100% - 0 sys - vcenter on
109 Sas2 (SYS) 465G 100% - 0 sys - vcenter on
110 datastore1 (3 457.8G 48% - 1 img vcenter vcenter on
111 datastore1 (3 0M - - 0 sys - vcenter on

from my sched.log I get this:

Thu Oct 19 15:36:34 2017 [Z0][HOST][D]: Discovered 1 enabled hosts.
Thu Oct 19 15:36:34 2017 [Z0][VM][D]: VMs in VMGroups:

Thu Oct 19 15:36:34 2017 [Z0][SCHED][D]: Dispatching VMs to hosts:
VMID Priority Host System DS
--------------------------------------------------------------

Could you point me where to look for issues? I think I have all cluster & host assigned properly.

As @jamdev12 resolved his problem with changing PUBLIC_CLOUD to NO. I did that too, but it didn’t work for me. My VM script has a different line instead of AUTOMATIC_REQUIREMENTS CLUSTER_ID = 100 & !(PUBLIC_CLOUD = YES). I only have AUTOMATIC_REQUIREMENTS = “!(PUBLIC_CLOUD = YES)”. What is this setting anyway ?

11

I managed to move something along, but now I’m getting this error

Thu Oct 19 16:59:18 2017 [Z0][VMM][D]: Message received: LOG I 377 ExitCode: 255

Thu Oct 19 16:59:18 2017 [Z0][VMM][D]: Message received: LOG I 377 Failed to execute virtualization driver operation: deploy.

Thu Oct 19 16:59:18 2017 [Z0][VMM][D]: Message received: DEPLOY FAILURE 377 Deploy of VM 377 on vCenter cluster CloudCustomers with /var/lib/one/vms/377/deployment.0 failed due to “ManagedObjectNotFound: The object has already been deleted or has not been completely created”

Which is even more cryptic than the other… :smiley:

12

I reimported the template and machine is building now. I guess something during upgrade didn’t quite workout.