Overprovisioning factor is applied twice to new hosts

daveparker · March 13, 2019, 10:37pm

For our staging cluster we are overprovisioning host resources with RESERVED_CPU="-50%" and RESERVED_MEM="-25% on the cluster template. In this case the overcommit values are only applied on the cluster template, the host template RESERVED_CPU and RESERVED_MEM are blank. When adding a host with 128GB RAM and 32 cores to the cluster I would expect the reported memory capacity to be 160GB (128 * 1.25) and reported CPU capacity to be 4800 (32 * 1.5 * 100).

What actually happens is that the “Overcommitment” section in Sunstone displays the expected values, but the capacity section shows that the memory and CPU overcommitment factors have been applied twice, so memory is reported as 196GB ( 128 * 1.25 * 1.25) and CPU is 7200 (32 * 1.5 * 1.5 * 100) (see attached).

ruben · March 14, 2019, 10:23am

Hi Dave,

I want to check if this is more a cosmetic issue or not. Could you please send the output of onehost show -x 51

Cheers

daveparker · March 14, 2019, 4:50pm

Hi Ruben,

I had to use that host so I provisioned a new one with 24 cores and 64GB RAM. Looking at the XML per your suggestion it looks like it is just a cosmetic issue with sunstone, as the values for MAX_MEM = 78.4GB (62.7 * 1.25) and MAX_CPU = 3600 (2400 * 1.5).

Full XML from onehost show -x 53 below:

<HOST>
  <ID>53</ID>
  <NAME>testhost</NAME>
  <STATE>2</STATE>
  <IM_MAD><![CDATA[kvm]]></IM_MAD>
  <VM_MAD><![CDATA[kvm]]></VM_MAD>
  <LAST_MON_TIME>1552581540</LAST_MON_TIME>
  <CLUSTER_ID>0</CLUSTER_ID>
  <CLUSTER>default</CLUSTER>
  <HOST_SHARE>
    <DISK_USAGE>0</DISK_USAGE>
    <MEM_USAGE>0</MEM_USAGE>
    <CPU_USAGE>0</CPU_USAGE>
    <TOTAL_MEM>65794420</TOTAL_MEM>
    <TOTAL_CPU>2400</TOTAL_CPU>
    <MAX_DISK>474625</MAX_DISK>
    <MAX_MEM>82243025</MAX_MEM>
    <MAX_CPU>3600</MAX_CPU>
    <FREE_DISK>472892</FREE_DISK>
    <FREE_MEM>65096388</FREE_MEM>
    <FREE_CPU>2400</FREE_CPU>
    <USED_DISK>1733</USED_DISK>
    <USED_MEM>698032</USED_MEM>
    <USED_CPU>0</USED_CPU>
    <RUNNING_VMS>0</RUNNING_VMS>
    <DATASTORES>
      <DS>
        <FREE_MB><![CDATA[472892]]></FREE_MB>
        <ID><![CDATA[0]]></ID>
        <TOTAL_MB><![CDATA[474625]]></TOTAL_MB>
        <USED_MB><![CDATA[1733]]></USED_MB>
      </DS>
    </DATASTORES>
    <PCI_DEVICES/>
  </HOST_SHARE>
  <VMS/>
  <TEMPLATE>
    <ARCH><![CDATA[x86_64]]></ARCH>
    <CPUSPEED><![CDATA[1600]]></CPUSPEED>
    <HOSTNAME><![CDATA[testhost]></HOSTNAME>
    <HYPERVISOR><![CDATA[kvm]]></HYPERVISOR>
    <IM_MAD><![CDATA[kvm]]></IM_MAD>
    <KVM_CPU_MODELS><![CDATA[486 pentium pentium2 pentium3 pentiumpro coreduo n270 core2duo qemu32 kvm32 cpu64-rhel5 cpu64-rhel6 kvm64 qemu64 Conroe Penryn Nehalem Nehalem-IBRS Westmere Westmere-IBRS SandyBridge SandyBridge-IBRS IvyBridge IvyBridge-IBRS Haswell-noTSX Haswell-noTSX-IBRS Haswell Haswell-IBRS Broadwell-noTSX Broadwell-noTSX-IBRS Broadwell Broadwell-IBRS Skylake-Client Skylake-Client-IBRS Skylake-Server Skylake-Server-IBRS athlon phenom Opteron_G1 Opteron_G2 Opteron_G3 Opteron_G4 Opteron_G5 EPYC EPYC-IBPB]]></KVM_CPU_MODELS>
    <KVM_MACHINES><![CDATA[pc-i440fx-rhel7.0.0 pc rhel6.0.0 rhel6.1.0 rhel6.2.0 rhel6.3.0 rhel6.4.0 rhel6.5.0 rhel6.6.0]]></KVM_MACHINES>
    <MODELNAME><![CDATA[Intel(R) Xeon(R) CPU           E5645  @ 2.40GHz]]></MODELNAME>
    <NETRX><![CDATA[308768663]]></NETRX>
    <NETTX><![CDATA[7598521]]></NETTX>
    <RESERVED_CPU><![CDATA[]]></RESERVED_CPU>
    <RESERVED_MEM><![CDATA[]]></RESERVED_MEM>
    <VERSION><![CDATA[5.6.1]]></VERSION>
    <VM_MAD><![CDATA[kvm]]></VM_MAD>
  </TEMPLATE>
</HOST>

Thanks for your help!

daveparker · June 16, 2019, 7:04pm

Hi @ruben,

This behavior is being exhibited again, this time in a different cluster where we are moving from 0% overcommit to 5% reserved. For example a host with CPU=2400 and a cluster RESERVED_CPU=5% should be 2280 (2400*.95), however sunstone is reporting that host as CPU=2166(2400*.95*.95) in the “Capacity” section of that host.

The issue is still cosmetic, as is verified by host xml in onehost show -x $HOSTID, however sunstone is the primary interaction point with ONE and sunstone is showing incorrect information. We have version 5.6.1 deployed, were you able to pinpoint what is causing this issue and do you know if this was addressed in a newer version?

Thanks,
Dave

tinova · June 17, 2019, 9:54am

Hi,

Can you send us the output of onehost show -x in order to reproduce the problem?

daveparker · June 18, 2019, 5:06am

Sure.
Here is the output of onehost show -x 0 when RESERVED_CPU=0 is set on the cluster. At this point Sunstone shows an allocated CPU value of 2100/2400 on the host.

onehost show -x 0
<HOST>
  <ID>0</ID>
  <NAME>host1</NAME>
  <STATE>2</STATE>
  <IM_MAD><![CDATA[kvm]]></IM_MAD>
  <VM_MAD><![CDATA[kvm]]></VM_MAD>
  <LAST_MON_TIME>1560833497</LAST_MON_TIME>
  <CLUSTER_ID>0</CLUSTER_ID>
  <CLUSTER>default</CLUSTER>
  <HOST_SHARE>
    <DISK_USAGE>0</DISK_USAGE>
    <MEM_USAGE>127926272</MEM_USAGE>
    <CPU_USAGE>2100</CPU_USAGE>
    <TOTAL_MEM>197789696</TOTAL_MEM>
    <TOTAL_CPU>2400</TOTAL_CPU>
    <MAX_DISK>1828435</MAX_DISK>
    <MAX_MEM>197761433</MAX_MEM>
    <MAX_CPU>2400</MAX_CPU>
    <FREE_DISK>1710027</FREE_DISK>
    <FREE_MEM>135249152</FREE_MEM>
    <FREE_CPU>2376</FREE_CPU>
    <USED_DISK>118409</USED_DISK>
    <USED_MEM>62540552</USED_MEM>
    <USED_CPU>24</USED_CPU>
    <RUNNING_VMS>5</RUNNING_VMS>
    <DATASTORES>
      <DS>
        <FREE_MB><![CDATA[1710027]]></FREE_MB>
        <ID><![CDATA[0]]></ID>
        <TOTAL_MB><![CDATA[1828435]]></TOTAL_MB>
        <USED_MB><![CDATA[118409]]></USED_MB>
      </DS>
    </DATASTORES>
    <PCI_DEVICES/>
  </HOST_SHARE>
  <VMS>
    <ID>2</ID>
    <ID>16</ID>
    <ID>22</ID>
    <ID>56</ID>
    <ID>67</ID>
  </VMS>
  <TEMPLATE>
    <ARCH><![CDATA[x86_64]]></ARCH>
    <CPUSPEED><![CDATA[2400]]></CPUSPEED>
    <DISKTYPE><![CDATA[SSD]]></DISKTYPE>
    <GEN><![CDATA[9]]></GEN>
    <HOSTNAME><![CDATA[host1]]></HOSTNAME>
        <HYPERVISOR><![CDATA[kvm]]></HYPERVISOR>
    <IM_MAD><![CDATA[kvm]]></IM_MAD>
    <KVM_CPU_MODELS><![CDATA[486 pentium pentium2 pentium3 pentiumpro coreduo n270 core2duo qemu32 kvm32 cpu64-rhel5 cpu64-rhel6 kvm64 qemu64 Conroe Penryn Nehalem Nehalem-IBRS Westmere Westmere-IBRS SandyBridge SandyBridge-IBRS IvyBridge
 IvyBridge-IBRS Haswell-noTSX Haswell-noTSX-IBRS Haswell Haswell-IBRS Broadwell-noTSX Broadwell-noTSX-IBRS Broadwell Broadwell-IBRS Skylake-Client Skylake-Client-IBRS Skylake-Server Skylake-Server-IBRS athlon phenom Opteron_G1 Opteron_G2
Opteron_G3 Opteron_G4 Opteron_G5 EPYC EPYC-IBPB]]></KVM_CPU_MODELS>
    <KVM_MACHINES><![CDATA[pc-i440fx-rhel7.0.0 pc rhel6.0.0 rhel6.1.0 rhel6.2.0 rhel6.3.0 rhel6.4.0 rhel6.5.0 rhel6.6.0]]></KVM_MACHINES>
    <LABELS><![CDATA[ssd]]></LABELS>
    <MANUFACTURER><![CDATA[HPE]]></MANUFACTURER>
    <MODEL><![CDATA[DL360]]></MODEL>
    <MODELNAME><![CDATA[Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz]]></MODELNAME>
    <NETRX><![CDATA[903553003273]]></NETRX>
    <NETTX><![CDATA[851226511928]]></NETTX>
    <RESERVED_CPU><![CDATA[]]></RESERVED_CPU>
    <RESERVED_MEM><![CDATA[28263]]></RESERVED_MEM>
    <VERSION><![CDATA[5.6.1]]></VERSION>
    <VM_MAD><![CDATA[kvm]]></VM_MAD>
  </TEMPLATE>
</HOST>

And this is the output of onehost show -x 0 when RESERVED_CPU=5% is set on the cluster. At this point Sunstone shows an Allocated CPU value of 2100/2166 on the host (where the max value should be 2280).

<HOST>
  <ID>0</ID>
  <NAME>host1</NAME>
  <STATE>2</STATE>
  <IM_MAD><![CDATA[kvm]]></IM_MAD>
  <VM_MAD><![CDATA[kvm]]></VM_MAD>
  <LAST_MON_TIME>1560833981</LAST_MON_TIME>
  <CLUSTER_ID>0</CLUSTER_ID>
  <CLUSTER>default</CLUSTER>
  <HOST_SHARE>
    <DISK_USAGE>0</DISK_USAGE>
    <MEM_USAGE>127926272</MEM_USAGE>
    <CPU_USAGE>2100</CPU_USAGE>
    <TOTAL_MEM>197789696</TOTAL_MEM>
    <TOTAL_CPU>2400</TOTAL_CPU>
    <MAX_DISK>1828435</MAX_DISK>
    <MAX_MEM>197761433</MAX_MEM>
    <MAX_CPU>2280</MAX_CPU>
    <FREE_DISK>1710010</FREE_DISK>
    <FREE_MEM>135242016</FREE_MEM>
    <FREE_CPU>2376</FREE_CPU>
    <USED_DISK>118426</USED_DISK>
    <USED_MEM>62547676</USED_MEM>
    <USED_CPU>24</USED_CPU>
    <RUNNING_VMS>5</RUNNING_VMS>
    <DATASTORES>
      <DS>
        <FREE_MB><![CDATA[1710010]]></FREE_MB>
        <ID><![CDATA[0]]></ID>
        <TOTAL_MB><![CDATA[1828435]]></TOTAL_MB>
        <USED_MB><![CDATA[118426]]></USED_MB>
      </DS>
    </DATASTORES>
    <PCI_DEVICES/>
  </HOST_SHARE>
  <VMS>
    <ID>2</ID>
    <ID>16</ID>
    <ID>22</ID>
    <ID>56</ID>
    <ID>67</ID>
  </VMS>
  <TEMPLATE>
    <ARCH><![CDATA[x86_64]]></ARCH>
    <CPUSPEED><![CDATA[2400]]></CPUSPEED>
    <DISKTYPE><![CDATA[SSD]]></DISKTYPE>
    <GEN><![CDATA[9]]></GEN>
    <HOSTNAME><![CDATA[host1]]></HOSTNAME>
    <HYPERVISOR><![CDATA[kvm]]></HYPERVISOR>
    <IM_MAD><![CDATA[kvm]]></IM_MAD>
    <KVM_CPU_MODELS><![CDATA[486 pentium pentium2 pentium3 pentiumpro coreduo n270 core2duo qemu32 kvm32 cpu64-rhel5 cpu64-rhel6 kvm64 qemu64 Conroe Penryn Nehalem Nehalem-IBRS Westmere Westmere-IBRS SandyBridge SandyBridge-IBRS IvyBridge
 IvyBridge-IBRS Haswell-noTSX Haswell-noTSX-IBRS Haswell Haswell-IBRS Broadwell-noTSX Broadwell-noTSX-IBRS Broadwell Broadwell-IBRS Skylake-Client Skylake-Client-IBRS Skylake-Server Skylake-Server-IBRS athlon phenom Opteron_G1 Opteron_G2
Opteron_G3 Opteron_G4 Opteron_G5 EPYC EPYC-IBPB]]></KVM_CPU_MODELS>
    <KVM_MACHINES><![CDATA[pc-i440fx-rhel7.0.0 pc rhel6.0.0 rhel6.1.0 rhel6.2.0 rhel6.3.0 rhel6.4.0 rhel6.5.0 rhel6.6.0]]></KVM_MACHINES>
    <LABELS><![CDATA[ssd]]></LABELS>
    <MANUFACTURER><![CDATA[HPE]]></MANUFACTURER>
    <MODEL><![CDATA[DL360]]></MODEL>
    <MODELNAME><![CDATA[Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz]]></MODELNAME>
    <NETRX><![CDATA[903770386307]]></NETRX>
    <NETTX><![CDATA[851411233610]]></NETTX>
    <RESERVED_CPU><![CDATA[]]></RESERVED_CPU>
    <RESERVED_MEM><![CDATA[28263]]></RESERVED_MEM>
    <VERSION><![CDATA[5.6.1]]></VERSION>
    <VM_MAD><![CDATA[kvm]]></VM_MAD>
  </TEMPLATE>
</HOST>

Thanks,
Dave

Topic		Replies	Views
CPU overcommit problem vCenter	14	1475	January 7, 2021
Host monitoring reporting incorrect allocation Community Support	2	2426	November 25, 2015
[VirtualMachineResize] host 0 does not have enough capacity Community Support	3	1264	April 25, 2016
Questions about RESERVED_MEM and RESERVED_CPU (Opennebula 4.8.0) Community Support	5	1562	February 20, 2015
Overcommit and XML-RPC API Development	3	613	January 15, 2018

Overprovisioning factor is applied twice to new hosts

Related topics