Node stuck with state MONITORING_INIT

Community Support
1

Before i get this state, i had MONITORING_ERROR state on second node from “onehost show 1” if i not mistaken(I forgot a little bit). Then i tried to enable the node with “onehost enable 1”. And then i get this init state with some details :

HOST 1 INFORMATION
ID                    : 1
NAME                  : opennebulanode2
CLUSTER               : default
STATE                 : MONITORING_INIT
IM_MAD                : kvm
VM_MAD                : kvm
LAST MONITORING TIME  : 01/18 16:58:25

HOST SHARES
RUNNING VMS           : 45
MEMORY
  TOTAL               : 78.6G
  TOTAL +/- RESERVED  : 157.1G
  USED (REAL)         : 32.1G
  USED (ALLOCATED)    : 95G
CPU
  TOTAL               : 3200
  TOTAL +/- RESERVED  : 6400
  USED (REAL)         : 32
  USED (ALLOCATED)    : 5100

LOCAL SYSTEM DATASTORE #0 CAPACITY
TOTAL:                : 360.7G
USED:                 : 297.9G
FREE:                 : 44.5G

MONITORING INFORMATION
ARCH="x86_64"
CPUSPEED="1294"
ERROR="Wed Jan 10 23:48:45 2018 : Error monitoring Host opennebulanode2 (1): error: failed to connect to the hypervisor
error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': No such file or directory
ERROR MESSAGE --8<------
Error executing kvm.rb
ERROR MESSAGE ------>8--
ARCH=x86_64
MODELNAME=\"Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz\"
"
HOSTNAME="opennebulanode2"
HYPERVISOR="kvm"
IM_MAD="kvm"
MODELNAME="Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz"
NETRX="8484511038"
NETTX="21123100924"
RESERVED_CPU="-3200"
RESERVED_MEM="-82371420"
VERSION="5.4.0"
VM_MAD="kvm"

On the opennebula state i got this error :

oneerror
oneerror.png948×161 9.25 KB

But, all vms inside the node are running fine. Is this something i should worry and start to fix? I searched for the similar problem on the internet. But, i can’t get enough information and still couldn’t fix the problem. Can you guys help me?

2

Hello,

I ran into a similar issue few days ago: Onehost sync temporary failure

All my hosts were in the INIT or UPDATE state. After pulling hair over this issue, I have rebooted the server where ONe daemons run, which I planned to do nevertheless because of Meltdown/Spectre updates. Since then, the host monitoring work as expected - the hosts are in the ON state. So maybe restarting all ONe daemons can help?

-Yenya

3

Can that issue be related with that one?

4

In my case, probably not. I have also ran into that issue a while ago, and now I have polkit disabled for libvirt the same way @vholer suggests.

5

Okay then, i will try to restart the system. but what’s with this error?

I got libvirt-sock up and running on the second host. Oneadmin is also on libvirtd group. So what exactly my problem is? Is it a bug on 5.4 version? Because i have opennebula 5.2 running fine on the other server.

6

All problem solved after restarting all node. But, then it comes up again after several days. How should i make the node on the “on” state?

7

Still facing the same issue in 2019, I am using open nebula 5.8.0. Restarted all the nodes as well as front end. Still stuck at init state and I dont want to lose all the vms.
No error monitored in /var/log/one/oned.log.
It simply shows
Mon May 27 11:12:00 2019 [Z0][InM][D]: Monitoring host node-4 192.168.2.90 (1)
Mon May 27 11:12:00 2019 [Z0][InM][D]: Monitoring host node-1 192.168.2.89 (3)

8

You can try to restart only libvirt, maybe it helps.

9

Upon restarting it I get:
Failed to restart libvirt.service: Unit libvirt.service not found.

Installed packages are:
libvirt-bin libvirt-guests

Still restarting them doesnt help much