Can not instant VM

keiji · January 26, 2018, 12:35pm

Please, describe the problem here and provide additional information below (if applicable) …

We built the front end and the node according to this procedure.

https://docs.opennebula.org/5.4/deployment/opennebula_installation/frontend_installation.html

then I downloaded the image from APPS.
and launched it.

I think that the key exchange is in the correct state.

Please tell me how to do the right thing.
Thank you

Versions of the related components and OS (frontend, hypervisors, VMs):
Ceotos7
opennebula5,4,6

Steps to reproduce:
onetemplate instantiate centos7kvm

Current results:
can not make VM

Expected results:
VM will be running

Fri Jan 26 21:20:16 2018 [Z0][VM][I]: New state is ACTIVE
Fri Jan 26 21:20:16 2018 [Z0][VM][I]: New LCM state is PROLOG
Fri Jan 26 21:20:17 2018 [Z0][TM][I]: Command execution fail: /var/lib/one/remotes/tm/ssh/clone test5:/var/lib/one//datastores/1/a2cbe1fb1d14b320bd1da81db7bfdf57 test6:/var/lib/one//datastores/0/23/disk.0 23 1
Fri Jan 26 21:20:17 2018 [Z0][TM][I]: clone: Cloning test5:/var/lib/one//datastores/1/a2cbe1fb1d14b320bd1da81db7bfdf57 in /var/lib/one/datastores/0/23/disk.0
Fri Jan 26 21:20:17 2018 [Z0][TM][E]: clone: Command “scp -r test5:/var/lib/one//datastores/1/a2cbe1fb1d14b320bd1da81db7bfdf57 test6:/var/lib/one//datastores/0/23/disk.0” failed: Host key verification failed.
Fri Jan 26 21:20:17 2018 [Z0][TM][E]: Error copying test5:/var/lib/one//datastores/1/a2cbe1fb1d14b320bd1da81db7bfdf57 to test6:/var/lib/one//datastores/0/23/disk.0
Fri Jan 26 21:20:17 2018 [Z0][TM][I]: ExitCode: 1
Fri Jan 26 21:20:17 2018 [Z0][TM][E]: Error executing image transfer script: Error copying test5:/var/lib/one//datastores/1/a2cbe1fb1d14b320bd1da81db7bfdf57 to test6:/var/lib/one//datastores/0/23/disk.0
Fri Jan 26 21:20:17 2018 [Z0][VM][I]: New LCM state is PROLOG_FAILURE

marcindulak · January 27, 2018, 4:26pm

This post contains the steps used to debug ssh connection problems https://forum.opennebula.io/t/error-executing-image-transfer-script-error-copying
Try also to read other posts that contain the errors above.

keiji · January 29, 2018, 1:59am

Thank you for answer. The problem was solved, but another issue came up.
please help me.

log is

Mon Jan 29 10:56:29 2018 [Z0][VM][I]: New state is ACTIVE
Mon Jan 29 10:56:29 2018 [Z0][VM][I]: New LCM state is PROLOG
Mon Jan 29 10:56:35 2018 [Z0][VM][I]: New LCM state is BOOT
Mon Jan 29 10:56:35 2018 [Z0][VMM][I]: Generating deployment file: /var/lib/one/vms/27/deployment.0
Mon Jan 29 10:56:36 2018 [Z0][VMM][I]: Successfully execute transfer manager driver operation: tm_context.
Mon Jan 29 10:56:36 2018 [Z0][VMM][I]: ExitCode: 0
Mon Jan 29 10:56:36 2018 [Z0][VMM][I]: Successfully execute network driver operation: pre.
Mon Jan 29 10:56:36 2018 [Z0][VMM][I]: Command execution fail: cat << EOT | /var/tmp/one/vmm/kvm/deploy ‘/var/lib/one//datastores/0/27/deployment.0’ ‘test6’ 27 test6
Mon Jan 29 10:56:36 2018 [Z0][VMM][I]: error: failed to connect to the hypervisor
Mon Jan 29 10:56:36 2018 [Z0][VMM][I]: error: unable to connect to server at ‘localhost:16509’: Connection refused
Mon Jan 29 10:56:36 2018 [Z0][VMM][E]: Could not create domain from /var/lib/one//datastores/0/27/deployment.0
Mon Jan 29 10:56:36 2018 [Z0][VMM][I]: ExitCode: 255
Mon Jan 29 10:56:36 2018 [Z0][VMM][I]: Failed to execute virtualization driver operation: deploy.
Mon Jan 29 10:56:36 2018 [Z0][VMM][E]: Error deploying virtual machine: Could not create domain from /var/lib/one//datastores/0/27/deployment.0
Mon Jan 29 10:56:36 2018 [Z0][VM][I]: New LCM state is BOOT_FAILURE

keiji · January 30, 2018, 2:32am

and this is oned.log. somebody help me! please

Tue Jan 30 11:31:10 2018 [Z0][ReM][D]: Req:1776 UID:0 one.zone.raftstatus invoked
Tue Jan 30 11:31:10 2018 [Z0][ReM][D]: Req:1776 UID:0 one.zone.raftstatus result SUCCESS, "-1<…"
Tue Jan 30 11:31:10 2018 [Z0][ReM][D]: Req:5280 UID:0 one.vmpool.info invoked , -2, -1, -1, -1
Tue Jan 30 11:31:10 2018 [Z0][ReM][D]: Req:5280 UID:0 one.vmpool.info result SUCCESS, "<VM_POOL>
37<…"
Tue Jan 30 11:31:10 2018 [Z0][ReM][D]: Req:7840 UID:0 one.vmpool.info invoked , -2, -1, -1, -1
Tue Jan 30 11:31:10 2018 [Z0][ReM][D]: Req:7840 UID:0 one.vmpool.info result SUCCESS, "<VM_POOL>
37<…"
Tue Jan 30 11:31:16 2018 [Z0][ONE][E]: Error parsing host information: syntax error, unexpected VARI
ABLE, expecting EQUAL or EQUAL_EMPTY at line 1, columns 8:14. Monitoring information:
error: failed to connect to the hypervisor
error: unable to connect to server at ‘localhost:16509’: Connection refused
ARCH=x86_64
MODELNAME="Intel® Xeon® CPU E5-2690 0 @ 2.90GHz"
HYPERVISOR=kvm
TOTALCPU=3200
CPUSPEED=1200
TOTALMEMORY=131824240
USEDMEMORY=1246856
FREEMEMORY=130577384
FREECPU=3200
USEDCPU=0
NETRX=0
NETTX=0
KVM_MACHINES="pc-i440fx-rhel7.0.0 pc rhel6.0.0 rhel6.1.0 rhel6.2.0 rhel6.3.0 rhel6.4.0 rhel6.5.0 rhe
l6.6.0"
KVM_CPU_MODELS="486 pentium pentium2 pentium3 pentiumpro coreduo n270 core2duo qemu32 kvm32 cpu64-rh
el5 cpu64-rhel6 kvm64 qemu64 Conroe Penryn Nehalem Nehalem-IBRS Westmere Westmere-IBRS SandyBridge S
andyBridge-IBRS IvyBridge IvyBridge-IBRS Haswell-noTSX Haswell-noTSX-IBRS Haswell Haswell-IBRS Broad
well-noTSX Broadwell-noTSX-IBRS Broadwell Broadwell-IBRS Skylake-Client Skylake-Client-IBRS athlon p
henom Opteron_G1 Opteron_G2 Opteron_G3 Opteron_G4 Opteron_G5"
DS_LOCATION_USED_MB=1826
DS_LOCATION_TOTAL_MB=204696
DS_LOCATION_FREE_MB=202871
DS = [
ID = 0,
USED_MB = 1826,
TOTAL_MB = 204696,
FREE_MB = 202871
]

marcindulak · January 30, 2018, 3:27am

From the frontend, what is the output of

sudo su - oneadmin -c 'onehost list'

sudo su - oneadmin -c 'onehost show 0'

And on the worker node:

sudo systemctl status libvirtd

keiji · January 30, 2018, 4:42am

thisi is log. thank you for your help.

[oneadmin@test5 ~]$ onehost list
ID NAME CLUSTER RVM ALLOCATED_CPU ALLOCATED_MEM STAT
4 test6 default 0 0 / 3200 (0%) 0K / 125.7G (0%) on

[oneadmin@test5 ~]$ onehost show 4
HOST 4 INFORMATION
ID : 4
NAME : test6
CLUSTER : default
STATE : MONITORED
IM_MAD : kvm
VM_MAD : kvm
LAST MONITORING TIME : 01/30 13:35:32

HOST SHARES
RUNNING VMS : 0
MEMORY
TOTAL : 125.7G
TOTAL +/- RESERVED : 125.7G
USED (REAL) : 15.3G
USED (ALLOCATED) : 0K
CPU
TOTAL : 3200
TOTAL +/- RESERVED : 3200
USED (REAL) : 0
USED (ALLOCATED) : 0

LOCAL SYSTEM DATASTORE #0 CAPACITY
TOTAL: : 199.9G
USED: : 4.8G
FREE: : 195.1G

MONITORING INFORMATION
ARCH=“x86_64”
CPUSPEED=“1200”
HOSTNAME=“test6”
HYPERVISOR=“kvm”
IM_MAD=“kvm”
KVM_CPU_MODELS=“486 pentium pentium2 pentium3 pentiumpro coreduo n270 core2duo qemu32 kvm32 cpu64-rhel5 cpu64-rhel6 kvm64 qemu64 Conroe Penryn Nehalem Nehalem-IBRS Westmere Westmere-IBRS SandyBridge SandyBridge-IBRS IvyBridge IvyBridge-IBRS Haswell-noTSX Haswell-noTSX-IBRS Haswell Haswell-IBRS Broadwell-noTSX Broadwell-noTSX-IBRS Broadwell Broadwell-IBRS Skylake-Client Skylake-Client-IBRS athlon phenom Opteron_G1 Opteron_G2 Opteron_G3 Opteron_G4 Opteron_G5”
KVM_MACHINES=“pc-i440fx-rhel7.0.0 pc rhel6.0.0 rhel6.1.0 rhel6.2.0 rhel6.3.0 rhel6.4.0 rhel6.5.0 rhel6.6.0”
MODELNAME=“Intel(R) Xeon(R) CPU E5-2690 0 @ 2.90GHz”
NETRX=“0”
NETTX=“0”
RESERVED_CPU=“”
RESERVED_MEM=“”
VERSION=“5.4.6”
VM_MAD=“kvm”

WILD VIRTUAL MACHINES

NAME IMPORT_ID CPU MEMORY

VIRTUAL MACHINES

ID USER     GROUP    NAME            STAT UCPU    UMEM HOST             TIME

[root@test6 kernel]# systemctl status libvirtd
● libvirtd.service - Virtualization daemon
Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled; vendor preset: enabled)
Active: active (running) since 火 2018-01-30 13:25:38 JST; 13min ago
Docs: man:libvirtd(8)
http://libvirt.org
Main PID: 51226 (libvirtd)
CGroup: /system.slice/libvirtd.service
├─ 1659 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/…
├─ 1660 /usr/sbin/dnsmasq --conf-file=/var/lib/libvirt/dnsmasq/default.conf --leasefile-ro --dhcp-script=/usr/…
└─51226 /usr/sbin/libvirtd

1月 30 13:25:37 test6 systemd[1]: Starting Virtualization daemon…
1月 30 13:25:38 test6 systemd[1]: Started Virtualization daemon.
1月 30 13:25:38 test6 dnsmasq[1659]: read /etc/hosts - 4 addresses
1月 30 13:25:38 test6 dnsmasq[1659]: read /var/lib/libvirt/dnsmasq/default.addnhosts - 0 addresses
1月 30 13:25:38 test6 dnsmasq-dhcp[1659]: read /var/lib/libvirt/dnsmasq/default.hostsfile

Node@test6

virsh -c qemu+tcp://host/system

error: unable to connect to server at ‘host:16509’: Connection refused
error: failed to connect to the hypervisor

marcindulak · January 30, 2018, 8:15pm

I don’t know why libvirt is trying to use the 16509 port - that should be commented out by default in /etc/libvirt/libvirtd.conf on test6. I imagine you have no problems creating a test VM from test6 manually with virt-install, as root?

virsh list --all
grep oneadmin /etc/libvirt/qemu.conf  # opennebula modifies this file
yum -y install wget virt-install virt-viewer
wget https://download.cirros-cloud.net/0.4.0/cirros-0.4.0-i386-disk.img -P /tmp
chown oneadmin:oneadmin /tmp/cirros-0.4.0-i386-disk.img
virt-install --os-variant=virtio26 --name=cirros --ram=64 --vcpus=1 --disk /tmp/cirros-0.4.0-i386-disk.img,format=qcow2 --import --nonetworks

In another terminal, with $DISPLAY available:

virt-viewer cirros  # just check if it starts fine
virsh destroy cirros
virsh undefine cirros

keiji · January 31, 2018, 12:47am

Than you for your help. I can create VM from test6 manually with virt-install,as root.

[root@test6 qemu]# virsh list --all

ID Name State

1 cirros running

[root@test6 qemu]# virsh console cirros

※I can login

marcindulak · January 31, 2018, 9:43am

I’m unable to reproduce the problem, maybe a developer can comment.

If you google search for “‘localhost:16509’: Connection refused” there are few posts that clarify what may be happening. If you find a solution please describe it here.

VURoland · February 1, 2018, 8:02pm

Could this be your problem?
see: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_deployment_and_administration_guide/sect-troubleshooting-common_libvirt_errors_and_troubleshooting#sect-Unable_to_connect_to_server_at_host16509_Connection_refused_…_error_failed_to_connect_to_the_hypervisor

 The libvirt daemon was started without the --listen option. Verify this by running this command:

# ps aux | grep libvirtd
root     10749  0.1  0.2 558276 18280 ?        Ssl  23:21   0:00 /usr/sbin/libvirtd

The output does not contain the --listen option.

Solution
Start the daemon with the --listen option.
To do this, modify the /etc/sysconfig/libvirtd file and uncomment the following line:

# LIBVIRTD_ARGS="--listen"

Then, restart the libvirtd service with this command:

# /bin/systemctl restart libvirtd.service

keiji · February 2, 2018, 11:16am

Thank you for your help.
I configured LIBVIRTD_ARGS="–listen". but
I could not start libvirtd.

I gave up Centos7 version.

I tried ubuntu16 version. now I perfectry set up opennebula.
I think ubuntu version better.

keiji · February 2, 2018, 11:18am

I will try again Centos7 version after a while.

Jehe · July 7, 2022, 1:53am

Maybe after wrong opennebula packages reinstallations you’ve lost /var/lib/one/.ssh/config file, which manages ssh connections while clonning VMs.

As it is, workaround you can create config file with the next content:

Host *
  StrictHostKeyChecking no

But much more better to restore native opennebula config content

Topic		Replies	Views
Driver Error Instanciate General	3	564	March 30, 2022
Can't create VM from template official v.6.2 Community Support	2	330	March 16, 2022
Unable to instanciate VM on a fresh installed system Community Support	11	2532	September 20, 2017
Problems starting VM from template Community Support	5	4016	June 10, 2015
Problems setting HA Community Support	7	960	October 3, 2017

Can not instant VM

virsh -c qemu+tcp://host/system

ID Name State

Related topics