Gateway of different range is not working in KVM node

Community Support
1

Gateway of diffrent range is not working in KVM node

I have configure KVM node in dedicated server from Hetner, it has one default NIC eno1 as below with one public IP

eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 4c:72:b9:43:37:99 brd ff:ff:ff:ff:ff:ff
inet 5.9.XX.236 peer 5.9.100.225/32 brd 5.9.100.236 scope global eno1
valid_lft forever preferred_lft forever
inet 5.9.100.236/32 scope global eno1
valid_lft forever preferred_lft forever
inet6 2a01:4f8:162:32c4::2/64 scope global
valid_lft forever preferred_lft forever
inet6 fe80::4e72:b9ff:fe43:3799/64 scope link
valid_lft forever preferred_lft forever

The configuration of the eno1 is

DEVICE=eno1
ONBOOT=yes
BOOTPROTO=none
IPADDR=5.9.XX.236
PREFIX=32
SCOPE=“peer 5.9.XX.225”
DEFROUTE=yes

IPV6INIT=yes
IPV6ADDR=2a01:4f8:162:32c4::2/64
IPV6_DEFAULTGW=fe80::1
IPV6_DEFROUTE=yes
IPV6_DEFAULTDEV=eno1

and I have created one bridge connection as br0

br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether fe:00:94:fb:92:51 brd ff:ff:ff:ff:ff:ff
inet 148.251.XX.80/28 brd 148.251.146.95 scope global br0
valid_lft forever preferred_lft forever
inet6 fe80::fc00:94ff:fefb:9251/64 scope link
valid_lft forever preferred_lft forever

The configuration of the bridge connection is

DEVICE=“br0”
BOOTPROTO=static
ONBOOT=yes
TYPE=Bridge
IPADDR=148.251.146.80
NETMASK=255.255.255.240
GATEWAY=5.9.XX.236
BROADCAST=148.251.146.95
DNS1=8.8.8.8
DNS2=8.8.4.4
DELAY=0

Also created one Virtual network in bridge modein frontend as br0 and assign 16 public IP of 148.251.XX.80/28 subnet

Attributes
BRIDGE br0
BRIDGE_TYPE linux
DESCRIPTION VNETWORK-HOST
DNS 8.8.8.8
GATEWAY 5.9.XX.236
GUEST_MTU 1500
INBOUND_AVG_BW 1000
INBOUND_PEAK_BW 1000
INBOUND_PEAK_KB 1000
NETWORK_ADDRESS 148.251.XX.80
NETWORK_MASK 255.255.255.240

After that I am able to create a VM through this virtual network, and the VM is also getting the public IP but the state of the nic link is showing UNKNOW and also the public IP is not accessible. After login to VM through VNC I checked the route of the VM but the route was not there and the NIC configuration of that VM also doesnot have GATEWAY assign to it. Is this any kind of bug that I have not allow to assign the gateway of diffrent range IP as th gateway is the public IP of the KVM node 5.9.XX.236 and the bridge connection has the public IP of 148.251.XX.80 range ?? Or am I missing something in my configuration? Any help will be appriciated.

NIC of the VM in KVM node

one-33-0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc htb master br0 state UNKNOWN group default qlen 1000
link/ether fe:00:94:fb:92:51 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc00:94ff:fefb:9251/64 scope link
valid_lft forever preferred_lft forever

Route of the created VM

[root@localhost]# route -n
Destination Gateway GenMask Iface
148.251.XX.80 0.0.0.0 255.255.255.240 eth0

While adding the route to the VM manually I get below error as well

[root@localhost]# ip route add default via 5.9.100.236
RTNETLINK answers: Network is unreachable

2

Hello, your problem is not related to OpenNebula nor KVM. The problem is with networking skills. VM has IP from 148.251.XX.xx subnet only, so it doesn’t know how to connect to 5.9.100.236.

I don’t know how is 148.251.XX.xx subnet routed to your server, but looks like via 5.9.XX.236. You assigned 148.251.146.80 to KVM hypervisor. So try to use this address as the default gateway in your VM.

3

Thank you @feldsam . I will check this.

4

@feldsam I have confirmed with Hetzner(Server Provider) that they have route 148.251.146.80/28 subnet statically with the server’s main IP i.e 5.9.100.236 as next hop address. So how should I proceed in this case? As the VM is not allowing to add this as a gateway .

5

Hi, as I wrote, you have configured 148.251.146.80 on kvm node bridge, so you can use it as gateway in VMs

1 Like
6

Hi @feldsam .Thank you. It helped, somehow it start working.

7

Glad to hear this, but it is not “somehow” :slight_smile: it is like networks works :wink:

1 Like
8

Thank you @feldsam .

9

Hi, again @feldsam . I have created windows sever which is getting different NIC and different IP. How can I make my VM to get the default NIC of the KVM node . How can I solve this issue?

10

Hi, it is too general question to answer. I don’t know your VM configration. Would you paste here full VM template and full network(s) templates(s)?

11

@feldsam Here is my VM template configuration .

CONTEXT = [
NETWORK = “YES”,
SSH_PUBLIC_KEY = “$USER[SSH_PUBLIC_KEY]” ]
CPU = “4”
DESCRIPTION = “WIN-2019-TEMP”
DISK = [
IMAGE = “WIN-2019-ISO”,
IMAGE_UNAME = “oneadmin” ]
DISK = [
IMAGE = “NEW-VIRT-IO_DRIVER”,
IMAGE_UNAME = “oneadmin” ]
DISK = [
IMAGE = “onecontext-new”,
IMAGE_UNAME = “oneadmin” ]
DISK = [
DEV_PREFIX = “vd”,
IMAGE = “WINDOWS-DATABLOCK”,
IMAGE_UNAME = “oneadmin” ]
GRAPHICS = [
LISTEN = “0.0.0.0”,
PASSWD = “niba”,
PORT = “5966”,
TYPE = “VNC” ]
HYPERVISOR = “kvm”
INPUT = [
BUS = “usb”,
TYPE = “tablet” ]
INPUTS_ORDER = “”
LOGO = “images/logos/windows8.png”
MEMORY = “4096”
MEMORY_UNIT_COST = “MB”
NIC = [
NETWORK = “VNetwork-KVM”,
NETWORK_UNAME = “oneadmin”,
SECURITY_GROUPS = “0” ]
NIC_DEFAULT = [
MODEL = “virtio” ]
OS = [
BOOT = “disk0” ]
VCPU = “4”

And below is my template of the VM, after the creation of the VM.

Template

AUTOMATIC_DS_REQUIREMENTS = “(“CLUSTERS/ID” @> 0)”
AUTOMATIC_NIC_REQUIREMENTS = “(“CLUSTERS/ID” @> 0)”
AUTOMATIC_REQUIREMENTS = “(CLUSTER_ID = 0) & !(PUBLIC_CLOUD = YES) & !(PIN_POLICY = PINNED)”
CONTEXT = [
DISK_ID = “4”,
ETH0_CONTEXT_FORCE_IPV4 = “”,
ETH0_DNS = “8.8.8.8”,
ETH0_EXTERNAL = “”,
ETH0_GATEWAY = “148.251.146.xx”,
ETH0_GATEWAY6 = “”,
ETH0_IP = “148.251.146.xx”,
ETH0_IP6 = “”,
ETH0_IP6_PREFIX_LENGTH = “”,
ETH0_IP6_ULA = “”,
ETH0_MAC = “02:00:94:fb:92:xx”,
ETH0_MASK = “255.255.255.240”,
ETH0_METRIC = “”,
ETH0_METRIC6 = “”,
ETH0_MTU = “1500”,
ETH0_NETWORK = “148.251.146.xx”,
ETH0_SEARCH_DOMAIN = “”,
ETH0_VLAN_ID = “”,
ETH0_VROUTER_IP = “”,
ETH0_VROUTER_IP6 = “”,
ETH0_VROUTER_MANAGEMENT = “”,
NETWORK = “YES”,
SSH_PUBLIC_KEY = “”,
TARGET = “hdd” ]
CPU = “4”
DISK = [
ALLOW_ORPHANS = “NO”,
CLONE = “NO”,
CLONE_TARGET = “SYSTEM”,
CLUSTER_ID = “0”,
DATASTORE = “WINDOWS-DATASTORE”,
DATASTORE_ID = “103”,
DEV_PREFIX = “hd”,
DISK_ID = “0”,
DISK_SNAPSHOT_TOTAL_SIZE = “0”,
DISK_TYPE = “FILE”,
DRIVER = “raw”,
IMAGE = “WIN-2019-ISO”,
IMAGE_ID = “40”,
IMAGE_STATE = “8”,
IMAGE_UNAME = “oneadmin”,
LN_TARGET = “SYSTEM”,
ORDER = “1”,
ORIGINAL_SIZE = “5052”,
READONLY = “YES”,
SAVE = “NO”,
SIZE = “5052”,
SOURCE = “/var/lib/one//datastores/103/cfd7e7838acb960fbebb950fe95fef0f”,
TARGET = “hda”,
TM_MAD = “shared”,
TM_MAD_SYSTEM = “ssh”,
TYPE = “CDROM” ]
DISK = [
ALLOW_ORPHANS = “NO”,
CLONE = “NO”,
CLONE_TARGET = “SYSTEM”,
CLUSTER_ID = “0”,
DATASTORE = “WINDOWS-DATASTORE”,
DATASTORE_ID = “103”,
DEV_PREFIX = “hd”,
DISK_ID = “1”,
DISK_SNAPSHOT_TOTAL_SIZE = “0”,
DISK_TYPE = “FILE”,
DRIVER = “raw”,
IMAGE = “NEW-VIRT-IO_DRIVER”,
IMAGE_ID = “32”,
IMAGE_STATE = “2”,
IMAGE_UNAME = “oneadmin”,
LN_TARGET = “SYSTEM”,
ORIGINAL_SIZE = “394”,
READONLY = “YES”,
SAVE = “NO”,
SIZE = “394”,
SOURCE = “/var/lib/one//datastores/103/66efbc41e0354580de0b5e5745d02205”,
TARGET = “hdb”,
TM_MAD = “shared”,
TM_MAD_SYSTEM = “ssh”,
TYPE = “CDROM” ]
DISK = [
ALLOW_ORPHANS = “NO”,
CLONE = “NO”,
CLONE_TARGET = “SYSTEM”,
CLUSTER_ID = “0”,
DATASTORE = “WINDOWS-DATASTORE”,
DATASTORE_ID = “103”,
DEV_PREFIX = “hd”,
DISK_ID = “2”,
DISK_SNAPSHOT_TOTAL_SIZE = “0”,
DISK_TYPE = “FILE”,
DRIVER = “raw”,
IMAGE = “onecontext-new”,
IMAGE_ID = “41”,
IMAGE_STATE = “2”,
IMAGE_UNAME = “oneadmin”,
LN_TARGET = “SYSTEM”,
ORIGINAL_SIZE = “1”,
READONLY = “YES”,
SAVE = “NO”,
SIZE = “1”,
SOURCE = “/var/lib/one//datastores/103/9c29a7f278ded800e0086b4b92035d42”,
TARGET = “hdc”,
TM_MAD = “shared”,
TM_MAD_SYSTEM = “ssh”,
TYPE = “CDROM” ]
DISK = [
ALLOW_ORPHANS = “NO”,
CLONE = “NO”,
CLONE_TARGET = “SYSTEM”,
CLUSTER_ID = “0”,
DATASTORE = “WINDOWS-DATASTORE”,
DATASTORE_ID = “103”,
DEV_PREFIX = “vd”,
DISK_ID = “3”,
DISK_SNAPSHOT_TOTAL_SIZE = “0”,
DISK_TYPE = “FILE”,
DRIVER = “raw”,
IMAGE = “WINDOWS-DATABLOCK”,
IMAGE_ID = “42”,
IMAGE_STATE = “8”,
IMAGE_UNAME = “oneadmin”,
LN_TARGET = “SYSTEM”,
ORIGINAL_SIZE = “51200”,
PERSISTENT = “YES”,
READONLY = “NO”,
SAVE = “YES”,
SIZE = “51200”,
SOURCE = “/var/lib/one//datastores/103/902e5b93e8a359db91182ed3023b610a”,
TARGET = “vda”,
TM_MAD = “shared”,
TM_MAD_SYSTEM = “ssh”,
TYPE = “FILE” ]
GRAPHICS = [
LISTEN = “0.0.0.0”,
PASSWD = “nvy”,
PORT = “5966”,
TYPE = “VNC” ]
INPUT = [
BUS = “usb”,
TYPE = “tablet” ]
MEMORY = “4096”
NIC = [
AR_ID = “0”,
BRIDGE = “br0”,
BRIDGE_TYPE = “linux”,
CLUSTER_ID = “0”,
INBOUND_AVG_BW = “1000”,
INBOUND_PEAK_BW = “1000”,
INBOUND_PEAK_KB = “1000”,
IP = “148.251.146.xx”,
MAC = “02:00:94:fb:92:xx”,
MODEL = “virtio”,
NAME = “NIC0”,
NETWORK = “VNetwork-KVM”,
NETWORK_ID = “7”,
NETWORK_UNAME = “oneadmin”,
NIC_ID = “0”,
OUTBOUND_AVG_BW = “1000”,
OUTBOUND_PEAK_BW = “1000”,
OUTBOUND_PEAK_KB = “1000”,
SECURITY_GROUPS = “0”,
TARGET = “one-85-0”,
VN_MAD = “bridge” ]
NIC_DEFAULT = [
MODEL = “virtio” ]
OS = [
BOOT = “disk0” ]
SECURITY_GROUP_RULE = [
PROTOCOL = “ALL”,
RULE_TYPE = “OUTBOUND”,
SECURITY_GROUP_ID = “0”,
SECURITY_GROUP_NAME = “default” ]
SECURITY_GROUP_RULE = [
PROTOCOL = “ALL”,
RULE_TYPE = “INBOUND”,
SECURITY_GROUP_ID = “0”,
SECURITY_GROUP_NAME = “default” ]
TEMPLATE_ID = “21”
TM_MAD_SYSTEM = “ssh”
VCPU = “4”
VMID = “85”

In this VM i was supposed to get the IP odf148.251.146.xx IP but the VM NIC is showing the IP of 169.254.1.159 inside the VM.

12

hi, 169.254.x.x is default IP, if DHCP doesnt assing anything. Looks like you dont have contexualization scripts installed.

1 Like
13

Hi, @feldsam .I have to run win virtio exec file manually after login into the VM. After that only the NIC is visible. And it was assigning 169.254.xx.xx range IP. After that I reboot the VM and it start working.
How can we run this win virtio exec automatically and run context script automatically? Do we have to do this manually every time we create a Windows server and reboot every time?

14

Hello @feldsam ,
I have got issue while creating a VR. I have use the VR image “Service Virtual Router” which I have downloaded from the opennebula app.
I have two VNetwork “Vnet NEW-VN-Private” and “VNet Private-Vnet-188.40” . The first VNetwork has network of 10.10.15.0/28 and other one has 10.10.15.16/28 network. These both VNetwork are attached to different KVM node. In the first KVM node there is also VNetwork for the public IP. So how can I get access in these two different Vnetwork VM through the private IP? How can I add the route to this VR template? Can you guide on this issue.

Why it is also giving this Error “Error allocating a virtual route” while creating a VR?

Here is my vrouter details from frontend.

$onevrouter show 13

VIRTUAL ROUTER 13 INFORMATION
ID : 13
NAME : VR4
USER : oneadmin
GROUP : oneadmin
LOCK : None

PERMISSIONS
OWNER : um-
GROUP : —
OTHER : —

VIRTUAL ROUTER NICS
ID NETWORK MANAGEMENT IP
0 Private-VNet-188.40. YES -
1 NEW-VN-PRIVATE YES -

TEMPLATE CONTENTS

VIRTUAL MACHINES

189687188_320226266159782_5676438985114218307_n
189687188_320226266159782_5676438985114218307_n850Ă—567 40.5 KB

15

You have to prepare a “base image” with these things installed.

1 Like
16

Why you are getting messy with Virtual routers when you don’t understand networks?

You have one KVM node with public IPs, so it can works as router, which actualy works (from your other post). You can just interconnect more kvm nodes with some private network on provider side and use first kvm node ip as gateway

1 Like
17

Hi @feldsam ,
Thanks for the reply. I am new to VR. As you have mention I have used the first KVM node IP as gateway for other node . The internet is working fine in other second KVM node. But, how can I make these two different private network communicate each other. Using same vlan or VPN is only solution or there is any other way ? Any suggestion?

18

Hi, You should contact your server provider and ask for private vlan

1 Like
19

Hello @feldsam ,
I have added the vlan in both the bridge connection of nodes. Still not able to access between private network. Any suggestion?

20

Hi, I don’t know how your topology looks, paste here infocnfig output on both nodes and network templates