Bridged virtual network with VLAN problem

Community Support
1

I have a virtual network defined like this. The VLAN ID 40 is allowed on the eth0 interface on my host.

BRIDGE="virbr0"
BRIDGE_TYPE="linux"
NETWORK_ADDRESS="192.168.122.0"
NETWORK_MASK="255.255.255.0"
OUTER_VLAN_ID=""
PHYDEV="eth0"
SECURITY_GROUPS="0"
VLAN_ID="40"
VN_MAD="802.1Q"

But when I attach the NIC to a VM and try to start it errors out with this message. Why is it trying to append the 40 ID to the physical device? That seems to go against my interpretation of the docs, which say the physical interface will be tagged and added to the bridge.

Sat Apr  4 18:33:42 2020 [Z0][VMM][E]: pre: Command "sudo ip link add link eth0.40 name eth0.40.40 mtu 1500 type vlan id 40 " failed.
Sat Apr  4 18:33:42 2020 [Z0][VMM][E]: pre: Cannot find device "eth0.40"
Sat Apr  4 18:33:42 2020 [Z0][VMM][E]: Cannot find device "eth0.40"
2

I’m trying to reproduce the failure but I’m not able to do it. Could you send the following information?

  1. Modifications done in OpenNebula configuration files if any
  2. The VM template information that you are using (onetemplate show -x <TEMPLATE_ID>)
3

I haven’t made any changes to the configuration files. Here’s my template:

<VMTEMPLATE>
  <ID>18</ID>
  <UID>0</UID>
  <GID>0</GID>
  <UNAME>oneadmin</UNAME>
  <GNAME>oneadmin</GNAME>
  <NAME>centos7-run1</NAME>
  <PERMISSIONS>
    <OWNER_U>1</OWNER_U>
    <OWNER_M>1</OWNER_M>
    <OWNER_A>0</OWNER_A>
    <GROUP_U>0</GROUP_U>
    <GROUP_M>0</GROUP_M>
    <GROUP_A>0</GROUP_A>
    <OTHER_U>0</OTHER_U>
    <OTHER_M>0</OTHER_M>
    <OTHER_A>0</OTHER_A>
  </PERMISSIONS>
  <REGTIME>1585631177</REGTIME>
  <TEMPLATE>
    <CONTEXT>
      <GROW_ROOTFS><![CDATA[NO]]></GROW_ROOTFS>
      <NETWORK><![CDATA[YES]]></NETWORK>
      <SET_HOSTNAME><![CDATA[$NAME]]></SET_HOSTNAME>
      <START_SCRIPT_BASE64><![CDATA[IyEvYmluL2Jhc2gKeXVtIC15IGluc3RhbGwgYW5zaWJsZQp5dW0gaW5zdGFsbCAteSBudHAKeXVtIGluc3RhbGwgLXkgbnRwZGF0ZQpudHBkYXRlIDAucG9vbC5udHAub3JnCnN5c3RlbWN0bCBlbmFibGUgbnRwZApzeXN0ZW1jdGwgc3RhcnQgbnRwZA==]]></START_SCRIPT_BASE64>
    </CONTEXT>
    <CPU><![CDATA[1]]></CPU>
    <DISK>
      <IMAGE><![CDATA[centos7-data-installed]]></IMAGE>
      <IMAGE_UNAME><![CDATA[oneadmin]]></IMAGE_UNAME>
      <SIZE><![CDATA[10240]]></SIZE>
    </DISK>
    <DISK>
      <FORMAT><![CDATA[raw]]></FORMAT>
      <SIZE><![CDATA[512]]></SIZE>
      <TYPE><![CDATA[swap]]></TYPE>
    </DISK>
    <GRAPHICS>
      <LISTEN><![CDATA[0.0.0.0]]></LISTEN>
      <TYPE><![CDATA[VNC]]></TYPE>
    </GRAPHICS>
    <INPUTS_ORDER><![CDATA[]]></INPUTS_ORDER>
    <MEMORY><![CDATA[4096]]></MEMORY>
    <MEMORY_UNIT_COST><![CDATA[MB]]></MEMORY_UNIT_COST>
    <NIC>
      <NETWORK><![CDATA[vmnet0]]></NETWORK>
      <NETWORK_UNAME><![CDATA[oneadmin]]></NETWORK_UNAME>
    </NIC>
    <OS>
      <ARCH><![CDATA[x86_64]]></ARCH>
      <BOOT><![CDATA[disk0]]></BOOT>
    </OS>
  </TEMPLATE>
</VMTEMPLATE>
4

Could you also send the following information?

  1. Virtual Network information. Output of:
    onevnet show -x <VNET_ID
  2. Your OpenNebula version. Output of:
    oned -v
5

Thanks-

<VNET>
  <ID>2</ID>
  <UID>0</UID>
  <GID>0</GID>
  <UNAME>oneadmin</UNAME>
  <GNAME>oneadmin</GNAME>
  <NAME>vmnet0</NAME>
  <PERMISSIONS>
    <OWNER_U>1</OWNER_U>
    <OWNER_M>1</OWNER_M>
    <OWNER_A>0</OWNER_A>
    <GROUP_U>0</GROUP_U>
    <GROUP_M>0</GROUP_M>
    <GROUP_A>0</GROUP_A>
    <OTHER_U>0</OTHER_U>
    <OTHER_M>0</OTHER_M>
    <OTHER_A>0</OTHER_A>
  </PERMISSIONS>
  <CLUSTERS>
    <ID>0</ID>
  </CLUSTERS>
  <BRIDGE><![CDATA[virbr0]]></BRIDGE>
  <BRIDGE_TYPE><![CDATA[linux]]></BRIDGE_TYPE>
  <PARENT_NETWORK_ID/>
  <VN_MAD><![CDATA[802.1Q]]></VN_MAD>
  <PHYDEV><![CDATA[eth0]]></PHYDEV>
  <VLAN_ID><![CDATA[40]]></VLAN_ID>
  <OUTER_VLAN_ID/>
  <VLAN_ID_AUTOMATIC>0</VLAN_ID_AUTOMATIC>
  <OUTER_VLAN_ID_AUTOMATIC>0</OUTER_VLAN_ID_AUTOMATIC>
  <USED_LEASES>1</USED_LEASES>
  <VROUTERS/>
  <TEMPLATE>
    <BRIDGE><![CDATA[virbr0]]></BRIDGE>
    <BRIDGE_TYPE><![CDATA[linux]]></BRIDGE_TYPE>
    <DNS><![CDATA[128.121.222.222]]></DNS>
    <GATEWAY><![CDATA[192.168.122.1]]></GATEWAY>
    <NETWORK_ADDRESS><![CDATA[192.168.122.0]]></NETWORK_ADDRESS>
    <NETWORK_MASK><![CDATA[255.255.255.0]]></NETWORK_MASK>
    <OUTER_VLAN_ID><![CDATA[]]></OUTER_VLAN_ID>
    <PHYDEV><![CDATA[eth0]]></PHYDEV>
    <SECURITY_GROUPS><![CDATA[0]]></SECURITY_GROUPS>
    <VLAN_ID><![CDATA[40]]></VLAN_ID>
    <VN_MAD><![CDATA[802.1Q]]></VN_MAD>
  </TEMPLATE>
  <AR_POOL>
    <AR>
      <AR_ID><![CDATA[0]]></AR_ID>
      <IP><![CDATA[192.168.122.10]]></IP>
      <MAC><![CDATA[02:00:c0:a8:7a:0a]]></MAC>
      <SIZE><![CDATA[20]]></SIZE>
      <TYPE><![CDATA[IP4]]></TYPE>
      <MAC_END><![CDATA[02:00:c0:a8:7a:1d]]></MAC_END>
      <IP_END><![CDATA[192.168.122.29]]></IP_END>
      <USED_LEASES>1</USED_LEASES>
      <LEASES>
        <LEASE>
          <IP><![CDATA[192.168.122.10]]></IP>
          <MAC><![CDATA[02:00:c0:a8:7a:0a]]></MAC>
          <VM><![CDATA[41]]></VM>
        </LEASE>
      </LEASES>
    </AR>
  </AR_POOL>
</VNET>

oned -v
Copyright 2002-2019, OpenNebula Project, OpenNebula Systems

OpenNebula 5.10.1 (45f6d6df) is distributed and licensed for use under the terms of the
Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0).

6

Still not able to reproduce the issue. Could you send the following information, please?

  1. Output of onevm show -x ID being ID the id of the failed VM.
  2. Output of ip a on node.
  3. Output of cat /etc/lsb-release on node.

Thanks.

7

Thanks for putting the time into looking at this.

I’m attaching the onevm xml output.

onevm_show_x.txt (13.6 KB)

I don’t have a /etc/lsb-release file, there are several files in a directory:

ls /etc/lsb-release.d/
core-4.1-amd64   cxx-4.1-amd64   desktop-4.1-amd64   languages-4.1-amd64   printing-4.1-amd64
core-4.1-noarch  cxx-4.1-noarch  desktop-4.1-noarch  languages-4.1-noarch  printing-4.1-noarch

 hpc-ctest2 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:21:28:8e:b2:74 brd ff:ff:ff:ff:ff:ff
    inet 10.125.255.122/24 brd 10.125.255.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::221:28ff:fe8e:b274/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:21:28:8e:b2:75 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:21:28:8e:b2:76 brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 00:21:28:8e:b2:77 brd ff:ff:ff:ff:ff:ff
6: usb0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 02:21:28:57:47:17 brd ff:ff:ff:ff:ff:ff
7: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:21:28:8e:b2:74 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
8: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
    link/ether 52:54:00:82:d8:27 brd ff:ff:ff:ff:ff:ff
9: eth0.40@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:21:28:8e:b2:74 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::221:28ff:fe8e:b274/64 scope link
       valid_lft forever preferred_lft forever
10: eth0.40.40@eth0.40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master virbr0 state UP group default qlen 1000
    link/ether 00:21:28:8e:b2:74 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::221:28ff:fe8e:b274/64 scope link
       valid_lft forever preferred_lft forever
8

Cannot figure out how you ended having <PHYDEV><![CDATA[eth0.40]]></PHYDEV> in VM info. How did you instantiate the VM? From Sunstone or CLI? Did you modify any attribute or pass any argument?

Meanwhile you can try instantiating again from template.

9

This whole time I’ve been provisioning through Sunstone. I just tried again and now things are working correctly. I’m not sure why, one provisioned the virtual NICs correctly this time. I’ll do some more testing. On my node:

7: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 00:21:28:8e:b2:74 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
8: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
    link/ether 52:54:00:82:d8:27 brd ff:ff:ff:ff:ff:ff
12: eth0.40@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master virbr0 state UP group default qlen 1000
    link/ether 00:21:28:8e:b2:74 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::221:28ff:fe8e:b274/64 scope link
       valid_lft forever preferred_lft forever
13: one-42-0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UNKNOWN group default qlen 1000
    link/ether fe:00:c0:a8:7a:0a brd ff:ff:ff:ff:ff:ff
    inet6 fe80::fc00:c0ff:fea8:7a0a/64 scope link
       valid_lft forever preferred_lft forever
10

i am facing the same issue. has your been resolved ? could you please guide ?

11

I’m not sure what happened, it just started working for me. I’m using the same onevnet configuration at the start of my post. On key thing seems to be to keep the configuration as simple as possible and test things before adding more configurations. Also, make sure your KVM host doesn’t have any VLAN tagged interfaces hanging around like something that looks like “eth0.40”. OpenNebula wants to create that itself when the VM is instantiated.

12

No reply.….?

13

Hi Kate, are you seeing something like I was seeing? On a KVM host with no VMs and the vnet I configured that’s posted at the start of the thread, the interfaces look like this:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:21:28:84:02:08 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:21:28:84:02:09 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:21:28:84:02:0a brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:21:28:84:02:0b brd ff:ff:ff:ff:ff:ff
6: usb0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 02:21:28:57:47:17 brd ff:ff:ff:ff:ff:ff
7: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default qlen 1000
    link/ether 52:54:00:94:7c:8e brd ff:ff:ff:ff:ff:ff
8: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT group default qlen 1000
    link/ether 52:54:00:94:7c:8e brd ff:ff:ff:ff:ff:ff

After I successfully start a VM (with ID ‘50’), I see this:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether 00:21:28:8e:b2:74 brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:21:28:8e:b2:75 brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:21:28:8e:b2:76 brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 00:21:28:8e:b2:77 brd ff:ff:ff:ff:ff:ff
6: usb0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/ether 02:21:28:57:47:17 brd ff:ff:ff:ff:ff:ff
7: virbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 00:21:28:8e:b2:74 brd ff:ff:ff:ff:ff:ff
8: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN mode DEFAULT group default qlen 1000
    link/ether 52:54:00:82:d8:27 brd ff:ff:ff:ff:ff:ff
12: eth0.40@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master virbr0 state UP mode DEFAULT group default qlen 1000
    link/ether 00:21:28:8e:b2:74 brd ff:ff:ff:ff:ff:ff
21: one-50-0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master virbr0 state UNKNOWN mode DEFAULT group default qlen 1000
    link/ether fe:00:c0:a8:7a:0a brd ff:ff:ff:ff:ff:ff

I’m not sure what happened, it just started working for me.