# VXLAN networking again

**URL:** <https://forum.opennebula.io/t/vxlan-networking-again/2826>\
**Category:** Product Support\
**Created:** [August 26, 2016, 3:32pm UTC](https://forum.opennebula.io/t/vxlan-networking-again/2826 "2016-08-26T15:32:48Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![paprika27](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@paprika27](https://forum.opennebula.io/u/paprika27)\
**Post date:** [August 26, 2016, 3:32pm UTC](https://forum.opennebula.io/t/vxlan-networking-again/2826/1 "2016-08-26T15:32:48Z")

</div>

Hello all,

I have a physical router 10.10.10.1 and a frontend&host 10.10.10.102.

```
[oneadmin@localhost ~]$ brctl show
bridge name bridge id STP enabled interfaces
br0 8000.001517895489 yes enp5s0f1
virbr0 8000.000000000000 yes
[oneadmin@localhost ~]$ ifconfig
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
    inet 10.10.10.102 netmask 255.0.0.0 broadcast 10.255.255.255
    inet6 fe80::215:17ff:fe89:5489 prefixlen 64 scopeid 0x20<link>
    ether 00:15:17:89:54:89 txqueuelen 0 (Ethernet)
    RX packets 1463 bytes 179277 (175.0 KiB)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 1868 bytes 562376 (549.1 KiB)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

enp5s0f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
    inet6 fe80::215:17ff:fe89:5489 prefixlen 64 scopeid 0x20<link>
    ether 00:15:17:89:54:89 txqueuelen 1000 (Ethernet)
    RX packets 9904 bytes 1776419 (1.6 MiB)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 10166 bytes 2642482 (2.5 MiB)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
    device interrupt 19 memory 0xb8800000-b8820000

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
    inet 127.0.0.1 netmask 255.0.0.0
    inet6 ::1 prefixlen 128 scopeid 0x10<host>
    loop txqueuelen 0 (Local Loopback)
    RX packets 21858 bytes 13623286 (12.9 MiB)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 21858 bytes 13623286 (12.9 MiB)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1460
    inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255
    ether 00:00:00:00:00:00 txqueuelen 0 (Ethernet)
    RX packets 29 bytes 5272 (5.1 KiB)
    RX errors 0 dropped 0 overruns 0 frame 0
    TX packets 26 bytes 4420 (4.3 KiB)
    TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[oneadmin@localhost ~]$ route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 10.10.10.1 0.0.0.0 UG 0 0 0 br0
10.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 br0
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0

```

Using

```
BRIDGE = "br0"
DNS = "10.10.10.1"
GATEWAY = "10.10.10.1"
GUEST_MTU = "1460"
NETWORK_ADDRESS = "10.10.10.0"
NETWORK_MASK = "255.0.0.0"
PHYDEV = ""
SECURITY_GROUPS = "0"
VLAN_ID = ""
VN_MAD = "dummy"`

```

my VM receives

```
alpine-ssh-base-51:~# ifconfig
eth0 Link encap:Ethernet HWaddr 02:00:0A:00:01:01
          inet addr:10.10.10.104 Bcast:0.0.0.0 Mask:255.0.0.0
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          RX packets:35 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2732 (2.6 KiB) TX bytes:1284 (1.2 KiB)

lo Link encap:Local Loopback
          inet addr:127.0.0.1 Mask:255.0.0.0
          inet6 addr: ::1%32571/128 Scope:Host
          UP LOOPBACK RUNNING MTU:65536 Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

```

apparently directly from the physical router.  
Now I would like to use

```
BRIDGE = "br0"
DNS = "10.10.10.1"
GATEWAY = "10.10.10.1"
GUEST_MTU = "1460"
MTU = "1460"
NETWORK_ADDRESS = "10.10.10.0"
NETWORK_MASK = "255.0.0.0"
PHYDEV = "enp5s0f1"
VN_MAD = "vxlan"

```

with a range in 10.0.1.x. However, I get these errors:

```
[Z0][VMM][D]: Message received: LOG I 52 Command execution fail: /var/tmp/one/vnm/vxlan/pre PFZNPjxJRD41M...
[Z0][VMM][D]: Message received: LOG E 52 pre: Command "sudo ip link add enp5s0f1.2 mtu 1460 type vxlan id 2 group 239.0.0.2 ttl 16 dev enp5s0f1" failed.
[Z0][VMM][D]: Message received: LOG E 52 pre: vxlan: destination port not specified
[Z0][VMM][D]: Message received: LOG I 52 Will use Linux kernel default (non-standard value)
[Z0][VMM][D]: Message received: LOG I 52 Use 'dstport 4789' to get the IANA assigned value
[Z0][VMM][D]: Message received: LOG I 52 Use 'dstport 0' to get default and quiet this message
[Z0][VMM][D]: Message received: LOG I 52 RTNETLINK answers: File exists
[Z0][VMM][D]: Message received: LOG E 52 vxlan: destination port not specified
[Z0][VMM][D]: Message received: LOG E 52 Will use Linux kernel default (non-standard value)
[Z0][VMM][D]: Message received: LOG E 52 Use 'dstport 4789' to get the IANA assigned value
[Z0][VMM][D]: Message received: LOG E 52 Use 'dstport 0' to get default and quiet this message
[Z0][VMM][D]: Message received: LOG E 52 RTNETLINK answers: File exists
[Z0][VMM][D]: Message received: LOG E 52
[Z0][VMM][D]: Message received: LOG I 52 ExitCode: 2
[Z0][VMM][D]: Message received: LOG I 52 Failed to execute network driver operation: pre.
[Z0][VMM][D]: Message received: DEPLOY FAILURE 52 vxlan: vxlan: destination port not specified
[Z0][VMM][D]: Message received: Will use Linux kernel default (non-standard value)
[Z0][VMM][D]: Message received: Use 'dstport 4789' to get the IANA assigned value
[Z0][VMM][D]: Message received: Use 'dstport 0' to get default and quiet this message
[Z0][VMM][D]: Message received: RTNETLINK answers: File exists

```

Unsurprisingly

```
[oneadmin@localhost ~]$ sudo /var/tmp/one/vnm/vxlan/pre PFZNPjxJRD41MjwvSUQ+...
ERROR: pre: Command "sudo ip link add enp5s0f1.2 mtu 1460 type vxlan id 2 group 239.0.0.2 ttl 16 dev enp5s0f1" failed.
ERROR: pre: vxlan: destination port not specified
Will use Linux kernel default (non-standard value)
Use 'dstport 4789' to get the IANA assigned value
Use 'dstport 0' to get default and quiet this message
RTNETLINK answers: File exists
ERROR MESSAGE --8<------
vxlan: destination port not specified
Will use Linux kernel default (non-standard value)
Use 'dstport 4789' to get the IANA assigned value
Use 'dstport 0' to get default and quiet this message
RTNETLINK answers: File exists
ERROR MESSAGE ------>8--

[oneadmin@localhost ~]$ sudo ip link add enp5s0f1.2 mtu 1460 type vxlan id 2 group 239.0.0.2 ttl 16 dev enp5s0f1
vxlan: destination port not specified
Will use Linux kernel default (non-standard value)
Use 'dstport 4789' to get the IANA assigned value
Use 'dstport 0' to get default and quiet this message
RTNETLINK answers: File exists
[oneadmin@localhost ~]$ sudo ip link add enp5s0f1.2 mtu 1460 type vxlan id 2 group 239.0.0.2 ttl 16 dev enp5s0f1 dstport 0
RTNETLINK answers: File exists

```

what now? Using

```
BRIDGE = "br0"
DNS = "10.10.10.1"
GATEWAY = "10.10.10.1"
GUEST_MTU = "1460"
NETWORK_ADDRESS = "10.10.10.0"
NETWORK_MASK = "255.0.0.0"
PHYDEV = ""
SECURITY_GROUPS = "0"
VN_MAD = "ovswitch"

```

yields

```
[Z0][VMM][D]: Message received: LOG I 52 Command execution fail: /var/tmp/one/vnm/ovswitch/post PFZNPjxJRD41MjwvSUQ+... '/dev/loop1
[Z0][VMM][D]: Message received: LOG I 52 one-52'
[Z0][VMM][D]: Message received: LOG I 52 No tap device found for nic 0
[Z0][VMM][D]: Message received: LOG I 52 ExitCode: 1
[Z0][ReM][D]: Req:6720 UID:0 VirtualMachineInfo invoked , 52
[Z0][ReM][D]: Req:6720 UID:0 VirtualMachineInfo result SUCCESS, "<VM><ID>52</ID><UID>..."
[Z0][InM][D]: Host head1 (2) successfully monitored.
[Z0][ReM][D]: Req:4064 UID:0 VirtualMachineMonitoring invoked , 52
[Z0][ReM][D]: Req:4064 UID:0 VirtualMachineMonitoring result SUCCESS, "<MONITORING_DATA><VM..."
[Z0][VMM][D]: Message received: LOG I 52 ExitCode: 0
[Z0][VMM][D]: Message received: LOG I 52 Successfully execute virtualization driver operation: cancel.
[Z0][VMM][D]: Message received: LOG I 52 Failed to execute network driver operation: post.
[Z0][VMM][D]: Message received: DEPLOY FAILURE 52 ovswitch: -

```

regardless of whether I put enp5s0f1 as PHYDEV.

I must not have understood something, please point out the right direction.

EDIT: I tried some more:

```
[oneadmin@localhost ~]$ ip -d link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 addrgenmode eui64
2: enp5s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
link/ether 00:15:17:89:54:88 brd ff:ff:ff:ff:ff:ff promiscuity 0 addrgenmode eui64
3: enp5s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP mode DEFAULT qlen 1000
link/ether 00:15:17:89:54:89 brd ff:ff:ff:ff:ff:ff promiscuity 1 addrgenmode eui64
5: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1460 qdisc noqueue state DOWN mode DEFAULT
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff promiscuity 0
bridge addrgenmode eui64
6: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 500
link/ether 52:54:00:2a:4b:55 brd ff:ff:ff:ff:ff:ff promiscuity 0
tun addrgenmode eui64
8: enp5s0f1.2: <BROADCAST,MULTICAST> mtu 1460 qdisc noqueue state DOWN mode DEFAULT
link/ether 4a:11:46:c2:71:11 brd ff:ff:ff:ff:ff:ff promiscuity 0
vxlan id 2 group 239.0.0.2 dev enp5s0f1 srcport 0 0 dstport 8472 ttl 16 ageing 300 addrgenmode eui64
20: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether 00:15:17:89:54:89 brd ff:ff:ff:ff:ff:ff promiscuity 0
bridge addrgenmode eui64
[oneadmin@localhost ~]$ sudo ip link del virbr0-nic
[oneadmin@localhost ~]$ sudo ip link del enp5s0f1.2
[oneadmin@localhost ~]$ ip -d link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 addrgenmode eui64
2: enp5s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
link/ether 00:15:17:89:54:88 brd ff:ff:ff:ff:ff:ff promiscuity 0 addrgenmode eui64
3: enp5s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP mode DEFAULT qlen 1000
link/ether 00:15:17:89:54:89 brd ff:ff:ff:ff:ff:ff promiscuity 1 addrgenmode eui64
5: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1460 qdisc noqueue state DOWN mode DEFAULT
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff promiscuity 0
bridge addrgenmode eui64
20: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether 00:15:17:89:54:89 brd ff:ff:ff:ff:ff:ff promiscuity 0
bridge addrgenmode eui64

```

deployment with vxlan regenerates enp5s0f1.2 and results in the same errors.  
A /var/spool/mail arrived with the following:

```
sudo: svncterm: command not found
[...]
sed: -e expression #1, char 1: unknown command: `,'

```

but I know as little as ever what to do

---

<div class="post-metadata">

**Author:** ![ruben](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ruben/32/19_2.png) [@ruben](https://forum.opennebula.io/u/ruben)\
**Post date:** [August 29, 2016, 10:21am UTC](https://forum.opennebula.io/t/vxlan-networking-again/2826/2 "2016-08-29T10:21:12Z")

</div>

It seems that the tagged interface cannot be created, please try to removed  
the br0, and start the tests in a clean status (or host) (i.e. without the  
bridged configuration).

---

<div class="post-metadata">

**Author:** ![paprika27](https://avatars.discourse-cdn.com/v4/letter/p/7feea3/32.png) [@paprika27](https://forum.opennebula.io/u/paprika27)\
**Post date:** [August 29, 2016, 6:14pm UTC](https://forum.opennebula.io/t/vxlan-networking-again/2826/3 "2016-08-29T18:14:16Z")

</div>

the vxlan [enp5s0f1.ID](http://enp5s0f1.ID) ie 2 does get created both with my linux br0 and the virsh NAT virbr0.  
would i remove br0 and still put it in the template? as I understood, I need a linux bridge like virsh’s created first and put that into the template for vxlan. Can you try to explain what the thing is trying to do or where I can find more detailed logs?

I got a step further when I generated an ovs bridge (instead of a linux bridge as virsh does). opennebula or whichever part of it was able to generate taps in that. with that come new problems, windows virtio nic vms getting absurd IPs from who knows where while linux vms and lxc connect fine. I guess the urgency is out of the problem, it’s not solved but I got around it. I also built svncterm taking care of the first of the two errors.

Is there any more detailed resource than the docs for me to learn from what should be going on?