VXLAN networking again

Hello all,

I have a physical router 10.10.10.1 and a frontend&host 10.10.10.102.

[oneadmin@localhost ~]$ brctl show
bridge name     bridge id               STP enabled     interfaces
br0             8000.001517895489       yes             enp5s0f1
virbr0          8000.000000000000       yes
[oneadmin@localhost ~]$ ifconfig
br0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet 10.10.10.102  netmask 255.0.0.0  broadcast 10.255.255.255
    inet6 fe80::215:17ff:fe89:5489  prefixlen 64  scopeid 0x20<link>
    ether 00:15:17:89:54:89  txqueuelen 0  (Ethernet)
    RX packets 1463  bytes 179277 (175.0 KiB)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 1868  bytes 562376 (549.1 KiB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp5s0f1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet6 fe80::215:17ff:fe89:5489  prefixlen 64  scopeid 0x20<link>
    ether 00:15:17:89:54:89  txqueuelen 1000  (Ethernet)
    RX packets 9904  bytes 1776419 (1.6 MiB)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 10166  bytes 2642482 (2.5 MiB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
    device interrupt 19  memory 0xb8800000-b8820000

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
    inet 127.0.0.1  netmask 255.0.0.0
    inet6 ::1  prefixlen 128  scopeid 0x10<host>
    loop  txqueuelen 0  (Local Loopback)
    RX packets 21858  bytes 13623286 (12.9 MiB)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 21858  bytes 13623286 (12.9 MiB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1460
    inet 192.168.122.1  netmask 255.255.255.0  broadcast 192.168.122.255
    ether 00:00:00:00:00:00  txqueuelen 0  (Ethernet)
    RX packets 29  bytes 5272 (5.1 KiB)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 26  bytes 4420 (4.3 KiB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

[oneadmin@localhost ~]$ route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.10.10.1      0.0.0.0         UG    0      0        0 br0
10.0.0.0        0.0.0.0         255.0.0.0       U     0      0        0 br0
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0

Using

BRIDGE = "br0"
DNS = "10.10.10.1"
GATEWAY = "10.10.10.1"
GUEST_MTU = "1460"
NETWORK_ADDRESS = "10.10.10.0"
NETWORK_MASK = "255.0.0.0"
PHYDEV = ""
SECURITY_GROUPS = "0"
VLAN_ID = ""
VN_MAD = "dummy"`

my VM receives

alpine-ssh-base-51:~# ifconfig
eth0      Link encap:Ethernet  HWaddr 02:00:0A:00:01:01
          inet addr:10.10.10.104  Bcast:0.0.0.0  Mask:255.0.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:35 errors:0 dropped:0 overruns:0 frame:0
          TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2732 (2.6 KiB)  TX bytes:1284 (1.2 KiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1%32571/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

apparently directly from the physical router.
Now I would like to use

BRIDGE = "br0"
DNS = "10.10.10.1"
GATEWAY = "10.10.10.1"
GUEST_MTU = "1460"
MTU = "1460"
NETWORK_ADDRESS = "10.10.10.0"
NETWORK_MASK = "255.0.0.0"
PHYDEV = "enp5s0f1"
VN_MAD = "vxlan"

with a range in 10.0.1.x. However, I get these errors:

[Z0][VMM][D]: Message received: LOG I 52 Command execution fail: /var/tmp/one/vnm/vxlan/pre PFZNPjxJRD41M...
[Z0][VMM][D]: Message received: LOG E 52 pre: Command "sudo ip link add enp5s0f1.2 mtu 1460 type vxlan id 2 group 239.0.0.2 ttl 16 dev enp5s0f1" failed.
[Z0][VMM][D]: Message received: LOG E 52 pre: vxlan: destination port not specified
[Z0][VMM][D]: Message received: LOG I 52 Will use Linux kernel default (non-standard value)
[Z0][VMM][D]: Message received: LOG I 52 Use 'dstport 4789' to get the IANA assigned value
[Z0][VMM][D]: Message received: LOG I 52 Use 'dstport 0' to get default and quiet this message
[Z0][VMM][D]: Message received: LOG I 52 RTNETLINK answers: File exists
[Z0][VMM][D]: Message received: LOG E 52 vxlan: destination port not specified
[Z0][VMM][D]: Message received: LOG E 52 Will use Linux kernel default (non-standard value)
[Z0][VMM][D]: Message received: LOG E 52 Use 'dstport 4789' to get the IANA assigned value
[Z0][VMM][D]: Message received: LOG E 52 Use 'dstport 0' to get default and quiet this message
[Z0][VMM][D]: Message received: LOG E 52 RTNETLINK answers: File exists
[Z0][VMM][D]: Message received: LOG E 52
[Z0][VMM][D]: Message received: LOG I 52 ExitCode: 2
[Z0][VMM][D]: Message received: LOG I 52 Failed to execute network driver operation: pre.
[Z0][VMM][D]: Message received: DEPLOY FAILURE 52  vxlan: vxlan: destination port not specified
[Z0][VMM][D]: Message received: Will use Linux kernel default (non-standard value)
[Z0][VMM][D]: Message received: Use 'dstport 4789' to get the IANA assigned value
[Z0][VMM][D]: Message received: Use 'dstport 0' to get default and quiet this message
[Z0][VMM][D]: Message received: RTNETLINK answers: File exists

Unsurprisingly

[oneadmin@localhost ~]$ sudo /var/tmp/one/vnm/vxlan/pre PFZNPjxJRD41MjwvSUQ+...
ERROR: pre: Command "sudo ip link add enp5s0f1.2 mtu 1460 type vxlan id 2 group 239.0.0.2 ttl 16 dev enp5s0f1" failed.
ERROR: pre: vxlan: destination port not specified
Will use Linux kernel default (non-standard value)
Use 'dstport 4789' to get the IANA assigned value
Use 'dstport 0' to get default and quiet this message
RTNETLINK answers: File exists
ERROR MESSAGE --8<------
vxlan: destination port not specified
Will use Linux kernel default (non-standard value)
Use 'dstport 4789' to get the IANA assigned value
Use 'dstport 0' to get default and quiet this message
RTNETLINK answers: File exists
ERROR MESSAGE ------>8--

[oneadmin@localhost ~]$ sudo ip link add enp5s0f1.2 mtu 1460 type vxlan id 2 group 239.0.0.2 ttl 16 dev enp5s0f1
vxlan: destination port not specified
Will use Linux kernel default (non-standard value)
Use 'dstport 4789' to get the IANA assigned value
Use 'dstport 0' to get default and quiet this message
RTNETLINK answers: File exists
[oneadmin@localhost ~]$ sudo ip link add enp5s0f1.2 mtu 1460 type vxlan id 2 group 239.0.0.2 ttl 16 dev enp5s0f1 dstport 0
RTNETLINK answers: File exists

what now? Using

BRIDGE = "br0"
DNS = "10.10.10.1"
GATEWAY = "10.10.10.1"
GUEST_MTU = "1460"
NETWORK_ADDRESS = "10.10.10.0"
NETWORK_MASK = "255.0.0.0"
PHYDEV = ""
SECURITY_GROUPS = "0"
VN_MAD = "ovswitch"

yields

[Z0][VMM][D]: Message received: LOG I 52 Command execution fail: /var/tmp/one/vnm/ovswitch/post PFZNPjxJRD41MjwvSUQ+... '/dev/loop1
[Z0][VMM][D]: Message received: LOG I 52 one-52'
[Z0][VMM][D]: Message received: LOG I 52 No tap device found for nic 0
[Z0][VMM][D]: Message received: LOG I 52 ExitCode: 1
[Z0][ReM][D]: Req:6720 UID:0 VirtualMachineInfo invoked , 52
[Z0][ReM][D]: Req:6720 UID:0 VirtualMachineInfo result SUCCESS, "<VM><ID>52</ID><UID>..."
[Z0][InM][D]: Host head1 (2) successfully monitored.
[Z0][ReM][D]: Req:4064 UID:0 VirtualMachineMonitoring invoked , 52
[Z0][ReM][D]: Req:4064 UID:0 VirtualMachineMonitoring result SUCCESS, "<MONITORING_DATA><VM..."
[Z0][VMM][D]: Message received: LOG I 52 ExitCode: 0
[Z0][VMM][D]: Message received: LOG I 52 Successfully execute virtualization driver operation: cancel.
[Z0][VMM][D]: Message received: LOG I 52 Failed to execute network driver operation: post.
[Z0][VMM][D]: Message received: DEPLOY FAILURE 52  ovswitch: -

regardless of whether I put enp5s0f1 as PHYDEV.

I must not have understood something, please point out the right direction.

EDIT: I tried some more:

[oneadmin@localhost ~]$ ip -d  link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 addrgenmode eui64
2: enp5s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
link/ether 00:15:17:89:54:88 brd ff:ff:ff:ff:ff:ff promiscuity 0 addrgenmode eui64
3: enp5s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP mode DEFAULT qlen 1000
link/ether 00:15:17:89:54:89 brd ff:ff:ff:ff:ff:ff promiscuity 1 addrgenmode eui64
5: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1460 qdisc noqueue state DOWN mode DEFAULT
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff promiscuity 0
bridge addrgenmode eui64
6: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast state DOWN mode DEFAULT qlen 500
link/ether 52:54:00:2a:4b:55 brd ff:ff:ff:ff:ff:ff promiscuity 0
tun addrgenmode eui64
8: enp5s0f1.2: <BROADCAST,MULTICAST> mtu 1460 qdisc noqueue state DOWN mode DEFAULT
link/ether 4a:11:46:c2:71:11 brd ff:ff:ff:ff:ff:ff promiscuity 0
vxlan id 2 group 239.0.0.2 dev enp5s0f1 srcport 0 0 dstport 8472 ttl 16 ageing 300 addrgenmode eui64
20: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether 00:15:17:89:54:89 brd ff:ff:ff:ff:ff:ff promiscuity 0
bridge addrgenmode eui64
[oneadmin@localhost ~]$ sudo ip link del virbr0-nic
[oneadmin@localhost ~]$ sudo ip link del enp5s0f1.2
[oneadmin@localhost ~]$ ip -d  link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 promiscuity 0 addrgenmode eui64
2: enp5s0f0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT qlen 1000
link/ether 00:15:17:89:54:88 brd ff:ff:ff:ff:ff:ff promiscuity 0 addrgenmode eui64
3: enp5s0f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master br0 state UP mode DEFAULT qlen 1000
link/ether 00:15:17:89:54:89 brd ff:ff:ff:ff:ff:ff promiscuity 1 addrgenmode eui64
5: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1460 qdisc noqueue state DOWN mode DEFAULT
link/ether 00:00:00:00:00:00 brd ff:ff:ff:ff:ff:ff promiscuity 0
bridge addrgenmode eui64
20: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT
link/ether 00:15:17:89:54:89 brd ff:ff:ff:ff:ff:ff promiscuity 0
bridge addrgenmode eui64

deployment with vxlan regenerates enp5s0f1.2 and results in the same errors.
A /var/spool/mail arrived with the following:

sudo: svncterm: command not found
[...]
sed: -e expression #1, char 1: unknown command: `,'

but I know as little as ever what to do

It seems that the tagged interface cannot be created, please try to removed
the br0, and start the tests in a clean status (or host) (i.e. without the
bridged configuration).

the vxlan enp5s0f1.ID ie 2 does get created both with my linux br0 and the virsh NAT virbr0.
would i remove br0 and still put it in the template? as I understood, I need a linux bridge like virsh’s created first and put that into the template for vxlan. Can you try to explain what the thing is trying to do or where I can find more detailed logs?

I got a step further when I generated an ovs bridge (instead of a linux bridge as virsh does). opennebula or whichever part of it was able to generate taps in that. with that come new problems, windows virtio nic vms getting absurd IPs from who knows where while linux vms and lxc connect fine. I guess the urgency is out of the problem, it’s not solved but I got around it. I also built svncterm taking care of the first of the two errors.

Is there any more detailed resource than the docs for me to learn from what should be going on?