Using virtual (bridge) networks over vms in different locations

Community Support Network
1

Hi out there. I just to make sure that I do not oversee somthing yet.

We have a few bare metal machine on different providers like Hetzner, OVH, AWS, ScaleWay. I have created several KVM nodes located in these data centers and have connected them all into a FireEdge Management. Currently I use only “private networks” in each host, with the host as a router and NAT and/or internal VPN. To allow “live migrations” I run the same virtual network on each host, so that the IP address of the VM is still the same when migrating. Due to the fact, that each host has the same onbr0 address and function as a gateway, also traffic continues when migrating.

The fact not using a bridged virtual networ is, that those bare metal achines have only one NIC and that is attached to the WAN and mostly configured by a MAC address, that this is the only address allowed to send data out. This is a rule by most providers. If you miss this, they should normally down their Port and you receive a message like “You are using not allowed traffic”.

So for that I do not see another option for in and out traffic than using the host as a gateway as well.

Do I miss some concepts here for running one network over public NICs and different locations?

Thanks for your replies.
Tom

2

Hello!
Your approach seems solid given the cloud provider constraints. The only limitation I can think of is the connectivity between VMs that, despite being attached to the same vNet, are deployed in different locations. If this isn’t an issue, your approach should work well.

If it is a problem, one alternative is to define an MP-BGP EVPN VXLAN vNet and use your OpenNebula FrontEnd as a BGP Route Reflector. This is a fairly complex network setup and can be difficult to troubleshoot if something goes wrong. There is some documentation explaining how it works:https://docs.opennebula.io/6.10/open_cluster_deployment/networking_setup/vxlan.html#using-vxlan-with-bgp-evpn

Additionally, its deployment can be automated using the One-Deploy Ansible playbooks: https://github.com/OpenNebula/one-deploy/wiki/arch_evpn

Cheers,
Daniel