Minione 802.1Q network: VM can't access internet

vdjurovic · November 7, 2020, 4:26pm

I’ve set up a 802.1Q virtual network, but when I deploy VM to it, it can’t reach internet. Here is my setup:

kernel module is loaded

lsmod | grep 802
8021q 40960 0
garp 16384 1 8021q
mrp 20480 1 8021q

Network configuration is as follows

VIRTUAL NETWORK 4 INFORMATION                                                   
ID                       : 4                   
NAME                     : test-net-001        
USER                     : oneadmin            
GROUP                    : oneadmin            
LOCK                     : None                
CLUSTERS                 : 0                   
BRIDGE                   : onebr4              
VN_MAD                   : 802.1Q              
PHYSICAL DEVICE          : enp0s31f6           
VLAN ID                  : 6                   
AUTOMATIC VLAN ID        : YES                 
AUTOMATIC OUTER VLAN ID  : NO                  
USED LEASES              : 0

PERMISSIONS
OWNER                    : um-
GROUP                    : ---
OTHER                    : ---

VIRTUAL NETWORK TEMPLATE
BRIDGE="onebr4"
BRIDGE_TYPE="linux"
DNS="10.0.5.1"
FILTER_IP_SPOOFING="YES"
FILTER_MAC_SPOOFING="YES"
GATEWAY="10.0.5.1"
GUEST_MTU="1500"
NETWORK_ADDRESS="10.0.5.255"
NETWORK_MASK="255.255.255.0"

OUTER_VLAN_ID=""
PHYDEV=“enp0s31f6”
SECURITY_GROUPS=“0”
VN_MAD=“802.1Q”

ADDRESS RANGE POOL
AR 0
SIZE           : 100
LEASES         : 0

RANGE                                   FIRST                               LAST
MAC                         02:00:0a:00:05:02                  02:00:0a:00:05:65
IP                                   10.0.5.2                         10.0.5.101

When I deploy VM, it shows correct IP address and routing information obtained from context:

ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default     qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
   valid_lft forever preferred_lft forever
inet6 ::1/128 scope host 
   valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 02:00:0a:00:05:02 brd ff:ff:ff:ff:ff:ff
inet 10.0.5.2/24 brd 10.0.5.255 scope global ens3
   valid_lft forever preferred_lft forever
inet6 fe80::aff:fe00:502/64 scope link 
   valid_lft forever preferred_lft forever


ip route
default via 10.0.5.1 dev ens3 onlink 
10.0.5.0/24 dev ens3 proto kernel scope link src 10.0.5.2 
169.254.0.0/16 dev ens3 scope link metric 1000

But, if I try to ping default gateway, it is not reachable:

ping 10.0.5.1
PING 10.0.5.1 (10.0.5.1) 56(84) bytes of data.
From 10.0.5.2 icmp_seq=1 Destination Host Unreachable
From 10.0.5.2 icmp_seq=2 Destination Host Unreachable
From 10.0.5.2 icmp_seq=3 Destination Host Unreachable

On host, it seems that both bridge and tagged interface are created correctly:

onebr4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet6 fe80::a4ec:72ff:fe0e:396  prefixlen 64  scopeid 0x20<link>
    ether 90:1b:0e:cd:79:32  txqueuelen 1000  (Ethernet)
    RX packets 149  bytes 12268 (11.9 KiB)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 6  bytes 540 (540.0 B)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

enp0s31f6.6: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet6 fe80::921b:eff:fecd:7932  prefixlen 64  scopeid 0x20<link>
    ether 90:1b:0e:cd:79:32  txqueuelen 1000  (Ethernet)
    RX packets 0  bytes 0 (0.0 B)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 139  bytes 12310 (12.0 KiB)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

one-25-0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
    inet6 fe80::fc00:aff:fe00:502  prefixlen 64  scopeid 0x20<link>
    ether fe:00:0a:00:05:02  txqueuelen 1000  (Ethernet)
    RX packets 151  bytes 14422 (14.0 KiB)
    RX errors 0  dropped 0  overruns 0  frame 0
    TX packets 8  bytes 720 (720.0 B)
    TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Now, if I set address 10.0.5.1 to onebr4 interface, then I can ping it from VM. But, I still can’t ping anything on internet.

ping -c 3 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.

--- 8.8.8.8 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2033ms

I checked ip tables config on host, and it seems OK:

iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
f2b-sshd   tcp  --  anywhere             anywhere             multiport dports ssh

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
opennebula  all  --  anywhere             anywhere             PHYSDEV match --physdev-is-bridged

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain f2b-sshd (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere

Chain opennebula (1 references)
target     prot opt source               destination
 one-25-0-o  all  --  anywhere             anywhere             PHYSDEV match --physdev-in one-25-0     --physdev-is-bridged
one-25-0-i  all  --  anywhere             anywhere             PHYSDEV match --physdev-out one-25-0 --physdev-is-bridged
ACCEPT     all  --  anywhere             anywhere
    
Chain one-25-0-i (1 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
RETURN     all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere

Chain one-25-0-o (1 references)
target     prot opt source               destination
DROP       all  --  anywhere             anywhere             MAC ! 02:00:0A:00:05:02
RETURN     udp  --  0.0.0.0              255.255.255.255      udp spt:bootpc dpt:bootps
DROP       all  --  anywhere             anywhere             ! match-set one-25-0-ip-spoofing src
RETURN     all  --  anywhere             anywhere             state RELATED,ESTABLISHED
RETURN     all  --  anywhere             anywhere
DROP       all  --  anywhere             anywhere

And also ebtables:

ebtables -L
Bridge table: filter
Bridge chain: INPUT, entries: 0, policy: ACCEPT
Bridge chain: FORWARD, entries: 0, policy: ACCEPT
Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

So, everything seems fine to me, but VM can’t access internet. I must be missing something, but I can’t figure out what.
Any pointer would be greatly appreciated.

vdjurovic · November 27, 2020, 11:07am

It’s been a while, and I forgot about this topic, but I found the answer in the meantime. Posting it here in case it could help someone.

What I did was the following:

manually assign address 10.0.5.1 to bridge interface (onebr4). This allows VM to ping default gateway
add the following iptables rule:

iptables -t nat -A POSTROUTING -s ‘10.0.5.0/24’ -o enp0s31f6 -j MASQUERADE

This allows VM to ping external IP addresses (eg. 8.8.8.8). But still DNS does not work. To fix, add the following to /etc/dnsmasq.conf file:

interfaces=lo,onebr<ID>

Now full network access is available. One thing I’m not sure about is if this is usual way to get outbound network access, or I’m just doing a hack. So, if anyone has some more insight, please share :-).

Topic		Replies	Views
Minione: Can not ping any host from VM Network solved	6	1649	August 6, 2020
[Solved] Error starting vm with 802.1Q network Community Support	6	1502	December 6, 2016
[Solved] Help with 802.1Q VLAN Community Support	9	5006	February 15, 2018
802.1Q VLAN network setup Community Support	3	1004	February 2, 2018
802.1q network setup option Community Support	0	343	October 21, 2019

Minione 802.1Q network: VM can't access internet

Related topics