Bridge with Security not working?

I cannot get bridge with security to work. However a standard bridge works fine…
If I set vn_mad to fw no traffic can get through. e.g. ping google.com. However if it is set to bridge it works fine.

I have OpenNebula running on Ubuntu Server 20.04

My netplan is:

network:
  ethernets:
    eno2:
      dhcp4: false
  version: 2
  bridges:
    eno2br:
      dhcp4: yes
      interfaces:
        - eno2
      addresses: [192.168.1.2/24]
      gateway4: 192.168.1.254
      nameservers:
        addresses:
        - 8.8.8.8

public bridge network template with security (not working)

BRIDGE = "eno2br"
BRIDGE_TYPE = "linux"
DNS = "8.8.8.8, 8.8.4.4"
GATEWAY = "192.168.1.254"
NETWORK_ADDRESS = "192.168.1.2"
NETWORK_MASK = "255.255.255.0"
OUTER_VLAN_ID = ""
PHYDEV = ""
SECURITY_GROUPS = "0"
VLAN_ID = ""
VN_MAD = "fw"

Default security template

RULE = [
  PROTOCOL = "TCP",
  RULE_TYPE = "inbound" ]
RULE = [
  PROTOCOL = "TCP",
  RULE_TYPE = "outbound" ]

However this template does work

BRIDGE = "eno2br"
BRIDGE_TYPE = "linux"
DNS = "8.8.8.8, 8.8.4.4"
GATEWAY = "192.168.1.254"
NETWORK_ADDRESS = "192.168.1.2"
NETWORK_MASK = "255.255.255.0"
OUTER_VLAN_ID = ""
PHYDEV = ""
SECURITY_GROUPS = "0"
VLAN_ID = ""
VN_MAD = "bridge"

Any advise on what may be wrong would be greatly appreciated.

Hi @Daryn_L ,

Take a look at this. In your case, the private network is 192.168.1.0/24.

Cheers.

Hi @rdiaz,

Thanks for your reply.

Following your answer allows my VM to ping google. However the security groups do not work. e.g. if I delete both rules in the default group I can still ping google from within the VM.
Furthermore, my VM cannot ping other VMs either.

I have decided to try and go through the minione setup again. Maybe there is some step missing…

Here are the exact (and only) steps I have done:

  1. Fresh install ubuntu server 20.04 with openssh installed
  2. Modified my netplan to set a static IP to my router
network:
  version: 2
  ethernets:
    eno1:
      dhcp4: no
      addresses:
        - 192.168.1.100/24
      gateway4: 192.168.1.254
      nameservers:
          addresses: [8.8.8.8, 8.8.4.4]

  1. update, upgrade and reboot
  2. sudo -s && cd
    ufw allow 80/tcp
    ufw allow 22/tcp
    ufw allow 2616/tcp
  3. wget ‘https://github.com/OpenNebula/minione/releases/latest/download/minione
  4. bash minione
### Main deployment steps:
Install OpenNebula frontend version 6.2
Install Terraform
Install Docker
Configure bridge minionebr with IP 172.16.100.1/24
Enable NAT over eno1
Modify AppArmor
Install OpenNebula KVM node
Export appliance and update VM template
Install  augeas-tools bridge-utils apt-transport-https iptables-persistent netfilter-persistent python3-pip unzip
Install pip 'ansible==2.9.9'

Everything ran OK

  1. login sunstone

  2. instantiate 2x instances of centos 7

  3. Test ping:
    instances can ping each other (172.16.100.3 from 172.16.100.2)
    instances can ping google.com

  4. network → security groups → default
    delete both default rules and update and commit
    onehost sync -f

  5. Test ping:
    can ping google.com (would presume I shouldn’t be able to?)
    cannot ping other vm (seems correct)

  6. network → security groups → default
    recreate both default ANY rules and update and commit

    onehost sync -f

  7. Test ping:
    can ping google.com
    cannot ping other vm (this should now work again?) (172.16.100.3 from 172.16.100.2)

  8. As per your suggestion
    as root:
    iptables -t nat -A POSTROUTING -s 172.16.100.0/24 ! -d 172.16.100.0/24 -j MASQUERADE

  9. Test ping:
    can ping google.com
    cannot ping other vm (172.16.100.3 from 172.16.100.2)

  10. delete VMs

  11. create 2x new VMs the same

  12. Test ping:
    can ping google.com
    cannot ping other vm (172.16.100.3 from 172.16.100.2)

As soon as I make a change to the security group in sunstone it basically breaks all communication between VMs. Security groups are also not working still…

  1. reboot
  2. Test ping:
    can ping google.com
    cannot ping other vm (172.16.100.3 from 172.16.100.2)

So basically, for me, tampering with the security groups breaks the whole system.

Any thoughts?

Best,
Daryn

Hi @Daryn_L ,

First of all, my apologies, I read your message too fast and understood you couldn’t ping google. You don’t need to explicitly run iptables -t nat -A POSTROUTING -s 172.16.100.0/24 ! -d 172.16.100.0/24 -j MASQUERADE (minione does it for you), and that is done to provide Internet connection to VMs.

Note that in step 12 you set Protocol TCP instead of all (ping is ICMP). If you use set all in Protocol, you’ll see the ping echo replies again in step 13.