Bridge with Security not working?

I cannot get bridge with security to work. However a standard bridge works fine…
If I set vn_mad to fw no traffic can get through. e.g. ping google.com. However if it is set to bridge it works fine.

I have OpenNebula running on Ubuntu Server 20.04

My netplan is:

network:
  ethernets:
    eno2:
      dhcp4: false
  version: 2
  bridges:
    eno2br:
      dhcp4: yes
      interfaces:
        - eno2
      addresses: [192.168.1.2/24]
      gateway4: 192.168.1.254
      nameservers:
        addresses:
        - 8.8.8.8

public bridge network template with security (not working)

BRIDGE = "eno2br"
BRIDGE_TYPE = "linux"
DNS = "8.8.8.8, 8.8.4.4"
GATEWAY = "192.168.1.254"
NETWORK_ADDRESS = "192.168.1.2"
NETWORK_MASK = "255.255.255.0"
OUTER_VLAN_ID = ""
PHYDEV = ""
SECURITY_GROUPS = "0"
VLAN_ID = ""
VN_MAD = "fw"

Default security template

RULE = [
  PROTOCOL = "TCP",
  RULE_TYPE = "inbound" ]
RULE = [
  PROTOCOL = "TCP",
  RULE_TYPE = "outbound" ]

However this template does work

BRIDGE = "eno2br"
BRIDGE_TYPE = "linux"
DNS = "8.8.8.8, 8.8.4.4"
GATEWAY = "192.168.1.254"
NETWORK_ADDRESS = "192.168.1.2"
NETWORK_MASK = "255.255.255.0"
OUTER_VLAN_ID = ""
PHYDEV = ""
SECURITY_GROUPS = "0"
VLAN_ID = ""
VN_MAD = "bridge"

Any advise on what may be wrong would be greatly appreciated.

Hi @Daryn_L ,

Take a look at this. In your case, the private network is 192.168.1.0/24.

Cheers.

Hi @rdiaz,

Thanks for your reply.

Following your answer allows my VM to ping google. However the security groups do not work. e.g. if I delete both rules in the default group I can still ping google from within the VM.
Furthermore, my VM cannot ping other VMs either.

I have decided to try and go through the minione setup again. Maybe there is some step missing…

Here are the exact (and only) steps I have done:

  1. Fresh install ubuntu server 20.04 with openssh installed
  2. Modified my netplan to set a static IP to my router
network:
  version: 2
  ethernets:
    eno1:
      dhcp4: no
      addresses:
        - 192.168.1.100/24
      gateway4: 192.168.1.254
      nameservers:
          addresses: [8.8.8.8, 8.8.4.4]

  1. update, upgrade and reboot
  2. sudo -s && cd
    ufw allow 80/tcp
    ufw allow 22/tcp
    ufw allow 2616/tcp
  3. wget ‘https://github.com/OpenNebula/minione/releases/latest/download/minione
  4. bash minione
### Main deployment steps:
Install OpenNebula frontend version 6.2
Install Terraform
Install Docker
Configure bridge minionebr with IP 172.16.100.1/24
Enable NAT over eno1
Modify AppArmor
Install OpenNebula KVM node
Export appliance and update VM template
Install  augeas-tools bridge-utils apt-transport-https iptables-persistent netfilter-persistent python3-pip unzip
Install pip 'ansible==2.9.9'

Everything ran OK

  1. login sunstone

  2. instantiate 2x instances of centos 7

  3. Test ping:
    instances can ping each other (172.16.100.3 from 172.16.100.2)
    instances can ping google.com

  4. network → security groups → default
    delete both default rules and update and commit
    onehost sync -f

  5. Test ping:
    can ping google.com (would presume I shouldn’t be able to?)
    cannot ping other vm (seems correct)

  6. network → security groups → default
    recreate both default ANY rules and update and commit

    onehost sync -f

  7. Test ping:
    can ping google.com
    cannot ping other vm (this should now work again?) (172.16.100.3 from 172.16.100.2)

  8. As per your suggestion
    as root:
    iptables -t nat -A POSTROUTING -s 172.16.100.0/24 ! -d 172.16.100.0/24 -j MASQUERADE

  9. Test ping:
    can ping google.com
    cannot ping other vm (172.16.100.3 from 172.16.100.2)

  10. delete VMs

  11. create 2x new VMs the same

  12. Test ping:
    can ping google.com
    cannot ping other vm (172.16.100.3 from 172.16.100.2)

As soon as I make a change to the security group in sunstone it basically breaks all communication between VMs. Security groups are also not working still…

  1. reboot
  2. Test ping:
    can ping google.com
    cannot ping other vm (172.16.100.3 from 172.16.100.2)

So basically, for me, tampering with the security groups breaks the whole system.

Any thoughts?

Best,
Daryn