Native nft Security Groups

[divanikus]

Hello.
I have ported iptables security groups to newer nftables. All types of filters are supported.
All operations are made on bridge family table, so unfortunatelly the resulting ruleset is stateless.

table bridge filter {
        chain INPUT {
                type filter hook input priority -200; policy accept;
                iifname "one-0-0" jump one-0-0-o
        }

        chain FORWARD {
                type filter hook forward priority -200; policy accept;
                oifname "one-0-0" jump one-0-0-i
                iifname "one-0-0" jump one-0-0-o
        }

        chain OUTPUT {
                type filter hook output priority -200; policy accept;
                oifname "one-0-0" jump one-0-0-i
        }

        chain one-0-0-i {
                ether type arp counter packets 4 bytes 112 accept
                icmpv6 type nd-router-advert counter packets 0 bytes 0 accept
                icmpv6 type nd-neighbor-solicit counter packets 6 bytes 432 accept
                icmpv6 type nd-neighbor-advert counter packets 0 bytes 0 accept
                icmpv6 type nd-redirect counter packets 0 bytes 0 accept
                counter packets 400 bytes 22226 return
                counter packets 0 bytes 0 drop
        }

        chain one-0-0-o {
                ether type arp counter packets 4 bytes 112 accept
                icmpv6 type nd-router-solicit counter packets 0 bytes 0 accept
                icmpv6 type nd-neighbor-solicit counter packets 0 bytes 0 accept
                icmpv6 type nd-neighbor-advert counter packets 0 bytes 0 accept
                ether saddr != 02:00:c0:a8:05:03 counter packets 0 bytes 0 drop
                counter packets 51 bytes 2566 return
                counter packets 0 bytes 0 drop
        }
}

Is anyone interested in such integration? How to properly integrate it with Opennebula?

[divanikus]

# -------------------------------------------------------------------------- #
# Copyright 2002-2019, OpenNebula Project, OpenNebula Systems                #
#                                                                            #
# Licensed under the Apache License, Version 2.0 (the "License"); you may    #
# not use this file except in compliance with the License. You may obtain    #
# a copy of the License at                                                   #
#                                                                            #
# http://www.apache.org/licenses/LICENSE-2.0                                 #
#                                                                            #
# Unless required by applicable law or agreed to in writing, software        #
# distributed under the License is distributed on an "AS IS" BASIS,          #
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.   #
# See the License for the specific language governing permissions and        #
# limitations under the License.                                             #
#--------------------------------------------------------------------------- #

module VNMMAD

# This module implements the SecurityGroup abstraction on top of nft
module SGNft

    ############################################################################
    # A Rule implemented with the nft Linux kernel facilities
    ############################################################################
    class RuleNft < VNMNetwork::Rule
        ########################################################################
        # Implementation of each rule type
        ########################################################################
        private

        # Implements the :protocol rule. Example:
        #   nft add rule ip filter one-3-0-i meta l4proto tcp counter return
        def process_protocol(cmds, vars)
            chain = @rule_type == :inbound ? vars[:chain_in] : vars[:chain_out]

            proto = @protocol == :all ? "" : "meta l4proto #{@protocol}"
            cmds.add :nft, "add rule bridge #{FILTER_TABLE} #{chain} #{proto} counter return"
        end

        # Implements the :portrange rule. Example:
        #   nft add rule ip filter one-3-0-o meta l4proto udp udp dport { 80,22 } counter return
        def process_portrange(cmds, vars)
            chain = @rule_type == :inbound ? vars[:chain_in] : vars[:chain_out]

            range = @range.gsub(/:/, '-')
            cmds.add :nft, "add rule bridge #{FILTER_TABLE} #{chain} meta l4proto #{@protocol}" \
                " #{@protocol} dport { #{range} } counter return"
        end

        # Implements the :icmp_type rule. Example:
        #   nft add rule ip filter one-3-0-o icmp type 8 counter return
        def process_icmp_type(cmds, vars)
            chain = @rule_type == :inbound ? vars[:chain_in] : vars[:chain_out]

            cmds.add :nft, "add rule bridge #{FILTER_TABLE} #{chain} icmp type #{@icmp_type}"\
                " counter return"
        end

        # Implements the :icmpv6_type rule. Example:
        #   nft add rule ip6 filter one-3-0-o meta l4proto ipv6-icmp icmpv6 type 128 counter return
        def process_icmpv6_type(cmds, vars)
            chain = @rule_type == :inbound ? vars[:chain_in] : vars[:chain_out]

            cmds.add :nft, "add rule bridge #{FILTER_TABLE} #{chain} meta l4proto ipv6-icmp icmpv6 type "\
                "#{@icmpv6_type} counter return"
        end

        # Implements the :net rule. Example:
        #   nft add rule bridge filter one-3-0-i meta l4proto tcp \
        #       ip saddr { 10.0.0.0/24 } counter return
        #       comment \"one-3-0-1-i-tcp-n-ip\"
        def process_net(cmds, vars)
            the_nets = net()

            return if the_nets.empty?

            sets  = {}
            proto = @protocol == :all ? "" : "meta l4proto #{@protocol}"

            the_nets.each do |n|
                if IPAddr.new(n).ipv6?
                    family  = "ip6"
                else
                    family  = "ip"
                end

                if @rule_type == :inbound
                    chain = vars[:chain_in]
                    set = "#{vars[:set_sg_in]}-#{@protocol}-n-#{family}"
                    dir = "#{family} saddr"
                else
                    chain = vars[:chain_out]
                    set = "#{vars[:set_sg_out]}-#{@protocol}-n-#{family}"
                    dir = "#{family} daddr"
                end

                sets[set] = {net: [], chain: chain, dir: dir} unless sets.key?(set)
                sets[set][:net].push(n)
            end

            sets.each do |k, v|
                cmds.add :nft, "add rule bridge #{FILTER_TABLE} #{v[:chain]} "\
                    "#{proto} #{v[:dir]} { #{v[:net].join(', ')} } "\
                    "counter return comment \\\"#{k}\\\""
            end
        end

        # Implements the :net_portrange rule. Example:
        #   nft add rule bridge filter one-3-0-i meta l4proto tcp \
        #       ip saddr { 10.0.0.0/24 } tcp dport { 80 } \
        #       counter return comment \"one-3-0-1-i-nr-ip\"
        def process_net_portrange(cmds, vars)
            the_nets = net()

            return if the_nets.empty?

            sets  = {}

            the_nets.each do |n|
                if IPAddr.new(n).ipv6?
                    family  = "ip6"
                else
                    family  = "ip"
                end

                if @rule_type == :inbound
                    chain = vars[:chain_in]
                    set = "#{vars[:set_sg_in]}-nr-#{family}"
                    dir = "#{family} saddr"
                else
                    chain = vars[:chain_out]
                    set = "#{vars[:set_sg_out]}-nr-#{family}"
                    dir = "#{family} daddr"
                end

                sets[set] = {net: [], chain: chain, dir: dir} unless sets.key?(set)
                sets[set][:net].push(n)
            end

            range = @range.gsub(/:/, '-')

            sets.each do |k, v|
                cmds.add :nft, "add rule bridge #{FILTER_TABLE} #{v[:chain]} "\
                    "meta l4proto #{@protocol} #{v[:dir]} { #{v[:net].join(', ')} } "\
                    "#{@protocol} dport { #{range} } "\
                    "counter return comment \\\"#{k}\\\""
            end
        end

        # Implements the :net_icmp_type rule. Example:
        #   nft add rule bridge filter one-3-0-i meta l4proto icmp \
        #       ip saddr { 10.0.0.0/24 } icmp type 8 \
        #       icmp code { 0 } counter return \
        #       comment \"one-3-0-1-i-ni\"
        def process_net_icmp_type(cmds, vars)
            if @rule_type == :inbound
                chain = vars[:chain_in]
                set = "#{vars[:set_sg_in]}-ni"
                dir = "ip saddr"
            else
                chain = vars[:chain_out]
                set = "#{vars[:set_sg_out]}-ni"
                dir = "ip daddr"
            end

            the_nets = net()
            return if the_nets.empty?

            codes = ICMP_TYPES_EXPANDED[@icmp_type.to_i]
            codes = "0" if codes.nil?

            cmds.add :nft, "add rule bridge #{FILTER_TABLE} #{chain} "\
                "meta l4proto icmp #{dir} { #{the_nets.join(', ')} } "\
                "icmp type #{@icmp_type} icmp code { #{codes.join(', ')} } "\
                "counter return comment \\\"#{set}\\\""
        end

        # Implements the :net_icmpv6_type rule. Example:
        #   nft add rule bridge filter one-3-0-i meta l4proto ipv6-icmp \
        #       ip6 saddr { fd00::/64 } icmpv6 type 128 \
        #       icmpv6 code { 0 } counter return \
        #       comment \"one-3-0-1-i-ni6\"
        def process_net_icmpv6_type(cmds, vars)
            if @rule_type == :inbound
                chain = vars[:chain_in]
                set = "#{vars[:set_sg_in]}-ni6"
                dir = "ip6 saddr"
            else
                chain = vars[:chain_out]
                set = "#{vars[:set_sg_out]}-ni6"
                dir = "ip6 daddr"
            end

            the_nets = net()
            return if the_nets.empty?

            codes = ICMPv6_TYPES_EXPANDED[@icmpv6_type.to_i]
            codes = "0" if codes.nil?

            cmds.add :nft, "add rule bridge #{FILTER_TABLE} #{chain} "\
                "meta l4proto ipv6-icmp #{dir} { #{the_nets.join(', ')} } "\
                "icmpv6 type #{@icmpv6_type} icmpv6 code { #{codes.join(', ')} } "\
                "counter return comment \\\"#{set}\\\""
        end
    end

    ############################################################################
    # This class represents a SecurityGroup implemented with nft
    # Kernel facilities.
    ############################################################################
    class SecurityGroupNft < VNMNetwork::SecurityGroup
        def initialize(vm, nic, sg_id, rules)
            super

            @vars = SGNft.vars(@vm, @nic, @sg_id)
        end

        def new_rule(rule)
            RuleNft.new(rule)
        end
    end

    ############################################################################
    # Methods to configure the hypervisor nft rules. All the rules are
    # added to the FILTER_TABLE table. By default this chain is "filter"
    ############################################################################

    FILTER_TABLE  = "filter"
    INPUT_CHAIN   = "INPUT"
    FORWARD_CHAIN = "FORWARD"
    OUTPUT_CHAIN  = "OUTPUT"

    # Get information from the current iptables rules and chains
    #   @return [Hash] with the following keys:
    #     - :forwards
    #     - :chains
    def self.info
        commands = VNMNetwork::Commands.new

        chains   = []
        forwards = []

        commands.add :nft, "list table bridge #{FILTER_TABLE} -a"
        table = commands.run!
        table.split("\n").each do |x| 
            x.match(/chain (\S+?) {/) { |m| chains.push(m[1]) }
            x.match(/ jump (\S+?) .*# handle (\d+)/) { |m| forwards.push([chains.last, m[1], m[2]]) }
            x.match(/set (\S+?) {/) { |m| sets.push(m[1]) }
        end

        {
            :forwards => forwards,
            :chains => chains,
        }
    end

    # Bootstrap the OpenNebula chains and rules. This method:
    #   1.- Creates the INPUT_CHAIN, FORWARD_CHAIN and OUTPUT_CHAIN chains
    #   2.- By default ACCEPT all traffic
    def self.global_bootstrap
        info = SGNft.info

        commands = VNMNetwork::Commands.new

        if !info[:chains].include?('INPUT')
            commands.add :nft, "add chain bridge #{FILTER_TABLE} #{INPUT_CHAIN} "\
                "{ type filter hook input priority -200\; policy accept\; }"
        end

        if !info[:chains].include?('FORWARD')
            commands.add :nft, "add chain bridge #{FILTER_TABLE} #{FORWARD_CHAIN} "\
                "{ type filter hook forward priority -200\; policy accept\; }"
        end

        if !info[:chains].include?('OUTPUT')
            commands.add :nft, "add chain bridge #{FILTER_TABLE} #{OUTPUT_CHAIN} "\
                "{ type filter hook output priority -200\; policy accept\; }"
        end

        commands.run! if !commands.empty?
    end

    # Returns the base chain and ipset names for the VM
    #   @param vm  [VM] the virtual machine
    #   @param nic [Nic] of the VM
    #   @param sg_id [Fixnum] ID of the SecurityGroup if any
    #
    #   @return [Hash] with the :chain, :chain_in, :chain_out chain names, and
    #   :set_sg_in and :set_seg_out ipset names.
    def self.vars(vm, nic, sg_id = nil)
        vm_id  = vm['ID']
        nic_id = nic[:nic_id]

        vars = {}

        vars[:nic]       = nic
        vars[:vm_id]     = vm_id
        vars[:nic_id]    = nic_id
        vars[:chain]     = "one-#{vm_id}-#{nic_id}"
        vars[:chain_in]  = "#{vars[:chain]}-i"
        vars[:chain_out] = "#{vars[:chain]}-o"

        if sg_id
            vars[:set_sg_in]  = "#{vars[:chain]}-#{sg_id}-i"
            vars[:set_sg_out] = "#{vars[:chain]}-#{sg_id}-o"
        end

        vars
    end

    #  Bootstrap NIC rules for the interface. It creates the :chain_in and
    #  :chain_out and sets up FORWARD rules to these chains for inbound and
    #  outbound traffic.
    #
    #  This method also sets mac_spoofing, and ip_spoofing rules
    #
    #  Example, for VM 3 and NIC 0
    #   nft add chain bridge filter one-3-0-i
    #   nft add chain bridge filter one-3-0-o
    #
    #   nft insert rule bridge filter INPUT iifname "vnet0" jump one-3-0-o
    #   nft insert rule bridge filter FORWARD iifname "vnet0" jump one-3-0-o
    #   nft insert rule bridge filter FORWARD oifname "vnet0" jump one-3-0-i
    #   nft insert rule bridge filter OUTPUT oifname "vnet0" jump one-3-0-i
    #
    #   Mac spoofing (no output traffic from a different MAC)
    #   nft add rule bridge filter one-3-0-o ether saddr != 02:00:00:00:00:01 counter drop
    #
    #   IP spoofing
    #   nft add rule bridge filter one-3-0-o ip saddr != { 10.0.0.1 } counter drop
    def self.nic_pre(vm, nic)
        commands = VNMNetwork::Commands.new

        vars = SGNft.vars(vm, nic)

        chain_in  = vars[:chain_in]
        chain_out = vars[:chain_out]

        # create chains
        commands.add :nft, "add chain bridge #{FILTER_TABLE} #{chain_in}"  # inbound
        commands.add :nft, "add chain bridge #{FILTER_TABLE} #{chain_out}" # outbound

        # Send traffic to the NIC chains
        commands.add :nft, "insert rule bridge #{FILTER_TABLE} #{INPUT_CHAIN} "\
            "iifname \"#{nic[:tap]}\" jump #{chain_out}"
        commands.add :nft, "insert rule bridge #{FILTER_TABLE} #{FORWARD_CHAIN} "\
            "iifname \"#{nic[:tap]}\" jump #{chain_out}"
        commands.add :nft, "insert rule bridge #{FILTER_TABLE} #{FORWARD_CHAIN} "\
            "oifname \"#{nic[:tap]}\" jump #{chain_in}"
        commands.add :nft, "insert rule bridge #{FILTER_TABLE} #{OUTPUT_CHAIN} "\
            "oifname \"#{nic[:tap]}\" jump #{chain_in}"

        # ARP for IPv4 (since we are on the bridge level)
        commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_in}  ether type arp counter accept"
        commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_out} ether type arp counter accept"
        
        # ICMPv6 Neighbor Discovery Protocol (ARP replacement for IPv6)
        ## Allow routers to send router advertisements
        commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_in} meta l4proto ipv6-icmp icmpv6 "\
            "type nd-router-advert counter accept"

        ## Allow neighbor solicitations to reach the host
        commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_in} meta l4proto ipv6-icmp icmpv6 "\
            "type nd-neighbor-solicit counter accept"

        ## Allow neighbor solicitations replies to reach the host
        commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_in} meta l4proto ipv6-icmp icmpv6 "\
            "type nd-neighbor-advert counter accept"

        ## Allow routers to send Redirect messages
        commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_in} meta l4proto ipv6-icmp icmpv6 "\
            "type nd-redirect counter accept"

        ## Allow the host to send a router solicitation
        commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_out} meta l4proto ipv6-icmp icmpv6 "\
            "type nd-router-solicit counter accept"

        ## Allow the host to send neighbor solicitation requests
        commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_out} meta l4proto ipv6-icmp icmpv6 "\
            "type nd-neighbor-solicit counter accept"

        ## Allow the host to send neighbor solicitation replies
        commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_out} meta l4proto ipv6-icmp icmpv6 "\
            "type nd-neighbor-advert counter accept"

        # Mac-spofing
        if nic[:filter_mac_spoofing] == "YES"
            commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_out} "\
                "ether saddr != #{nic[:mac]} counter drop"
        end

        # IP-spofing
        if nic[:filter_ip_spoofing] == "YES"
            ipv4s = Array.new

            [:ip, :vrouter_ip].each do |key|
                ipv4s << nic[key] if !nic[key].nil? && !nic[key].empty?
            end

            if !ipv4s.empty?
                #bootp
                commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_out} "\
                    "ip daddr 255.255.255.255 udp sport 68 udp dport 67 "\
                    "counter return"

                commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_out} ip saddr != "\
                    "{ #{ipv4s.join(', ')} } counter drop"
            else # If there are no IPv4 addresses allowed, block all
                commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_out} counter "\
                    "drop"
            end

            ipv6s = Array.new

            [:ip6, :ip6_global, :ip6_link, :ip6_ula].each do |key|
                ipv6s << nic[key] if !nic[key].nil? && !nic[key].empty?
            end

            if !ipv6s.empty?
                commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_out} ip6 saddr != "\
                    "{ #{ipv6s.join(', ')} } counter drop"
            else # If there are no IPv6 addresses allowed, block all
                commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_out} counter drop"
            end
        end

        # ct is not supported by table bridge
        # Related, Established
        #commands.add :nft, "add rule ip #{FILTER_TABLE} #{chain_in}"\
        #    " ct state related,established counter return"
        #commands.add :nft, "add rule ip #{FILTER_TABLE} #{chain_out}"\
        #    " ct state related,established counter return"
        #commands.add :nft, "add rule ip6 #{FILTER_TABLE} #{chain_in}"\
        #    " ct state related,established counter return"
        #commands.add :nft, "add rule ip6 #{FILTER_TABLE} #{chain_out}"\
        #    " ct state related,established counter return"

        commands.run!
    end

    # Sets the default policy to DROP for the NIC rules. Example
    #   nft add rule bridge filter one-3-0-i counter drop
    #   nft add rule bridge filter one-3-0-o counter drop
    def self.nic_post(vm, nic)
        vars      = SGNft.vars(vm, nic)
        chain_in  = vars[:chain_in]
        chain_out = vars[:chain_out]

        commands = VNMNetwork::Commands.new

        commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_in} counter drop"
        commands.add :nft, "add rule bridge #{FILTER_TABLE} #{chain_out} counter drop"

        commands.run!
    end

    # Removes all the rules associated to a VM and NIC
    def self.nic_deactivate(vm, nic)
        vars      = SGNft.vars(vm, nic)
        chain     = vars[:chain]
        chain_in  = vars[:chain_in]
        chain_out = vars[:chain_out]

        info = self.info

        forwards        = info[:forwards]
        chains          = info[:chains]

        commands = VNMNetwork::Commands.new

        forwards.reverse_each do |fields|
            if [chain_in, chain_out].include?(fields[1])
                c = fields[0]
                n = fields[2]
                commands.add :nft, "delete rule bridge #{FILTER_TABLE} #{c} handle #{n}"
            end
        end

        remove_chains = chains.grep(/^#{chain}(-|$)/)
        remove_chains.each {|c| commands.add :nft, "flush chain bridge #{FILTER_TABLE} #{c}" }
        remove_chains.each {|c| commands.add :nft, "delete chain bridge #{FILTER_TABLE} #{c}" }

        commands.run!
    end
end

end

[ruben (Rubén S. Montero)]

That’s awesome, Woud you mind to open a PR at https://github.com/OpenNebula/one/pulls

In any case Thanks!!!

[Nico_Schottelius]

Just joining the discussion here, as I created https://github.com/OpenNebula/one/issues/4739. Essentially it seems that OpenNebula’s security groups fail on Debian 10, because Debian 10 replaced ip(6)tables with nft.

Is there any public progress of above patch?