VM's Secondary IP address isn't working (solved in general)

Community Support
1

Hello I have this:

Two virtual mashines in one Host

When I try to ping each other by primary ip address all working fine.
When I try to ping secondary address of VM 103 from VM 92 - ping loss 100%

At the same time traffic dump interface one-92-0 contains ARP request/reply about secondary address, but I didn’t see ARP reply on interface one-103-0.

Please HELP

Opennebula 5.0.1
Host Ubuntu 14.04.5 LTS
VM 92 - Debian 8.4
VM 103 Ubuntu 14.04.5 LTS

UPDATE
I’ve solved it!

First I’ve used Network with IP address pool. Opennebula gave one IP address from pool. (IP spoofing prevent, and MAC spoofing were not activated)
Now I’ve changed network to network with mac-address only pool, I’ve assigned first and second IP addresses inside VM. All IP adressess have started work correctly.

UPDATE2
The reason of this
/var/log/VM.log

Wed Aug 24 09:01:49 2016 [Z0][VMM][I]: post: Executed "sudo ovs-vsctl set Port one-102-0 tag=4045".
Wed Aug 24 09:01:49 2016 [Z0][VMM][I]: post: Executed "sudo ovs-ofctl add-flow ovsbr0 in_port=21,dl_src=02:00:01:a0:33:42,priority=40000,actions=normal".
Wed Aug 24 09:01:49 2016 [Z0][VMM][I]: post: Executed "sudo ovs-ofctl add-flow ovsbr0 in_port=21,priority=39000,actions=drop".
2

Glad you solved it. I have the feeling this might just be a problem with the routing tables.

3

No it’s opennebula features :slight_smile:
When I was using network with IP address pool, the opennebula added this in openvswitch configuration:

Fri Aug 19 08:37:08 2016 [Z0][VMM][I]: post: Executed “sudo ovs-vsctl set Port one-101-0 tag=4045”.
Fri Aug 19 08:37:08 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=17,arp,dl_src=02:00:ac:14:15:b4,priority=45000,actions=drop”.
Fri Aug 19 08:37:08 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=17,arp,dl_src=02:00:ac:14:15:b4,nw_src=172.20.21.180,priority=46000,actions=normal”.
Fri Aug 19 08:37:08 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=17,dl_src=02:00:ac:14:15:b4,priority=40000,actions=normal”.
Fri Aug 19 08:37:08 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=17,priority=39000,actions=drop”.

It’s restrict combination port+mac+IP, so ip address added by VRRP isn’t working without reconfiguration openvswitch on the node.

When I was using address pool with only mac-address pool, opennebula did this:

Wed Aug 24 09:01:49 2016 [Z0][VMM][I]: post: Executed “sudo ovs-vsctl set Port one-102-0 tag=4045”.
Wed Aug 24 09:01:49 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=21,dl_src=02:00:01:a0:33:42,priority=40000,actions=normal”.
Wed Aug 24 09:01:49 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=21,priority=39000,actions=drop”.

In this case strict rule contains only mac+port combination, and you can use any IP addresses on the VM’s interface/

4

Oh! understood. Thanks for pulling the threads :slight_smile:

5

Sorry to hijack a thread, I wonder if I’m running into the same problem. I’m using OpenVSwitch, and I have a VM with the following config:

Public 1
Public 2
Private 1
Private 2

Public 1 and Private 1 both work, but Public 2 and Private 2 do not.

Thanks!

6

Hello, my new research:)
I have tried to use network with and without settings IP spoofing, Mac spoofing checkbox. The result was the same:

Mon Oct 10 08:22:46 2016 [Z0][VMM][I]: post: Executed “sudo ovs-vsctl set Port one-167-0 tag=4043”.
Mon Oct 10 08:22:46 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=12,arp,dl_src=02:00:c0:a8:9a:0e,priority=45000,actions=drop”.
Mon Oct 10 08:22:46 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=12,arp,dl_src=02:00:c0:a8:9a:0e,nw_src=192.168.154.14,priority=46000,actions=normal”.
Mon Oct 10 08:22:46 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=12,dl_src=02:00:c0:a8:9a:0e,priority=40000,actions=normal”.
Mon Oct 10 08:22:46 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=12,priority=39000,actions=drop”.

Specially I have created new VM after change network properties:

Mon Oct 10 08:19:19 2016 [Z0][VMM][I]: post: Executed “sudo ovs-vsctl set Port one-166-0 tag=4043”.
Mon Oct 10 08:19:19 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=11,arp,dl_src=02:00:c0:a8:9a:0d,priority=45000,actions=drop”.
Mon Oct 10 08:19:19 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=11,arp,dl_src=02:00:c0:a8:9a:0d,nw_src=192.168.154.13,priority=46000,actions=normal”.
Mon Oct 10 08:19:19 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=11,dl_src=02:00:c0:a8:9a:0d,priority=40000,actions=normal”.
Mon Oct 10 08:19:19 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=11,priority=39000,actions=drop”.

So, I thought about problem with vmn driver:
I have found:
file: /var/lib/one/remotes/vnm/ovswitch/OpenvSwitch.rb (line 59)

arp_cache_poisoning if CONF[:arp_cache_poisoning] && @nic[:ip]

file: /var/lib/one/remotes/vnm/vnmmad.rb

begin
CONF = YAML.load_file(
File.join(File.dirname(FILE), “OpenNebulaNetwork.conf”)
)
rescue
# Default configuration values
CONF = {
:arp_cache_poisoning => true,
:vxlan_mc => “239.0.0.0”,
:vxlan_ttl => “16”
}
end

Thereby, arp_cache_poisoning depends on the default value=true, and value in file ./OpenNebulaNetwork.conf.

And now, file: /var/lib/one/remotes/vnm/OpenNebulaNetwork.conf

:arp_cache_poisoning: true

I have changed this optionsе to false, ran onehost sync --force (mandatory operation!!!), and Secondary IP’s have started works.
In VM,s log file:

Mon Oct 10 09:10:39 2016 [Z0][VMM][I]: post: Executed “sudo ovs-vsctl set Port one-166-0 tag=4043”.
Mon Oct 10 09:10:39 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=14,dl_src=02:00:c0:a8:9a:0d,priority=40000,actions=normal”.
Mon Oct 10 09:10:39 2016 [Z0][VMM][I]: post: Executed “sudo ovs-ofctl add-flow ovsbr0 in_port=14,priority=39000,actions=drop”.

So the checkboxes “Ip spoofing”, “mac spoofing” have not matters. You can permit secondary IP’s, for all Networks!
It’s not useful.
It would be fine, if developers fixed influence IP/mac spoofing checkboxes for each network. Because global setting in OpenNebulaNetwork.conf reduces network security in general.

1 Like
7

Please, see VM's Secondary IP address isn't working (solved in general)

8

I’ve filled an issue to look at this

http://dev.opennebula.org/issues/4862

THANKS for the detailed feedback!

1 Like
9

I just hit same problem on implementing VRRP with floating IP’s inside VMs. Thanks for sharing.
Looking forward to 5.4 to have peace in my soul, that this will be fixed. Meanwhile we applied Fox’s “workround”.

10

Hello,

I believe the problem everybody experiences here is that the FILTER_IP_SPOOFING isn’t supported with the Open vSwitch driver, but internal ARP Cache Poisoning prevention rules may act as the IP spoofing filter on the ARP layer.

We have fixed the driver for the OpenNebula 5.6, so that:

  • FILTER_IP_SPOOFING is supported
  • ARP cache prevention rules are added inside FILTER_IP_SPOOFING and/or FILTER_MAC_SPOOFING rules only if they are enabled, only for that particular part (IP and/or MAC).

So, basically, if you disable FILTER_IP_SPOOFING to have floating IPs under your control, no rules on the ARP level should block this anymore. If FILTER_MAC_SPOOFING is still enabled with ARP spoofing rules, these rules are just related to the MAC address part, as expected.

Also, it’s still possible to disable the ARP poisoning prevention rules globally.

Best regards,
Vlastimil Holer

https://github.com/OpenNebula/one/commit/bcc172311d290628388a99471abc57f15fe04d6a