We are deploying OpenNebula Service Appliances (OneKE 1.31) in a proxied environment. While VNF nodes showing complete configuration successfully, but master and worker remain stuck at from last one day:
2/3 Configuration step is in progress…
used Vnets for oneke
Public vnet = IP: 10.159.192.67 | 10.159.192.75
private vnet = IP: 192.168.200.2 | 192.168.200.101
oneadmin@abc06:~$ onevm list
ID USER GROUP NAME STAT CPU MEM HOST TIME
26 oneadmin oneadmin worker_0_(service_9) runn 2 3G localhost 0d 18h41
25 oneadmin oneadmin master_0_(service_9) runn 2 3G localhost 0d 18h41
24 oneadmin oneadmin vnf_0_(service_9) runn 1 512M localhost 0d 18h45
oneadmin@abc06:~$ oneflow list
ID USER GROUP NAME STARTTIME STAT
9 oneadmin oneadmin Service OneKE 1.31 07/31 12:01:59 RUNNING
oneadmin@abc06:~$
oneadmin@abc06:~$ ssh -A -J root@10.159.192.67 root@192.168.200.4
Welcome to Ubuntu 22.04.5 LTS (GNU/Linux 5.15.0-126-generic x86_64)
- Documentation: https://help.ubuntu.com
- Management: https://landscape.canonical.com
- Support: https://ubuntu.com/pro
System information as of Fri Aug 1 06:49:24 UTC 2025
System load: 0.0 Processes: 98
Usage of /: 6.8% of 24.05GB Users logged in: 0
Memory usage: 5% IPv4 address for eth0: 192.168.200.4
Swap usage: 0%
Expanded Security Maintenance for Applications is not enabled.
72 updates can be applied immediately.
66 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable
2 additional security updates can be applied with ESM Apps.
Learn more about enabling ESM Apps service at Ubuntu Expanded Security Maintenance | Security | Ubuntu
Failed to connect to https://changelogs.ubuntu.com/meta-release-lts. Check your Internet connection or proxy settings
/ _ \ | '_ \ / _ \ OpenNebula Service Appliance
| () || | | || /
_/ || |_| ___|
2/3 Configuration step is in progress…
- PLEASE WAIT *
Last login: Fri Aug 1 06:15:10 2025 from 192.168.200.2
root@localhost:~#
VNF vm
oneadmin@abc06:~$ ssh root@10.159.192.67
/ _ \ | '_ \ / _ \ OpenNebula Service Appliance
| () || | | || /
_/ || |_| ___|
All set and ready to serve 8)
vrouter:~#
worker node
oneadmin@abc06:~$ ssh root@192.168.200.6
Expanded Security Maintenance for Applications is not enabled.
72 updates can be applied immediately.
66 of these updates are standard security updates.
To see these additional updates run: apt list --upgradable
1 additional security update can be applied with ESM Apps.
Learn more about enabling ESM Apps service at Ubuntu Expanded Security Maintenance | Security | Ubuntu
Failed to connect to https://changelogs.ubuntu.com/meta-release-lts. Check your Internet connection or proxy settings
/ _ \ | '_ \ / _ \ OpenNebula Service Appliance
| () || | | || /
_/ || |_| ___|
2/3 Configuration step is in progress…
- PLEASE WAIT *
Last login: Fri Aug 1 06:44:34 2025 from 10.159.198.9
root@localhost:~#
Vnf node logs
oneadmin@abc06:~$ ssh root@10.159.192.67
/ _ \ | '_ \ / _ \ OpenNebula Service Appliance
| () || | | || /
_/ || |_| ___|
All set and ready to serve 8)
vrouter:~# cd /var/log/
vrouter:/var/log# ls
acpid.log dmesg messages one-appliance qemu-ga.log unbound wtmp
vrouter:/var/log# cd one-appliance/
vrouter:/var/log/one-appliance# ls
bootstrap.log configure.log install.log one-failover.err one-failover.log one-haproxy.log one-nat4.log one-router4.log
vrouter:/var/log/one-appliance# tail -f one-failover.err
from /etc/one-appliance/service.d/VRouter/Failover/execute.rb:96:in process_events' from /etc/one-appliance/service.d/VRouter/Failover/execute.rb:143:in execute’
from -e:1:in `’
D, [2025-08-01T06:10:38.437635 #3603] DEBUG – : {:event=>{:type=>“INSTANCE”, :name=>“ETH1”, :state=>“BACKUP”, :priority=>“100”}, :from=>“BACKUP”, :to=>“BACKUP”, :direction=>:stay, :ignored=>true}
D, [2025-08-01T06:10:38.437863 #3603] DEBUG – : :STAY (pid = 3603)
D, [2025-08-01T06:10:38.437987 #3603] DEBUG – : {:event=>{:type=>“INSTANCE”, :name=>“ETH1”, :state=>“MASTER”, :priority=>“100”}, :from=>“BACKUP”, :to=>“MASTER”, :direction=>:stay, :ignored=>true}
D, [2025-08-01T06:10:38.438044 #3603] DEBUG – : :STAY (pid = 3603)
D, [2025-08-01T06:10:38.438246 #3603] DEBUG – : {:event=>{:type=>“GROUP”, :name=>“VRouter”, :state=>“MASTER”, :priority=>“0”}, :from=>“BACKUP”, :to=>“MASTER”, :direction=>:up, :ignored=>false}
D, [2025-08-01T06:10:38.438350 #3603] DEBUG – : :UP (pid = 3603)
^C
vrouter:/var/log/one-appliance# tail -f one-failover.log
I, [2025-08-01T06:10:40.635616 #3603] INFO – : one-dns(:enabled)
- Checking /etc/unbound/unbound.conf … [ ok ]
- Starting unbound daemon … [ ok ]
I, [2025-08-01T06:10:40.911556 #3603] INFO – : one-dhcp4v2(:disabled)
I, [2025-08-01T06:10:40.925063 #3603] INFO – : coredhcp(:disabled)
I, [2025-08-01T06:10:40.945698 #3603] INFO – : one-wg(:disabled)
^C
vrouter:/var/log/one-appliance# tail -f configure.log
I, [2025-08-01T06:10:32.872891 #3514] INFO – : SDNAT4::configure
I, [2025-08-01T06:10:32.872953 #3514] INFO – : SDNAT4::toggle([:stop])
I, [2025-08-01T06:10:32.884283 #3514] INFO – : SDNAT4::toggle([:disable])
I, [2025-08-01T06:10:32.896955 #3514] INFO – : WireGuard::configure
I, [2025-08-01T06:10:32.897002 #3514] INFO – : WireGuard::toggle([:stop])
I, [2025-08-01T06:10:32.913996 #3514] INFO – : WireGuard::toggle([:disable])
^C
vrouter:/var/log/one-appliance# tail -f one-haproxy.log
from /etc/one-appliance/service.d/VRouter/vrouter.rb:45:in service_show' from /etc/one-appliance/service.d/VRouter/vrouter.rb:392:in get_service_vms’
from /etc/one-appliance/service.d/VRouter/HAProxy/execute.rb:71:in call' from /etc/one-appliance/service.d/VRouter/HAProxy/execute.rb:71:in block in execute’
from internal:kernel:187:in loop' from /etc/one-appliance/service.d/VRouter/HAProxy/execute.rb:70:in execute’
from -e:1:in `’
- Reloading haproxy …
- Checking /etc/haproxy … [ ok ]
[ ok ]
^C
vrouter:/var/log/one-appliance# tail -f one-nat4.log - Checking rules … [ ok ]
- Flushing firewall … [ ok ]
- Loading iptables state and starting firewall … [ ok ]
I, [2025-08-01T06:10:39.384147 #4200] INFO – : NAT4::execute
I, [2025-08-01T06:10:39.568242 #4200] INFO – : NAT4::toggle([:save]) - Saving iptables state … [ ok ]
I, [2025-08-01T06:10:39.613021 #4200] INFO – : NAT4::toggle([:reload]) - Checking rules … [ ok ]
- Flushing firewall … [ ok ]
- Loading iptables state and starting firewall … [ ok ]
^C
vrouter:/var/log/one-appliance# tail -f one-router4.log
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
kernel.unprivileged_bpf_disabled = 1
net.ipv4.ip_forward = 0
net.ipv4.conf.all.forwarding = 0
net.ipv4.conf.default.forwarding = 0
net.ipv4.conf.eth0.forwarding = 1
net.ipv4.conf.eth1.forwarding = 1
net.ipv4.conf.eth2.forwarding = 0
net.ipv4.conf.eth3.forwarding = 0