can you try forwarding both 9869 and 29876?
2 Likes
With forwarding you are referring to using https on that port?
I have just checked, on our instance the proxy is actually reachable from externally directly (probably not a good idea?):
[23:53:10] entrance:~# netstat -anp | grep 29876
tcp 0 0 0.0.0.0:29876 0.0.0.0:* LISTEN 7467/python
Hi Nico, I also using nginx as reverse proxy and VNC is working. I have issues if I use non-ssl access respectively when I access sunstone direct via IP and certificane doesnt match. So verify that you have valid certificate. Here is my config:
nginx:
server {
listen 80 default_server;
server_name _;
return 301 https://$host$request_uri;
}
server {
listen 443;
server_name _;
ssl on;
ssl_certificate /etc/nginx/feldhost.cz.nginx.crt;
ssl_certificate_key /etc/nginx/feldhost.cz.nginx.key;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
proxy_pass http://127.0.0.1:9869;
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
location /nginx_status {
# Turn on nginx stats
stub_status on;
# I do not need logs for stats
access_log off;
# Security: Only allow access from localhost IP #
allow 127.0.0.1;
# Send rest of the world to /dev/null #
deny all;
}
}
and sunstone-server vnc conf
:vnc_proxy_port: 29876
:vnc_proxy_support_wss: only
:vnc_proxy_cert: /etc/one/ssl/feldhost.cz.nginx.crt
:vnc_proxy_key: /etc/one/ssl/feldhost.cz.nginx.key
:vnc_proxy_ipv6: false
Hope to help you.
I also added my private network to firewall zone “trusted”, so sunstone can conenct without limitations to compute nodes.
Thanks for following up, Kristian! We found out that it was two problems in our case:
a) firewall setup (sigh!)
b) changing the url from ws:// to wss://
It works like charm now!
Hello guys.
I installed the SSL certificates on the server nginx and sunstone. It worked.
I updated the Ubuntu sunstone. I updated my nginx server.
Configured nginx following this post.
I tested it with firefox and chrome. In both the message:
“VNC Server disconnected (code: 1006)”
I’ve changed the parameter (: vnc_proxy_support_wss:) file (vi /etc/one/sunstone-server.conf) to “yes”, “no” and “only”. None worked.
Does anyone know what could be?
Thank you.
Hi César,
I think you are missing two things:
- For one side, I think you need to define the attribute “VNC_WSS = no” for oneadmin user.
- For other side, maybe you need to access to your sunstone UI through the web browser and to the port 29876 to accept your certificate. Example: https://sunstone-UI:29876/
I hope it helps 
Cheers,
Esteban
By the way, after the update, have you made a restart/reload of nginx/sunstone services?
Cheers,
Esteban
Hi. Thanks.
I should add that line down on my file?
/etc/one/sunstone-server.conf
“VNC_WSS = no”
Yes. i restarted my machine 3 times.
Hi @cesarufmt,
Exactly, on your sunstone UI you need to go to users, look for oneadmin user and add “VNC_WSS no” to the attributes for oneadmin user. For that, you just need to enter “VNC_WSS” on the left field and “no” on the right field on the attributes fields for oneadmin user and finally, just make click on the “add” button.
If it still does not work after it, maybe you need to access to your sunstone UI through the
web browser and to the port 29876 to accept your certificate. Example: https://sunstone-UI:29876/
I hope it helps
Cheers,
Esteban
Hi. Thanks for you help.
So what did I have to add that the “VNC_WSS” and this value “on” in the oneadmin OpenNebula user settings in the web interface. Would it be this?
My sunstone:
################################################################################
# UI Settings
################################################################################
# :vnc_proxy_
# port: port where the vnc proxy will listen
# support_wss: no | yes | only. For yes and only, provide path to
# cert and key. “yes” means both ws and wss connections will be
# supported.
# vnc_proxy_cert: Certificate to encrypt wss connections.
# vnc_proxy_key: Key for wss connections. Only necessary if not included in cert.
# vnc_proxy_ipv6: Enable ipv6 support for novnc-server
:vnc_proxy_port: 29876
#:vnc_proxy_support_wss: no #era default
:vnc_proxy_support_wss: only
#:vnc_proxy_cert: #era default
#:vnc_proxy_key: #era default
:vnc_proxy_cert: /home/cesar/certificados/ic.ufmt.br.crt
:vnc_proxy_key: /home/cesar/certificados/www.ic.ufmt.br.key
:vnc_proxy_ipv6: false
My nginx file:
upstream opennebula.ic.ufmt.br {
ip_hash;
server 10.5.0.1:9869;
}
Parte HTTP
server {
listen 80;
server_name opennebula.ic.ufmt.br;
return 301 https://$host$request_uri;
}
HTTPS virtual host
server {
listen 443;
server_name opennebula.ic.ufmt.br;
### SSL Parameters
ssl on;
ssl_certificate /etc/nginx/certificados/ic.ufmt.br.crt;
ssl_certificate_key /etc/nginx/certificados/www.ic.ufmt.br.key;
Proxy request to upstream
location / {
proxy_pass http://opennebula.ic.ufmt.br;
}
}
Hey,
what we did to get vnc+ssl running behind nginx:
reconfigure /etc/one/sunstone-server.conf
:vnc_proxy_support_wss: yes
:vnc_proxy_cert: /etc/ssl/certs/star.ungleich.ch.crt
:vnc_proxy_key: /etc/ssl/private/star.ungleich.ch.key
restart sunstone server
service opennebula-sunstone restart
reconfigure user configuration in sunstone GUI to use wss (User Settings -> tab conf -> check box VNC Secure websockets)
HTH!
Nico
hi all,
just sharing how we set things up with nginx ssl proxy for sunstone and encrypted vnc.
here is the nginx config we use:
# No squealing.
server_tokens off;
# OpenNebula Sunstone upstream
upstream sunstone {
server 127.0.0.1:9869;
}
# HTTP virtual host, redirect to HTTPS
server {
listen 80 default_server;
return 301 https://$server_name:443;
}
# HTTPS virtual host, proxy to Sunstone
server {
listen 443 ssl default_server;
ssl_certificate /etc/ssl/certs/opennebula-certchain.pem;
ssl_certificate_key /etc/ssl/private/opennebula-key.pem;
ssl_stapling on;
}
here is the relevant part of sunstone conf:
UI Settings
:vnc_proxy_port: 29876
:vnc_proxy_support_wss: only
:vnc_proxy_cert: /etc/one/ssl/opennebula-certchain.pem
:vnc_proxy_key: /etc/one/ssl/opennebula-key.pem
:vnc_proxy_ipv6: false
When this is set, reload sunstone. Then, go in web interface to “settings” > “info” and change VNC_WSS from no to yes (4.12.1) older versions have a checkmark under “conf” to set VNC to secure sockets.
NOTE: if using a selfsigned cert, the connection to VNC window in Sunstone will fail, either get a real cert, or manually accept the selfsigned cert in your browser before trying it with Sunstone.
Now, VNC sessions should show “encrypted” in the title.
Extra protection:
To better protect VNC we also change the default listen address in templates from “0.0.0.0” to the word “vnc-if”. Then on every host in the cluster we make sure “vnc-if” points to a local IP by editing /etc/hosts for every server in the cluster. (like localhost points to 127.0.0.1, vnc-if points to 10.0.0.1 on server 1, to 10.0.0.2 on server 2, etc etc), which is a non-public management network. This way the VNC listen port is only available for that secure network, and people cant view screens they shouldnt, without the need for a firewall.
This will change the VNC ports for running VMs on opennebula hypervisors from:
tcp 0 0 0.0.0.0:6102 0.0.0.0:* LISTEN 36021
tcp 0 0 0.0.0.0:6135 0.0.0.0:* LISTEN 35370
tcp 0 0 0.0.0.0:6109 0.0.0.0:* LISTEN 33822
tcp 0 0 0.0.0.0:6144 0.0.0.0:* LISTEN 7273
tcp 0 0 0.0.0.0:6145 0.0.0.0:* LISTEN 11997
into:
tcp 0 0 10.149.16.12:6102 0.0.0.0:* LISTEN 36021
tcp 0 0 10.149.16.12:6135 0.0.0.0:* LISTEN 35370
tcp 0 0 10.149.16.12:6109 0.0.0.0:* LISTEN 33822
tcp 0 0 10.149.16.12:6144 0.0.0.0:* LISTEN 7273
tcp 0 0 10.149.16.12:6145 0.0.0.0:* LISTEN 11997
Which makes it a LOT easier to secure VNC. With all hosts setup correctly with “vnc-if” pointing to their local IP, stuff like live migration still works, because “vnc-if” always points to the correct IP on any host in opennebula.
Hope this helps anyone, lots of good info in this thread 
EDIT: forgot to mention, with the above config, you only need to enable 443 (HTTPS) and 29876 (VNC proxy) in your (physical) firewall of Sunstone.
Thank you all for this great info, we will include it in the docs, so other users can benefit from it.
Make sure that if you use vnc over ssl (wss) and strict-transport-security (which you should!) in your ssl config in nginx you will need to include your domain there too or else your browser will not allow to open the VNC window. This took me a while to figure out.
# extra security settings
add_header X-Frame-Options SAMEORIGIN;
add_header X-Content-Type-Options nosniff;
add_header X-XSS-Protection "1; mode=block";
add_header Strict-Transport-Security max-age=31536000;
add_header Content-Security-Policy "default-src 'self'; connect-src 'self' ws://localhost wss://www.example.com:29876
Hey Martijn,
we did not configure our nginx with these headers, however things work
perfectly. I am not sure, why you needed to add the domain - are you
accessing sunstone via various domains?
Cheers,
Nico
No but it adds extra security to your SSL setup, we like the A+ at ssllabs 
https://www.ssllabs.com/ssltest/analyze.html?d=ui.hpccloud.surfsara.nl&latest
Hi,
If I proxy cluster IP from NAT to the public get the error “VNC noVNC ready: native WebSockets, canvas rendering”
For example.
Cloud-1 - 192.168.2.2
Cloud-2 - 192.168.2.3
Cloud-3 - 192.168.2.4
Cluster IP under Corosync - 192.168.2.5
And I proxy it with nginx to public 111.111.111.111 with domain mydomain.com
server {
listen 80;
server_name mydomain.com;
rewrite ^(.*)$ https://mydomain.com$1 permanent;
}
server {
listen 443;
ssl on;
ssl_certificate /etc/letsencrypt/live/mydomain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/mydomain.com/privkey.pem;
location / {
proxy_pass http://192.168.2.5:9869/;
proxy_redirect off;
log_not_found off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-FOR $proxy_add_x_forwarded_for;
}
}
I tried with wss, but it`s doesnt work.
Roman opennebula@discoursemail.com writes:
Hi,
If I proxy cluster IP from NAT to the public get the error “VNC noVNC ready: native WebSockets, canvas rendering”
[…]
I tried with wss, but it`s doesnt work.
Hello,
We opened a feature request some time ago[1] for this.
Regards.
Footnotes:
[1] https://dev.opennebula.org/issues/3538
I have working VNC + Nginx + SSL without any problems.
pid /run/nginx.pid;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
client_max_body_size 5000m;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
....
server {
listen 80 default_server;
server_name _;
return 301 https://$host$request_uri;
}
server {
listen 443;
server_name _;
ssl on;
ssl_certificate /etc/ssl/certs/cert.crt;
ssl_certificate_key /etc/ssl/certs/cert.key;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
location / {
proxy_pass http://127.0.0.1:9869;
}
error_page 404 /404.html;
location = /40x.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
location /nginx_status {
# Turn on nginx stats
stub_status on;
# I do not need logs for stats
access_log off;
# Security: Only allow access from localhost IP #
allow 127.0.0.1;
# Send rest of the world to /dev/null #
deny all;
}
}
}
:vnc_proxy_port: 29876
:vnc_proxy_support_wss: only
:vnc_proxy_cert: /etc/ssl/certs/cert.crt
:vnc_proxy_key: /etc/ssl/certs/cert.key
:vnc_proxy_ipv6: false
:vnc_request_password: true
Hi,
You proxied sunstone with ClusterIP?
I have 3 nodes with configuration as you wrote. (192.168.2.1, 192.168.2.2, 192.168.2.3)
Corosync unites this servers with ClusterIP 192.168.2.5
I proxies ClusterIP 192.168.2.1 with NGINX+SSL to public network with domain myopennebula.example
NGINX.CONF
upstream sunstone {
server 192.168.2.5:9869;
}
upstream websocketproxy {
server 192.168.2.5:29876;
}
server {
listen *:80;
server_name myopennebula.example;
rewrite ^(.*)$ https://myopennebula.example$1 permanent;
}
server {
listen 443;
ssl on;
ssl_certificate /etc/letsencrypt/live/myopennebula.example/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/myopennebula.example/privkey.pem;
access_log /var/log/nginx/opennebula-sunstone-access.log;
error_log /var/log/nginx/opennebula-sunstone-error.log;
# To upload ISO files, increase for VMs images
client_max_body_size 1G;
location / {
proxy_pass http://sunstone;
proxy_redirect off;
log_not_found off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-FOR $proxy_add_x_forwarded_for;
}
location /websockify {
proxy_pass http://websocketproxy;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
COROSYNC.CONF
# Please read the corosync.conf.5 manual page
totem {
version: 2
crypto_cipher: none
crypto_hash: none
interface {
ringnumber: 0
bindnetaddr: 192.168.2.0
mcastport: 5405
ttl: 1
}
transport: udpu
}
logging {
fileline: off
to_stderr: no
to_logfile: yes
to_syslog: yes
logfile: /var/log/cluster/corosync.log
debug: off
timestamp: on
logger_subsys {
subsys: QUORUM
debug: off
}
}
quorum {
provider: corosync_votequorum
}
service {
name: pacemaker
ver: 1
}
nodelist {
node {
ring0_addr: CloudKVM-1
nodeid: 1
}
node {
ring0_addr: CloudKVM-2
nodeid: 2
}
node {
ring0_addr: CloudKVM-3
nodeid: 3
}
COROSYNC SHOW CONFIGURATION
node 1: CloudKVM-1
node 2: CloudKVM-2
node 3: CloudKVM-3
primitive ClusterIP IPaddr2 \
params ip=192.168.2.5 cidr_netmask=23 \
op monitor interval=5s
primitive opennebula-novnc_p systemd:opennebula-novnc \
op monitor interval=10s timeout=100s \
op start interval=0 timeout=100s \
op stop interval=0 timeout=100s
primitive opennebula-sunstone_p systemd:opennebula-sunstone \
op monitor interval=10s timeout=100s \
op start interval=0 timeout=100s \
op stop interval=0 timeout=100s
primitive opennebula_p systemd:opennebula \
op monitor interval=10s timeout=100s \
op start interval=0 timeout=100s \
op stop interval=0 timeout=100s
group Opennebula_HA ClusterIP opennebula_p opennebula-sunstone_p opennebula-novnc_p
location cli-prefer-Opennebula_HA Opennebula_HA role=Started inf: CloudKVM-3
property cib-bootstrap-options: \
have-watchdog=false \
dc-version=1.1.15-11.el7_3.2-e174ec8 \
cluster-infrastructure=corosync \
stonith-enabled=false \
no-quorum-policy=ignore \
last-lrm-refresh=1486635972
