Proxied novnc server token issue


Version 5.8.1

I have a HA frontend environment. The novnc wss:// on the cluster leader IP is being proxied by nginx. The sunstone services are being load balanced and are configured to make api calls against the leader xmlrpc. Encrypted novnc will work initially, but when clicking through each of my test VMs to confirm its functionality, it eventually fails. Which oned server is the leader is not a deterministic factor.


[error] 78120#0: *1272 upstream prematurely closed connection while reading response header from upstream, client: IP, server: , request: "GET /?token=q8khpkup76njr0ekygmg HTTP/1.1", upstream: "https://IPLeader:29876/?token=q8khpkup76njr0ekygmg", host: "IPLB:29876"

leader /var/log/one/novnc.log:

Token 'q8khpkup76njr0ekygmg' not found

A successful log is: - - [23/Jun/2019 14:19:59] SSL/TLS (wss://) WebSoc ket connection - - [23/Jun/2019 14:19:59] Version hybi-13, base64 : ‘False’ - - [23/Jun/2019 14:19:59] Path: ‘/?token=899ukkla 9cuvrl637ii2’ - - [23/Jun/2019 14:19:59] connecting to:

Can I get some additional information regarding the novnc token lifespan? Or where else to troubleshoot? Restarting opennebula-novnc seems to help, but then it fails again. The system seems to be querying for the token before it is even created. Oned log is not helpful, and sunstone does not throw logs when the vnc fails.


  upstream stream_backend {
    server IP:443;
    server IP:443;
    server IP:443;
  server {
    listen 443;
    proxy_pass stream_backend;
upstream wss_backend {
    server IP:29876;
server {
    listen 29876;
    ssl on;
    ssl_certificate /etc/nginx/cert.pem;
    ssl_certificate_key /etc/nginx/key.pem;

    location / {
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Proxy "";
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_pass https://wss_backend;


:one_xmlrpc: http://leaderIP:2633/RPC2
:one_xmlrpc_timeout: 60
:memcache_port: 11211
:memcache_namespace: opennebula.sunstone
:vnc_proxy_port: 29876
:vnc_proxy_support_wss: only
:vnc_proxy_cert: /var/lib/one/cert.pem
:vnc_proxy_key: /var/lib/one/key.pem
:vnc_proxy_ipv6: false
:vnc_request_password: false