Not able to start SunStone server after upgrading from 4.12.1 to 4.14 Beta 1

Hi all,

We are performing the upgrade from OpenNebula 4.12.1 to 4.14 Beta1 in an OpenNebula testbed which we have set up with a similar configuration that the one that we have on production and following the release notes through the following link, http://docs.opennebula.org/4.14/release_notes/release_notes/upgrade_412.html

As I already commented in other posts, we have decided to separate the OpenNebula, Sunstone and APP-Market in three different nodes. On the OpenNebula head node everything went smooth but on the Sunstone node, I am getting the following issue when trying to start it (I also need to mention that we use nginx):

[ 2015-08-20 10:21:41.1068 9257/7fcf74738700 App/Implementation.cpp:287 ]: Could not spawn process for     
application /usr/lib/one/sunstone: An error occured while starting up the preloader.

Error ID: d0f600d5

Error details saved to: /tmp/passenger-error-69COer.html
Message from application: cannot load such file -- vcenter_driver (LoadError)
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:53:in `require'
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:53:in `require'
/usr/lib/one/sunstone/routes/vcenter.rb:26:in `<top (required)>'
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:53:in `require'
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:53:in `require'
/usr/lib/one/sunstone/sunstone-server.rb:363:in `block in <top (required)>'
/usr/lib/one/sunstone/sunstone-server.rb:362:in `each'
/usr/lib/one/sunstone/sunstone-server.rb:362:in `<top (required)>'
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:53:in `require'
/usr/share/rubygems/rubygems/core_ext/kernel_require.rb:53:in `require'
config.ru:9:in `block in <main>'
/usr/local/share/gems/gems/rack-1.6.4/lib/rack/builder.rb:55:in `instance_eval'
/usr/local/share/gems/gems/rack-1.6.4/lib/rack/builder.rb:55:in `initialize'
config.ru:1:in `new'
config.ru:1:in `<main>'
/opt/passenger-5.0.4/helper-scripts/rack-preloader.rb:111:in `eval'
/opt/passenger-5.0.4/helper-scripts/rack-preloader.rb:111:in `preload_app'
/opt/passenger-5.0.4/helper-scripts/rack-preloader.rb:157:in `<module:App>'
/opt/passenger-5.0.4/helper-scripts/rack-preloader.rb:29:in `<module:PhusionPassenger>'
/opt/passenger-5.0.4/helper-scripts/rack-preloader.rb:28:in `<main>'

Also to mention that /tmp/passenger-error-DIVUeY.html file does not exist on the sunstone node:

[root@test-sunstone4 ~]# ll /tmp/passenger-error-DIVUeY.html
ls: cannot access /tmp/passenger-error-DIVUeY.html: No such file or directory

Also, on sunstone-server.conf, I have added appmarket to the routes (not sure if it is related but I prefer to comment it)

:routes:
   - oneflow
   - vcenter
   - support
   - appmarket

Maybe, it is an issue from my side when restoring sunstone-server.conf configuration but in any case, I would appreciate any help. From the output, I understand we are missing a gem. Did anyone else experiment this issue?

Thanks in advance,
Esteban

The problem is this one:

Message from application: cannot load such file -- vcenter_driver (LoadError)

I have to check but it may be a packaging problem. vcenter_driver.rb file existing in OpenNebula server package and not in the ruby package.

Can you try installing opennebula-server (if using RHEL based distro) or opennebula (in Debian based distro) and try again? OpenNebula does not be running in that node.

Hi Javi,

First of all, thank you very much for answering me so fast! :smile:

Yes, we are using RHEL based distro (CentOS Linux release 7.0.1406 (Core).

Indeed, we have installed opennebula-server package on the Sunstone node and now, it looks like everything is running correctly (I still need to check the new features, what I am wishing to do…hehe :smile: )

yum clean all;yum install opennebula-server

Aug 20 11:19:45 Installed: opennebula-4.13.80-1.x86_64
Aug 20 11:19:45 Installed: libusal-1.1.11-22.el7.x86_64
Aug 20 11:19:45 Installed: genisoimage-1.1.11-22.el7.x86_64
Aug 20 11:19:45 Installed: xmlrpc-c-1.32.5-1905.svn2451.el7.x86_64
Aug 20 11:19:46 opennebula-server-4.13.80-1.x86_64: 100

[root@test-sunstone4 yum.repos.d]# systemctl status nginx
nginx.service - The nginx HTTP and reverse proxy server
Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled)
Active: active (running) since Thu 2015-08-20 11:25:19 CEST; 2s ago
Process: 11595 ExecStop=/bin/kill -s QUIT $MAINPID (code=exited, status=0/SUCCESS)
Process: 11668 ExecStart=/usr/local/sbin/nginx (code=exited, status=0/SUCCESS)
Process: 11647 ExecStartPre=/usr/local/sbin/nginx -t (code=exited, status=0/SUCCESS)
Main PID: 11691 (nginx)
CGroup: /system.slice/nginx.service
       ├─11669 PassengerAgent watchdog
       ├─11672 PassengerAgent server
       ├─11679 PassengerAgent logger
       ├─11691 nginx: master process /usr/local/sbin/nginx
       ├─11692 nginx: worker process
       ├─11693 nginx: worker process
       ├─11694 nginx: worker process
       ├─11695 nginx: worker process
       ├─11696 /usr/bin/ruby /opt/passenger-5.0.4/helper-scripts/prespawn https://test-sunstone4.oort.surfsara.nl/
       └─11701 /usr/bin/ruby /opt/passenger-5.0.4/helper-scripts/rack-preloader.rb

       Aug 20 11:25:19 test-sunstone4.oort.surfsara.nl nginx[11647]: nginx: the configuration file /etc/nginx/nginx.conf 
       syntax is ok
       Aug 20 11:25:19 test-sunstone4.oort.surfsara.nl nginx[11647]: nginx: configuration file /etc/nginx/nginx.conf test 
       is successful
       Aug 20 11:25:19 test-sunstone4.oort.surfsara.nl systemd[1]: Started The nginx HTTP and reverse proxy server.

By the way Javi, now, I am seeing on the error log from nginx related to the sunstone server the following warning:

App 11701 stderr: WARN: tilt autoloading 'tilt/erb' in a non thread-safe way; explicit require 'tilt/erb' suggested.

Is it normal? or do I need to change anything on the configuration?

Thanks in advance,
Esteban

Hi Javi

We have the same issue here as @esfreire said, the new sunstone standalone service requires opennebula-server package to work properly. Probably these ruby files should be shipped with opennebula-ruby or core package instead opennebula-server (the opennebula client also complains with the same issue)

Cheers
Alvaro

1 Like

We’ve filled an issue to fix this for 4.14 final.

http://dev.opennebula.org/issues/3951

THANKS for testing!!!

Hi @ruben,

Thank you very much for tracking this issue and fix it for the 4.14 final :smile:

Cheers,
Esteban