I would like to announce that I have just pushed code for OpenNebula into ElastiCluster 's “master” branch. I’m very new to ONE, any comment, feedback or criticism is very welcome!
ElastiCluster is a command-line tool (backed by a Python library and a large collection of Ansible playbooks) to spin up computational clusters of different kinds with minimal manual intervention. Currently supported are SLURM, Grid Engine, PBSPro, HTCondor, Spark/Hadoop, and Kubernetes – see a complete list at https://elasticluster.readthedocs.io/en/latest/playbooks.html cloud backends include AWS, Azure, Google, OpenStack and now OpenNebula.
If you have any questions, I’ll be happy to discuss further either here (if appropriate) or on the ElastiCluster mailing list or Gitter Chat (sorry cannot post the links as I’m a new user…)
P.S. I apologize in advance if this is not the right forum category for these posts; I was unsure where announcement about “friend” projects belonged…
I’m not sure if the docs are clear enough – would be interested in feedback on that as well!
Here is a sample configuration file for creating a SLURM cluster on ONE; credentials, template and network IDs would have to be filled in according to your local installation, of course.
#
# Example: SLURM 18.08 cluster on CentOS7 on OpenNebula
#
# For more details about ElastiCluster's configuration, see:
# http://elasticluster.readthedocs.io/en/latest/configure.html
#
# For more examples, see:
# https://github.com/elasticluster/elasticluster/tree/master/examples
#
[cloud/one]
provider=opennebula
# comment this out to use the value of the ONE_URL env variable
endpoint=http://localhost:2633/RPC2
# env vars `ONE_USERNAME` and `ONE_PASSWORD` can be used instead of
# hard-coded values here; if both `username=` and the env variable
# `ONE_USERNAME` are unset, then credentials are read from the value
# pointed to by the environmental variable `ONE_AUTH` or -if that is
# unset too-, file `~/.one/one_auth`
username=me
password=secret
[setup/slurm]
provider=ansible
master_groups=slurm_master
worker_groups=slurm_worker
# only on CentOS 7
global_var_slurm_version=18.08
global_var_multiuser_cluster=no
[cluster/slurm]
setup=slurm
cloud=one
# `flavor` may use a template ID, name, or specify key:value pairs
flavor=large-vm-template
# boot disk images can be specified by numerical ID or `username/network-name`
image_id=oneadmin/centos7
# networks can be specified by numerical ID or `username/network-name`
network_ids=oneadmin/public-network
security_group=default
login=centos
master_nodes=1
worker_nodes=4
# override the worker node template to be sure they have enough CPU
[cluster/slurm/worker]
flavor=large-vm-template,cpu:8
# override the master node template to be sure it has enough disk space
[cluster/slurm/worker]
flavor=large-vm-template,disk.size:50000
[login/centos]
image_user=centos
image_user_sudo=root
image_sudo=True
user_key_name=elasticluster
user_key_private=~/.ssh/id_rsa
user_key_public=~/.ssh/id_rsa.pub
Many thanks for letting us know you’ve been working on this, it sounds terrific! It’s always good to hear from people using OpenNebula or expanding its connections with other platforms.
We’d very much like to know more details about the integration, and I think this deserves at least a post on our blog to explain your contribution and, more generally, the ElastiCluster project.
I’ll send you an email to stay in touch. Thank you again and welcome to the OpenNebula Community!
It’s always good to hear from people using OpenNebula or expanding its connections with other platforms.
