Deploying VMs via YAML config files

Hi all,

Thank you for looking into my topic and if this topic has been discussed in some other topic please point me in the right direction. On to my questions:

So at work we have been using Openstack to deploy our VMs via templates configured via yaml files. We can deploy an entire application stack in this model (200+ VMs per stack), but the complexities of Openstack make it really difficult to address the networking issue with Openstack and VMware. Hence the reason why we are looking for an alternative solution and was looking into Opennebula. What I have not seen though is how to address this problem with Opennebula. Is there a way to deploy VMs in mass like I’m currently able to do with Openstack? Are the configuration files different for input? How do I let Opennebula know how to use these input files? Is there documentation on the structure of the config files and do they take bash/scripting code as well?

Thanks again for looking into my topic and I hope to hear from this community soon.

Jesus

Hi Jesus,

Let’s see if I understood your question correctly…

In OpenNebula you first need to define the resources you are going to use, mainly the image (typically an OS image) and a network. Once you have that, you need to create a template, the template defines VM capacity (CPU, Memory, …) you will need to select an image and a network. Once you have the template you can instantiate it multiple times at once, either with:

$ onetemplate instantiate -m <number_of_vms> <template_name|template_id>

Remember to check $ onetemplate instantiate --help to read all the available options:

$ onetemplate instantiate --help
[...]    
-m, --multiple x          Instance multiple VMs
[...]    

You can also do that with Sunstone:

Is this what you’re asking?

Hi Jamie,

Thank you for your response. Very much appreciated. Yes this is what I’m talking about. Based on what you gave me. It seems the options are all command line driven which is cool. I can start working with this. Looking at the options it seems that the command takes in XML input. Can definitions for all the VMs be given in XML format? Here is what a YAML file under Openstack looks like:

heat_template_version: 2013-05-23

description: Server Template

resources:
server01:
type: OS::Nova::Server
properties:
name: "server01"
image: "centos65min"
flavor: "app_4g"
networks: [{“fixed_ip”: “172.16.21.91”, “network”: “9059d3bb-c0b7-4f0d-8daf-6ca3fc3d4d51”}]
user_data: |
#!/bin/bash -v
bash <(curl -L http://172.16.20.25/heat/example/register.sh) ‘objectives,example’ > /tmp/register.log
sed -i ‘s/^Defaults requiretty/#Defaults requiretty/’ /etc/sudoers
IP=ifconfig |grep '172.16.21' |awk '{print $2}' | cut -f2 -d':'
cat > /etc/sysconfig/network-scripts/ifcfg-eth0 <<EOF
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
IPADDR=$IP
NETMASK=255.255.254.0
GATEWAY=172.16.20.1
DNS1=172.16.21.34
DNS2=172.16.21.35
EOF
sed -i ‘s/dhcp01/example/g’ /etc/sysconfig/network
cat > /etc/resolv.conf <<EOF
search example.com
nameserver 172.16.21.34
nameserver 172.16.21.35
EOF
HOSTNAME=hostname
DOMAIN=dnsdomainname
hostname ${HOSTNAME}.${DOMAIN}
if [ -d “/usr/local/data/XDM_DATA” ]; then
echo "XDM_DATA found"
else
mkdir -p /usr/local/data/XDM_DATA
echo "XDM_DATA directory created."
fi
if [ fgrep -q mount /etc/rc.local; echo $? == 1 ]; then
cat >> /etc/rc.local <<EOF
mount nfs01:/nfs_data/archive /usr/local/data/archive
mount nfs01:/nfs_data/XDM_DATA /usr/local/data/XDM_DATA
EOF
fi
echo "Mount NFS file systems…"
mount nfs01:/nfs_data/archive /usr/local/data/archive
mount nfs01:/nfs_data/XDM_DATA /usr/local/data/XDM_DATA
yum install -y freetype fontconfig
tar -zxf /usr/local/data/archive/dist/common/fonts.tgz -C /usr/share/fonts/
fc-cache
/etc/init.d/network restart
chef-client -E example -W
chef-client -E example -o hostname -s | tr -d [0-9]
/etc/init.d/xas-server start
user_data_format: RAW
config_drive: “True”

server02:
type: OS::Nova::Server
properties:
name: "server02"
image: "centos65min"
flavor: "app_4g"
networks: [{“fixed_ip”: “172.16.21.92”, “network”: “9059d3bb-c0b7-4f0d-8daf-6ca3fc3d4d51”}]
user_data: |
#!/bin/bash
bash <(curl -L http://172.16.20.25/heat/example/register.sh) ‘objectives,example’ > /tmp/register.log
sed -i ‘s/^Defaults requiretty/#Defaults requiretty/’ /etc/sudoers
IP=ifconfig |grep '172.16.21' |awk '{print $2}' | cut -f2 -d':'
cat > /etc/sysconfig/network-scripts/ifcfg-eth0 <<EOF
DEVICE=eth0
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=static
IPADDR=$IP
NETMASK=255.255.254.0
GATEWAY=172.16.20.1
DNS1=172.16.21.34
DNS2=172.16.21.35
EOF
sed -i ‘s/dhcp01/example/g’ /etc/sysconfig/network
cat > /etc/resolv.conf <<EOF
search example.com
nameserver 172.16.21.34
nameserver 172.16.21.35
EOF
HOSTNAME=hostname
DOMAIN=dnsdomainname
hostname ${HOSTNAME}.${DOMAIN}
if [ -d “/usr/local/data/XDM_DATA” ]; then
echo "XDM_DATA found"
else
mkdir -p /usr/local/data/XDM_DATA
echo "XDM_DATA directory created."
fi
if [ fgrep -q mount /etc/rc.local; echo $? == 1 ]; then
cat >> /etc/rc.local <<EOF
mount nfs01:/nfs_data/archive /usr/local/data/archive
mount nfs01:/nfs_data/XDM_DATA /usr/local/data/XDM_DATA
EOF
fi
echo "Mount NFS file systems…"
mount nfs01:/nfs_data/archive /usr/local/data/archive
mount nfs01:/nfs_data/XDM_DATA /usr/local/data/XDM_DATA
yum install -y freetype fontconfig
tar -zxf /usr/local/data/archive/dist/common/fonts.tgz -C /usr/share/fonts/
fc-cache
/etc/init.d/network restart
chef-client -E example -W
chef-client -E example -o hostname -s | tr -d [0-9]
/etc/init.d/xas-server start
user_data_format: RAW
config_drive: “True”

What I’m doing here is quite simple. This can get really complex. Does Opennebula support something like this via its XML input?

Thanks again for your reply.

Jesus

Hi Jesus,

your messages have been detected as spam by the forum’s spam filter, that’s why they’re being deleted. This should be fixed now. I manually removed all your previous messages except the first one, which includes all the relevant information.

Hi Jaime,

Thanks and I’m sorry for misspelling your name.

Jesus

As far as I can tell all the things you mention can be done very easily in OpenNebula. However, things are done differently than in OpenStack, and imho in a much neater way :slight_smile:

You can define templates in either XML, the OpenNebula native syntax, or in the graphical user interface:

$ cat test.xml
<TEMPLATE>
  <NAME>test</NAME>
  <CONTEXT>
    <FILES_DS><![CDATA[$FILE[IMAGE_ID=2]]]></FILES_DS>
    <INIT_SCRIPTS><![CDATA[init.sh]]></INIT_SCRIPTS>
    <NETWORK><![CDATA[YES]]></NETWORK>
    <SSH_PUBLIC_KEY><![CDATA[$USER[SSH_PUBLIC_KEY]]]></SSH_PUBLIC_KEY>
  </CONTEXT>
  <CPU><![CDATA[1.0]]></CPU>
  <DISK>
    <IMAGE><![CDATA[c6]]></IMAGE>
  </DISK>
  <GRAPHICS>
    <LISTEN><![CDATA[0.0.0.0]]></LISTEN>
    <TYPE><![CDATA[vnc]]></TYPE>
  </GRAPHICS>
  <MEMORY><![CDATA[512]]></MEMORY>
  <NIC>
    <NETWORK><![CDATA[net_172]]></NETWORK>
  </NIC>
  <OS>
    <ARCH><![CDATA[x86_64]]></ARCH>
  </OS>
</TEMPLATE>

$ onetemplate create test.xml

# or ...

$ cat test.tpl
CONTEXT=[
  INIT_SCRIPTS="init.sh",
  FILES_DS="$FILE[IMAGE_ID=2]",
  SSH_PUBLIC_KEY="$USER[SSH_PUBLIC_KEY]" ]
CPU="0.1"
DISK=[
  IMAGE="ttylinux" ]
GRAPHICS=[
  LISTEN="0.0.0.0",
  TYPE="vnc" ]
MEMORY="64"
NIC=[
  NETWORK="net_172_vlan",
  NETWORK_UNAME="oneadmin" ]
OS=[
  ARCH="x86_64" ]

$ onetemplate create test.tpl

You will notice that there is no embedded script in the template, that’s because we do it differently in OpenNebula. In OpenNebula we register the script separately and specify in the template that it should be included into the VM and executed. The FILES_DS parameter indicates what files to include into the VM, and the INIT_SCRIPTS what files to execute.

$ oneimage create --name init.sh --path init.sh --type context -d files
ID: 2

All of this will only work if the Virtual Machine OS image has the OpenNebula Context Packages installed.

Many of the stuff that you have in the user_data section can be done directly with OpenNebula, without putting them in a script. For instance, networking will be automatically configured, no need to do that manually in the script. The network information is sent to the VM via contextualization mechanism and that is transparent to the user. More info here.

Also note that all these things can be in Sunstone, the web gui, very easily.

Disclaimer: I haven’t tested any of the templates or commands, just wrote them off the top of my head, there are probably missing attributes and syntax errors everywhere.

Jaime,

I have to say thank you very much for taking the time and letting me know how to do this in OpenNebula. This is AWESOME!!! :smile:

This gives me everything I need to get started and will provide feedback on what I find and any corrections to this, but overall seems very much doable and like you said, looks much simpler than Openstack. :smile:

Thanks,
Jesus