# Contribution: OpenNebula backup script for QCOW2 datastores **URL:** \ **Category:** Integration Support\ **Created:** [December 3, 2017, 12:09am UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368 "2017-12-03T00:09:50Z")\ **Posts on this page:** 19\ **Page:** 2
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [July 8, 2018, 6:35pm UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/22 "2018-07-08T18:35:34Z")
Hello @mptek, I explained nature of non-persistent and persistent images/instances in my prev posts. I already discuss this problem with few my customers. It all depends on usage of opennebula. If you use it in “cloud” way, them it is like amazon ec2 instances (vm instances), ami images (non-persistent images) and ebs volumes (persistent images). So customer deploys new instances based on ami image (non-persistent image) and configure everything “infrastructure as code” using tools like ansible. That programatic configuration also includes attaching and mounting ebs volume (persistent image) which can be backuped. So in case of crash, they can easily deploys and configure everything by running “program”. Important data will be on ebs volumes, so preserved. On the other hand, many people use openenbula as “vps” like hosting and deploys non-persistent instances/images. In this case there are two possible solutions (in my subjective opinion). First one is to make every customer VPS persistent, so clone base image and make it persistent, clone base VM template, map it to that image, configure persistent IP, etc… and deploy vm. Second one is deploying non persistent instances and do backups of that “non-persistent” images created on background. Personally I use first approach. ---
**Author:** ![uli](https://avatars.discourse-cdn.com/v4/letter/u/ccd318/32.png) [@uli](https://forum.opennebula.io/u/uli)\ **Post date:** [July 25, 2018, 8:30pm UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/23 "2018-07-25T20:30:26Z")
Hi Kristian, I am currently using ssh transfer datastores on some of my hosts. Images are deployed persistent and with driver=qcow2. Unfortunately your backup script does not cover this scenario (TM\_MAD=ssh, Image persistent, Image “Driver = SSH”). I believe technically it should work with the same commands from your script. Am I correct? If yes, do you see a chance to implement this? Thanks Uli ---
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [July 25, 2018, 8:33pm UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/24 "2018-07-25T20:33:25Z")
Hello, I did support in develop branch, but not tested. you can try it and let me know what happens ---
**Author:** ![uli](https://avatars.discourse-cdn.com/v4/letter/u/ccd318/32.png) [@uli](https://forum.opennebula.io/u/uli)\ **Post date:** [July 26, 2018, 5:07am UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/25 "2018-07-26T05:07:31Z")
Hi, the version in the develop branch does not backup the image as I expected. To be more precise about my goal: I deployed a VM to a KVM node with local storage and a system datastore (TMD\_MAD=ssh). I checked the box “persistent” while deploying the VM. Now I would like to backup the image located in the system datastore from the running VM directly from this KVM node. When I run your script (pointing to the system datastore using a label) it just does nothing. Best Regards Uli ---
**Author:** ![uli](https://avatars.discourse-cdn.com/v4/letter/u/ccd318/32.png) [@uli](https://forum.opennebula.io/u/uli)\ **Post date:** [July 27, 2018, 6:16am UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/26 "2018-07-27T06:16:57Z")
Ok, I did some more testing and can report following results. In my first tests I only included the system ds (ID: 109) on the kvm node in the backup run. In my oppinion this should be have been sufficient because the whole persistent image is located there. The backup.sh just did nothing in this tests. In my next tests I included only the image in the backup run (based on label, not the complete ds) and I think I found a bug in your script: 1. Snapshot is created succesfully 2. rsync tries to copy the image file from datastore id: 110, which is the originating image datastore only existing in the frontend node. It should copy from id: 109, which is the ssh datastore on the kvm\_node. Here is the ouptut (look at the bold part): [oneadmin@backup addon-image-backup]$ ./backup.sh Create live snapshot of image 5 named testpersistent-disk-0 attached to VM 16 as disk sda running on kvm01.inf.corp.loc Run cmd: ssh oneadmin@kvm01.inf.corp.loc ‘touch /var/lib/one/snapshot\_image\_backup/one-16-weekly-backup’ Run cmd: ssh oneadmin@kvm01.inf.corp.loc ‘virsh -c qemu:///system snapshot-create-as --domain one-16 weekly-backup --diskspec sda,file=/var/lib/one/snapshot\_image\_backup/one-16-weekly-backup --disk-only --atomic --no-metadata --quiesce’ || ssh oneadmin@kvm01.inf.corp.loc ‘virsh -c qemu:///system snapshot-create-as --domain one-16 weekly-backup --diskspec sda,file=/var/lib/one/snapshot\_image\_backup/one-16-weekly-backup --disk-only --atomic --no-metadata’ Domain snapshot weekly-backup created Run cmd: mkdir -p /mnt/image\_backup/110/6a32b4fe232caa0b36287e2cbb91cc8e.snap Run cmd: rsync -aHAXxWv --inplace --numeric-ids --progress -e “ssh -T -o Compression=no -x” oneadmin@kvm01.inf.corp.loc:/var/lib/one//datastores/ **110** /6a32b4fe232caa0b36287e2cbb91cc8e /mnt/image\_backup/110/6a32b4fe232caa0b36287e2cbb91cc8e.tmp receiving incremental file list rsync: change\_dir “/var/lib/one//datastores/110” failed: No such file or directory (2) sent 8 bytes received 100 bytes 43.20 bytes/sec total size is 0 speedup is 0.00 rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1650) [Receiver=3.1.2] rsync: [Receiver] write error: Broken pipe (32) ---
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [August 7, 2018, 8:11am UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/27 "2018-08-07T08:11:31Z")
Hello, would you print here your datastores? If I understand correcttly, them persistent image is created in system? datastore on specific compute node? or it is image datastore? ---
**Author:** ![mptek](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@mptek](https://forum.opennebula.io/u/mptek)\ **Post date:** [October 28, 2018, 10:39am UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/28 "2018-10-28T10:39:54Z")
Hi Kristian, I apologize for the delay in your reply. I’m trying to better understand the operation of ON so as not to waste anyone’s time. In your previous post write: > [@feldsam](#): > > ### Non-persistent > > You can deploy many VMs using same non-persistent image. On the background, oned creates new qcow2 image which have backing image of that non-persistent one. That new image is deployed to system datastore. > When you terminate VM instance, that image deployed to system datastore will be deleted. So from nature of this functionality, VMs deployed from non-persistent images have character of instances of base image and should not persist data and should not be backuped. This is ideal for applications, which can run several times at once for horizontal scaling. These instances usually connects to same shared database server and/or shared file server (nfs) to persist data. When VM instance need to persist data, them there should be attached persistent datablock or VM should be instatiated as persistent. Literally I understand that the VM created “non-persistent” will have as a backing-file the image of the VM template. As the example of “Snowman”: > [@Snowman](#): > > VM disk image looks like this : > qemu-img info 0 > image: 0 > file format: qcow2 > virtual size: 10G (10737418240 bytes) > disk size: 9.7G > cluster\_size: 65536 > backing file: /var/lib/one/datastores/103/cdfd110a37e569165315b96f913293a2 > Format specific information: > compat: 1.1 > lazy refcounts: false > refcount bits: 16 > corrupt: false But if I do a qemu-img info of my qcow2 disks (of non-persistent VM) i get this ( **no backing-file** ): ``` root @ sto1-a: / var / lib / one / datastores / 0/83 # qemu-img info disk.0 image: disk.0 file format: qcow2 virtual size: 200G (214748364800 bytes) disk size: 12G cluster_size: 65536 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false ``` **Why?** It looks like a copy of the template image (complete). The first case for the solution you listed is IMHO laborious. As I use it, opennebula assigns various parameters automatically (eg IPv4, IPv6, hostname, rootpwd, etc …). If you open sunstone to internet my clients could create VMs by setting only the parameters strictly necessary leaving others decided by ON (eg IPs). Thanks a lot anyway. Very kind. Greetings Mirko ---
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [October 28, 2018, 10:55am UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/29 "2018-10-28T10:55:47Z")
Hello, I think that it deponds on datastore type. I use shared qcow2 files datastore for images and for system too. I have clustered setup with gfs2 running on top of it. So it uses backing files for non-persistent VM and are in system DS. Persistent VMs each have own image in image DS. If you use non shared SSH DS, them image is copied from frontend node via ssh to compute node, so no backing file. My backup script supports only qcow2 datastore or shared datastores, not ssh/local ---
**Author:** ![mptek](https://avatars.discourse-cdn.com/v4/letter/m/f1d935/32.png) [@mptek](https://forum.opennebula.io/u/mptek)\ **Post date:** [November 4, 2018, 12:36pm UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/30 "2018-11-04T12:36:55Z")
Hello Feldsam. You are right. I use shared QCOW2 for SYS DS and shared (only) for IMG. ON copy the base qcow2 file to SYS DS (without backing file). But i prefer this mode. Every VM have separate and indepndent storage file. ``` root@sto1-a:~# onedatastore list ID NAME SIZE AVAIL CLUSTERS IMAGES TYPE DS TM STAT 0 Storage_VM 13.9T 95% 0 0 sys - qcow2 on 1 Immagini_VM 13.9T 95% 0 2 img fs shared on 2 Files 13.9T 95% 0 0 fil fs shared on ``` I will look to modify your script to “accomodate” my setup. Do you think is a heavy mods or simple? Thank you anyway. Very very kind. Hello ---
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [November 4, 2018, 12:53pm UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/31 "2018-11-04T12:53:14Z")
Hello, I also prefer to have separate image for each VM, but I use persistent disks instead of non-persistent. My backup script also counts with backuping persistent disks only. In case of use persistent disks, oned use system ds for just deployment files (virsh xml, symlink to image datastore) ---
**Author:** ![FileneTaylor](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/filenetaylor/32/7981_2.png) [@FileneTaylor](https://forum.opennebula.io/u/FileneTaylor)\ **Post date:** [February 17, 2024, 9:06pm UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/32 "2024-02-17T21:06:42Z")
Hey, Kristian Feldsam, I joined OpenNebula’s forums looking for a script that would do exactly what another user you spoke to was hoping for. This other user wanted to Borg store a bunch of System states from OpenNebula. You said you back them up, and you have a script. This script, what you share here, I hope to adopt to my needs 6 years after its release. Thank you for your work, and your generosity. I think I’ll be leaving OpenStack (as it can’t capture live states) and joining OpenNebula thanks to you. 🙇‍♀️ Sincerely, Filene Taylor ---
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [February 19, 2024, 2:12pm UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/33 "2024-02-19T14:12:08Z")
Hello @FileneTaylor I rewritten my original script to python and I am using borg backup. I also have implemented restic and kopia for testing purposes. It is also opensource by it is tied to opennebula 3par driver, so it doesnt backup qcow2 - not implemented. > **[Files · parallel · FeldHost Public Projects / OpenNebula 3PAR Backup · GitLab](https://gitlab.feldhost.cz/feldhost-public/one-addon-backup-3par/-/tree/parallel?ref_type=heads)** > > FeldHost™ Git Hosting Btw, in meantime, opennebula implements own backup mechanism, but I dont have time/prioritu to look into it ---
**Author:** ![FileneTaylor](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/filenetaylor/32/7981_2.png) [@FileneTaylor](https://forum.opennebula.io/u/FileneTaylor)\ **Post date:** [February 21, 2024, 2:17am UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/34 "2024-02-21T02:17:24Z")
Thank you so much, Kristian! I am very excited to do the work and get Open Nebula working, configured, and backing up regularly! I did expect to use QCow2, and a lot has changed like their own Backup function… I have no idea if I’ll prefer 3Par! Thank you deeply for sharing your kind gifts and efforts with us!! I truly hope the OpenNebula and Open Source communities recognize you and support you and your endeavors!! ( feldhost.cz !! 😃 ) I have a lot to research… I’ll gladly accept any quick notes you have left to share. 🙇‍♀️💖. Thank you again. ---
**Author:** ![ruben](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/ruben/32/19_2.png) [@ruben](https://forum.opennebula.io/u/ruben)\ **Post date:** [February 21, 2024, 11:58am UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/35 "2024-02-21T11:58:59Z")
Hi Filene Probably if you are using qcow2 with shared or ssh drivers, it is worth looking the stock backup solution. Any feedback is welcome! 😉 And yes thanks @feldsam for sharing this awesome drivers 😍 ---
**Author:** ![FileneTaylor](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/filenetaylor/32/7981_2.png) [@FileneTaylor](https://forum.opennebula.io/u/FileneTaylor)\ **Post date:** [February 21, 2024, 1:59pm UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/36 "2024-02-21T13:59:07Z")
Hi there, Rubén Montero, and thank you for replying! One of the largest reasons I found this thread is because of the Borg backup feature. Part of my plan for using Borg isn’t just because of how it reduces used space, but because I plan on distributing backups of my virtual machine! If it’s possible, or already included, to use Borg backups within OpenNebula for saving, loading, and restoring backups, that will save me a lot of trouble using Kristian Feldsam’s scripts for Node or Python. However, I do think my need is such an edge case I will be lucky to find that OpenNebula is already attuned for my needs. I think other users will either need to use the OpenNebula restore-from-backups feature to load my distributions, or they’ll need to find a way to use the backup snapshot as OpenNebula captures it. So far, this is the closest means to what I’m working for. OpenStack doesn’t capture running states, for example. Being able to capture a running state, while being compatible with QEMU and libvirt puts OpenNebula ahead of other non-open source solutions for me. (Open Source is an important part in being assured of the contents of what I’m running. 🙂 Since I have a bunch of confusion I’m still busy clearing up, the help of others in Open Nebula would be of great assistance to me. Thank you both, utterly and sincerely. 🙇‍♀️🙇‍♀️🔥 ---
**Author:** ![feldsam](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/feldsam/32/4441_2.png) [@feldsam](https://forum.opennebula.io/u/feldsam)\ **Post date:** [February 21, 2024, 4:10pm UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/37 "2024-02-21T16:10:35Z")
@FileneTaylor From what I know, there is only restic implementation in enterprise - paid version. I am open to help you with developing custom backup driver, to use opennebula native backup system with borg as backend. If you are interested, please contact me on feldsam@feldhost.cz ---
**Author:** ![FileneTaylor](https://yyz1.discourse-cdn.com/flex031/user_avatar/forum.opennebula.io/filenetaylor/32/7981_2.png) [@FileneTaylor](https://forum.opennebula.io/u/FileneTaylor)\ **Post date:** [February 24, 2024, 7:15pm UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/38 "2024-02-24T19:15:14Z")
Hi there Kristian! I plan on contacting you after I have my home server reimaged in the next week. 🙂 Thank you for offering your help, I’ll start a new thread and link to it to help marry the latest from OpenNebula and the aid all your versions of work provide me. 🙇‍♀️🔥✊ Sincerely, Filene! ---
**Author:** ![bebezeek](https://avatars.discourse-cdn.com/v4/letter/b/f17d59/32.png) [@bebezeek](https://forum.opennebula.io/u/bebezeek)\ **Post date:** [March 18, 2024, 4:11pm UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/39 "2024-03-18T16:11:10Z")
I can confirm both options are working in our environment (also with nested labes like “images/protected”). ---
**Author:** ![multiprefi](https://avatars.discourse-cdn.com/v4/letter/m/ba9def/32.png) [@multiprefi](https://forum.opennebula.io/u/multiprefi)\ **Post date:** [April 2, 2024, 8:40am UTC](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368/40 "2024-04-02T08:40:40Z")
Ok, thanks. That link to the Tuning Guide makes it very clear, although I have some security related doubts with this setup (non-authenticated and non-encrypted). Network isolation should be bullet proof. Just my opinion… [Previous page](https://forum.opennebula.io/t/contribution-opennebula-backup-script-for-qcow2-datastores/5368.md?page=1)