Image data couldn't came back to image file in SSH image datastore

Community Support
1

Hello. I use ssh system and image datastore.
I have created empty datablock image and template for future VM.
Next I instantiated new VM, and installed OS.
Then I noticed, that my working image transferring only between system datastores Frontend and host machines. Image file in image datastore is empty.
I have saved new image by disk save-as command and have used it for new vm’s. Then I created new VM for modify template image, and saved it to.
My problem - I have some unused images, and I can’t remove images, because vm’s on this images count not 0. Hovewer data in this images won’t be use.

2

could you try onedb fsck?

3

Sorry, may be I was uncorrect. I mean, that persistent and non-persistent images are working equally, as non-persistent in other variants datastores. If I create cleanly datablock image and mark its as persistent and install OS, I will use this image only as cleanly datablock. Unless shared storage, where persistent image in system datastore is link to image storage.

4

In ssh, the image should be copied back to the front-end, unless you use onevm delete. Using onevm shutdown you should see an epilog phase, where the datablock is copied back to the image datastore in the front-end.

5

Hm, I have:

  • created image, as datablock size 3GB, and mark it persistent;
  • created template, and VM (modify time is 13:30);
  • deployed VM;
  • installed OS into VM;
  • shut down VM in (14:10). Access time of file disk.0 on hosts system datastore was 14:10, but access time in Frontend image store was old (13:30). I compared files by data, it was different;
  • ran VM;
  • stopped VM. Access time in Frontend image store was old (13:30), and VMs file transferred to system datastore in Frontend. Thus it means, that data from disk image in Sestem DS and Image DS is different.

I guess correctly that persistent image in this case using only for guarantee “exclusivity” (one persistent image - one VM), and as “starting point for further changes” ?

Thanks

6

No it should, copy back the image, could you paste the transfer.0.epilog
file under the vm directory /var/lib/one/vms/<vm_id>. Also could you paste
the configuration of the image and system datastore?

7

There isn’t file .epilog
ls -l /var/lib/one/vms/22/
-rw-rw-r-- 1 oneadmin oneadmin 1293 мая 18 11:56 deployment.0
-rw-rw-r-- 1 oneadmin oneadmin 1293 мая 18 14:22 deployment.1
-rw-rw-r-- 1 oneadmin oneadmin 1293 мая 18 14:31 deployment.2
-rw-rw-r-- 1 oneadmin oneadmin 1293 мая 18 17:32 deployment.4
-rw-rw-r-- 1 oneadmin oneadmin 300 мая 18 11:55 transfer.0.prolog
-rw-rw-r-- 1 oneadmin oneadmin 367 мая 18 14:33 transfer.2.stop
-rw-rw-r-- 1 oneadmin oneadmin 367 мая 18 17:21 transfer.3.resume
-rw-rw-r-- 1 oneadmin oneadmin 367 мая 18 17:34 transfer.4.stop

VIRTUAL MACHINE 22 INFORMATION
ID : 22
NAME : vm-1-22
USER : vadim
GROUP : oneadmin
STATE : ACTIVE
LCM_STATE : EPILOG_STOP
RESCHED : No
HOST : pamir.aoe.ntk.novotelecom.ru
CLUSTER ID : 101
CLUSTER : NTK-office-sas
START TIME : 05/18 11:54:52
END TIME : -
DEPLOY ID : one-22

VIRTUAL MACHINE MONITORING
CPU : 0.0
MEMORY : 0K

PERMISSIONS
OWNER : um-
GROUP : —
OTHER : —

VM DISKS
ID DATASTORE TARGET IMAGE SIZE TYPE SAVE
0 NTK-office vda vm1 2.9G/2.9G file YES
1 NTK-office hda ubuntu-14.04.3-server-amd64.iso 574M/574M cdro NO

VIRTUAL MACHINE HISTORY
SEQ HOST ACTION DS START TIME PROLOG
0 pamir.aoe.ntk.n poweroff 103 05/18 11:55:09 0d 02h18m 0h01m16s
1 pamir.aoe.ntk.n poweroff 103 05/18 14:22:09 0d 00h04m 0h00m00s
2 pamir.aoe.ntk.n stop 103 05/18 14:31:00 0d 00h03m 0h00m00s
3 pamir.aoe.ntk.n poweroff 103 05/18 17:21:28 0d 00h05m 0h01m35s
4 pamir.aoe.ntk.n stop 103 05/18 17:32:51 0d 00h03m 0h00m00s

USER TEMPLATE
HYPERVISOR="kvm"
LOGO="images/logos/ubuntu.png"
SCHED_REQUIREMENTS="CLUSTER_ID=“101"”

VIRTUAL MACHINE TEMPLATE
AUTOMATIC_REQUIREMENTS=“CLUSTER_ID = 101 & !(PUBLIC_CLOUD = YES)“
CPU=“2"
GRAPHICS=[
LISTEN=“0.0.0.0”,
PORT=“5922”,
TYPE=“SPICE” ]
MEMORY=“1024"
OS=[
BOOT=“hd,cdrom” ]
RAW=[
DATA=””””,
TYPE=“kvm” ]
TEMPLATE_ID="5"
VCPU="2"
VMID="22"
oneadmin@nola1:~/vms/22$
oneadmin@nola1:~/vms/22$ onevm show 22 | less
DEPLOY ID : one-22

VIRTUAL MACHINE MONITORING
CPU : 0.0
MEMORY : 0K

PERMISSIONS
OWNER : um-
GROUP : —
OTHER : —

VM DISKS
ID DATASTORE TARGET IMAGE SIZE TYPE SAVE
0 NTK-office vda vm1 2.9G/2.9G file YES
1 NTK-office hda ubuntu-14.04.3-server-amd64.iso 574M/574M cdro NO

VIRTUAL MACHINE HISTORY
SEQ HOST ACTION DS START TIME PROLOG
0 pamir.aoe.ntk.n poweroff 103 05/18 11:55:09 0d 02h18m 0h01m16s
1 pamir.aoe.ntk.n poweroff 103 05/18 14:22:09 0d 00h04m 0h00m00s
2 pamir.aoe.ntk.n stop 103 05/18 14:31:00 0d 00h03m 0h00m00s
3 pamir.aoe.ntk.n poweroff 103 05/18 17:21:28 0d 00h05m 0h01m35s
4 pamir.aoe.ntk.n stop 103 05/18 17:32:51 0d 00h03m 0h00m00s

USER TEMPLATE
HYPERVISOR="kvm"
LOGO="images/logos/ubuntu.png"
SCHED_REQUIREMENTS="CLUSTER_ID=“101"”

VIRTUAL MACHINE TEMPLATE
AUTOMATIC_REQUIREMENTS=“CLUSTER_ID = 101 & !(PUBLIC_CLOUD = YES)“
CPU=“2"
GRAPHICS=[
LISTEN=“0.0.0.0”,
PORT=“5922”,
TYPE=“SPICE” ]
MEMORY=“1024"
OS=[
BOOT=“hd,cdrom” ]
RAW=[
DATA=””””,
TYPE=“kvm” ]
TEMPLATE_ID="5"
VCPU="2"
VMID=“22”

IMAGE 7 INFORMATION
ID : 7
NAME : vm1
USER : vadim
GROUP : oneadmin
DATASTORE : NTK-office-SAS-IDS
TYPE : DATABLOCK
REGISTER TIME : 05/18 08:02:27
PERSISTENT : Yes
SOURCE : /var/lib/one//datastores/104/488d75e84135b3378b0fe21a60a6602f
FSTYPE : qcow2
SIZE : 2.9G
STATE : used
RUNNING_VMS : 1

PERMISSIONS
OWNER : um-
GROUP : —
OTHER : —

IMAGE TEMPLATE
DEV_PREFIX="vd"
DRIVER=“qcow2”

VIRTUAL MACHINES

ID USER     GROUP    NAME            STAT UCPU    UMEM HOST             TIME
22 vadim    oneadmin vm-1-22         poff  0.0      0K pamir.aoe.   0d 05h34

DATASTORE 103 INFORMATION
ID : 103
NAME : NTK-office-SAS-SDS
USER : vadim
GROUP : office-admins
CLUSTER : NTK-office-sas
TYPE : SYSTEM
DS_MAD : -
TM_MAD : ssh
BASE PATH : /var/lib/one//datastores/103
DISK_TYPE : FILE
STATE : READY

DATASTORE CAPACITY
TOTAL: : -
FREE: : -
USED: : -
LIMIT: : -

PERMISSIONS
OWNER : uma
GROUP : um-
OTHER : —

DATASTORE TEMPLATE
BASE_PATH="/var/lib/one//datastores/"
SHARED="NO"
TM_MAD="ssh"
TYPE=“SYSTEM_DS”

IMAGES

DATASTORE 104 INFORMATION
ID : 104
NAME : NTK-office-SAS-IDS
USER : vadim
GROUP : oneadmin
CLUSTER : NTK-office-sas
TYPE : IMAGE
DS_MAD : fs
TM_MAD : ssh
BASE PATH : /var/lib/one//datastores/104
DISK_TYPE : FILE
STATE : READY

DATASTORE CAPACITY
TOTAL: : 2T
FREE: : 2T
USED: : 685M
LIMIT: : -

PERMISSIONS
OWNER : um-
GROUP : u–
OTHER : —

DATASTORE TEMPLATE
BASE_PATH="/var/lib/one//datastores/"
CLONE_TARGET="SYSTEM"
DISK_TYPE="FILE"
DS_MAD="fs"
LN_TARGET="SYSTEM"
TM_MAD="ssh"
TYPE=“IMAGE_DS”

IMAGES
7
8
9

8

Do you use onevm shutdown? Note that delete is aimed to recover stuck VMs
and not to terminate a VM.

9

I have worked with web interface and pressed buttons “POWEROFF” and “STOP”, and “Undeploy”.
Now I have started VM, and executed “onevm shutdown 22” from console.
Now I see epilog file:

/var/lib/one/vms/22/transfer.5.epilog
MVDS ssh pamir.aoe.ntk.novotelecom.ru:/var/lib/one//datastores/103/22/disk.0 /var/lib/one//datastores/104/488d75e84135b3378b0fe21a60a6602f 22 104
DELETE ssh pamir.aoe.ntk.novotelecom.ru:/var/lib/one//datastores/103/22/disk.1 22 104
DELETE ssh pamir.aoe.ntk.novotelecom.ru:/var/lib/one//datastores/103/22 22 103

Which button in Sunstone makes correctly shutdown operation? Poweroff, stop, undeploy?
And I see errors in vm.log name. I attached it22.log (21.0 KB)

10

As follows

To terminate a VM life cycle:

  • shutdown - terminates (ordered, copying disks back to the image DS) the
    VM,

To “pause” the execution of a VM, this can be in the host
(poweroff/suspend) or in the front-end (undeply/stop):

  • poweroff - poweroff the VM in the host, does not make any disk transfer
    or free resources (i.e. allocated capacity)

  • suspend - same as above but also generates a checkpoint file to resume
    the execution from a given state

  • undeploy - equivalent to poweroff but host capacity is freed and disks
    moved to the frontend, BUT kept in the system ds. The VM can be resumed
    from that copy in another hypervisor. Note that the VM is still alive

  • stop - same as above, but also with a checkpoint.

Cheers

11

Oh! Sorry, I’m really missed. I didn’t pay attention, that shutdown command places in “Trash” icon list.

12

And one note :slight_smile:
I had an erorrs in /var/log/one/22.log

MVDS ssh pamir.aoe.ntk.novotelecom.ru:/var/lib/one//datastores/103/22/disk.0 /var/lib/one//datastores/104/488d75e84135b3378b0fe21a60a6602f 22 104
…
cp: cannot create regular file ‘/var/lib/one//datastores/104/488d75e84135b3378b0fe21a60a6602f’: No such file or directory

I have created ‘/var/lib/one//datastores/104’ on host and saw, image after SHUTDOWN (disk.0) was moving to /var/lib/one//datastores/104 on host, not in frontend. I have controlled it by image-files mtime.

For cleanly experiment I have created new fs-ssh image storage 105, and repeated cycle with new vm (create-deploy-shutdown). I had the same errors, and action with image.

Could you tell me, what went wrong?

13

It could be this bug :

http://dev.opennebula.org/issues/4235

it has been backported to the one4.14 branch so you can take the scritp
from there