[Resolved] Copy Error while Instantiating VM template 4.14Beta2

Community Support
1

Hello Guys,

I have added 2 ESXi 5.5 host and downloaded vmware-tty template from marketplace to test if the vms are deployed.

However while deploying the template I get copy error in logs.

Thu Sep 24 01:35:10 2015 [Z0][VM][I]: New state is ACTIVE
Thu Sep 24 01:35:10 2015 [Z0][VM][I]: New LCM state is PROLOG
Thu Sep 24 01:35:11 2015 [Z0][TM][I]: Command execution fail: /var/lib/one/remotes/tm/shared/clone cloudnode:/var/lib/one//datastores/1/4b53962827da5683a9187309cef54729 192.168.4.112:/var/lib/one//datastores/0/7/disk.0 7 1
Thu Sep 24 01:35:11 2015 [Z0][TM][I]: /var/lib/one/remotes/tm/shared/clone: line 87: [: : integer expression expected
Thu Sep 24 01:35:11 2015 [Z0][TM][I]: clone: Cloning /var/lib/one/datastores/1/4b53962827da5683a9187309cef54729 in 192.168.4.112:/var/lib/one//datastores/0/7/disk.0
Thu Sep 24 01:35:11 2015 [Z0][TM][E]: clone: Command "cd /var/lib/one/datastores/0/7; cp /var/lib/one/datastores/1/4b53962827da5683a9187309cef54729 /var/lib/one/datastores/0/7/disk.0 " failed: cp: can’t stat ‘/var/lib/one/datastores/1/4b53962827da5683a9187309cef54729’: No such file or directory
Thu Sep 24 01:35:11 2015 [Z0][TM][E]: Error copying cloudnode:/var/lib/one//datastores/1/4b53962827da5683a9187309cef54729 to 192.168.4.112:/var/lib/one//datastores/0/7/disk.0
Thu Sep 24 01:35:11 2015 [Z0][TM][I]: ExitCode: 1
Thu Sep 24 01:35:11 2015 [Z0][TM][E]: Error executing image transfer script: Error copying cloudnode:/var/lib/one//datastores/1/4b53962827da5683a9187309cef54729 to 192.168.4.112:/var/lib/one//datastores/0/7/disk.0
Thu Sep 24 01:35:11 2015 [Z0][VM][I]: New LCM state is PROLOG_FAILURE

ONED.LOG

Thu Sep 24 01:35:10 2015 [Z0][DiM][D]: Deploying VM 7
Thu Sep 24 01:35:10 2015 [Z0][ReM][D]: Req:4416 UID:0 VirtualMachineDeploy result SUCCESS, 7
Thu Sep 24 01:35:10 2015 [Z0][ReM][D]: Req:5376 UID:0 VirtualMachineInfo invoked , 7
Thu Sep 24 01:35:10 2015 [Z0][ReM][D]: Req:5376 UID:0 VirtualMachineInfo result SUCCESS, “70…”
Thu Sep 24 01:35:11 2015 [Z0][TM][D]: Message received: LOG I 7 Command execution fail: /var/lib/one/remotes/tm/shared/clone cloudnode:/var/lib/one//datastores/1/4b5396
2827da5683a9187309cef54729 192.168.4.112:/var/lib/one//datastores/0/7/disk.0 7 1

Thu Sep 24 01:35:11 2015 [Z0][TM][D]: Message received: LOG I 7 /var/lib/one/remotes/tm/shared/clone: line 87: [: : integer expression expected

Thu Sep 24 01:35:11 2015 [Z0][TM][D]: Message received: LOG I 7 clone: Cloning /var/lib/one/datastores/1/4b53962827da5683a9187309cef54729 in 192.168.4.112:/var/lib/one/
/datastores/0/7/disk.0

Thu Sep 24 01:35:11 2015 [Z0][TM][D]: Message received: LOG E 7 clone: Command "cd /var/lib/one/datastores/0/7; cp /var/lib/one/datastores/1/4b53962827da5683a91
87309cef54729 /var/lib/one/datastores/0/7/disk.0 " failed: cp: can’t stat ‘/var/lib/one/datastores/1/4b53962827da5683a9187309cef54729’: No such file or director
y

Thu Sep 24 01:35:11 2015 [Z0][TM][D]: Message received: LOG E 7 Error copying cloudnode:/var/lib/one//datastores/1/4b53962827da5683a9187309cef54729 to 192.168.4.112:/va
r/lib/one//datastores/0/7/disk.0

Thu Sep 24 01:35:11 2015 [Z0][TM][D]: Message received: LOG I 7 ExitCode: 1

Thu Sep 24 01:35:11 2015 [Z0][TM][D]: Message received: TRANSFER FAILURE 7 Error copying cloudnode:/var/lib/one//datastores/1/4b53962827da5683a9187309cef54729 to 192.16
8.4.112:/var/lib/one//datastores/0/7/disk.0

Thu Sep 24 01:35:11 2015 [Z0][InM][D]: Host Esxi5-5-Cluster-16 (1) successfully monitored.
Thu Sep 24 01:35:15 2015 [Z0][AuM][D]: Message received: AUTHENTICATE SUCCESS 7 -

Thu Sep 24 01:35:15 2015 [Z0][ReM][D]: Req:1696 UID:0 VirtualMachinePoolInfo invoked , -2, 0, -2000, -1
Thu Sep 24 01:35:15 2015 [Z0][ReM][D]: Req:1696 UID:0 VirtualMachinePoolInfo result SUCCESS, “<VM_POOL>7</…”
Thu Sep 24 01:35:15 2015 [Z0][InM][D]: Monitoring datastore default (1)
Thu Sep 24 01:35:15 2015 [Z0][InM][D]: Monitoring datastore files (2)
Thu Sep 24 01:35:16 2015 [Z0][ImM][D]: Datastore files (2) successfully monitored.
Thu Sep 24 01:35:16 2015 [Z0][ImM][D]: Datastore default (1) successfully monitored.
Thu Sep 24 01:35:17 2015 [Z0][ReM][D]: Req:4432 UID:0 VirtualMachinePoolInfo invoked , -2, 0, -2000, -1
Thu Sep 24 01:35:17 2015 [Z0][ReM][D]: Req:4432 UID:0 VirtualMachinePoolInfo result SUCCESS, “<VM_POOL>7</…”
Thu Sep 24 01:35:17 2015 [Z0][ReM][D]: Req:2256 UID:0 VirtualMachineInfo invoked , 7

Could you kindly guide me what is missing so that I can fix it.

Regards
Kiran Ranjane

2

You need to configure the ‘vmfs’ storage drivers instead of the ‘shared’ ones you seem to be using from your logs, as explained here:

http://docs.opennebula.org/4.14/administration/storage/vmware_ds.html

3

Hi Tino,

Thanks for the reply, I missed that and now I have updated it, but getting another error

ONED LOG

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 else

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 if [ ! -d /vmfs/volumes/0/12 ]; then

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 mkdir -p /vmfs/volumes/0/12

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 fi

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 rm -f /vmfs/volumes/0/12/disk.0

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 cp /var/lib/one/datastores/1/5559bf49d7cb79fd7fd19f461e270f1c /vmfs/volumes/0/12/disk.0

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 rm -f /vmfs/volumes/0/12/disk.0.iso

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 ln -s /vmfs/volumes/0/12/disk.0 /vmfs/volumes/0/12/disk.0.iso

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 fi" failed: sh: cd: line 2: can’t cd to /vmfs/volumes/0/12

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 mkdir: can’t create directory ‘/vmfs/volumes/0/12/’: Permission denied

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 Failed to delete virtual disk: The system cannot find the file specified (25).

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 DiskLib_Check() failed for source disk The system cannot find the file specified (25).

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG E 12 Error cloning cloudnode:/var/lib/one//datastores/1/5559bf49d7cb79fd7fd19f461e270f1c to 192.168.4.112:/vmfs/volumes/0/12/disk.0

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: LOG I 12 ExitCode: 255

Thu Sep 24 21:05:22 2015 [Z0][TM][D]: Message received: TRANSFER FAILURE 12 Error cloning cloudnode:/var/lib/one//datastores/1/5559bf49d7cb79fd7fd19f461e270f1c to 192.168.4.112:/vmfs/volumes/0/12/disk.0

Thu Sep 24 21:05:24 2015 [Z0][AuM][D]: Message received: AUTHENTICATE SUCCESS 27 -

Thu Sep 24 21:05:24 2015 [Z0][ReM][D]: Req:6928 UID:0 VirtualMachinePoolInfo invoked , -2, 0, -2000, -1

I am Using NFS Share and the datastores are mounted on ESXI as 0 and 1

Here is the exports file content, I am using Ubuntu 14.04 OS

/var/lib/one/datastores/0 *(rw,sync,no_subtree_check,root_squash)
/var/lib/one/datastores/1 *(rw,sync,no_subtree_check,root_squash)

Regards
Kiran Ranjane

4

While trying to create a folder in the datastore that is mounted on ESXi, I get this error

/vmfs/volumes # cd 0
/vmfs/volumes/c1d16545-f2ea4e42 # mkdir kiran
mkdir: can’t create directory ‘kiran’: Permission denied
/vmfs/volumes/c1d16545-f2ea4e42 #

How do I fix this?

Regards
Kiran Ranjane

5

The oneadmin user of the ESX hosts needs to be able to write in the datastore. You may need to add the user to the root group as explained here:

http://docs.opennebula.org/4.14/administration/virtualization/evmwareg.html#users-groups

6

HI Tino,

I think ESXi 5.1 and later does not support local groups. Groups have been depreciated in ESXi 5.1 onwards.

The group memebership tab is empty and does not give me to add a new local group. Here’s the screenshot.

vmware-local group.png796×410 9.9 KB

Is there any other way to add oneadmin to root group in ESXi5.5?

Regards
Kiran Ranjane

7

There are several ways to tackle this. I suggest making sure that user oneadmin in the front-end shares the UID 1000 that is assigned on the ESX, or alternatively using the anonuid and anongid options in the NFS server, as explained here:

http://www.troubleshooters.com/linux/nfs.htm

8

Hi Tino,

Issues was fixed by adding anonuid and anongid options in the NFS and by adding domain to the hostname.

This is for Ubuntu 14.04 OS, Here are the step that might help.

The vital clue showed up in /var/log/syslog:

Sep 16 13:11:07 client nfsidmap[7340]: nss_getpwnam: name ‘cloudnode@localdomain’ does not map into domain ‘localdomain’

combined with this excerpt from NFSv4Howto

If all directory listings show just “nobody” and “nogroup” instead
of real user and group names, then you might want to check the Domain
parameter set in /etc/idmapd.conf. NFSv4 client and server should be in
the same domain.

and this comment in /etc/idmap.conf:

set your own domain here, if id differs from FQDN minus hostname
Domain = localdomain

So turns out my ESXi server thought its domain was localhost.localdomain:

localhost$ hostname -f
localhost.locadomain

localhost$ grep server /etc/hosts
127.0.1.1 localhost.locadomain localhost

while my client didn’t have a domain:

cloudnode$ hostname -f
cloudnode

cloudnode$ grep client /etc/hosts
127.0.1.1 cloudnode

The fix: edit /etc/hosts on the client to read

127.0.1.1 cloudnode.localdomain cloudnode

and rebooted, mounted the nfs on ESXi 5.5 and VMs were deployed.

Thanks Tino for your guidance.

Regards
Kiran Ranjane