Hi All!
I have been trying out some of the econe interfaces, and am running into problems when uploading some of our images.
The image is about 11G and i upload it normally via the econe-upload tool.
At first i thought that it just died, but that was not the case: in the background one is still uploading the file…
So i debugged the issue a bit:
- Upload from the command line:
oneadmin@service01 ~ $ date && time econe-upload -U XXX -K YYY -S ZZZ /tmp/contextulize-image.raw
Mon Jan 4 17:38:44 CET 2016
/var/lib/gems/1.9.1/gems/curb-0.9.0/lib/curl/easy.rb:73:in `perform': Server returned nothing (no headers, no data) (Curl::Err::GotNothingError)
from /usr/lib/one/ruby/cloud/econe/EC2QueryClient.rb:254:in `http_post'
from /usr/lib/one/ruby/cloud/econe/EC2QueryClient.rb:254:in `upload_image'
from /usr/bin/econe-upload:74:in `block (2 levels) in <main>'
from /usr/lib/one/ruby/cli/command_parser.rb:449:in `call'
from /usr/lib/one/ruby/cli/command_parser.rb:449:in `run'
from /usr/lib/one/ruby/cli/command_parser.rb:76:in `initialize'
from /usr/bin/econe-upload:44:in `new'
from /usr/bin/econe-upload:44:in `<main>'
real 3m33.775s
user 0m2.659s
sys 0m19.796s
(note the time is approx. 17:41)
2.)
Check the /var/tmp for files:
oneadmin@service01 ~ $ ls -lah /var/tmp/
total 16G
drwxrwxrwt 3 root root 4.0K Jan 4 17:41 .
drwxr-xr-x 13 root root 4.0K May 27 2015 ..
drwx------ 2 root root 16K Oct 7 14:39 lost+found
-rw------- 1 oneadmin oneadmin 4.5G Jan 4 17:42 RackMultipart20160104-8268-1unujwz
-rw------- 1 oneadmin oneadmin 11G Jan 4 17:41 thin-body20160104-8268-11eefhk
3.) log output
I added this line:
logger.error { "FU: #{erb_version} - #{response.result(binding)}" }
after this line:
Mon Jan 04 17:46:41 2016 [I]: Updating user pool cache.
Mon Jan 04 17:46:43 2016 [E]: FU: 2008-12-01 - <RegisterImageResponse xmlns="http://ec2.amazonaws.com/doc/2008-12-01/">
<requestId>3e522bcb-29c3-40fa-bab5-bfd91bf4ada3</requestId>
<imageId>ami-00000588</imageId>
</RegisterImageResponse>
Mon Jan 04 17:46:43 2016 [E]: UploadImage
Mon Jan 04 17:46:43 2016 [I]: 10.1.3.100 - - [04/Jan/2016 17:46:43] "POST / HTTP/1.1" 200 186 150.9169
note the time is alot later than when the error occured… namely when (as can be seen next) the RackMultipart* file(s) finished being uploaded… Its also the time that the image shows up in the image list in state locked.
4.) Still uploading according to files:
oneadmin@service01 ~ $ ls -lah /var/tmp/
total 23G
drwxrwxrwt 3 root root 4.0K Jan 4 17:46 .
drwxr-xr-x 13 root root 4.0K May 27 2015 ..
-rw-rw-r-- 1 oneadmin oneadmin 1.2G Jan 4 17:47 5f2ca86da9534a0e231a4264eee13692
drwx------ 2 root root 16K Oct 7 14:39 lost+found
-rw------- 1 oneadmin oneadmin 11G Jan 4 17:46 RackMultipart20160104-8268-1lvww3i
-rw------- 1 oneadmin oneadmin 11G Jan 4 17:44 RackMultipart20160104-8268-1unujwz
Problem also persists with multipart-post:
oneadmin@service01 ~ $ econe-upload -M -U XXX -K YYY -S ZZZ /tmp/contextulize-image.raw
/usr/lib/ruby/vendor_ruby/xmlsimple.rb:990:in `find_xml_file': File does not exist: end of file reached. (ArgumentError)
from /usr/lib/ruby/vendor_ruby/xmlsimple.rb:168:in `xml_in'
from /usr/lib/ruby/vendor_ruby/xmlsimple.rb:203:in `xml_in'
from /usr/lib/ruby/vendor_ruby/AWS/responses.rb:15:in `parse'
from /usr/lib/one/ruby/cloud/econe/EC2QueryClient.rb:286:in `upload_image'
from /usr/bin/econe-upload:74:in `block (2 levels) in <main>'
from /usr/lib/one/ruby/cli/command_parser.rb:449:in `call'
from /usr/lib/one/ruby/cli/command_parser.rb:449:in `run'
from /usr/lib/one/ruby/cli/command_parser.rb:76:in `initialize'
from /usr/bin/econe-upload:44:in `new'
from /usr/bin/econe-upload:44:in `<main>'
Anyone know why this is happening?
Also, why is the file uploaded twice:
-rw------- 1 oneadmin oneadmin 11G Jan 4 17:46 RackMultipart20160104-8268-1lvww3i
-rw------- 1 oneadmin oneadmin 11G Jan 4 17:44 RackMultipart20160104-8268-1unujwz
and then not removed afterwards?
Thanks!
Jason
Running on 4.14.2 latest (i just upgraded, just to make sure…)
)