Different auth between AWS CLI and ECONE CLI

Hi,
Im using OpenNebula(4.12.3) and configured the econe-server.conf as mentioned in all the articles.

This is the part of the config:

Host and port where econe server will run
:host: 192.168.11.2
:port: 4567

According to [this][1] i should be able to communicate via any EC2 Client to OpenNebula.
" The econe-server is a component on top of the OpenNebula core that translates Amazon EC2 API calls into OpenNebula actions, therefore a user will be able to interact with an OpenNebula cloud using any of the available EC2 clients or tools."

I wan´t to use the Amazon Java API or the Amazon CLI to communicate with OpenNebula but i get the following Error:
Nov 04, 2015 11:36:25 AM com.amazonaws.http.AmazonHttpClient parseClockSkewOffset
WARNUNG: Unable to parse clock skew offset from errmsg: The username or password is not correct (Service: AmazonEC2; Status Code: 401; Error Code: AuthFailure; Request ID: null)
Nov 04, 2015 11:36:25 AM com.amazonaws.http.AmazonHttpClient parseClockSkewOffset
WARNUNG: Unable to parse clock skew offset from errmsg: The username or password is not correct (Service: AmazonEC2; Status Code: 401; Error Code: AuthFailure; Request ID: null)
Nov 04, 2015 11:36:26 AM com.amazonaws.http.AmazonHttpClient parseClockSkewOffset
WARNUNG: Unable to parse clock skew offset from errmsg: The username or password is not correct (Service: AmazonEC2; Status Code: 401; Error Code: AuthFailure; Request ID: null)
Exception in thread “main” com.amazonaws.AmazonServiceException: The username or password is not correct (Service: AmazonEC2; Status Code: 401; Error Code: AuthFailure; Request ID: null)
at com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1219)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:803)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:505)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:317)
at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:11901)
at com.amazonaws.services.ec2.AmazonEC2Client.describeImages(AmazonEC2Client.java:6091)
at init.main(init.java:43)

I used oneadmin as accesskey and the hashed password from oneuser show oneadmin.

now there is some strange behavior on OpenNebula when i run netcat and try to connect with Amazon EC2 CLI to describe images:

*CLIENT-COMMAND
/ec2-describe-images -O oneadmin -W *********** -v -H --debug
Setting User-Agent to [ec2-api-tools 1.7.5.1]
Using AWS acces key: oneadmin
2015-11-04 12:09:26,508 [main] DEBUG org.apache.http.wire - >> "POST / HTTP/1.1[\r][\n]"
2015-11-04 12:09:26,511 [main] DEBUG org.apache.http.wire - >> "Host: 192.168.11.2:4567[\r][\n]"
2015-11-04 12:09:26,512 [main] DEBUG org.apache.http.wire - >> "X-Amz-Date: 20151104T110926Z[\r][\n]"
2015-11-04 12:09:26,512 [main] DEBUG org.apache.http.wire - >> "Authorization: AWS4-HMAC-SHA256 Credential=oneadmin/20151104/us-east-1/ec2/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signature=f70e0406ce2d16f8587decdbff33c6ddbd44e4a339fafe997f251774d22f70e4[\r][\n]"
2015-11-04 12:09:26,512 [main] DEBUG org.apache.http.wire - >> "User-Agent: ec2-api-tools 1.7.5.1, aws-sdk-java/unknown-version Linux/4.1.0-2-amd64 OpenJDK_64-Bit_Server_VM/24.91-b01/1.7.0_91[\r][\n]"
2015-11-04 12:09:26,513 [main] DEBUG org.apache.http.wire - >> "Content-Type: application/x-www-form-urlencoded; charset=utf-8[\r][\n]"
2015-11-04 12:09:26,513 [main] DEBUG org.apache.http.wire - >> "Content-Length: 53[\r][\n]"
2015-11-04 12:09:26,513 [main] DEBUG org.apache.http.wire - >> "Connection: Keep-Alive[\r][\n]"
2015-11-04 12:09:26,513 [main] DEBUG org.apache.http.wire - >> "[\r][\n]"
2015-11-04 12:09:26,514 [main] DEBUG org.apache.http.wire - >> "Action=DescribeImages&Version=2015-04-15&Owner.1=self"
Unknown problem connecting to host: 'http://192.168.11.2:4567
Unable to execute HTTP request: Read timed out
com.amazonaws.AmazonClientException: Unable to execute HTTP request: Read timed out
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:467)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:295)
at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:8184)
at com.amazonaws.services.ec2.AmazonEC2Client.describeImages(AmazonEC2Client.java:4303)
at com.amazon.aes.webservices.client.Jec2SdkImpl.describeImages(Jec2SdkImpl.java:390)
at com.amazon.aes.webservices.client.cmd.DescribeImages.invokeOnline(DescribeImages.java:158)
at com.amazon.aes.webservices.client.cmd.BaseCmd.invoke(BaseCmd.java:1187)
at com.amazon.aes.webservices.client.cmd.DescribeImages.main(DescribeImages.java:182)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:152)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:166)
at org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:90)
at org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:281)
at org.apache.http.impl.conn.LoggingSessionInputBuffer.readLine(LoggingSessionInputBuffer.java:115)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:92)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:62)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:254)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:289)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:252)
at org.apache.http.impl.conn.ManagedClientConnectionImpl.receiveResponseHeader(ManagedClientConnectionImpl.java:191)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:300)
at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doReceiveResponse(SdkHttpRequestExecutor.java:66)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:127)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:717)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:522)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:906)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:805)
at com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:681)
at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:456)
… 7 more
ec2-api-tools-1.7.5.1/bin$ packet_write_wait: Connection to 132.231.8.240: Broken pipe

*SERVER-COMMAND
command: netcat -vlp 4567

listening on [any] 4567 …
connect to [192.168.11.2] XXXXXXX [192.168.11.2] 40027
POST / HTTP/1.1
Host: 192.168.11.2:4567
X-Amz-Date: 20151104T110926Z
Authorization: AWS4-HMAC-SHA256 Credential=oneadmin/20151104/us-east-1/ec2/aws4_request, SignedHeaders=host;user-agent;x-amz-date, Signature=f70e0406ce2d16f8587decdbff33c6ddbd44e4a339fafe997f251774d22f70e4
User-Agent: ec2-api-tools 1.7.5.1, aws-sdk-java/unknown-version Linux/4.1.0-2-amd64 OpenJDK_64-Bit_Server_VM/24.91-b01/1.7.0_91
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: 53
Connection: Keep-Alive

Action=DescribeImages&Version=2015-04-15&Owner.1=self

now if i try to connect with the OpenNebula EC2 CLI (ECONE), the Host receives following and the connection is established:

*CLIENT COMMAND:
ec2-api-tools-1.7.5.1/bin$ econe-describe-images -K oneadmin -S *********

SERVER COMMAND:
netcat -vlp 4567

listening on [any] 4567 …
connect to [192.168.11.2] from XXXXXXX [192.168.11.2] 39973
POST / HTTP/1.1
Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
Accept: /
User-Agent: github-amazon-ec2-ruby-gem
Content-Type: application/x-www-form-urlencoded
Host: 192.168.11.2:4567
Content-Length: 204

AWSAccessKeyId=oneadmin&Action=DescribeImages&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2015-11-04T11%3A08%3A00Z&Version=2010-08-31&Signature=3QXOb68NHEExQtmeyUzPO%2FN438fxs6JxhNfNHBZvFlU%3Dro

so you can see, there are differences between the different tools on the Authentication and the Auth is not working if i use any EC2 client (same problem with hybridfox!). So i´m not able to use EC2 clients as mentioned in [1].
Can anybody explain me, why this is happens and how to solve it to use the Amazone EC2 CLI ?

Thank you!
[1]: http://opennebula.org/opennebula-4-4-enhanced-amazon-ec2-api-implementation/

Hi,

I have tested and indeed it is not working. I remember that years ago if you try to use an endpoint different to an official ec2 url, the cli used a default amazon url to generate the url, probably the issue is related to this. I have opened a ticket to take a look into this in detail
http://dev.opennebula.org/issues/4147

On the other hand, euc2ools should work as expected with econe-server:

thank you !

the problem is even any other EC2 client is not working with opennebula.
i have to write some code in java using jclouds and have to use the standart ec2 api for this.
so i can´t use euca2ools .

i´ve tested the VirtualBox tryout image and started my javacode with the configuration (std ec2 api) to this econe-server and it works… that´s strange.(same econe.conf, same oned.conf)
meantime, i updated to 4.14 but still the same issue.

whats the difference between the virtualbox image and the installation on the server ?

Could you check the amazon-ec2 gem version in both machines?

on the server, where econe is not working:

amazon-ec2 (0.9.17)
aws-sdk (2.1.23, 1.33.0)
aws-sdk-core (2.1.23)
aws-sdk-resources (2.1.23)

and on the virtualbox tryout image, where it works:
amazon-ec2 (0.9.17)
aws-sdk (1.33.0)
aws-sdk-core (2.1.30, 2.0.29)
aws-sdk-resources (2.1.30, 2.0.29)

aws-sdk is not used for the econe server, just for the hybrid driver, thus same gem version.

Could you check if the system clock is synced on the server, where econe is not working?

the date command shows the same time on the econe-server machine as on the client, which wants to access.

dmolina you mentioned euca2ools. now i tried to connect but there is still the same issue.

$ euca-describe-images -U http://*******:80/ -I oneadmin -S *********
euca-describe-images: error (AuthFailure): The username or password is not correct

as password i used the one given by oneuser show 0 AND the "cleartextpassword | sha1sum"
no one works.

is any other EC2-interface than econe-tools useable with opennebula ?

AWS Java API = not working
JCLOUDS = login working , listNodes and any more NOT working because of an dateformat exception between Opennebula and the AWS-code in JCLOUDS)
ECONE-Tools = working
AWS CLI = not working
euca2ools = not working

thanks

Is this problem also solved with the blocked port issue?

no , it´s still not working.

can you test the euca2ools on your system ? maybe i´m doing something wrong ?

It looks like the problem is related to the Signature version used by official tools. I have opened this issue to fix it:
http://dev.opennebula.org/issues/4165

I have uploaded an implementation for signature version 4. Could you try replacing the EC2CloudAuth.rb file in your installation and check if it works for you. I have tested it using the latest euca2ools version.

Thank you for your work, but if i replace this file in /usr/lib/one/ruby/cloud/CloudAuth and restart econe-server there is a problem:

econe-server.error:
/usr/lib/one/ruby/cloud/CloudAuth/EC2CloudAuth.rb:4: syntax error, unexpected ‘<’

^
/usr/lib/one/ruby/cloud/CloudAuth/EC2CloudAuth.rb:5: syntax error, unexpected ‘<’

^ /usr/lib/one/ruby/cloud/CloudAuth/EC2CloudAuth.rb:5: syntax error, unexpected keyword_class, expecting$ ^

Hi All!
I have also been trying this out: I didnt have an issue at all. Just replaced /usr/lib/one/ruby/cloud/CloudAuth/EC2CloudAuth.rb, restarted the econe daemon and started digging.
@s3bb0: Are you sure you dont have any copy/paste character issues?

@dmolina
I am still having some issues though when connecting from fog-aws based drivers.
In the simplest case i call:

require 'fog'
ec2 = Fog::Compute.new :provider => 'AWS', :endpoint => "http://fqdn_url:4567/",:aws_access_key_id => 'test', :aws_secret_access_key => 'hashed_password'
ec2.describe_images

from an irb shell and get the same 401 auth errors.

I have tracked the issue down into the signature_v4 function, where it seems like something is not being signed correctly. See https://github.com/OpenNebula/one/blob/master/src/cloud/common/CloudAuth/EC2CloudAuth.rb#L46

auth_attrs["Signature"] never equals signature_v4(req_env, opts) although my password in line 40 is correct.
(horray for echoing passwords into the log files :wink: )

When comparing with the fog-aws code i have not been able to find anything obvious in the difference of the signing algorithm and it works fine against amazon’s ec2 endpoints. :confused:

Is it just me?
Jason

Hi Jason,

Thank you for your feedback. After trying this out with fog, I found out that the X-Amz-Content-Sha256 header is not included in the request. I have included code to generate the body digest if this header is not provided.

A new version including this commit is available in the repo if you want to try it out.

status 200

perfect! Thanks for the quick reply and the fix!