You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@libcloud.apache.org by Rudolf Streif <rs...@linuxfoundation.org> on 2013/03/12 16:34:36 UTC
Newbie questions
Hello World, :)
I am new to libcloud and have been using it for a little bit. I very much
like the driver concept for abstracting the underlying cloud provider APIs
which is very similar to the RDBMS drivers.
So far I have mainly worked with the 'compute' drivers and I have a couple
of newbie questions:
1. It seems odd to me that the NodeDriver class only defines methods for
create_node, reboot_node and destroy_node but no methods for start_node and
stop_node. I can see that some cloud APIs do not provide such functionality
but very common ones such as EC2, CloudSigma, etc. do. Consequently,
someone has implemented those methods as extensions in the particular
drivers. Is that a design consideration? Why not making them common in the
NodeDriver class?
2. Similar to 1 for the Node class. There are methods for rebooting and
destroying a node but none for starting and stopping it. There is also no
wait_until_running method in the Node class. Why is that?
I have cloned the git repo and am adding the functionality I need for my
application such as the above and other stuff. I am happy to contribute it
back if there is interest.
Cheers,
Rudi
Re: Newbie questions
Posted by Charles Loomis <lo...@lal.in2p3.fr>.
For the StratusLab image management, we allow image creators to provide metadata about the image itself. Information like the SSH username could be put into the image metadata and passed into Libcloud from the NodeImage objects.
There is already an "extra" map available for extra driver information, but it might be worthwhile to provide an optional "metadata" dict on the NodeImage object with agreed schema for information like the SSH username. If the information existed in the image metadata, then it could be used directly; if not, then the username cycling method could be used instead.
Cal
On Mar 15, 2013, at 14:09 , Joseph Hall <pe...@gmail.com> wrote:
> Cycling through a list of possible usernames (after verifying that the
> SSH port is accessible) is exactly how we handle this issue in Salt
> Cloud:
>
> https://github.com/saltstack/salt-cloud/blob/develop/saltcloud/clouds/libcloud_aws.py#L169-L189
> https://github.com/saltstack/salt-cloud/blob/develop/saltcloud/utils/__init__.py#L207-L250
>
> It's a horrible solution and it drives me crazy, but I have yet to
> think of another one. It would be pretty awesome to have it solved
> more gracefully on the libcloud layer.
>
>
> On Fri, Mar 15, 2013 at 4:52 AM, Chris Psaltis <cp...@mist.io> wrote:
>> We bumped into this as well as few days ago.
>>
>> The issue is that you can't suppose beforehand what is the ssh_username for
>> a specific image. We currently solve it in our application by creating a map
>> of ami ids -> ssh_usernames and passing the correct one to libcloud.
>> However, I think this should be handled on the libcloud level. One idea is
>> to try/except for several known ssh_usernames, e.g. ec2-user, ubuntu etc. It
>> is not very graceful, but probably it's the most robust solution that will
>> require less maintenance.
>>
>> What ideas do you have? I'm willing to implement it next week.
>>
>> Thanks,
>>
>> Chris
>>
>>
>>
>> On 2013-03-15 00:12, Rudolf Streif wrote:
>>>
>>> Thank you, Tomaz. I actually had both issues, username and key. And once I
>>> added ssh_key to the mix it actually started working. :)
>>>
>>>
>>>
>>>
>>> On Thu, Mar 14, 2013 at 11:50 AM, Tomaz Muraus <to...@apache.org> wrote:
>>>
>>>> Yes, you need to pass ssh_username="ubuntu" kwarg to the deploy_node
>>>> method.
>>>>
>>>> I honestly thought this was documented on the website somewhere, but I
>>>> couldn't find it just by quickly glancing over "getting started" and
>>>> "docs". This means that even if it is documented somewhere it's not
>>>> documented / presented in a good way if I can't find it myself. We will
>>>> definitely work on improving the docs and pointing this out
>>>> more prominently.
>>>>
>>>> This thing is also one of the improvements I'm working on for the next
>>>> release and some of the improvements are already available in trunk and
>>>> 0.12.x branch.
>>>>
>>>> Currently, even if you use LIBCLOUD_DEBUG functionality, you get a bad /
>>>> unrelated exception if you use a correct key, but an invalid SSH
>>>> username.
>>>>
>>>> paramiko library log messages are especially confusing. They don't
>>>> indicate
>>>> any error and one of them even says something along the lines of
>>>> "authentication successful".
>>>>
>>>> All of this makes for a bad user experience so I will try to figure out
>>>> the
>>>> best way to detect this issue and throw a more specific exceptions so
>>>> people won't be confused and need to waste a lot of time on this.
>>>>
>>>> On Thu, Mar 14, 2013 at 11:39 AM, Rudolf Streif <
>>>> rstreif@linuxfoundation.org
>>>>>
>>>>> wrote:
>>>>
>>>>
>>>>> Thanks, Tomaz.
>>>>>
>>>>> I tried to use deploy_node with EC2. I used a key pair already existing
>>>>> and set the security group right. However, it fails when logging into
>>>>> the
>>>>> node and trying to deploy the scripts. I am doing this:
>>>>>
>>>>> node = ec2driver.deploy_node(name='test-3', image=image, size=size,
>>>>> deploy=msd, ex_keyname="key")
>>>>>
>>>>> The image is a Ubuntu 12.10 64-bit, ami-7539b41c
>>>>> The size is t1.micro
>>>>>
>>>>> The error I am getting is libcloud.compute.types.DeploymentError:
>>>>> <DeploymentError: node=i-11f73563, error=Bad authentication type
>>>>> (allowed_types=['publickey']),
>>>>> driver=<libcloud.compute.drivers.ec2.EC2NodeDriver object at 0xdb0b90>>
>>>>>
>>>>> I can log in just fine using ssh. The issue could be that the image
>>>>> expects logins as user "ubuntu" rather than root. Can I set the user
>>>>> with
>>>>> the kwargs?
>>>>>
>>>>> Thanks,
>>>>> Rudi
>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>
>>
>> --
>> mist.io
>> down to earth cloud computing
>
>
>
> --
> "In order to create, you have to have the willingness, the desire to
> be challenged, to be learning." -- Ferran Adria (speaking at Harvard,
> 2011)
Re: Newbie questions
Posted by Joseph Hall <pe...@gmail.com>.
Our code only does the username detection when spinning up and
provisioning a machine, but it does store the username detected for
use throughout that process. After the machine is spun up, it is
generally assumed to be managed with Salt (no SSH required!).
One nice thing is that once you've settled upon an OS and an image
provider, you can reasonably expect the username to remain consistent.
For instance, Bitnami images will always use "bitnami" as their
username. Images provided by Canonical will always use "ubuntu" as the
username. Anything that Amazon gives you will always use "ec2-user",
and a number of other image makers follow suit.
A number of image makers also just use "root", even on EC2. And it
should be noted that if you find an image you like, you can make a
copy that can be modified as needed, including allowing remote root
login, and then just use that as your base image. The caveat here is
that you have to maintain your own base image.
On Fri, Mar 15, 2013 at 10:13 AM, Rudolf Streif
<rs...@linuxfoundation.org> wrote:
> I suppose there is not good solution to this and trying multiple possible
> user names is of course time consuming, and I agree ugly. I wish the image
> providers would always allow root access as a minimum. Then I could
> configure it the way I want it from there. It's not more insecure than
> using a regular username and have it in sudoers anyway.
>
> Joseph, I suppose your code, which I have not looked at yet, then stores
> the successful username somewhere for the next time.
>
> In production I am going to work around the issue with using my own image.
> That of course allows me to control the user name and/or allow root access
> via ssh. And it may save me if the provider removes a particular default
> image.
>
>
> On Fri, Mar 15, 2013 at 6:09 AM, Joseph Hall <pe...@gmail.com> wrote:
>
>> Cycling through a list of possible usernames (after verifying that the
>> SSH port is accessible) is exactly how we handle this issue in Salt
>> Cloud:
>>
>>
>> https://github.com/saltstack/salt-cloud/blob/develop/saltcloud/clouds/libcloud_aws.py#L169-L189
>>
>> https://github.com/saltstack/salt-cloud/blob/develop/saltcloud/utils/__init__.py#L207-L250
>>
>> It's a horrible solution and it drives me crazy, but I have yet to
>> think of another one. It would be pretty awesome to have it solved
>> more gracefully on the libcloud layer.
>>
>>
>> On Fri, Mar 15, 2013 at 4:52 AM, Chris Psaltis <cp...@mist.io> wrote:
>> > We bumped into this as well as few days ago.
>> >
>> > The issue is that you can't suppose beforehand what is the ssh_username
>> for
>> > a specific image. We currently solve it in our application by creating a
>> map
>> > of ami ids -> ssh_usernames and passing the correct one to libcloud.
>> > However, I think this should be handled on the libcloud level. One idea
>> is
>> > to try/except for several known ssh_usernames, e.g. ec2-user, ubuntu
>> etc. It
>> > is not very graceful, but probably it's the most robust solution that
>> will
>> > require less maintenance.
>> >
>> > What ideas do you have? I'm willing to implement it next week.
>> >
>> > Thanks,
>> >
>> > Chris
>> >
>> >
>> >
>> > On 2013-03-15 00:12, Rudolf Streif wrote:
>> >>
>> >> Thank you, Tomaz. I actually had both issues, username and key. And
>> once I
>> >> added ssh_key to the mix it actually started working. :)
>> >>
>> >>
>> >>
>> >>
>> >> On Thu, Mar 14, 2013 at 11:50 AM, Tomaz Muraus <to...@apache.org>
>> wrote:
>> >>
>> >>> Yes, you need to pass ssh_username="ubuntu" kwarg to the deploy_node
>> >>> method.
>> >>>
>> >>> I honestly thought this was documented on the website somewhere, but I
>> >>> couldn't find it just by quickly glancing over "getting started" and
>> >>> "docs". This means that even if it is documented somewhere it's not
>> >>> documented / presented in a good way if I can't find it myself. We will
>> >>> definitely work on improving the docs and pointing this out
>> >>> more prominently.
>> >>>
>> >>> This thing is also one of the improvements I'm working on for the next
>> >>> release and some of the improvements are already available in trunk and
>> >>> 0.12.x branch.
>> >>>
>> >>> Currently, even if you use LIBCLOUD_DEBUG functionality, you get a bad
>> /
>> >>> unrelated exception if you use a correct key, but an invalid SSH
>> >>> username.
>> >>>
>> >>> paramiko library log messages are especially confusing. They don't
>> >>> indicate
>> >>> any error and one of them even says something along the lines of
>> >>> "authentication successful".
>> >>>
>> >>> All of this makes for a bad user experience so I will try to figure out
>> >>> the
>> >>> best way to detect this issue and throw a more specific exceptions so
>> >>> people won't be confused and need to waste a lot of time on this.
>> >>>
>> >>> On Thu, Mar 14, 2013 at 11:39 AM, Rudolf Streif <
>> >>> rstreif@linuxfoundation.org
>> >>>>
>> >>>> wrote:
>> >>>
>> >>>
>> >>>> Thanks, Tomaz.
>> >>>>
>> >>>> I tried to use deploy_node with EC2. I used a key pair already
>> existing
>> >>>> and set the security group right. However, it fails when logging into
>> >>>> the
>> >>>> node and trying to deploy the scripts. I am doing this:
>> >>>>
>> >>>> node = ec2driver.deploy_node(name='test-3', image=image, size=size,
>> >>>> deploy=msd, ex_keyname="key")
>> >>>>
>> >>>> The image is a Ubuntu 12.10 64-bit, ami-7539b41c
>> >>>> The size is t1.micro
>> >>>>
>> >>>> The error I am getting is libcloud.compute.types.DeploymentError:
>> >>>> <DeploymentError: node=i-11f73563, error=Bad authentication type
>> >>>> (allowed_types=['publickey']),
>> >>>> driver=<libcloud.compute.drivers.ec2.EC2NodeDriver object at
>> 0xdb0b90>>
>> >>>>
>> >>>> I can log in just fine using ssh. The issue could be that the image
>> >>>> expects logins as user "ubuntu" rather than root. Can I set the user
>> >>>> with
>> >>>> the kwargs?
>> >>>>
>> >>>> Thanks,
>> >>>> Rudi
>> >>>>
>> >>>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >
>> >
>> > --
>> > mist.io
>> > down to earth cloud computing
>>
>>
>>
>> --
>> "In order to create, you have to have the willingness, the desire to
>> be challenged, to be learning." -- Ferran Adria (speaking at Harvard,
>> 2011)
>>
>
>
>
> --
> --
> *Rudolf J. Streif*
> Director of Embedded Solutions
> The Linux Foundation
>
> rudolf.streif@linux.com
> Phone: +1.619.631.5383
> Skype: rudolfstreif
> PGP: RSA 2048/2048 D6E7D28B
>
> Linux Foundation Events Schedule: events.linuxfoundation.org
> Linux Foundation Training Schedule: training.linuxfoundation.org
--
"In order to create, you have to have the willingness, the desire to
be challenged, to be learning." -- Ferran Adria (speaking at Harvard,
2011)
Re: Newbie questions
Posted by Rudolf Streif <rs...@linuxfoundation.org>.
I suppose there is not good solution to this and trying multiple possible
user names is of course time consuming, and I agree ugly. I wish the image
providers would always allow root access as a minimum. Then I could
configure it the way I want it from there. It's not more insecure than
using a regular username and have it in sudoers anyway.
Joseph, I suppose your code, which I have not looked at yet, then stores
the successful username somewhere for the next time.
In production I am going to work around the issue with using my own image.
That of course allows me to control the user name and/or allow root access
via ssh. And it may save me if the provider removes a particular default
image.
On Fri, Mar 15, 2013 at 6:09 AM, Joseph Hall <pe...@gmail.com> wrote:
> Cycling through a list of possible usernames (after verifying that the
> SSH port is accessible) is exactly how we handle this issue in Salt
> Cloud:
>
>
> https://github.com/saltstack/salt-cloud/blob/develop/saltcloud/clouds/libcloud_aws.py#L169-L189
>
> https://github.com/saltstack/salt-cloud/blob/develop/saltcloud/utils/__init__.py#L207-L250
>
> It's a horrible solution and it drives me crazy, but I have yet to
> think of another one. It would be pretty awesome to have it solved
> more gracefully on the libcloud layer.
>
>
> On Fri, Mar 15, 2013 at 4:52 AM, Chris Psaltis <cp...@mist.io> wrote:
> > We bumped into this as well as few days ago.
> >
> > The issue is that you can't suppose beforehand what is the ssh_username
> for
> > a specific image. We currently solve it in our application by creating a
> map
> > of ami ids -> ssh_usernames and passing the correct one to libcloud.
> > However, I think this should be handled on the libcloud level. One idea
> is
> > to try/except for several known ssh_usernames, e.g. ec2-user, ubuntu
> etc. It
> > is not very graceful, but probably it's the most robust solution that
> will
> > require less maintenance.
> >
> > What ideas do you have? I'm willing to implement it next week.
> >
> > Thanks,
> >
> > Chris
> >
> >
> >
> > On 2013-03-15 00:12, Rudolf Streif wrote:
> >>
> >> Thank you, Tomaz. I actually had both issues, username and key. And
> once I
> >> added ssh_key to the mix it actually started working. :)
> >>
> >>
> >>
> >>
> >> On Thu, Mar 14, 2013 at 11:50 AM, Tomaz Muraus <to...@apache.org>
> wrote:
> >>
> >>> Yes, you need to pass ssh_username="ubuntu" kwarg to the deploy_node
> >>> method.
> >>>
> >>> I honestly thought this was documented on the website somewhere, but I
> >>> couldn't find it just by quickly glancing over "getting started" and
> >>> "docs". This means that even if it is documented somewhere it's not
> >>> documented / presented in a good way if I can't find it myself. We will
> >>> definitely work on improving the docs and pointing this out
> >>> more prominently.
> >>>
> >>> This thing is also one of the improvements I'm working on for the next
> >>> release and some of the improvements are already available in trunk and
> >>> 0.12.x branch.
> >>>
> >>> Currently, even if you use LIBCLOUD_DEBUG functionality, you get a bad
> /
> >>> unrelated exception if you use a correct key, but an invalid SSH
> >>> username.
> >>>
> >>> paramiko library log messages are especially confusing. They don't
> >>> indicate
> >>> any error and one of them even says something along the lines of
> >>> "authentication successful".
> >>>
> >>> All of this makes for a bad user experience so I will try to figure out
> >>> the
> >>> best way to detect this issue and throw a more specific exceptions so
> >>> people won't be confused and need to waste a lot of time on this.
> >>>
> >>> On Thu, Mar 14, 2013 at 11:39 AM, Rudolf Streif <
> >>> rstreif@linuxfoundation.org
> >>>>
> >>>> wrote:
> >>>
> >>>
> >>>> Thanks, Tomaz.
> >>>>
> >>>> I tried to use deploy_node with EC2. I used a key pair already
> existing
> >>>> and set the security group right. However, it fails when logging into
> >>>> the
> >>>> node and trying to deploy the scripts. I am doing this:
> >>>>
> >>>> node = ec2driver.deploy_node(name='test-3', image=image, size=size,
> >>>> deploy=msd, ex_keyname="key")
> >>>>
> >>>> The image is a Ubuntu 12.10 64-bit, ami-7539b41c
> >>>> The size is t1.micro
> >>>>
> >>>> The error I am getting is libcloud.compute.types.DeploymentError:
> >>>> <DeploymentError: node=i-11f73563, error=Bad authentication type
> >>>> (allowed_types=['publickey']),
> >>>> driver=<libcloud.compute.drivers.ec2.EC2NodeDriver object at
> 0xdb0b90>>
> >>>>
> >>>> I can log in just fine using ssh. The issue could be that the image
> >>>> expects logins as user "ubuntu" rather than root. Can I set the user
> >>>> with
> >>>> the kwargs?
> >>>>
> >>>> Thanks,
> >>>> Rudi
> >>>>
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >
> >
> > --
> > mist.io
> > down to earth cloud computing
>
>
>
> --
> "In order to create, you have to have the willingness, the desire to
> be challenged, to be learning." -- Ferran Adria (speaking at Harvard,
> 2011)
>
--
--
*Rudolf J. Streif*
Director of Embedded Solutions
The Linux Foundation
rudolf.streif@linux.com
Phone: +1.619.631.5383
Skype: rudolfstreif
PGP: RSA 2048/2048 D6E7D28B
Linux Foundation Events Schedule: events.linuxfoundation.org
Linux Foundation Training Schedule: training.linuxfoundation.org
Re: Newbie questions
Posted by Joseph Hall <pe...@gmail.com>.
Cycling through a list of possible usernames (after verifying that the
SSH port is accessible) is exactly how we handle this issue in Salt
Cloud:
https://github.com/saltstack/salt-cloud/blob/develop/saltcloud/clouds/libcloud_aws.py#L169-L189
https://github.com/saltstack/salt-cloud/blob/develop/saltcloud/utils/__init__.py#L207-L250
It's a horrible solution and it drives me crazy, but I have yet to
think of another one. It would be pretty awesome to have it solved
more gracefully on the libcloud layer.
On Fri, Mar 15, 2013 at 4:52 AM, Chris Psaltis <cp...@mist.io> wrote:
> We bumped into this as well as few days ago.
>
> The issue is that you can't suppose beforehand what is the ssh_username for
> a specific image. We currently solve it in our application by creating a map
> of ami ids -> ssh_usernames and passing the correct one to libcloud.
> However, I think this should be handled on the libcloud level. One idea is
> to try/except for several known ssh_usernames, e.g. ec2-user, ubuntu etc. It
> is not very graceful, but probably it's the most robust solution that will
> require less maintenance.
>
> What ideas do you have? I'm willing to implement it next week.
>
> Thanks,
>
> Chris
>
>
>
> On 2013-03-15 00:12, Rudolf Streif wrote:
>>
>> Thank you, Tomaz. I actually had both issues, username and key. And once I
>> added ssh_key to the mix it actually started working. :)
>>
>>
>>
>>
>> On Thu, Mar 14, 2013 at 11:50 AM, Tomaz Muraus <to...@apache.org> wrote:
>>
>>> Yes, you need to pass ssh_username="ubuntu" kwarg to the deploy_node
>>> method.
>>>
>>> I honestly thought this was documented on the website somewhere, but I
>>> couldn't find it just by quickly glancing over "getting started" and
>>> "docs". This means that even if it is documented somewhere it's not
>>> documented / presented in a good way if I can't find it myself. We will
>>> definitely work on improving the docs and pointing this out
>>> more prominently.
>>>
>>> This thing is also one of the improvements I'm working on for the next
>>> release and some of the improvements are already available in trunk and
>>> 0.12.x branch.
>>>
>>> Currently, even if you use LIBCLOUD_DEBUG functionality, you get a bad /
>>> unrelated exception if you use a correct key, but an invalid SSH
>>> username.
>>>
>>> paramiko library log messages are especially confusing. They don't
>>> indicate
>>> any error and one of them even says something along the lines of
>>> "authentication successful".
>>>
>>> All of this makes for a bad user experience so I will try to figure out
>>> the
>>> best way to detect this issue and throw a more specific exceptions so
>>> people won't be confused and need to waste a lot of time on this.
>>>
>>> On Thu, Mar 14, 2013 at 11:39 AM, Rudolf Streif <
>>> rstreif@linuxfoundation.org
>>>>
>>>> wrote:
>>>
>>>
>>>> Thanks, Tomaz.
>>>>
>>>> I tried to use deploy_node with EC2. I used a key pair already existing
>>>> and set the security group right. However, it fails when logging into
>>>> the
>>>> node and trying to deploy the scripts. I am doing this:
>>>>
>>>> node = ec2driver.deploy_node(name='test-3', image=image, size=size,
>>>> deploy=msd, ex_keyname="key")
>>>>
>>>> The image is a Ubuntu 12.10 64-bit, ami-7539b41c
>>>> The size is t1.micro
>>>>
>>>> The error I am getting is libcloud.compute.types.DeploymentError:
>>>> <DeploymentError: node=i-11f73563, error=Bad authentication type
>>>> (allowed_types=['publickey']),
>>>> driver=<libcloud.compute.drivers.ec2.EC2NodeDriver object at 0xdb0b90>>
>>>>
>>>> I can log in just fine using ssh. The issue could be that the image
>>>> expects logins as user "ubuntu" rather than root. Can I set the user
>>>> with
>>>> the kwargs?
>>>>
>>>> Thanks,
>>>> Rudi
>>>>
>>>>
>>>
>>
>>
>>
>> --
>
>
> --
> mist.io
> down to earth cloud computing
--
"In order to create, you have to have the willingness, the desire to
be challenged, to be learning." -- Ferran Adria (speaking at Harvard,
2011)
Re: Newbie questions
Posted by Tomaz Muraus <to...@apache.org>.
Thanks,
I'll have a look as soon as possible.
On Wed, Mar 20, 2013 at 5:39 AM, Chris Psaltis <cp...@mist.io> wrote:
> Hi all,
>
> I just opened a ticket for this and submitted a patch. Take a look at
> https://issues.apache.org/**jira/browse/LIBCLOUD-309<https://issues.apache.org/jira/browse/LIBCLOUD-309>
>
> Thanks,
>
> Chris
>
>
> On 2013-03-18 17:28, Rudolf Streif wrote:
>
>> Short of fixing paramiko which is probably not what we want, imho it
>> would be beneficial to just catch the paramiko exception and throw a
>> libcloud exception that just adds some additional hints as to what
>> could have failed, such as username and key. For most cases that would
>> go a long way.
>>
>> :rjs
>>
>> On Fri, Mar 15, 2013 at 9:07 PM, Tomaz Muraus <to...@apache.org> wrote:
>>
>> I just spent some time looking into it. It looks like paramiko throws
>>> "not
>>> a valid DSA private key file" exception if you specify a valid ssh key,
>>> but
>>> an invalid username.
>>>
>>> This is unfortunate, because it makes it impossible to differentiate
>>> between actual invalid key file and an invalid username.
>>>
>>> From the paramiko debug logs:
>>>
>>> DEBUG:paramiko.transport:**Trying key 4334c61b988ac5eac5152721d2acae**08
>>> from
>>>
>>>> test2.pem
>>>> DEBUG:paramiko.transport:**userauth is OK
>>>> INFO:paramiko.transport:**Authentication (publickey) failed.
>>>>
>>>
>>> Keep the following things in mind:
>>>
>>> - test2.pem is in fact a valid and working private key
>>> - SSH agent is not used when you specify a password or a key file (we
>>> pass
>>> 'look_for_keys': False, and 'allow_agent': False to paramiko).
>>>
>>> This actually seems like a paramiko issue so I will try to dig some more
>>> there.
>>>
>>> On Fri, Mar 15, 2013 at 8:27 PM, Tomaz Muraus <to...@apache.org> wrote:
>>>
>>> Thanks for your feedback.
>>>>
>>>> I personally think that the first thing we (Libcloud) need to do it
>>>> detect
>>>> and propagate correct exception to the caller.
>>>>
>>>> For example, if a user provides an invalid username, password or private
>>>> key we should throw InvalidAuthenticationCredentia**ls and if an
>>>> invalid
>>>> authentication method is provided we should throw
>>>> InvalidAuthenticationMethod.
>>>>
>>>> Because of how SSH protocol works (
>>>> http://tools.ietf.org/html/**rfc4252#section-5.1<http://tools.ietf.org/html/rfc4252#section-5.1>[1]) we can't detect other or
>>>>
>>>> a more specific errors (please correct me if I'm wrong).
>>>>
>>>> I also agree that "cycling through usernames" is a hacky solution, but
>>>> it's not really much we can do in this case. I'm personally open to
>>>> adding
>>>> something like that once #1 has been figured out as long as a user can
>>>> disable it or maybe it should even be off by default.
>>>>
>>>> On Fri, Mar 15, 2013 at 3:52 AM, Chris Psaltis <cp...@mist.io>
>>>> wrote:
>>>>
>>>> We bumped into this as well as few days ago.
>>>>>
>>>>> The issue is that you can't suppose beforehand what is the ssh_username
>>>>> for a specific image. We currently solve it in our application by
>>>>> creating
>>>>> a map of ami ids -> ssh_usernames and passing the correct one to
>>>>> libcloud.
>>>>> However, I think this should be handled on the libcloud level. One
>>>>> idea is
>>>>> to try/except for several known ssh_usernames, e.g. ec2-user, ubuntu
>>>>> etc.
>>>>> It is not very graceful, but probably it's the most robust solution
>>>>> that
>>>>> will require less maintenance.
>>>>>
>>>>> What ideas do you have? I'm willing to implement it next week.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Chris
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 2013-03-15 00:12, Rudolf Streif wrote:
>>>>>
>>>>> Thank you, Tomaz. I actually had both issues, username and key. And
>>>>>> once
>>>>>> I
>>>>>> added ssh_key to the mix it actually started working. :)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 14, 2013 at 11:50 AM, Tomaz Muraus <to...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>> Yes, you need to pass ssh_username="ubuntu" kwarg to the deploy_node
>>>>>>
>>>>>>> method.
>>>>>>>
>>>>>>> I honestly thought this was documented on the website somewhere, but
>>>>>>> I
>>>>>>> couldn't find it just by quickly glancing over "getting started" and
>>>>>>> "docs". This means that even if it is documented somewhere it's not
>>>>>>> documented / presented in a good way if I can't find it myself. We
>>>>>>> will
>>>>>>> definitely work on improving the docs and pointing this out
>>>>>>> more prominently.
>>>>>>>
>>>>>>> This thing is also one of the improvements I'm working on for the
>>>>>>> next
>>>>>>> release and some of the improvements are already available in trunk
>>>>>>> and
>>>>>>> 0.12.x branch.
>>>>>>>
>>>>>>> Currently, even if you use LIBCLOUD_DEBUG functionality, you get a
>>>>>>> bad /
>>>>>>> unrelated exception if you use a correct key, but an invalid SSH
>>>>>>> username.
>>>>>>>
>>>>>>> paramiko library log messages are especially confusing. They don't
>>>>>>> indicate
>>>>>>> any error and one of them even says something along the lines of
>>>>>>> "authentication successful".
>>>>>>>
>>>>>>> All of this makes for a bad user experience so I will try to figure
>>>>>>> out
>>>>>>> the
>>>>>>> best way to detect this issue and throw a more specific exceptions so
>>>>>>> people won't be confused and need to waste a lot of time on this.
>>>>>>>
>>>>>>> On Thu, Mar 14, 2013 at 11:39 AM, Rudolf Streif <
>>>>>>> rstreif@linuxfoundation.org
>>>>>>>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>> Thanks, Tomaz.
>>>>>>>
>>>>>>>>
>>>>>>>> I tried to use deploy_node with EC2. I used a key pair already
>>>>>>>> existing
>>>>>>>> and set the security group right. However, it fails when logging
>>>>>>>> into
>>>>>>>> the
>>>>>>>> node and trying to deploy the scripts. I am doing this:
>>>>>>>>
>>>>>>>> node = ec2driver.deploy_node(name='****test-3', image=image,
>>>>>>>> size=size,
>>>>>>>> deploy=msd, ex_keyname="key")
>>>>>>>>
>>>>>>>> The image is a Ubuntu 12.10 64-bit, ami-7539b41c
>>>>>>>> The size is t1.micro
>>>>>>>>
>>>>>>>> The error I am getting is libcloud.compute.types.****
>>>>>>>> DeploymentError:
>>>>>>>> <DeploymentError: node=i-11f73563, error=Bad authentication type
>>>>>>>> (allowed_types=['publickey']),
>>>>>>>> driver=<libcloud.compute.****drivers.ec2.EC2NodeDriver object at
>>>>>>>> 0xdb0b90>>
>>>>>>>>
>>>>>>>> I can log in just fine using ssh. The issue could be that the image
>>>>>>>> expects logins as user "ubuntu" rather than root. Can I set the user
>>>>>>>> with
>>>>>>>> the kwargs?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Rudi
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>>
>>>>> --
>>>>> mist.io [2]
>>>>>
>>>>> down to earth cloud computing
>>>>>
>>>>>
>>>>
>>>>
>> --
>> --
>> RUDOLF J. STREIF
>>
>> Director of Embedded Solutions
>> The Linux Foundation
>>
>> rudolf.streif@linux.com
>> Phone: +1.619.631.5383
>> Skype: rudolfstreif
>> PGP: RSA 2048/2048 D6E7D28B
>>
>> Linux Foundation Events Schedule: events.linuxfoundation.org [3]
>> Linux Foundation Training Schedule: training.**linuxfoundation.org<http://training.linuxfoundation.org>[4]
>>
>>
>>
>> Links:
>> ------
>> [1] http://tools.ietf.org/html/**rfc4252#section-5.1<http://tools.ietf.org/html/rfc4252#section-5.1>
>> [2] http://mist.io
>> [3] http://events.linuxfoundation.**org/<http://events.linuxfoundation.org/>
>> [4] http://training.**linuxfoundation.org/<http://training.linuxfoundation.org/>
>>
>
> --
> mist.io
> down to earth cloud computing
>
Re: Newbie questions
Posted by Chris Psaltis <cp...@mist.io>.
Hi all,
I just opened a ticket for this and submitted a patch. Take a look at
https://issues.apache.org/jira/browse/LIBCLOUD-309
Thanks,
Chris
On 2013-03-18 17:28, Rudolf Streif wrote:
> Short of fixing paramiko which is probably not what we want, imho it
> would be beneficial to just catch the paramiko exception and throw a
> libcloud exception that just adds some additional hints as to what
> could have failed, such as username and key. For most cases that would
> go a long way.
>
> :rjs
>
> On Fri, Mar 15, 2013 at 9:07 PM, Tomaz Muraus <to...@apache.org>
> wrote:
>
>> I just spent some time looking into it. It looks like paramiko throws
>> "not
>> a valid DSA private key file" exception if you specify a valid ssh
>> key, but
>> an invalid username.
>>
>> This is unfortunate, because it makes it impossible to differentiate
>> between actual invalid key file and an invalid username.
>>
>> From the paramiko debug logs:
>>
>> DEBUG:paramiko.transport:Trying key 4334c61b988ac5eac5152721d2acae08
>> from
>>> test2.pem
>>> DEBUG:paramiko.transport:userauth is OK
>>> INFO:paramiko.transport:Authentication (publickey) failed.
>>
>> Keep the following things in mind:
>>
>> - test2.pem is in fact a valid and working private key
>> - SSH agent is not used when you specify a password or a key file (we
>> pass
>> 'look_for_keys': False, and 'allow_agent': False to paramiko).
>>
>> This actually seems like a paramiko issue so I will try to dig some
>> more
>> there.
>>
>> On Fri, Mar 15, 2013 at 8:27 PM, Tomaz Muraus <to...@apache.org>
>> wrote:
>>
>>> Thanks for your feedback.
>>>
>>> I personally think that the first thing we (Libcloud) need to do it
>>> detect
>>> and propagate correct exception to the caller.
>>>
>>> For example, if a user provides an invalid username, password or
>>> private
>>> key we should throw InvalidAuthenticationCredentials and if an
>>> invalid
>>> authentication method is provided we should throw
>>> InvalidAuthenticationMethod.
>>>
>>> Because of how SSH protocol works (
>>> http://tools.ietf.org/html/rfc4252#section-5.1 [1]) we can't detect
>>> other or
>>> a more specific errors (please correct me if I'm wrong).
>>>
>>> I also agree that "cycling through usernames" is a hacky solution,
>>> but
>>> it's not really much we can do in this case. I'm personally open to
>>> adding
>>> something like that once #1 has been figured out as long as a user
>>> can
>>> disable it or maybe it should even be off by default.
>>>
>>> On Fri, Mar 15, 2013 at 3:52 AM, Chris Psaltis <cp...@mist.io>
>>> wrote:
>>>
>>>> We bumped into this as well as few days ago.
>>>>
>>>> The issue is that you can't suppose beforehand what is the
>>>> ssh_username
>>>> for a specific image. We currently solve it in our application by
>>>> creating
>>>> a map of ami ids -> ssh_usernames and passing the correct one to
>>>> libcloud.
>>>> However, I think this should be handled on the libcloud level. One
>>>> idea is
>>>> to try/except for several known ssh_usernames, e.g. ec2-user,
>>>> ubuntu etc.
>>>> It is not very graceful, but probably it's the most robust solution
>>>> that
>>>> will require less maintenance.
>>>>
>>>> What ideas do you have? I'm willing to implement it next week.
>>>>
>>>> Thanks,
>>>>
>>>> Chris
>>>>
>>>>
>>>>
>>>>
>>>> On 2013-03-15 00:12, Rudolf Streif wrote:
>>>>
>>>>> Thank you, Tomaz. I actually had both issues, username and key.
>>>>> And once
>>>>> I
>>>>> added ssh_key to the mix it actually started working. :)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Mar 14, 2013 at 11:50 AM, Tomaz Muraus <to...@apache.org>
>>>>> wrote:
>>>>>
>>>>> Yes, you need to pass ssh_username="ubuntu" kwarg to the
>>>>> deploy_node
>>>>>> method.
>>>>>>
>>>>>> I honestly thought this was documented on the website somewhere,
>>>>>> but I
>>>>>> couldn't find it just by quickly glancing over "getting started"
>>>>>> and
>>>>>> "docs". This means that even if it is documented somewhere it's
>>>>>> not
>>>>>> documented / presented in a good way if I can't find it myself.
>>>>>> We will
>>>>>> definitely work on improving the docs and pointing this out
>>>>>> more prominently.
>>>>>>
>>>>>> This thing is also one of the improvements I'm working on for the
>>>>>> next
>>>>>> release and some of the improvements are already available in
>>>>>> trunk and
>>>>>> 0.12.x branch.
>>>>>>
>>>>>> Currently, even if you use LIBCLOUD_DEBUG functionality, you get
>>>>>> a bad /
>>>>>> unrelated exception if you use a correct key, but an invalid SSH
>>>>>> username.
>>>>>>
>>>>>> paramiko library log messages are especially confusing. They
>>>>>> don't
>>>>>> indicate
>>>>>> any error and one of them even says something along the lines of
>>>>>> "authentication successful".
>>>>>>
>>>>>> All of this makes for a bad user experience so I will try to
>>>>>> figure out
>>>>>> the
>>>>>> best way to detect this issue and throw a more specific
>>>>>> exceptions so
>>>>>> people won't be confused and need to waste a lot of time on this.
>>>>>>
>>>>>> On Thu, Mar 14, 2013 at 11:39 AM, Rudolf Streif <
>>>>>> rstreif@linuxfoundation.org
>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>
>>>>>> Thanks, Tomaz.
>>>>>>>
>>>>>>> I tried to use deploy_node with EC2. I used a key pair already
>>>>>>> existing
>>>>>>> and set the security group right. However, it fails when logging
>>>>>>> into
>>>>>>> the
>>>>>>> node and trying to deploy the scripts. I am doing this:
>>>>>>>
>>>>>>> node = ec2driver.deploy_node(name='**test-3', image=image,
>>>>>>> size=size,
>>>>>>> deploy=msd, ex_keyname="key")
>>>>>>>
>>>>>>> The image is a Ubuntu 12.10 64-bit, ami-7539b41c
>>>>>>> The size is t1.micro
>>>>>>>
>>>>>>> The error I am getting is
>>>>>>> libcloud.compute.types.**DeploymentError:
>>>>>>> <DeploymentError: node=i-11f73563, error=Bad authentication type
>>>>>>> (allowed_types=['publickey']),
>>>>>>> driver=<libcloud.compute.**drivers.ec2.EC2NodeDriver object at
>>>>>>> 0xdb0b90>>
>>>>>>>
>>>>>>> I can log in just fine using ssh. The issue could be that the
>>>>>>> image
>>>>>>> expects logins as user "ubuntu" rather than root. Can I set the
>>>>>>> user
>>>>>>> with
>>>>>>> the kwargs?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Rudi
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>
>>>> --
>>>> mist.io [2]
>>>> down to earth cloud computing
>>>>
>>>
>>>
>
> --
> --
> RUDOLF J. STREIF
> Director of Embedded Solutions
> The Linux Foundation
>
> rudolf.streif@linux.com
> Phone: +1.619.631.5383
> Skype: rudolfstreif
> PGP: RSA 2048/2048 D6E7D28B
>
> Linux Foundation Events Schedule: events.linuxfoundation.org [3]
> Linux Foundation Training Schedule: training.linuxfoundation.org [4]
>
>
>
> Links:
> ------
> [1] http://tools.ietf.org/html/rfc4252#section-5.1
> [2] http://mist.io
> [3] http://events.linuxfoundation.org/
> [4] http://training.linuxfoundation.org/
--
mist.io
down to earth cloud computing
Re: Newbie questions
Posted by Rudolf Streif <rs...@linuxfoundation.org>.
Short of fixing paramiko which is probably not what we want, imho it would
be beneficial to just catch the paramiko exception and throw a libcloud
exception that just adds some additional hints as to what could have
failed, such as username and key. For most cases that would go a long way.
:rjs
On Fri, Mar 15, 2013 at 9:07 PM, Tomaz Muraus <to...@apache.org> wrote:
> I just spent some time looking into it. It looks like paramiko throws "not
> a valid DSA private key file" exception if you specify a valid ssh key, but
> an invalid username.
>
> This is unfortunate, because it makes it impossible to differentiate
> between actual invalid key file and an invalid username.
>
> From the paramiko debug logs:
>
> DEBUG:paramiko.transport:Trying key 4334c61b988ac5eac5152721d2acae08 from
> > test2.pem
> > DEBUG:paramiko.transport:userauth is OK
> > INFO:paramiko.transport:Authentication (publickey) failed.
>
>
> Keep the following things in mind:
>
> - test2.pem is in fact a valid and working private key
> - SSH agent is not used when you specify a password or a key file (we pass
> 'look_for_keys': False, and 'allow_agent': False to paramiko).
>
> This actually seems like a paramiko issue so I will try to dig some more
> there.
>
> On Fri, Mar 15, 2013 at 8:27 PM, Tomaz Muraus <to...@apache.org> wrote:
>
> > Thanks for your feedback.
> >
> > I personally think that the first thing we (Libcloud) need to do it
> detect
> > and propagate correct exception to the caller.
> >
> > For example, if a user provides an invalid username, password or private
> > key we should throw InvalidAuthenticationCredentials and if an invalid
> > authentication method is provided we should throw
> > InvalidAuthenticationMethod.
> >
> > Because of how SSH protocol works (
> > http://tools.ietf.org/html/rfc4252#section-5.1) we can't detect other or
> > a more specific errors (please correct me if I'm wrong).
> >
> > I also agree that "cycling through usernames" is a hacky solution, but
> > it's not really much we can do in this case. I'm personally open to
> adding
> > something like that once #1 has been figured out as long as a user can
> > disable it or maybe it should even be off by default.
> >
> > On Fri, Mar 15, 2013 at 3:52 AM, Chris Psaltis <cp...@mist.io> wrote:
> >
> >> We bumped into this as well as few days ago.
> >>
> >> The issue is that you can't suppose beforehand what is the ssh_username
> >> for a specific image. We currently solve it in our application by
> creating
> >> a map of ami ids -> ssh_usernames and passing the correct one to
> libcloud.
> >> However, I think this should be handled on the libcloud level. One idea
> is
> >> to try/except for several known ssh_usernames, e.g. ec2-user, ubuntu
> etc.
> >> It is not very graceful, but probably it's the most robust solution that
> >> will require less maintenance.
> >>
> >> What ideas do you have? I'm willing to implement it next week.
> >>
> >> Thanks,
> >>
> >> Chris
> >>
> >>
> >>
> >>
> >> On 2013-03-15 00:12, Rudolf Streif wrote:
> >>
> >>> Thank you, Tomaz. I actually had both issues, username and key. And
> once
> >>> I
> >>> added ssh_key to the mix it actually started working. :)
> >>>
> >>>
> >>>
> >>>
> >>> On Thu, Mar 14, 2013 at 11:50 AM, Tomaz Muraus <to...@apache.org>
> wrote:
> >>>
> >>> Yes, you need to pass ssh_username="ubuntu" kwarg to the deploy_node
> >>>> method.
> >>>>
> >>>> I honestly thought this was documented on the website somewhere, but I
> >>>> couldn't find it just by quickly glancing over "getting started" and
> >>>> "docs". This means that even if it is documented somewhere it's not
> >>>> documented / presented in a good way if I can't find it myself. We
> will
> >>>> definitely work on improving the docs and pointing this out
> >>>> more prominently.
> >>>>
> >>>> This thing is also one of the improvements I'm working on for the next
> >>>> release and some of the improvements are already available in trunk
> and
> >>>> 0.12.x branch.
> >>>>
> >>>> Currently, even if you use LIBCLOUD_DEBUG functionality, you get a
> bad /
> >>>> unrelated exception if you use a correct key, but an invalid SSH
> >>>> username.
> >>>>
> >>>> paramiko library log messages are especially confusing. They don't
> >>>> indicate
> >>>> any error and one of them even says something along the lines of
> >>>> "authentication successful".
> >>>>
> >>>> All of this makes for a bad user experience so I will try to figure
> out
> >>>> the
> >>>> best way to detect this issue and throw a more specific exceptions so
> >>>> people won't be confused and need to waste a lot of time on this.
> >>>>
> >>>> On Thu, Mar 14, 2013 at 11:39 AM, Rudolf Streif <
> >>>> rstreif@linuxfoundation.org
> >>>>
> >>>>> wrote:
> >>>>>
> >>>>
> >>>> Thanks, Tomaz.
> >>>>>
> >>>>> I tried to use deploy_node with EC2. I used a key pair already
> existing
> >>>>> and set the security group right. However, it fails when logging into
> >>>>> the
> >>>>> node and trying to deploy the scripts. I am doing this:
> >>>>>
> >>>>> node = ec2driver.deploy_node(name='**test-3', image=image, size=size,
> >>>>> deploy=msd, ex_keyname="key")
> >>>>>
> >>>>> The image is a Ubuntu 12.10 64-bit, ami-7539b41c
> >>>>> The size is t1.micro
> >>>>>
> >>>>> The error I am getting is libcloud.compute.types.**DeploymentError:
> >>>>> <DeploymentError: node=i-11f73563, error=Bad authentication type
> >>>>> (allowed_types=['publickey']),
> >>>>> driver=<libcloud.compute.**drivers.ec2.EC2NodeDriver object at
> >>>>> 0xdb0b90>>
> >>>>>
> >>>>> I can log in just fine using ssh. The issue could be that the image
> >>>>> expects logins as user "ubuntu" rather than root. Can I set the user
> >>>>> with
> >>>>> the kwargs?
> >>>>>
> >>>>> Thanks,
> >>>>> Rudi
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>>
> >>>
> >>> --
> >>>
> >>
> >> --
> >> mist.io
> >> down to earth cloud computing
> >>
> >
> >
>
--
--
*Rudolf J. Streif*
Director of Embedded Solutions
The Linux Foundation
rudolf.streif@linux.com
Phone: +1.619.631.5383
Skype: rudolfstreif
PGP: RSA 2048/2048 D6E7D28B
Linux Foundation Events Schedule: events.linuxfoundation.org
Linux Foundation Training Schedule: training.linuxfoundation.org
Re: Newbie questions
Posted by Tomaz Muraus <to...@apache.org>.
I just spent some time looking into it. It looks like paramiko throws "not
a valid DSA private key file" exception if you specify a valid ssh key, but
an invalid username.
This is unfortunate, because it makes it impossible to differentiate
between actual invalid key file and an invalid username.
>From the paramiko debug logs:
DEBUG:paramiko.transport:Trying key 4334c61b988ac5eac5152721d2acae08 from
> test2.pem
> DEBUG:paramiko.transport:userauth is OK
> INFO:paramiko.transport:Authentication (publickey) failed.
Keep the following things in mind:
- test2.pem is in fact a valid and working private key
- SSH agent is not used when you specify a password or a key file (we pass
'look_for_keys': False, and 'allow_agent': False to paramiko).
This actually seems like a paramiko issue so I will try to dig some more
there.
On Fri, Mar 15, 2013 at 8:27 PM, Tomaz Muraus <to...@apache.org> wrote:
> Thanks for your feedback.
>
> I personally think that the first thing we (Libcloud) need to do it detect
> and propagate correct exception to the caller.
>
> For example, if a user provides an invalid username, password or private
> key we should throw InvalidAuthenticationCredentials and if an invalid
> authentication method is provided we should throw
> InvalidAuthenticationMethod.
>
> Because of how SSH protocol works (
> http://tools.ietf.org/html/rfc4252#section-5.1) we can't detect other or
> a more specific errors (please correct me if I'm wrong).
>
> I also agree that "cycling through usernames" is a hacky solution, but
> it's not really much we can do in this case. I'm personally open to adding
> something like that once #1 has been figured out as long as a user can
> disable it or maybe it should even be off by default.
>
> On Fri, Mar 15, 2013 at 3:52 AM, Chris Psaltis <cp...@mist.io> wrote:
>
>> We bumped into this as well as few days ago.
>>
>> The issue is that you can't suppose beforehand what is the ssh_username
>> for a specific image. We currently solve it in our application by creating
>> a map of ami ids -> ssh_usernames and passing the correct one to libcloud.
>> However, I think this should be handled on the libcloud level. One idea is
>> to try/except for several known ssh_usernames, e.g. ec2-user, ubuntu etc.
>> It is not very graceful, but probably it's the most robust solution that
>> will require less maintenance.
>>
>> What ideas do you have? I'm willing to implement it next week.
>>
>> Thanks,
>>
>> Chris
>>
>>
>>
>>
>> On 2013-03-15 00:12, Rudolf Streif wrote:
>>
>>> Thank you, Tomaz. I actually had both issues, username and key. And once
>>> I
>>> added ssh_key to the mix it actually started working. :)
>>>
>>>
>>>
>>>
>>> On Thu, Mar 14, 2013 at 11:50 AM, Tomaz Muraus <to...@apache.org> wrote:
>>>
>>> Yes, you need to pass ssh_username="ubuntu" kwarg to the deploy_node
>>>> method.
>>>>
>>>> I honestly thought this was documented on the website somewhere, but I
>>>> couldn't find it just by quickly glancing over "getting started" and
>>>> "docs". This means that even if it is documented somewhere it's not
>>>> documented / presented in a good way if I can't find it myself. We will
>>>> definitely work on improving the docs and pointing this out
>>>> more prominently.
>>>>
>>>> This thing is also one of the improvements I'm working on for the next
>>>> release and some of the improvements are already available in trunk and
>>>> 0.12.x branch.
>>>>
>>>> Currently, even if you use LIBCLOUD_DEBUG functionality, you get a bad /
>>>> unrelated exception if you use a correct key, but an invalid SSH
>>>> username.
>>>>
>>>> paramiko library log messages are especially confusing. They don't
>>>> indicate
>>>> any error and one of them even says something along the lines of
>>>> "authentication successful".
>>>>
>>>> All of this makes for a bad user experience so I will try to figure out
>>>> the
>>>> best way to detect this issue and throw a more specific exceptions so
>>>> people won't be confused and need to waste a lot of time on this.
>>>>
>>>> On Thu, Mar 14, 2013 at 11:39 AM, Rudolf Streif <
>>>> rstreif@linuxfoundation.org
>>>>
>>>>> wrote:
>>>>>
>>>>
>>>> Thanks, Tomaz.
>>>>>
>>>>> I tried to use deploy_node with EC2. I used a key pair already existing
>>>>> and set the security group right. However, it fails when logging into
>>>>> the
>>>>> node and trying to deploy the scripts. I am doing this:
>>>>>
>>>>> node = ec2driver.deploy_node(name='**test-3', image=image, size=size,
>>>>> deploy=msd, ex_keyname="key")
>>>>>
>>>>> The image is a Ubuntu 12.10 64-bit, ami-7539b41c
>>>>> The size is t1.micro
>>>>>
>>>>> The error I am getting is libcloud.compute.types.**DeploymentError:
>>>>> <DeploymentError: node=i-11f73563, error=Bad authentication type
>>>>> (allowed_types=['publickey']),
>>>>> driver=<libcloud.compute.**drivers.ec2.EC2NodeDriver object at
>>>>> 0xdb0b90>>
>>>>>
>>>>> I can log in just fine using ssh. The issue could be that the image
>>>>> expects logins as user "ubuntu" rather than root. Can I set the user
>>>>> with
>>>>> the kwargs?
>>>>>
>>>>> Thanks,
>>>>> Rudi
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>>
>>> --
>>>
>>
>> --
>> mist.io
>> down to earth cloud computing
>>
>
>
Re: Newbie questions
Posted by Tomaz Muraus <to...@apache.org>.
Thanks for your feedback.
I personally think that the first thing we (Libcloud) need to do it detect
and propagate correct exception to the caller.
For example, if a user provides an invalid username, password or private
key we should throw InvalidAuthenticationCredentials and if an invalid
authentication method is provided we should throw
InvalidAuthenticationMethod.
Because of how SSH protocol works (
http://tools.ietf.org/html/rfc4252#section-5.1) we can't detect other or a
more specific errors (please correct me if I'm wrong).
I also agree that "cycling through usernames" is a hacky solution, but it's
not really much we can do in this case. I'm personally open to adding
something like that once #1 has been figured out as long as a user can
disable it or maybe it should even be off by default.
On Fri, Mar 15, 2013 at 3:52 AM, Chris Psaltis <cp...@mist.io> wrote:
> We bumped into this as well as few days ago.
>
> The issue is that you can't suppose beforehand what is the ssh_username
> for a specific image. We currently solve it in our application by creating
> a map of ami ids -> ssh_usernames and passing the correct one to libcloud.
> However, I think this should be handled on the libcloud level. One idea is
> to try/except for several known ssh_usernames, e.g. ec2-user, ubuntu etc.
> It is not very graceful, but probably it's the most robust solution that
> will require less maintenance.
>
> What ideas do you have? I'm willing to implement it next week.
>
> Thanks,
>
> Chris
>
>
>
>
> On 2013-03-15 00:12, Rudolf Streif wrote:
>
>> Thank you, Tomaz. I actually had both issues, username and key. And once I
>> added ssh_key to the mix it actually started working. :)
>>
>>
>>
>>
>> On Thu, Mar 14, 2013 at 11:50 AM, Tomaz Muraus <to...@apache.org> wrote:
>>
>> Yes, you need to pass ssh_username="ubuntu" kwarg to the deploy_node
>>> method.
>>>
>>> I honestly thought this was documented on the website somewhere, but I
>>> couldn't find it just by quickly glancing over "getting started" and
>>> "docs". This means that even if it is documented somewhere it's not
>>> documented / presented in a good way if I can't find it myself. We will
>>> definitely work on improving the docs and pointing this out
>>> more prominently.
>>>
>>> This thing is also one of the improvements I'm working on for the next
>>> release and some of the improvements are already available in trunk and
>>> 0.12.x branch.
>>>
>>> Currently, even if you use LIBCLOUD_DEBUG functionality, you get a bad /
>>> unrelated exception if you use a correct key, but an invalid SSH
>>> username.
>>>
>>> paramiko library log messages are especially confusing. They don't
>>> indicate
>>> any error and one of them even says something along the lines of
>>> "authentication successful".
>>>
>>> All of this makes for a bad user experience so I will try to figure out
>>> the
>>> best way to detect this issue and throw a more specific exceptions so
>>> people won't be confused and need to waste a lot of time on this.
>>>
>>> On Thu, Mar 14, 2013 at 11:39 AM, Rudolf Streif <
>>> rstreif@linuxfoundation.org
>>>
>>>> wrote:
>>>>
>>>
>>> Thanks, Tomaz.
>>>>
>>>> I tried to use deploy_node with EC2. I used a key pair already existing
>>>> and set the security group right. However, it fails when logging into
>>>> the
>>>> node and trying to deploy the scripts. I am doing this:
>>>>
>>>> node = ec2driver.deploy_node(name='**test-3', image=image, size=size,
>>>> deploy=msd, ex_keyname="key")
>>>>
>>>> The image is a Ubuntu 12.10 64-bit, ami-7539b41c
>>>> The size is t1.micro
>>>>
>>>> The error I am getting is libcloud.compute.types.**DeploymentError:
>>>> <DeploymentError: node=i-11f73563, error=Bad authentication type
>>>> (allowed_types=['publickey']),
>>>> driver=<libcloud.compute.**drivers.ec2.EC2NodeDriver object at
>>>> 0xdb0b90>>
>>>>
>>>> I can log in just fine using ssh. The issue could be that the image
>>>> expects logins as user "ubuntu" rather than root. Can I set the user
>>>> with
>>>> the kwargs?
>>>>
>>>> Thanks,
>>>> Rudi
>>>>
>>>>
>>>>
>>>
>>
>>
>> --
>>
>
> --
> mist.io
> down to earth cloud computing
>
Re: Newbie questions
Posted by Chris Psaltis <cp...@mist.io>.
We bumped into this as well as few days ago.
The issue is that you can't suppose beforehand what is the ssh_username
for a specific image. We currently solve it in our application by
creating a map of ami ids -> ssh_usernames and passing the correct one
to libcloud. However, I think this should be handled on the libcloud
level. One idea is to try/except for several known ssh_usernames, e.g.
ec2-user, ubuntu etc. It is not very graceful, but probably it's the
most robust solution that will require less maintenance.
What ideas do you have? I'm willing to implement it next week.
Thanks,
Chris
On 2013-03-15 00:12, Rudolf Streif wrote:
> Thank you, Tomaz. I actually had both issues, username and key. And
> once I
> added ssh_key to the mix it actually started working. :)
>
>
>
>
> On Thu, Mar 14, 2013 at 11:50 AM, Tomaz Muraus <to...@apache.org>
> wrote:
>
>> Yes, you need to pass ssh_username="ubuntu" kwarg to the deploy_node
>> method.
>>
>> I honestly thought this was documented on the website somewhere, but
>> I
>> couldn't find it just by quickly glancing over "getting started" and
>> "docs". This means that even if it is documented somewhere it's not
>> documented / presented in a good way if I can't find it myself. We
>> will
>> definitely work on improving the docs and pointing this out
>> more prominently.
>>
>> This thing is also one of the improvements I'm working on for the
>> next
>> release and some of the improvements are already available in trunk
>> and
>> 0.12.x branch.
>>
>> Currently, even if you use LIBCLOUD_DEBUG functionality, you get a
>> bad /
>> unrelated exception if you use a correct key, but an invalid SSH
>> username.
>>
>> paramiko library log messages are especially confusing. They don't
>> indicate
>> any error and one of them even says something along the lines of
>> "authentication successful".
>>
>> All of this makes for a bad user experience so I will try to figure
>> out the
>> best way to detect this issue and throw a more specific exceptions so
>> people won't be confused and need to waste a lot of time on this.
>>
>> On Thu, Mar 14, 2013 at 11:39 AM, Rudolf Streif <
>> rstreif@linuxfoundation.org
>>> wrote:
>>
>>> Thanks, Tomaz.
>>>
>>> I tried to use deploy_node with EC2. I used a key pair already
>>> existing
>>> and set the security group right. However, it fails when logging
>>> into the
>>> node and trying to deploy the scripts. I am doing this:
>>>
>>> node = ec2driver.deploy_node(name='test-3', image=image, size=size,
>>> deploy=msd, ex_keyname="key")
>>>
>>> The image is a Ubuntu 12.10 64-bit, ami-7539b41c
>>> The size is t1.micro
>>>
>>> The error I am getting is libcloud.compute.types.DeploymentError:
>>> <DeploymentError: node=i-11f73563, error=Bad authentication type
>>> (allowed_types=['publickey']),
>>> driver=<libcloud.compute.drivers.ec2.EC2NodeDriver object at
>>> 0xdb0b90>>
>>>
>>> I can log in just fine using ssh. The issue could be that the image
>>> expects logins as user "ubuntu" rather than root. Can I set the user
>>> with
>>> the kwargs?
>>>
>>> Thanks,
>>> Rudi
>>>
>>>
>>
>
>
>
> --
--
mist.io
down to earth cloud computing
Re: Newbie questions
Posted by Rudolf Streif <rs...@linuxfoundation.org>.
Thank you, Tomaz. I actually had both issues, username and key. And once I
added ssh_key to the mix it actually started working. :)
On Thu, Mar 14, 2013 at 11:50 AM, Tomaz Muraus <to...@apache.org> wrote:
> Yes, you need to pass ssh_username="ubuntu" kwarg to the deploy_node
> method.
>
> I honestly thought this was documented on the website somewhere, but I
> couldn't find it just by quickly glancing over "getting started" and
> "docs". This means that even if it is documented somewhere it's not
> documented / presented in a good way if I can't find it myself. We will
> definitely work on improving the docs and pointing this out
> more prominently.
>
> This thing is also one of the improvements I'm working on for the next
> release and some of the improvements are already available in trunk and
> 0.12.x branch.
>
> Currently, even if you use LIBCLOUD_DEBUG functionality, you get a bad /
> unrelated exception if you use a correct key, but an invalid SSH username.
>
> paramiko library log messages are especially confusing. They don't indicate
> any error and one of them even says something along the lines of
> "authentication successful".
>
> All of this makes for a bad user experience so I will try to figure out the
> best way to detect this issue and throw a more specific exceptions so
> people won't be confused and need to waste a lot of time on this.
>
> On Thu, Mar 14, 2013 at 11:39 AM, Rudolf Streif <
> rstreif@linuxfoundation.org
> > wrote:
>
> > Thanks, Tomaz.
> >
> > I tried to use deploy_node with EC2. I used a key pair already existing
> > and set the security group right. However, it fails when logging into the
> > node and trying to deploy the scripts. I am doing this:
> >
> > node = ec2driver.deploy_node(name='test-3', image=image, size=size,
> > deploy=msd, ex_keyname="key")
> >
> > The image is a Ubuntu 12.10 64-bit, ami-7539b41c
> > The size is t1.micro
> >
> > The error I am getting is libcloud.compute.types.DeploymentError:
> > <DeploymentError: node=i-11f73563, error=Bad authentication type
> > (allowed_types=['publickey']),
> > driver=<libcloud.compute.drivers.ec2.EC2NodeDriver object at 0xdb0b90>>
> >
> > I can log in just fine using ssh. The issue could be that the image
> > expects logins as user "ubuntu" rather than root. Can I set the user with
> > the kwargs?
> >
> > Thanks,
> > Rudi
> >
> >
>
--
--
*Rudolf J. Streif*
Director of Embedded Solutions
The Linux Foundation
rudolf.streif@linux.com
Phone: +1.619.631.5383
Skype: rudolfstreif
PGP: RSA 2048/2048 D6E7D28B
Linux Foundation Events Schedule: events.linuxfoundation.org
Linux Foundation Training Schedule: training.linuxfoundation.org
Re: Newbie questions
Posted by Tomaz Muraus <to...@apache.org>.
Yes, you need to pass ssh_username="ubuntu" kwarg to the deploy_node method.
I honestly thought this was documented on the website somewhere, but I
couldn't find it just by quickly glancing over "getting started" and
"docs". This means that even if it is documented somewhere it's not
documented / presented in a good way if I can't find it myself. We will
definitely work on improving the docs and pointing this out
more prominently.
This thing is also one of the improvements I'm working on for the next
release and some of the improvements are already available in trunk and
0.12.x branch.
Currently, even if you use LIBCLOUD_DEBUG functionality, you get a bad /
unrelated exception if you use a correct key, but an invalid SSH username.
paramiko library log messages are especially confusing. They don't indicate
any error and one of them even says something along the lines of
"authentication successful".
All of this makes for a bad user experience so I will try to figure out the
best way to detect this issue and throw a more specific exceptions so
people won't be confused and need to waste a lot of time on this.
On Thu, Mar 14, 2013 at 11:39 AM, Rudolf Streif <rstreif@linuxfoundation.org
> wrote:
> Thanks, Tomaz.
>
> I tried to use deploy_node with EC2. I used a key pair already existing
> and set the security group right. However, it fails when logging into the
> node and trying to deploy the scripts. I am doing this:
>
> node = ec2driver.deploy_node(name='test-3', image=image, size=size,
> deploy=msd, ex_keyname="key")
>
> The image is a Ubuntu 12.10 64-bit, ami-7539b41c
> The size is t1.micro
>
> The error I am getting is libcloud.compute.types.DeploymentError:
> <DeploymentError: node=i-11f73563, error=Bad authentication type
> (allowed_types=['publickey']),
> driver=<libcloud.compute.drivers.ec2.EC2NodeDriver object at 0xdb0b90>>
>
> I can log in just fine using ssh. The issue could be that the image
> expects logins as user "ubuntu" rather than root. Can I set the user with
> the kwargs?
>
> Thanks,
> Rudi
>
>
Re: Newbie questions
Posted by Rudolf Streif <rs...@linuxfoundation.org>.
Thanks, Tomaz.
I tried to use deploy_node with EC2. I used a key pair already existing and
set the security group right. However, it fails when logging into the node
and trying to deploy the scripts. I am doing this:
node = ec2driver.deploy_node(name='test-3', image=image, size=size,
deploy=msd, ex_keyname="key")
The image is a Ubuntu 12.10 64-bit, ami-7539b41c
The size is t1.micro
The error I am getting is libcloud.compute.types.DeploymentError:
<DeploymentError: node=i-11f73563, error=Bad authentication type
(allowed_types=['publickey']),
driver=<libcloud.compute.drivers.ec2.EC2NodeDriver object at 0xdb0b90>>
I can log in just fine using ssh. The issue could be that the image expects
logins as user "ubuntu" rather than root. Can I set the user with the
kwargs?
Thanks,
Rudi
Re: Newbie questions
Posted by Tomaz Muraus <to...@apache.org>.
Welcome to the libcloud community :)
First thank you for your feedback.
And now answers / feedback to your questions.
1. One of the main goals of Libcloud is to make using and switching between
multiple providers easier.
To make this possible, base API only exposes functionality which is
supported by majority of providers we have drivers for.
In RDBMS abstraction terms this usually means that the base API only
supports functionality which is for example defined in the ANSI-89 SQL
standard and your are discouraged from using other (e.g. Postgresql /
Oracle specific) functionality.
In Libcloud we do support some providers specific functionality
using extension methods (methods prefixed with "ex_"), but we discourage
users from using them if they use or want to use multiple providers in the
future.
2.
Keep in mind that the big chunk of the base API is now 4+ years old. A lot
has changes since them and it might makes sense to move more methods into
the base API.
Judging by my memory, having node start and stop method in the base API
does seems reasonable. Next step would be checking documentation of all the
providers we support and determine if majority of them actually supports
this functionality.
As far as _wait_until_running method goes - this method has been "promoted"
to the base API in the latest (0.12.1) release.
P.S. Here is a link to a thread from a couple of weeks ago which contains
answers to similar questions -
http://mail-archives.apache.org/mod_mbox/libcloud-users/201302.mbox/%3C1A1F3532-20E5-49D0-9F73-C75D1D36F9DA%40lal.in2p3.fr%3E
Let us know if you have any more questions.
On Tue, Mar 12, 2013 at 8:34 AM, Rudolf Streif
<rs...@linuxfoundation.org>wrote:
> Hello World, :)
>
> I am new to libcloud and have been using it for a little bit. I very much
> like the driver concept for abstracting the underlying cloud provider APIs
> which is very similar to the RDBMS drivers.
>
> So far I have mainly worked with the 'compute' drivers and I have a couple
> of newbie questions:
>
>
> 1. It seems odd to me that the NodeDriver class only defines methods for
> create_node, reboot_node and destroy_node but no methods for start_node
> and
> stop_node. I can see that some cloud APIs do not provide such
> functionality
> but very common ones such as EC2, CloudSigma, etc. do. Consequently,
> someone has implemented those methods as extensions in the particular
> drivers. Is that a design consideration? Why not making them common in
> the
> NodeDriver class?
>
> 2. Similar to 1 for the Node class. There are methods for rebooting and
> destroying a node but none for starting and stopping it. There is also
> no
> wait_until_running method in the Node class. Why is that?
>
> I have cloned the git repo and am adding the functionality I need for my
> application such as the above and other stuff. I am happy to contribute it
> back if there is interest.
>
> Cheers,
> Rudi
>