StratusLab compute driver

[Charles Loomis <lo...@lal.in2p3.fr>]

Hello,

I wanted to announce that there is now a StratusLab compute driver available (see below) and to ask a few questions that came up when developing the driver.  Here are the questions, putting the more technical ones first: 
 
* Why is there no list_volumes() method in NodeDriver?  StratusLab treats volumes as separate resources that can exist independently of a running virtual machine.  Without a list_volumes() function, there really isn't a way to find existing volumes and then use them by attaching them to a node.

* Why is there no CPU (and/or core) fields in NodeSize?  For the StratusLab driver, I've created a subclass with this information, but this seems like a strange quantity to be missing when defining machine resources.

* An attribute is used in a Node to hold the state.  For us, this is a dynamic quantity that can change independently of the Node object.  Because of this, I've also subclassed Node and turned the state attribute into an active property (calling out to the server when the state is requested).  This works, but is probably a bit misleading to users because a "cheap" attribute reference has been turned into an "expensive" function call.  In deploy_node(), it seems that list_nodes() is used to get the updated state of running nodes.  Wouldn't it be better to expose the state information via a method?  

* Why does list_nodes() not take a location?  Our reference StratusLab cloud contains two federated site (locations).  It would seem more reasonable to have list_nodes() follow the same pattern as the other list_*() functions and allow a way to get the nodes from a single location.

* The StratusLab driver depends on our own python code as well as a few external libraries.  As a 'third-party' driver this isn't currently a problem, but does this raise any issues if this code will eventually be contributed directly to Libcloud?  Are there any constraints other than licence considerations? 

* When testing deploy_node(), I had problems with using RSA ssh keys.  It seems that paramiko prefers (requires?) DSA keys.  I didn't dig very deep with this as I was trying to get a demo to work, but has anyone else run into something similar? 

Cheers.

Cal


P.S. The driver is available from PyPi:

https://pypi.python.org/pypi/stratuslab-libcloud-drivers

The code in GitHub:

https://github.com/StratusLab/libcloud-drivers

And a news item with some usage information here:

http://stratuslab.eu//news/2013/02/20/news-libcloud.html

Re: StratusLab compute driver

[Tomaz Muraus <to...@apache.org>]

First sorry for delay.

My answers are in-line.

On Wed, Feb 20, 2013 at 9:08 AM, Charles Loomis <lo...@lal.in2p3.fr> wrote:

> Hello,
>
> I wanted to announce that there is now a StratusLab compute driver
> available (see below) and to ask a few questions that came up when
> developing the driver.  Here are the questions, putting the more technical
> ones first:
>
> * Why is there no list_volumes() method in NodeDriver?  StratusLab treats
> volumes as separate resources that can exist independently of a running
> virtual machine.  Without a list_volumes() function, there really isn't a
> way to find existing volumes and then use them by attaching them to a node.
>

To be honest, I have no clue. I do agree with your and think this method
should be included in the base NodeDriver class.


> * Why is there no CPU (and/or core) fields in NodeSize?  For the
> StratusLab driver, I've created a subclass with this information, but this
> seems like a strange quantity to be missing when defining machine resources.
>

One of the goals of NodeSize was to only expose attributes which are common
to majority of the providers we support. When we started most of the
providers didn't expose this information and that's why we don't expose it.

I think that it makes sense to re-evaluate this decision now and think
about at least including "vcpus" attribute on the NodeSize. This attribute
would contain a number of virtual CPUs this size exposes.


> * An attribute is used in a Node to hold the state.  For us, this is a
> dynamic quantity that can change independently of the Node object.  Because
> of this, I've also subclassed Node and turned the state attribute into an
> active property (calling out to the server when the state is requested).
>  This works, but is probably a bit misleading to users because a "cheap"
> attribute reference has been turned into an "expensive" function call.  In
> deploy_node(), it seems that list_nodes() is used to get the updated state
> of running nodes.  Wouldn't it be better to expose the state information
> via a method?
>

I do agree that it's usually not good to do expensive stuff in the
property, especially if you mask a normal attribute and user is not aware
of it.

Adding a new Node.get_current_state method sounds reasonable to me,
although issue with "state" Node attribute being out of date still exists.

I think this could be fine as long as we document that "state" attribute is
dynamic and if you hold a reference to a Node object and you want to get
the latest state you either need to call driver.list_nodes or
node.get_current_state.


> * Why does list_nodes() not take a location?  Our reference StratusLab
> cloud contains two federated site (locations).  It would seem more
> reasonable to have list_nodes() follow the same pattern as the other
> list_*() functions and allow a way to get the nodes from a single location.
>

It's an old artifact / legacy code. We are slowly moving away from this in
trunk with a "datacenter" attribute in the driver constructor.


> * The StratusLab driver depends on our own python code as well as a few
> external libraries.  As a 'third-party' driver this isn't currently a
> problem, but does this raise any issues if this code will eventually be
> contributed directly to Libcloud?  Are there any constraints other than
> licence considerations?
>

None of the existing drivers in Libcloud currently have an external
dependency. We try to avoid external dependencies (especially provider
Python bindings) so we can control the whole user experience and make sure
it's good (tests, etc.) and works with all the supported versions.

As far as the license goes - if the code is to be included in Libcloud it
can also be licensed under the following licenses
http://www.apache.org/legal/3party.html#category-a although we prefer
Apache 2.0.


> * When testing deploy_node(), I had problems with using RSA ssh keys.  It
> seems that paramiko prefers (requires?) DSA keys.  I didn't dig very deep
> with this as I was trying to get a demo to work, but has anyone else run
> into something similar?
>

I'm not familiar with any issues with RSA SSH keys. Did you try running
your deploy script with LIBCLOUD_DEBUG=/dev/stderr and inspecting the debug
output?

If you are still stuck please gist the debug output and post it here and we
can have a look.


> Cheers.
>
> Cal
>

Thanks!

This post includes a lot of really good questions and exposes things we can
improve in Libcloud.


>
>
> P.S. The driver is available from PyPi:
>
> https://pypi.python.org/pypi/stratuslab-libcloud-drivers
>
> The code in GitHub:
>
> https://github.com/StratusLab/libcloud-drivers
>
> And a news item with some usage information here:
>
> http://stratuslab.eu//news/2013/02/20/news-libcloud.html
>
>