[dev] [RFC] Refactoring of the base OpenStack compute drivers

[Tomaz Muraus <to...@apache.org>]

Howdy,

Currently base OpenStack driver class (OpenStack_1_1_NodeDriver) from which
other OpenStack based provider drivers such as Rackspace and HP Cloud
inherit from implements all the common functionality.

In addition to that, it also implements some functionality offered by
OpenStack extensions such as floating IPs and floating IP pools. Those
extensions might not be supported by all the OpenStack based providers.

This means that if you will try to use this method on a provider driver
which doesn't support it, you will get an ugly error message back.

Because of that, I have decided to refactor OpenStack classes a bit and
make the following changes:

1. New OpenStack_1_1_BaseNodeDriver base class which only implements common
OpenStack functionality exposed by the OpenStack API v1.1.

This class should never be instantiated directly. It should only be used as
a base class for other provider driver such as Rackspace and HP Cloud to
inherit from.

2. I've moved OpenStack extensions functionality to separate mixin classes.

This means that now all the OpenStack providers which implement floating IP
functionality should, in addition to the base class, also inherit
from OpenStackFloatingIPsExtensionMixin class.

3. I've updated affected OpenStack drivers (Rackspace and HP Cloud).

4. Modified OpenStack_1_1_NodeDriver to inherit from the base class and all
the extension mixin classes. This way this change is fully backward
compatible and this class will behave exactly the same as it did in the
past.

Proposed implementation is available at
https://github.com/Kami/libcloud/compare/trunk...openstack_driver_extensions_refactor

Benefits of this approach over the existing one:

1. It makes code nicer and easier to reuse.

2. It makes programmatic introspection easier (better automatic generated
documentation, etc.).

You can now know that some OpenStack based provider supports floating IP
functionality because those methods are present on the driver class.
Previously methods were always present so this was not always the case.

Open questions (not necessary blockers for getting those changes merged
into trunk):

1. We need to decide how to make this work better for people who use
OpenStack driver to connect to the OpenStack based cloud or OpenStack
installation.

Currently the OpenStack driver implements all the methods which is not
ideal.

Couple of options I can think of from top of my head:

1. Allow user to pass "extensions" argument to the OpenStack driver
constructor (e.g. extensions=['floating_ips']).

This argument specify which extension methods should be available to the
driver.

2. IIRC, OpenStack API exposes an endpoint which lists all the supported
extensions.

This means we could detect supported extensions during run-time and throw
NotImplemented exception or do something along those lines if some
extension method is not supported.

The disadvantage of this approach is that it's less explicit and makes
programmatic instrospection impossible since "capability detection" happens
during run time.

Questions? Comments? Feedback?

Thanks

Re: [dev] [RFC] Refactoring of the base OpenStack compute drivers

[Andrew Mann <an...@divvycloud.com>]

I think it's vital that the such information be queried if possible,
embedded if not, and only require the user to provide the information if
there's no other option.  There are a few cases I can think of off the top
of my head:

1) OpenStack regions are installation-specific, but today you need to pass
a region to libcloud to instantiate a driver. This puts a burden on the
user to coordinate a list of available regions separately from the
OpenStack install they are working with.
2) OpenStack network drivers are exposed through the service catalog, but
are not explicitly exposed through the OpenStack UI.  In order for a user
to know what network system is in use they would have to look up their
OpenStack version, then go dive into the OS docs to find out whether
Quantum or Neutron was the network system.
3) Today libcloud already exposes the different types of compute nodes
available in AWS regions.  I haven't looked to see if AWS has exposed this
through an API recently, but there's already precedent for putting this
kind of list-of-configuration-options in the libcloud layer even if it's
not exposed via API from the provider.
4) HP Cloud is based off OpenStack IceHouse, but they have disabled the
deprecated compute networking API calls so they return a 404.  As far as I
can tell there is no way for the end user to determine this without trial
and error. In this case the HP Cloud driver(s) should reflect this
configuration option directly by making the expected calls route to a
Neutron-compatible call even though the underlying base OpenStack driver
may still have the compute calls.





On Mon, Jun 2, 2014 at 10:49 AM, Tomaz Muraus <to...@apache.org> wrote:

> On Mon, Jun 2, 2014 at 8:58 AM, Sebastien Goasguen <ru...@gmail.com>
> wrote:
>
> >
> > On May 30, 2014, at 7:08 AM, Tomaz Muraus <to...@apache.org> wrote:
> >
> > > Howdy,
> > >
> > > Currently base OpenStack driver class (OpenStack_1_1_NodeDriver) from
> > which
> > > other OpenStack based provider drivers such as Rackspace and HP Cloud
> > > inherit from implements all the common functionality.
> > >
> > > In addition to that, it also implements some functionality offered by
> > > OpenStack extensions such as floating IPs and floating IP pools. Those
> > > extensions might not be supported by all the OpenStack based providers.
> > >
> > > This means that if you will try to use this method on a provider driver
> > > which doesn't support it, you will get an ugly error message back.
> > >
> >
> > We kind f have the same issue in CloudStack. When you in install
> > cloudstack, you can choose what type of
> > zone configuration you want (basic or advanced). The choice is based on
> > networking setup.
> > Some methods don't make sense in basic mode. So right now, if you
> > instantiate the cloudstack driver and try
> > certain methods (e.g ex_add_port_forwarding_rule) you will get an error.
> >
>
> Yeah, I didn't explicitly mention this in my original email, but my
> intention was for us to come up with a unified approach of handling this
> which can also be used in other drivers.
>
> I will also look at updating the CloudStack driver & friends (Exoscale,
> Ikoula) in my pull request. It would make it easier for me if you can list
> which methods should be available in basic and which in advanced networking
> setup.
>
>
> > > Because of that, I have decided to refactor OpenStack classes a bit and
> > > make the following changes:
> > >
> > > 1. New OpenStack_1_1_BaseNodeDriver base class which only implements
> > common
> > > OpenStack functionality exposed by the OpenStack API v1.1.
> > >
> > > This class should never be instantiated directly. It should only be
> used
> > as
> > > a base class for other provider driver such as Rackspace and HP Cloud
> to
> > > inherit from.
> > >
> > > 2. I've moved OpenStack extensions functionality to separate mixin
> > classes.
> > >
> > > This means that now all the OpenStack providers which implement
> floating
> > IP
> > > functionality should, in addition to the base class, also inherit
> > > from OpenStackFloatingIPsExtensionMixin class.
> > >
> > > 3. I've updated affected OpenStack drivers (Rackspace and HP Cloud).
> > >
> > > 4. Modified OpenStack_1_1_NodeDriver to inherit from the base class and
> > all
> > > the extension mixin classes. This way this change is fully backward
> > > compatible and this class will behave exactly the same as it did in the
> > > past.
> >
> > +1
> >
> > >
> > > Proposed implementation is available at
> > >
> >
> https://github.com/Kami/libcloud/compare/trunk...openstack_driver_extensions_refactor
> > >
> > > Benefits of this approach over the existing one:
> > >
> > > 1. It makes code nicer and easier to reuse.
> > >
> > > 2. It makes programmatic introspection easier (better automatic
> generated
> > > documentation, etc.).
> > >
> > > You can now know that some OpenStack based provider supports floating
> IP
> > > functionality because those methods are present on the driver class.
> > > Previously methods were always present so this was not always the case.
> > >
> > > Open questions (not necessary blockers for getting those changes merged
> > > into trunk):
> > >
> > > 1. We need to decide how to make this work better for people who use
> > > OpenStack driver to connect to the OpenStack based cloud or OpenStack
> > > installation.
> > >
> > > Currently the OpenStack driver implements all the methods which is not
> > > ideal.
> > >
> > > Couple of options I can think of from top of my head:
> > >
> > > 1. Allow user to pass "extensions" argument to the OpenStack driver
> > > constructor (e.g. extensions=['floating_ips']).
> > >
> > > This argument specify which extension methods should be available to
> the
> > > driver.
> > >
> > > 2. IIRC, OpenStack API exposes an endpoint which lists all the
> supported
> > > extensions.
> > >
> > > This means we could detect supported extensions during run-time and
> throw
> > > NotImplemented exception or do something along those lines if some
> > > extension method is not supported.
> > >
> > > The disadvantage of this approach is that it's less explicit and makes
> > > programmatic instrospection impossible since "capability detection"
> > happens
> > > during run time.
> > >
> > > Questions? Comments? Feedback?
> > >
> > > Thanks
> >
> >
>



-- 
Andrew Mann
DivvyCloud Inc.
www.divvycloud.com

Re: [dev] [RFC] Refactoring of the base OpenStack compute drivers

[Tomaz Muraus <to...@apache.org>]

On Mon, Jun 2, 2014 at 8:58 AM, Sebastien Goasguen <ru...@gmail.com> wrote:

>
> On May 30, 2014, at 7:08 AM, Tomaz Muraus <to...@apache.org> wrote:
>
> > Howdy,
> >
> > Currently base OpenStack driver class (OpenStack_1_1_NodeDriver) from
> which
> > other OpenStack based provider drivers such as Rackspace and HP Cloud
> > inherit from implements all the common functionality.
> >
> > In addition to that, it also implements some functionality offered by
> > OpenStack extensions such as floating IPs and floating IP pools. Those
> > extensions might not be supported by all the OpenStack based providers.
> >
> > This means that if you will try to use this method on a provider driver
> > which doesn't support it, you will get an ugly error message back.
> >
>
> We kind f have the same issue in CloudStack. When you in install
> cloudstack, you can choose what type of
> zone configuration you want (basic or advanced). The choice is based on
> networking setup.
> Some methods don't make sense in basic mode. So right now, if you
> instantiate the cloudstack driver and try
> certain methods (e.g ex_add_port_forwarding_rule) you will get an error.
>

Yeah, I didn't explicitly mention this in my original email, but my
intention was for us to come up with a unified approach of handling this
which can also be used in other drivers.

I will also look at updating the CloudStack driver & friends (Exoscale,
Ikoula) in my pull request. It would make it easier for me if you can list
which methods should be available in basic and which in advanced networking
setup.


> > Because of that, I have decided to refactor OpenStack classes a bit and
> > make the following changes:
> >
> > 1. New OpenStack_1_1_BaseNodeDriver base class which only implements
> common
> > OpenStack functionality exposed by the OpenStack API v1.1.
> >
> > This class should never be instantiated directly. It should only be used
> as
> > a base class for other provider driver such as Rackspace and HP Cloud to
> > inherit from.
> >
> > 2. I've moved OpenStack extensions functionality to separate mixin
> classes.
> >
> > This means that now all the OpenStack providers which implement floating
> IP
> > functionality should, in addition to the base class, also inherit
> > from OpenStackFloatingIPsExtensionMixin class.
> >
> > 3. I've updated affected OpenStack drivers (Rackspace and HP Cloud).
> >
> > 4. Modified OpenStack_1_1_NodeDriver to inherit from the base class and
> all
> > the extension mixin classes. This way this change is fully backward
> > compatible and this class will behave exactly the same as it did in the
> > past.
>
> +1
>
> >
> > Proposed implementation is available at
> >
> https://github.com/Kami/libcloud/compare/trunk...openstack_driver_extensions_refactor
> >
> > Benefits of this approach over the existing one:
> >
> > 1. It makes code nicer and easier to reuse.
> >
> > 2. It makes programmatic introspection easier (better automatic generated
> > documentation, etc.).
> >
> > You can now know that some OpenStack based provider supports floating IP
> > functionality because those methods are present on the driver class.
> > Previously methods were always present so this was not always the case.
> >
> > Open questions (not necessary blockers for getting those changes merged
> > into trunk):
> >
> > 1. We need to decide how to make this work better for people who use
> > OpenStack driver to connect to the OpenStack based cloud or OpenStack
> > installation.
> >
> > Currently the OpenStack driver implements all the methods which is not
> > ideal.
> >
> > Couple of options I can think of from top of my head:
> >
> > 1. Allow user to pass "extensions" argument to the OpenStack driver
> > constructor (e.g. extensions=['floating_ips']).
> >
> > This argument specify which extension methods should be available to the
> > driver.
> >
> > 2. IIRC, OpenStack API exposes an endpoint which lists all the supported
> > extensions.
> >
> > This means we could detect supported extensions during run-time and throw
> > NotImplemented exception or do something along those lines if some
> > extension method is not supported.
> >
> > The disadvantage of this approach is that it's less explicit and makes
> > programmatic instrospection impossible since "capability detection"
> happens
> > during run time.
> >
> > Questions? Comments? Feedback?
> >
> > Thanks
>
>

Re: [dev] [RFC] Refactoring of the base OpenStack compute drivers

[Sebastien Goasguen <ru...@gmail.com>]

On May 30, 2014, at 7:08 AM, Tomaz Muraus <to...@apache.org> wrote:

> Howdy,
> 
> Currently base OpenStack driver class (OpenStack_1_1_NodeDriver) from which
> other OpenStack based provider drivers such as Rackspace and HP Cloud
> inherit from implements all the common functionality.
> 
> In addition to that, it also implements some functionality offered by
> OpenStack extensions such as floating IPs and floating IP pools. Those
> extensions might not be supported by all the OpenStack based providers.
> 
> This means that if you will try to use this method on a provider driver
> which doesn't support it, you will get an ugly error message back.
> 

We kind f have the same issue in CloudStack. When you in install cloudstack, you can choose what type of
zone configuration you want (basic or advanced). The choice is based on networking setup.
Some methods don't make sense in basic mode. So right now, if you instantiate the cloudstack driver and try 
certain methods (e.g ex_add_port_forwarding_rule) you will get an error.

> Because of that, I have decided to refactor OpenStack classes a bit and
> make the following changes:
> 
> 1. New OpenStack_1_1_BaseNodeDriver base class which only implements common
> OpenStack functionality exposed by the OpenStack API v1.1.
> 
> This class should never be instantiated directly. It should only be used as
> a base class for other provider driver such as Rackspace and HP Cloud to
> inherit from.
> 
> 2. I've moved OpenStack extensions functionality to separate mixin classes.
> 
> This means that now all the OpenStack providers which implement floating IP
> functionality should, in addition to the base class, also inherit
> from OpenStackFloatingIPsExtensionMixin class.
> 
> 3. I've updated affected OpenStack drivers (Rackspace and HP Cloud).
> 
> 4. Modified OpenStack_1_1_NodeDriver to inherit from the base class and all
> the extension mixin classes. This way this change is fully backward
> compatible and this class will behave exactly the same as it did in the
> past.

+1

> 
> Proposed implementation is available at
> https://github.com/Kami/libcloud/compare/trunk...openstack_driver_extensions_refactor
> 
> Benefits of this approach over the existing one:
> 
> 1. It makes code nicer and easier to reuse.
> 
> 2. It makes programmatic introspection easier (better automatic generated
> documentation, etc.).
> 
> You can now know that some OpenStack based provider supports floating IP
> functionality because those methods are present on the driver class.
> Previously methods were always present so this was not always the case.
> 
> Open questions (not necessary blockers for getting those changes merged
> into trunk):
> 
> 1. We need to decide how to make this work better for people who use
> OpenStack driver to connect to the OpenStack based cloud or OpenStack
> installation.
> 
> Currently the OpenStack driver implements all the methods which is not
> ideal.
> 
> Couple of options I can think of from top of my head:
> 
> 1. Allow user to pass "extensions" argument to the OpenStack driver
> constructor (e.g. extensions=['floating_ips']).
> 
> This argument specify which extension methods should be available to the
> driver.
> 
> 2. IIRC, OpenStack API exposes an endpoint which lists all the supported
> extensions.
> 
> This means we could detect supported extensions during run-time and throw
> NotImplemented exception or do something along those lines if some
> extension method is not supported.
> 
> The disadvantage of this approach is that it's less explicit and makes
> programmatic instrospection impossible since "capability detection" happens
> during run time.
> 
> Questions? Comments? Feedback?
> 
> Thanks