[dev] Common pattern for drivers which support multiple API versions

[Tomaž Muraus <to...@apache.org>]

Hi all,

Recently a need for a single driver to support multiple API versions has
arose.

Good example of this is OpenStack driver - currently we support version 1.0
of the API, but we also plan to support version 1.1. Another example is
OpenNebula driver - currently we support version 1.4 of the API, but
versions 2.0 and 3.0 are already out and some of our users would like to use
them.

Currently we have no good and undefined way to handle this, so here is my
proposal.

Each provider driver which supports multiple API versions needs to have at
least two different classes:

- base driver class - this class implements common / api version agnostic
provider functionality
- version specific class - this class inherits from the base one and
implements version specific functionality

Base class constructor method should also act as a factory method and accept
"version" argument. Based on this version argument, the method would
instantiate a correct version specific class. If no version is provided we
should probably default to the latest version (?).

This approach also works OK with our current get_driver and provider
constants functionality - we still only need 1 constant per provider

Mike Nerone has already implemented something like this in his original
OpenStack branch - https://github.com/apache/libcloud/pull/26/files#L5R107

Feedback is welcome.

Thanks,
Tomaz

Re: [dev] Common pattern for drivers which support multiple API versions

[Tomaž Muraus <to...@apache.org>]

Hi,

Yes, we have decided to go with the factory approach. You can see an example
in the OpenStack driver which has been merged recently -
https://svn.apache.org/viewvc/libcloud/trunk/libcloud/compute/drivers/openstack.py?r1=1183722&r2=1183721&pathrev=1183722
.

For now, I have just used a simple if statement inside the __new__ method.
Later on we should implement some kind of per-driver level class registry.

For example, BaseDriver class would have
_register_api_version_class(api_version, cls) method and this would add an
entry to the dictionary on the driver class.

When this is done we can also add _get_driver_for_api_version method to the
BaseDriver class and all the driver classes which support multiple API
versions would just call this method inside __new__.

In general I'm not too opinionated about the "one file per class", but for
now I don't really want to mess with the directory structure too much and I
would prefer to keep everything inside a single file.

Thanks,
Tomaz

On Sun, Oct 16, 2011 at 11:16 PM, Hutson Betts <hu...@tamu.edu> wrote:

> To All,
>
> I actually wrote the e-mail contents, those surrounded by equal signs,
> in response to the multiple API question mid last week, but I had to fly
> out the next morning and didn't have an opportunity to submit my patch
> or response.
>
> ======================================================================
> As part of my own current work, I am developing an application for
> OpenNebula 3.0 utilizing libcloud as my cloud management tool. However,
> libcloud is currently designed against OpenNebula 1.4, which was
> released in December 2009, and has been superseded by 2.0, 2.2, and 3.0.
>
> Now, the OpenNebula situation is slightly convoluted because the
> OpenNebula API, as implemented by libcloud, is actually based on the
> OCCI API which has been implemented based on the OCCI Working Group
> Draft. Therefore, OpenNebula's 1.4 API was based on the OCCI draft at
> that time, and their latest version, 3.0, is based on the current draft
> version. Therefore, there's no definitive publication to base an API
> spec on.
>
> With regard to my input on this matter, I believe that either of Mike
> Nerone's solutions,
> https://github.com/apache/libcloud/pull/28
> https://github.com/apache/libcloud/pull/26
> Are acceptable solutions for supporting the on-going evolution of APIs
> and their derivatives.
>
> However, I believe a method must be decided upon before specialized
> forks have to be created to support the new features needed by users of
> libcloud.
>
> I guess the major decision is what paradigm to use when implementing
> multiple API versions.
>
> I agree with Tomaz's and Mike's proposals for the establishing of a base
> class, with version specific classes inheriting the base class while
> implementing API specific deviations.
>
> It is my opinion that Pull 26 is more aesthetically pleasing since it
> creates a dedicated folder to hold version files and then utilizes a
> factory method to instantiate the appropriate class. With this method a
> single file is not cluttered with multiple versions, only a single
> provider constant is required, and it does not become the responsibility
> of the user to instantiate a version specific class name.
>
> To further the matter, I've created a ticket associated with this matter
> with regard to OpenNebula, and I've attached a patch which implements a
> similar approach as Mike's pull request 26. (I based my patch on Mike's
> work).
> ======================================================================
>
> The ticket I've created is LIBCLOUD-120. As mentioned in the ticket, the
> patch currently allows me to communicate with an OpenNebula 2.2
> installation using the 3.0 API. The 1.4 API won't work with anything
> greater than v1.4.
>
> I hope the patch, or a condensed version of it, will be merged into
> trunk soon.
>
> If test fixtures are requested, or a condensed version is desired,
> please let me know.
>
> --
> Hutson Betts
> Computer Science and Engineering
> Texas A&M University
>
>
>
> On Thu, 2011-10-13 at 01:24 +0200, Tomaž Muraus wrote:
> > Hi all,
> >
> > Recently a need for a single driver to support multiple API versions has
> > arose.
> >
> > Good example of this is OpenStack driver - currently we support version
> 1.0
> > of the API, but we also plan to support version 1.1. Another example is
> > OpenNebula driver - currently we support version 1.4 of the API, but
> > versions 2.0 and 3.0 are already out and some of our users would like to
> use
> > them.
> >
> > Currently we have no good and undefined way to handle this, so here is my
> > proposal.
> >
> > Each provider driver which supports multiple API versions needs to have
> at
> > least two different classes:
> >
> > - base driver class - this class implements common / api version agnostic
> > provider functionality
> > - version specific class - this class inherits from the base one and
> > implements version specific functionality
> >
> > Base class constructor method should also act as a factory method and
> accept
> > "version" argument. Based on this version argument, the method would
> > instantiate a correct version specific class. If no version is provided
> we
> > should probably default to the latest version (?).
> >
> > This approach also works OK with our current get_driver and provider
> > constants functionality - we still only need 1 constant per provider
> >
> > Mike Nerone has already implemented something like this in his original
> > OpenStack branch -
> https://github.com/apache/libcloud/pull/26/files#L5R107
> >
> > Feedback is welcome.
> >
> > Thanks,
> > Tomaz
>
>

Re: [dev] Common pattern for drivers which support multiple API versions

[Hutson Betts <hu...@tamu.edu>]

To All,

I actually wrote the e-mail contents, those surrounded by equal signs,
in response to the multiple API question mid last week, but I had to fly
out the next morning and didn't have an opportunity to submit my patch
or response.

======================================================================
As part of my own current work, I am developing an application for
OpenNebula 3.0 utilizing libcloud as my cloud management tool. However,
libcloud is currently designed against OpenNebula 1.4, which was
released in December 2009, and has been superseded by 2.0, 2.2, and 3.0.

Now, the OpenNebula situation is slightly convoluted because the
OpenNebula API, as implemented by libcloud, is actually based on the
OCCI API which has been implemented based on the OCCI Working Group
Draft. Therefore, OpenNebula's 1.4 API was based on the OCCI draft at
that time, and their latest version, 3.0, is based on the current draft
version. Therefore, there's no definitive publication to base an API
spec on.

With regard to my input on this matter, I believe that either of Mike
Nerone's solutions, 
https://github.com/apache/libcloud/pull/28
https://github.com/apache/libcloud/pull/26
Are acceptable solutions for supporting the on-going evolution of APIs
and their derivatives.

However, I believe a method must be decided upon before specialized
forks have to be created to support the new features needed by users of
libcloud.

I guess the major decision is what paradigm to use when implementing
multiple API versions.

I agree with Tomaz's and Mike's proposals for the establishing of a base
class, with version specific classes inheriting the base class while
implementing API specific deviations.

It is my opinion that Pull 26 is more aesthetically pleasing since it
creates a dedicated folder to hold version files and then utilizes a
factory method to instantiate the appropriate class. With this method a
single file is not cluttered with multiple versions, only a single
provider constant is required, and it does not become the responsibility
of the user to instantiate a version specific class name.

To further the matter, I've created a ticket associated with this matter
with regard to OpenNebula, and I've attached a patch which implements a
similar approach as Mike's pull request 26. (I based my patch on Mike's
work).
======================================================================

The ticket I've created is LIBCLOUD-120. As mentioned in the ticket, the
patch currently allows me to communicate with an OpenNebula 2.2
installation using the 3.0 API. The 1.4 API won't work with anything
greater than v1.4.

I hope the patch, or a condensed version of it, will be merged into
trunk soon.

If test fixtures are requested, or a condensed version is desired,
please let me know.

-- 
Hutson Betts
Computer Science and Engineering
Texas A&M University



On Thu, 2011-10-13 at 01:24 +0200, Tomaž Muraus wrote: 
> Hi all,
> 
> Recently a need for a single driver to support multiple API versions has
> arose.
> 
> Good example of this is OpenStack driver - currently we support version 1.0
> of the API, but we also plan to support version 1.1. Another example is
> OpenNebula driver - currently we support version 1.4 of the API, but
> versions 2.0 and 3.0 are already out and some of our users would like to use
> them.
> 
> Currently we have no good and undefined way to handle this, so here is my
> proposal.
> 
> Each provider driver which supports multiple API versions needs to have at
> least two different classes:
> 
> - base driver class - this class implements common / api version agnostic
> provider functionality
> - version specific class - this class inherits from the base one and
> implements version specific functionality
> 
> Base class constructor method should also act as a factory method and accept
> "version" argument. Based on this version argument, the method would
> instantiate a correct version specific class. If no version is provided we
> should probably default to the latest version (?).
> 
> This approach also works OK with our current get_driver and provider
> constants functionality - we still only need 1 constant per provider
> 
> Mike Nerone has already implemented something like this in his original
> OpenStack branch - https://github.com/apache/libcloud/pull/26/files#L5R107
> 
> Feedback is welcome.
> 
> Thanks,
> Tomaz