You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@libcloud.apache.org by Alex Gaynor <al...@gmail.com> on 2013/06/27 00:16:27 UTC
[dev] Block storage
Hi all,
I'm interested in adding block storage support to libcloud (for the
providers that suppor it of course). To that end, I've drawn up a simple
design for the driver and related classes for the features that seem to be
supported generally (I reviewed Rackspace and Amazon).
You can find the design here: https://gist.github.com/alex/5872132, I think
it's mostly self-explanatory and consistent with the rest of libcloud, as
well as the (thankfully consistent!) terminology used by the providers.
Let me know what you think, and if there's general interest I'll go ahead
and start working on implementations for a few providers!
Alex
PS: In the interest of full disclosure, I work at Rackspace and have a
bunch of work time to help make libcloud awesome!
--
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
Re: [dev] Block storage
Posted by Jed Smith <je...@jedsmith.org>.
Oh, and, without a doubt the mount point behavior was the single most
confusing thing about Amazon's API. I worry about that being a string,
but there is basically no other solution. Will be interesting to see
how this can be made to support attaching to Windows instances, as
well...
--
Jed Smith
jed@jedsmith.org
Re: [dev] Block storage
Posted by Alex Gaynor <al...@gmail.com>.
On Wed, Jun 26, 2013 at 8:01 PM, Jed Smith <je...@jedsmith.org> wrote:
> This looks great, Alex! I haven't studied intently, but I have a
> couple quick comments:
>
> Line 16: Yes, I think it should be a parameter on create_volume. That
> makes the implementation less awkward if a provider does not support
> snapshotting.
>
> Line 70: What does node= do on detach()? Is it a sanity check in case
> your idea of state is wrong, or supporting attachment to multiple
> nodes at a time?
>
>
Both!
> I think some of the naming is wordy. As one example, you'll end up as
> a client with something like:
>
> node = ...
> volume = ...
> result = driver.node_attach_volume(node, volume, '/dev/xvdq')
>
>
This was largely inspired by the load balancer code, I imagine in practice
people will do `node.attach_volume(volume, '/dev/dfdasf')` assuming we're
ok with that type of cross-system API?
> To steal a Zed Shaw term, feels a bit like a feature chant ("node Node
> NODE"), but it fits with the rest of the APIs in my eyes, so probably
> a broader concern than just this design.
>
> --
> Jed Smith
> jed@jedsmith.org
>
--
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
Re: [dev] Block storage
Posted by Jed Smith <je...@jedsmith.org>.
This looks great, Alex! I haven't studied intently, but I have a
couple quick comments:
Line 16: Yes, I think it should be a parameter on create_volume. That
makes the implementation less awkward if a provider does not support
snapshotting.
Line 70: What does node= do on detach()? Is it a sanity check in case
your idea of state is wrong, or supporting attachment to multiple
nodes at a time?
I think some of the naming is wordy. As one example, you'll end up as
a client with something like:
node = ...
volume = ...
result = driver.node_attach_volume(node, volume, '/dev/xvdq')
To steal a Zed Shaw term, feels a bit like a feature chant ("node Node
NODE"), but it fits with the rest of the APIs in my eyes, so probably
a broader concern than just this design.
--
Jed Smith
jed@jedsmith.org
Re: [dev] Block storage
Posted by Ed Leafe <ed...@rackspace.com>.
On Jul 2, 2013, at 8:26 AM, Alex Gaynor <al...@gmail.com>
wrote:
> I think I've been swayed to favor (1) (putting it on compute drivers). a)
> it limits the code churn, which is nice, b) it's obvious they need *some*
> integration with each other, and there aren't well established patterns for
> that right now, rather than invent something new, grouping them together
> makes a lot of sense.
The only way to attach a block storage volume to an instance in OpenStack is via the compute API (at least for now), so integrating with compute makes a lot of sense.
-- Ed Leafe
Re: [dev] Block storage
Posted by Alex Gaynor <al...@gmail.com>.
I think I've been swayed to favor (1) (putting it on compute drivers). a)
it limits the code churn, which is nice, b) it's obvious they need *some*
integration with each other, and there aren't well established patterns for
that right now, rather than invent something new, grouping them together
makes a lot of sense.
Alex
On Sat, Jun 29, 2013 at 3:29 AM, John Carr <jo...@me.com> wrote:
> In order i'd probably vote 1, 3, 2.
>
> For me, the guide is "how does the user express this in as few lines of
> code as possible" and "how does the user express this elegantly".
>
> So I am absolutely convinced the compute driver needs to be aware of block
> storage. I think that (2) makes the code prettier for us libcloud
> contributors, but makes code that uses libcloud more annoying.
>
> (3) loses some points because I would have to instance a second Driver
> class with almost the same arguments and somehow pass that to the helper
> methods on the compute Node or to the compute driver itself. That isn't
> very elegant. Anything but the most light integration would probably lead
> to abstraction leakage.
>
> (And I can see users managing block storage too, without thinking of
> compute nodes, but I just think it should be through the compute driver. I
> don't think we should get hung up on taxonomy, block storage and S3 are
> different beasts.)
>
> Cheers,
> John
>
> On 29 Jun 2013, at 04:16, Chris Psaltis <cp...@mist.io> wrote:
>
> > From the options offered by Tomaz:
> >
> >> 1. Keep the existing API which is coupled to the compute API, iterate
> on it
> >> / improve it if necessary
> >> 2. Add a new, top-level Block Storage API, remove existing block storage
> >> related methods from the compute API
> >> 3. Add a new, top-level Block Storage API and update existing block
> storage
> >> related methods on the compute API to call into methods on the new top
> >> level Block Storage API
> >
> > Option (3), although more complicated, seems to be the best one.
> >
> > I'll use AWS terminology to illustrate my point. EBS is about storage
> and clearly belongs to the storage drivers, like S3. In terms of usage, I
> can imagine people listing EBS volumes, deleting them etc without even
> bothering about the nodes to which these volumes are attached to. On the
> other hand, I can also imagine someone trying to do node.detach_volume(),
> node.attach_volume(), node.create_snapshot() etc. In this sense I think
> option 3 is the best one because it deals with both use cases.
> >
> > This does come with maintainability issues though. If I had to choose
> from the other 2, I'd go with the a new top level API. This way you can
> deal with list_volumes() and juggle your way around attach/detach with
> something like volume.attach_to_node(node).
> >
> >
> >
> > Tomaz offers is best suited in this case. Initially it seems
> >
> > On 2013-06-28 19:45, Alex Gaynor wrote:
> >> So the biggest missing piece from the existing block-storage APIs that I
> >> see is snapshotting (Creation from a volume, creating a new volume from
> a
> >> snapshot, listing, and deletion). I don't have a good sense of whether
> they
> >> make the compute driver "too big" or whether it's fine to just add them.
> >> What do other people think?
> >> Alex
> >> On Fri, Jun 28, 2013 at 9:39 AM, Tomaz Muraus <to...@apache.org> wrote:
> >>> Yeah.
> >>> I also think there is some merit in having Block Storage as an
> additional,
> >>> separate top level API. It's been a while since I've last researched
> all
> >>> the existing block storage APIs so I would need to research it again to
> >>> refresh my memory, but I would also be interested in hearing other
> people's
> >>> take on it.
> >>> From this perspective, we have the following options:
> >>> 1. Keep the existing API which is coupled to the compute API, iterate
> on it
> >>> / improve it if necessary
> >>> 2. Add a new, top-level Block Storage API, remove existing block
> storage
> >>> related methods from the compute API
> >>> 3. Add a new, top-level Block Storage API and update existing block
> storage
> >>> related methods on the compute API to call into methods on the new top
> >>> level Block Storage API
> >>> Alex, since you've done this research fairly recently, what's your
> take on
> >>> this?
> >>> On Fri, Jun 28, 2013 at 6:23 PM, Alex Gaynor <al...@gmail.com>
> >>> wrote:
> >>>> That looks fine to me (I see only a few drivers implement it). I guess
> >>> this
> >>>> means we'd move all the methods from a new set of drivers onto
> compute?
> >>>> Alex
> >>>> On Fri, Jun 28, 2013 at 9:19 AM, Tomaz Muraus <to...@apache.org>
> wrote:
> >>>>> Heh, so it turns out I totally forgot that Libcloud already defines a
> >>>>> limited API for managing block storage:
> >>>>> *
> >>>
> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L368
> >>>>> *
> >>>
> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L705
> >>>>> Current block storage API is pretty tightly coupled to the compute
> API,
> >>>> but
> >>>>> it's pretty similar to what Alex proposed so we can probably use it
> as
> >>> a
> >>>>> good starting point.
> >>>>> Alex - what do you think?
> >>>>> On Thu, Jun 27, 2013 at 12:16 AM, Alex Gaynor <alex.gaynor@gmail.com
> >
> >>>>> wrote:
> >>>>>> Hi all,
> >>>>>> I'm interested in adding block storage support to libcloud (for the
> >>>>>> providers that suppor it of course). To that end, I've drawn up a
> >>>> simple
> >>>>>> design for the driver and related classes for the features that seem
> >>> to
> >>>>> be
> >>>>>> supported generally (I reviewed Rackspace and Amazon).
> >>>>>> You can find the design here: https://gist.github.com/alex/5872132,
> >>> I
> >>>>>> think
> >>>>>> it's mostly self-explanatory and consistent with the rest of
> >>> libcloud,
> >>>> as
> >>>>>> well as the (thankfully consistent!) terminology used by the
> >>> providers.
> >>>>>> Let me know what you think, and if there's general interest I'll go
> >>>> ahead
> >>>>>> and start working on implementations for a few providers!
> >>>>>> Alex
> >>>>>> PS: In the interest of full disclosure, I work at Rackspace and have
> >>> a
> >>>>>> bunch of work time to help make libcloud awesome!
> >>>>>> --
> >>>>>> "I disapprove of what you say, but I will defend to the death your
> >>>> right
> >>>>> to
> >>>>>> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> >>>>>> "The people's good is the highest law." -- Cicero
> >>>>>> GPG Key fingerprint: 125F 5C67 DFE9 4084
> >>>> --
> >>>> "I disapprove of what you say, but I will defend to the death your
> right
> >>> to
> >>>> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> >>>> "The people's good is the highest law." -- Cicero
> >>>> GPG Key fingerprint: 125F 5C67 DFE9 4084
> >
> > --
> > mist.io
> > cloud management in your pocket
>
>
--
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
Re: [dev] Block storage
Posted by John Carr <jo...@me.com>.
In order i'd probably vote 1, 3, 2.
For me, the guide is "how does the user express this in as few lines of code as possible" and "how does the user express this elegantly".
So I am absolutely convinced the compute driver needs to be aware of block storage. I think that (2) makes the code prettier for us libcloud contributors, but makes code that uses libcloud more annoying.
(3) loses some points because I would have to instance a second Driver class with almost the same arguments and somehow pass that to the helper methods on the compute Node or to the compute driver itself. That isn't very elegant. Anything but the most light integration would probably lead to abstraction leakage.
(And I can see users managing block storage too, without thinking of compute nodes, but I just think it should be through the compute driver. I don't think we should get hung up on taxonomy, block storage and S3 are different beasts.)
Cheers,
John
On 29 Jun 2013, at 04:16, Chris Psaltis <cp...@mist.io> wrote:
> From the options offered by Tomaz:
>
>> 1. Keep the existing API which is coupled to the compute API, iterate on it
>> / improve it if necessary
>> 2. Add a new, top-level Block Storage API, remove existing block storage
>> related methods from the compute API
>> 3. Add a new, top-level Block Storage API and update existing block storage
>> related methods on the compute API to call into methods on the new top
>> level Block Storage API
>
> Option (3), although more complicated, seems to be the best one.
>
> I'll use AWS terminology to illustrate my point. EBS is about storage and clearly belongs to the storage drivers, like S3. In terms of usage, I can imagine people listing EBS volumes, deleting them etc without even bothering about the nodes to which these volumes are attached to. On the other hand, I can also imagine someone trying to do node.detach_volume(), node.attach_volume(), node.create_snapshot() etc. In this sense I think option 3 is the best one because it deals with both use cases.
>
> This does come with maintainability issues though. If I had to choose from the other 2, I'd go with the a new top level API. This way you can deal with list_volumes() and juggle your way around attach/detach with something like volume.attach_to_node(node).
>
>
>
> Tomaz offers is best suited in this case. Initially it seems
>
> On 2013-06-28 19:45, Alex Gaynor wrote:
>> So the biggest missing piece from the existing block-storage APIs that I
>> see is snapshotting (Creation from a volume, creating a new volume from a
>> snapshot, listing, and deletion). I don't have a good sense of whether they
>> make the compute driver "too big" or whether it's fine to just add them.
>> What do other people think?
>> Alex
>> On Fri, Jun 28, 2013 at 9:39 AM, Tomaz Muraus <to...@apache.org> wrote:
>>> Yeah.
>>> I also think there is some merit in having Block Storage as an additional,
>>> separate top level API. It's been a while since I've last researched all
>>> the existing block storage APIs so I would need to research it again to
>>> refresh my memory, but I would also be interested in hearing other people's
>>> take on it.
>>> From this perspective, we have the following options:
>>> 1. Keep the existing API which is coupled to the compute API, iterate on it
>>> / improve it if necessary
>>> 2. Add a new, top-level Block Storage API, remove existing block storage
>>> related methods from the compute API
>>> 3. Add a new, top-level Block Storage API and update existing block storage
>>> related methods on the compute API to call into methods on the new top
>>> level Block Storage API
>>> Alex, since you've done this research fairly recently, what's your take on
>>> this?
>>> On Fri, Jun 28, 2013 at 6:23 PM, Alex Gaynor <al...@gmail.com>
>>> wrote:
>>>> That looks fine to me (I see only a few drivers implement it). I guess
>>> this
>>>> means we'd move all the methods from a new set of drivers onto compute?
>>>> Alex
>>>> On Fri, Jun 28, 2013 at 9:19 AM, Tomaz Muraus <to...@apache.org> wrote:
>>>>> Heh, so it turns out I totally forgot that Libcloud already defines a
>>>>> limited API for managing block storage:
>>>>> *
>>> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L368
>>>>> *
>>> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L705
>>>>> Current block storage API is pretty tightly coupled to the compute API,
>>>> but
>>>>> it's pretty similar to what Alex proposed so we can probably use it as
>>> a
>>>>> good starting point.
>>>>> Alex - what do you think?
>>>>> On Thu, Jun 27, 2013 at 12:16 AM, Alex Gaynor <al...@gmail.com>
>>>>> wrote:
>>>>>> Hi all,
>>>>>> I'm interested in adding block storage support to libcloud (for the
>>>>>> providers that suppor it of course). To that end, I've drawn up a
>>>> simple
>>>>>> design for the driver and related classes for the features that seem
>>> to
>>>>> be
>>>>>> supported generally (I reviewed Rackspace and Amazon).
>>>>>> You can find the design here: https://gist.github.com/alex/5872132,
>>> I
>>>>>> think
>>>>>> it's mostly self-explanatory and consistent with the rest of
>>> libcloud,
>>>> as
>>>>>> well as the (thankfully consistent!) terminology used by the
>>> providers.
>>>>>> Let me know what you think, and if there's general interest I'll go
>>>> ahead
>>>>>> and start working on implementations for a few providers!
>>>>>> Alex
>>>>>> PS: In the interest of full disclosure, I work at Rackspace and have
>>> a
>>>>>> bunch of work time to help make libcloud awesome!
>>>>>> --
>>>>>> "I disapprove of what you say, but I will defend to the death your
>>>> right
>>>>> to
>>>>>> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
>>>>>> "The people's good is the highest law." -- Cicero
>>>>>> GPG Key fingerprint: 125F 5C67 DFE9 4084
>>>> --
>>>> "I disapprove of what you say, but I will defend to the death your right
>>> to
>>>> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
>>>> "The people's good is the highest law." -- Cicero
>>>> GPG Key fingerprint: 125F 5C67 DFE9 4084
>
> --
> mist.io
> cloud management in your pocket
Re: [dev] Block storage
Posted by Chris Psaltis <cp...@mist.io>.
From the options offered by Tomaz:
> 1. Keep the existing API which is coupled to the compute API, iterate
> on it
> / improve it if necessary
> 2. Add a new, top-level Block Storage API, remove existing block
> storage
> related methods from the compute API
> 3. Add a new, top-level Block Storage API and update existing block
> storage
> related methods on the compute API to call into methods on the new top
> level Block Storage API
Option (3), although more complicated, seems to be the best one.
I'll use AWS terminology to illustrate my point. EBS is about storage
and clearly belongs to the storage drivers, like S3. In terms of usage,
I can imagine people listing EBS volumes, deleting them etc without even
bothering about the nodes to which these volumes are attached to. On the
other hand, I can also imagine someone trying to do
node.detach_volume(), node.attach_volume(), node.create_snapshot() etc.
In this sense I think option 3 is the best one because it deals with
both use cases.
This does come with maintainability issues though. If I had to choose
from the other 2, I'd go with the a new top level API. This way you can
deal with list_volumes() and juggle your way around attach/detach with
something like volume.attach_to_node(node).
Tomaz offers is best suited in this case. Initially it seems
On 2013-06-28 19:45, Alex Gaynor wrote:
> So the biggest missing piece from the existing block-storage APIs that
> I
> see is snapshotting (Creation from a volume, creating a new volume
> from a
> snapshot, listing, and deletion). I don't have a good sense of whether
> they
> make the compute driver "too big" or whether it's fine to just add
> them.
> What do other people think?
>
> Alex
>
>
> On Fri, Jun 28, 2013 at 9:39 AM, Tomaz Muraus <to...@apache.org>
> wrote:
>
>> Yeah.
>>
>> I also think there is some merit in having Block Storage as an
>> additional,
>> separate top level API. It's been a while since I've last researched
>> all
>> the existing block storage APIs so I would need to research it again
>> to
>> refresh my memory, but I would also be interested in hearing other
>> people's
>> take on it.
>>
>> From this perspective, we have the following options:
>>
>> 1. Keep the existing API which is coupled to the compute API, iterate
>> on it
>> / improve it if necessary
>> 2. Add a new, top-level Block Storage API, remove existing block
>> storage
>> related methods from the compute API
>> 3. Add a new, top-level Block Storage API and update existing block
>> storage
>> related methods on the compute API to call into methods on the new
>> top
>> level Block Storage API
>>
>> Alex, since you've done this research fairly recently, what's your
>> take on
>> this?
>>
>> On Fri, Jun 28, 2013 at 6:23 PM, Alex Gaynor <al...@gmail.com>
>> wrote:
>>
>>> That looks fine to me (I see only a few drivers implement it). I
>>> guess
>> this
>>> means we'd move all the methods from a new set of drivers onto
>>> compute?
>>>
>>> Alex
>>>
>>>
>>> On Fri, Jun 28, 2013 at 9:19 AM, Tomaz Muraus <to...@apache.org>
>>> wrote:
>>>
>>>> Heh, so it turns out I totally forgot that Libcloud already defines
>>>> a
>>>> limited API for managing block storage:
>>>>
>>>> *
>>>>
>>>
>> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L368
>>>> *
>>>>
>>>
>> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L705
>>>>
>>>> Current block storage API is pretty tightly coupled to the compute
>>>> API,
>>> but
>>>> it's pretty similar to what Alex proposed so we can probably use it
>>>> as
>> a
>>>> good starting point.
>>>>
>>>> Alex - what do you think?
>>>>
>>>> On Thu, Jun 27, 2013 at 12:16 AM, Alex Gaynor
>>>> <al...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> I'm interested in adding block storage support to libcloud (for
>>>>> the
>>>>> providers that suppor it of course). To that end, I've drawn up a
>>> simple
>>>>> design for the driver and related classes for the features that
>>>>> seem
>> to
>>>> be
>>>>> supported generally (I reviewed Rackspace and Amazon).
>>>>>
>>>>> You can find the design here:
>>>>> https://gist.github.com/alex/5872132,
>> I
>>>>> think
>>>>> it's mostly self-explanatory and consistent with the rest of
>> libcloud,
>>> as
>>>>> well as the (thankfully consistent!) terminology used by the
>> providers.
>>>>>
>>>>> Let me know what you think, and if there's general interest I'll
>>>>> go
>>> ahead
>>>>> and start working on implementations for a few providers!
>>>>>
>>>>> Alex
>>>>>
>>>>> PS: In the interest of full disclosure, I work at Rackspace and
>>>>> have
>> a
>>>>> bunch of work time to help make libcloud awesome!
>>>>>
>>>>>
>>>>> --
>>>>> "I disapprove of what you say, but I will defend to the death your
>>> right
>>>> to
>>>>> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
>>>>> "The people's good is the highest law." -- Cicero
>>>>> GPG Key fingerprint: 125F 5C67 DFE9 4084
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> "I disapprove of what you say, but I will defend to the death your
>>> right
>> to
>>> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
>>> "The people's good is the highest law." -- Cicero
>>> GPG Key fingerprint: 125F 5C67 DFE9 4084
>>>
>>
--
mist.io
cloud management in your pocket
Re: [dev] Block storage
Posted by Alex Gaynor <al...@gmail.com>.
So the biggest missing piece from the existing block-storage APIs that I
see is snapshotting (Creation from a volume, creating a new volume from a
snapshot, listing, and deletion). I don't have a good sense of whether they
make the compute driver "too big" or whether it's fine to just add them.
What do other people think?
Alex
On Fri, Jun 28, 2013 at 9:39 AM, Tomaz Muraus <to...@apache.org> wrote:
> Yeah.
>
> I also think there is some merit in having Block Storage as an additional,
> separate top level API. It's been a while since I've last researched all
> the existing block storage APIs so I would need to research it again to
> refresh my memory, but I would also be interested in hearing other people's
> take on it.
>
> From this perspective, we have the following options:
>
> 1. Keep the existing API which is coupled to the compute API, iterate on it
> / improve it if necessary
> 2. Add a new, top-level Block Storage API, remove existing block storage
> related methods from the compute API
> 3. Add a new, top-level Block Storage API and update existing block storage
> related methods on the compute API to call into methods on the new top
> level Block Storage API
>
> Alex, since you've done this research fairly recently, what's your take on
> this?
>
> On Fri, Jun 28, 2013 at 6:23 PM, Alex Gaynor <al...@gmail.com>
> wrote:
>
> > That looks fine to me (I see only a few drivers implement it). I guess
> this
> > means we'd move all the methods from a new set of drivers onto compute?
> >
> > Alex
> >
> >
> > On Fri, Jun 28, 2013 at 9:19 AM, Tomaz Muraus <to...@apache.org> wrote:
> >
> > > Heh, so it turns out I totally forgot that Libcloud already defines a
> > > limited API for managing block storage:
> > >
> > > *
> > >
> >
> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L368
> > > *
> > >
> >
> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L705
> > >
> > > Current block storage API is pretty tightly coupled to the compute API,
> > but
> > > it's pretty similar to what Alex proposed so we can probably use it as
> a
> > > good starting point.
> > >
> > > Alex - what do you think?
> > >
> > > On Thu, Jun 27, 2013 at 12:16 AM, Alex Gaynor <al...@gmail.com>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'm interested in adding block storage support to libcloud (for the
> > > > providers that suppor it of course). To that end, I've drawn up a
> > simple
> > > > design for the driver and related classes for the features that seem
> to
> > > be
> > > > supported generally (I reviewed Rackspace and Amazon).
> > > >
> > > > You can find the design here: https://gist.github.com/alex/5872132,
> I
> > > > think
> > > > it's mostly self-explanatory and consistent with the rest of
> libcloud,
> > as
> > > > well as the (thankfully consistent!) terminology used by the
> providers.
> > > >
> > > > Let me know what you think, and if there's general interest I'll go
> > ahead
> > > > and start working on implementations for a few providers!
> > > >
> > > > Alex
> > > >
> > > > PS: In the interest of full disclosure, I work at Rackspace and have
> a
> > > > bunch of work time to help make libcloud awesome!
> > > >
> > > >
> > > > --
> > > > "I disapprove of what you say, but I will defend to the death your
> > right
> > > to
> > > > say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> > > > "The people's good is the highest law." -- Cicero
> > > > GPG Key fingerprint: 125F 5C67 DFE9 4084
> > > >
> > >
> >
> >
> >
> > --
> > "I disapprove of what you say, but I will defend to the death your right
> to
> > say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> > "The people's good is the highest law." -- Cicero
> > GPG Key fingerprint: 125F 5C67 DFE9 4084
> >
>
--
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
Re: [dev] Block storage
Posted by Tomaz Muraus <to...@apache.org>.
Yeah.
I also think there is some merit in having Block Storage as an additional,
separate top level API. It's been a while since I've last researched all
the existing block storage APIs so I would need to research it again to
refresh my memory, but I would also be interested in hearing other people's
take on it.
>From this perspective, we have the following options:
1. Keep the existing API which is coupled to the compute API, iterate on it
/ improve it if necessary
2. Add a new, top-level Block Storage API, remove existing block storage
related methods from the compute API
3. Add a new, top-level Block Storage API and update existing block storage
related methods on the compute API to call into methods on the new top
level Block Storage API
Alex, since you've done this research fairly recently, what's your take on
this?
On Fri, Jun 28, 2013 at 6:23 PM, Alex Gaynor <al...@gmail.com> wrote:
> That looks fine to me (I see only a few drivers implement it). I guess this
> means we'd move all the methods from a new set of drivers onto compute?
>
> Alex
>
>
> On Fri, Jun 28, 2013 at 9:19 AM, Tomaz Muraus <to...@apache.org> wrote:
>
> > Heh, so it turns out I totally forgot that Libcloud already defines a
> > limited API for managing block storage:
> >
> > *
> >
> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L368
> > *
> >
> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L705
> >
> > Current block storage API is pretty tightly coupled to the compute API,
> but
> > it's pretty similar to what Alex proposed so we can probably use it as a
> > good starting point.
> >
> > Alex - what do you think?
> >
> > On Thu, Jun 27, 2013 at 12:16 AM, Alex Gaynor <al...@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > I'm interested in adding block storage support to libcloud (for the
> > > providers that suppor it of course). To that end, I've drawn up a
> simple
> > > design for the driver and related classes for the features that seem to
> > be
> > > supported generally (I reviewed Rackspace and Amazon).
> > >
> > > You can find the design here: https://gist.github.com/alex/5872132, I
> > > think
> > > it's mostly self-explanatory and consistent with the rest of libcloud,
> as
> > > well as the (thankfully consistent!) terminology used by the providers.
> > >
> > > Let me know what you think, and if there's general interest I'll go
> ahead
> > > and start working on implementations for a few providers!
> > >
> > > Alex
> > >
> > > PS: In the interest of full disclosure, I work at Rackspace and have a
> > > bunch of work time to help make libcloud awesome!
> > >
> > >
> > > --
> > > "I disapprove of what you say, but I will defend to the death your
> right
> > to
> > > say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> > > "The people's good is the highest law." -- Cicero
> > > GPG Key fingerprint: 125F 5C67 DFE9 4084
> > >
> >
>
>
>
> --
> "I disapprove of what you say, but I will defend to the death your right to
> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
>
Re: [dev] Block storage
Posted by Alex Gaynor <al...@gmail.com>.
That looks fine to me (I see only a few drivers implement it). I guess this
means we'd move all the methods from a new set of drivers onto compute?
Alex
On Fri, Jun 28, 2013 at 9:19 AM, Tomaz Muraus <to...@apache.org> wrote:
> Heh, so it turns out I totally forgot that Libcloud already defines a
> limited API for managing block storage:
>
> *
> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L368
> *
> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L705
>
> Current block storage API is pretty tightly coupled to the compute API, but
> it's pretty similar to what Alex proposed so we can probably use it as a
> good starting point.
>
> Alex - what do you think?
>
> On Thu, Jun 27, 2013 at 12:16 AM, Alex Gaynor <al...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > I'm interested in adding block storage support to libcloud (for the
> > providers that suppor it of course). To that end, I've drawn up a simple
> > design for the driver and related classes for the features that seem to
> be
> > supported generally (I reviewed Rackspace and Amazon).
> >
> > You can find the design here: https://gist.github.com/alex/5872132, I
> > think
> > it's mostly self-explanatory and consistent with the rest of libcloud, as
> > well as the (thankfully consistent!) terminology used by the providers.
> >
> > Let me know what you think, and if there's general interest I'll go ahead
> > and start working on implementations for a few providers!
> >
> > Alex
> >
> > PS: In the interest of full disclosure, I work at Rackspace and have a
> > bunch of work time to help make libcloud awesome!
> >
> >
> > --
> > "I disapprove of what you say, but I will defend to the death your right
> to
> > say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> > "The people's good is the highest law." -- Cicero
> > GPG Key fingerprint: 125F 5C67 DFE9 4084
> >
>
--
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
Re: [dev] Block storage
Posted by Tomaz Muraus <to...@apache.org>.
Heh, so it turns out I totally forgot that Libcloud already defines a
limited API for managing block storage:
*
https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L368
*
https://github.com/apache/libcloud/blob/trunk/libcloud/compute/base.py#L705
Current block storage API is pretty tightly coupled to the compute API, but
it's pretty similar to what Alex proposed so we can probably use it as a
good starting point.
Alex - what do you think?
On Thu, Jun 27, 2013 at 12:16 AM, Alex Gaynor <al...@gmail.com> wrote:
> Hi all,
>
> I'm interested in adding block storage support to libcloud (for the
> providers that suppor it of course). To that end, I've drawn up a simple
> design for the driver and related classes for the features that seem to be
> supported generally (I reviewed Rackspace and Amazon).
>
> You can find the design here: https://gist.github.com/alex/5872132, I
> think
> it's mostly self-explanatory and consistent with the rest of libcloud, as
> well as the (thankfully consistent!) terminology used by the providers.
>
> Let me know what you think, and if there's general interest I'll go ahead
> and start working on implementations for a few providers!
>
> Alex
>
> PS: In the interest of full disclosure, I work at Rackspace and have a
> bunch of work time to help make libcloud awesome!
>
>
> --
> "I disapprove of what you say, but I will defend to the death your right to
> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
>
Re: [dev] Block storage
Posted by Alex Gaynor <al...@gmail.com>.
On Thu, Jun 27, 2013 at 3:00 AM, Tomaz Muraus <to...@apache.org> wrote:
> Cool.
>
> I have some comments and questions:
>
> 1. We should probably get rid of the "Base" suffix.
>
> All of the bases API interfaces (libcloud.compute.base, libcloud.dns.base,
> libcloud.storage.base, etc.), don't contain this suffix.
>
>
Ok (I wrote this without checking the class names apparently).
> 2. As far as line 16 goes - Imo, having a separate method makes it more
> explicit and easier for developers if a provider doesn't support
> snapshotting.
>
> In this case, developer doesn't need to do anything and driver will
> automatically throw "not implement error" if a user calls
> "snapshot_create_volume" method and this method is unimplemented.
>
> In case of an argument to the create_volume method, developer will need to
> define a method without a "snapshot" argument.
>
> I think that's fine, but I would imagine most of the developers will just
> copy the base API and leave "snapshot=None" argument there and just ignore
> it if a user passes it to the method. Imo, that's confusing to the end
> user.
>
> 3. Heh, Jed already said some method names are wordy, but I personally like
> "create_volume_from_snapshot" more than "snapshot_create_volume".
>
> I'm not to opinionated about this one and I'm totally fine going with
> "snapshot_create_volume" if more people think it's better though.
>
>
I don't have a particular opinion, I would probably always write
`snapshot.create_volume()` anyways.
> 3. How do we plan to integrate this with the compute API? Do we even plan
> to do it?
>
> I think there is value in adding some integration with the compute API and
> some of the existing compute drivers already expose methods for managing
> block storage: For example:
>
> 1.
>
> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/gandi.py#L369
> 2.
>
> https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/cloudstack.py#L429
>
> A couple of drivers implement read-only functionality (e.g. list_volumes)
> and a couple of other drivers implement full CRUD functionality using
> extension methods.
>
I think there should be some integration, I'm not sure how much (maybe it
will become clearer once I start implementing)?
>
> On Thu, Jun 27, 2013 at 12:16 AM, Alex Gaynor <al...@gmail.com>
> wrote:
>
> > Hi all,
> >
> > I'm interested in adding block storage support to libcloud (for the
> > providers that suppor it of course). To that end, I've drawn up a simple
> > design for the driver and related classes for the features that seem to
> be
> > supported generally (I reviewed Rackspace and Amazon).
> >
> > You can find the design here: https://gist.github.com/alex/5872132, I
> > think
> > it's mostly self-explanatory and consistent with the rest of libcloud, as
> > well as the (thankfully consistent!) terminology used by the providers.
> >
> > Let me know what you think, and if there's general interest I'll go ahead
> > and start working on implementations for a few providers!
> >
> > Alex
> >
> > PS: In the interest of full disclosure, I work at Rackspace and have a
> > bunch of work time to help make libcloud awesome!
> >
> >
> > --
> > "I disapprove of what you say, but I will defend to the death your right
> to
> > say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> > "The people's good is the highest law." -- Cicero
> > GPG Key fingerprint: 125F 5C67 DFE9 4084
> >
>
--
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero
GPG Key fingerprint: 125F 5C67 DFE9 4084
Re: [dev] Block storage
Posted by Tomaz Muraus <to...@apache.org>.
Cool.
I have some comments and questions:
1. We should probably get rid of the "Base" suffix.
All of the bases API interfaces (libcloud.compute.base, libcloud.dns.base,
libcloud.storage.base, etc.), don't contain this suffix.
2. As far as line 16 goes - Imo, having a separate method makes it more
explicit and easier for developers if a provider doesn't support
snapshotting.
In this case, developer doesn't need to do anything and driver will
automatically throw "not implement error" if a user calls
"snapshot_create_volume" method and this method is unimplemented.
In case of an argument to the create_volume method, developer will need to
define a method without a "snapshot" argument.
I think that's fine, but I would imagine most of the developers will just
copy the base API and leave "snapshot=None" argument there and just ignore
it if a user passes it to the method. Imo, that's confusing to the end user.
3. Heh, Jed already said some method names are wordy, but I personally like
"create_volume_from_snapshot" more than "snapshot_create_volume".
I'm not to opinionated about this one and I'm totally fine going with
"snapshot_create_volume" if more people think it's better though.
3. How do we plan to integrate this with the compute API? Do we even plan
to do it?
I think there is value in adding some integration with the compute API and
some of the existing compute drivers already expose methods for managing
block storage: For example:
1.
https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/gandi.py#L369
2.
https://github.com/apache/libcloud/blob/trunk/libcloud/compute/drivers/cloudstack.py#L429
A couple of drivers implement read-only functionality (e.g. list_volumes)
and a couple of other drivers implement full CRUD functionality using
extension methods.
On Thu, Jun 27, 2013 at 12:16 AM, Alex Gaynor <al...@gmail.com> wrote:
> Hi all,
>
> I'm interested in adding block storage support to libcloud (for the
> providers that suppor it of course). To that end, I've drawn up a simple
> design for the driver and related classes for the features that seem to be
> supported generally (I reviewed Rackspace and Amazon).
>
> You can find the design here: https://gist.github.com/alex/5872132, I
> think
> it's mostly self-explanatory and consistent with the rest of libcloud, as
> well as the (thankfully consistent!) terminology used by the providers.
>
> Let me know what you think, and if there's general interest I'll go ahead
> and start working on implementations for a few providers!
>
> Alex
>
> PS: In the interest of full disclosure, I work at Rackspace and have a
> bunch of work time to help make libcloud awesome!
>
>
> --
> "I disapprove of what you say, but I will defend to the death your right to
> say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
> GPG Key fingerprint: 125F 5C67 DFE9 4084
>