You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Sudha Ponnaganti <su...@citrix.com> on 2013/01/16 00:40:13 UTC
S3 Based secondary storage - Stories 509 and 714
Hi Nitin/John,
There are two stories listed in JIRA - CLOUDSTACK 509 and CLOUDSTACK 714. Looks like 714 is duplicate of 509.
Can I mark 714 as duplicate of 509??
Also need response from John B/Prasanna, if any tests are added to Marvin. If so have they been run??
Thanks
/Sudha
Re: S3 Based secondary storage - Stories 509 and 714
Posted by David Nalley <da...@gnsa.us>.
On Tue, Jan 15, 2013 at 6:40 PM, Sudha Ponnaganti
<su...@citrix.com> wrote:
> Hi Nitin/John,
>
> There are two stories listed in JIRA - CLOUDSTACK 509 and CLOUDSTACK 714. Looks like 714 is duplicate of 509.
> Can I mark 714 as duplicate of 509??
>
> Also need response from John B/Prasanna, if any tests are added to Marvin. If so have they been run??
>
IIRC 509 is about adding support for S3 storage as secondary storage -
akin to what is in place now with Swift support.
714 is about eliminating a dependence on NFS as a buffer for object storage.
--David
Re: S3 Based secondary storage - Stories 509 and 714
Posted by John Burwell <jb...@basho.com>.
Hari,
The bottom line is that the secondary storage architecture makes explicit assumptions that a secondary storage device will support a file system. Until we complete a redesign of the storage subsystem, all object stores will require an intermediate filesystem to operate with CloudStack.
Thanks,
-John
On Jan 16, 2013, at 10:59 AM, Hari Kannan <ha...@citrix.com> wrote:
> Hi John,
>
> Wish to revisit this topic -
>
> Any reason you didn't take this approach indicated on this thread? http://markmail.org/message/ccsurkqhvxunqe26
>
> Also, are any of the requirements listed here https://cwiki.apache.org/confluence/display/CLOUDSTACK/S3-based+secondary+Storage
> addressed in your implementation?
>
> Thanks!
>
> Hari Kannan
>
> -----Original Message-----
> From: John Burwell [mailto:jburwell@basho.com]
> Sent: Wednesday, January 16, 2013 3:51 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: S3 Based secondary storage - Stories 509 and 714
>
> Marcus,
>
> The integration uses the S3 client. Therefore, any storage service that supports the S3 API can be used by specifying a different end point in the CloudStack configuration. To that point, I test against both Amazon S3 and Riak CS.
>
> Thanks,
> -John
>
> On Jan 16, 2013, at 2:19 AM, Marcus Sorensen <sh...@gmail.com> wrote:
>
>> Just for clarification, this feature isn't tied to amazon's S3, right?
>> Any
>> S3 compatible gateway would do?
>> On Jan 15, 2013 5:52 PM, "John Burwell" <jb...@basho.com> wrote:
>>
>>> Sudha,
>>>
>>> The issue with adding Marvin tests for S3 is the requirement and
>>> associated costs for an S3 account. For obvious reasons, I can not
>>> put my account credentials into a test case checked into a public
>>> repo used many people. Also, the indeterminate nature of
>>> synchronization process (i.e. polling every sixty seconds and the
>>> bandwidth available to S3 for download/upload operations) makes it
>>> very difficult to programmatically verify. Therefore, S3 is only
>>> testable manually at this time.
>>>
>>> Thanks,
>>> -John
>>>
>>> On Jan 15, 2013, at 6:40 PM, Sudha Ponnaganti
>>> <su...@citrix.com> wrote:
>>>
>>>> Hi Nitin/John,
>>>>
>>>> There are two stories listed in JIRA - CLOUDSTACK 509 and CLOUDSTACK
>>> 714. Looks like 714 is duplicate of 509.
>>>> Can I mark 714 as duplicate of 509??
>>>>
>>>> Also need response from John B/Prasanna, if any tests are added to
>>> Marvin. If so have they been run??
>>>>
>>>>
>>>> Thanks
>>>> /Sudha
>>>
RE: S3 Based secondary storage - Stories 509 and 714
Posted by Hari Kannan <ha...@citrix.com>.
Hi John,
Wish to revisit this topic -
Any reason you didn't take this approach indicated on this thread? http://markmail.org/message/ccsurkqhvxunqe26
Also, are any of the requirements listed here https://cwiki.apache.org/confluence/display/CLOUDSTACK/S3-based+secondary+Storage
addressed in your implementation?
Thanks!
Hari Kannan
-----Original Message-----
From: John Burwell [mailto:jburwell@basho.com]
Sent: Wednesday, January 16, 2013 3:51 AM
To: cloudstack-dev@incubator.apache.org
Subject: Re: S3 Based secondary storage - Stories 509 and 714
Marcus,
The integration uses the S3 client. Therefore, any storage service that supports the S3 API can be used by specifying a different end point in the CloudStack configuration. To that point, I test against both Amazon S3 and Riak CS.
Thanks,
-John
On Jan 16, 2013, at 2:19 AM, Marcus Sorensen <sh...@gmail.com> wrote:
> Just for clarification, this feature isn't tied to amazon's S3, right?
> Any
> S3 compatible gateway would do?
> On Jan 15, 2013 5:52 PM, "John Burwell" <jb...@basho.com> wrote:
>
>> Sudha,
>>
>> The issue with adding Marvin tests for S3 is the requirement and
>> associated costs for an S3 account. For obvious reasons, I can not
>> put my account credentials into a test case checked into a public
>> repo used many people. Also, the indeterminate nature of
>> synchronization process (i.e. polling every sixty seconds and the
>> bandwidth available to S3 for download/upload operations) makes it
>> very difficult to programmatically verify. Therefore, S3 is only
>> testable manually at this time.
>>
>> Thanks,
>> -John
>>
>> On Jan 15, 2013, at 6:40 PM, Sudha Ponnaganti
>> <su...@citrix.com> wrote:
>>
>>> Hi Nitin/John,
>>>
>>> There are two stories listed in JIRA - CLOUDSTACK 509 and CLOUDSTACK
>> 714. Looks like 714 is duplicate of 509.
>>> Can I mark 714 as duplicate of 509??
>>>
>>> Also need response from John B/Prasanna, if any tests are added to
>> Marvin. If so have they been run??
>>>
>>>
>>> Thanks
>>> /Sudha
>>
Re: S3 Based secondary storage - Stories 509 and 714
Posted by John Burwell <jb...@basho.com>.
Marcus,
The integration uses the S3 client. Therefore, any storage service
that supports the S3 API can be used by specifying a different end
point in the CloudStack configuration. To that point, I test against
both Amazon S3 and Riak CS.
Thanks,
-John
On Jan 16, 2013, at 2:19 AM, Marcus Sorensen <sh...@gmail.com> wrote:
> Just for clarification, this feature isn't tied to amazon's S3, right? Any
> S3 compatible gateway would do?
> On Jan 15, 2013 5:52 PM, "John Burwell" <jb...@basho.com> wrote:
>
>> Sudha,
>>
>> The issue with adding Marvin tests for S3 is the requirement and
>> associated costs for an S3 account. For obvious reasons, I can not
>> put my account credentials into a test case checked into a public repo
>> used many people. Also, the indeterminate nature of synchronization
>> process (i.e. polling every sixty seconds and the bandwidth available
>> to S3 for download/upload operations) makes it very difficult to
>> programmatically verify. Therefore, S3 is only testable manually at
>> this time.
>>
>> Thanks,
>> -John
>>
>> On Jan 15, 2013, at 6:40 PM, Sudha Ponnaganti
>> <su...@citrix.com> wrote:
>>
>>> Hi Nitin/John,
>>>
>>> There are two stories listed in JIRA - CLOUDSTACK 509 and CLOUDSTACK
>> 714. Looks like 714 is duplicate of 509.
>>> Can I mark 714 as duplicate of 509??
>>>
>>> Also need response from John B/Prasanna, if any tests are added to
>> Marvin. If so have they been run??
>>>
>>>
>>> Thanks
>>> /Sudha
>>
Re: S3 Based secondary storage - Stories 509 and 714
Posted by Marcus Sorensen <sh...@gmail.com>.
Just for clarification, this feature isn't tied to amazon's S3, right? Any
S3 compatible gateway would do?
On Jan 15, 2013 5:52 PM, "John Burwell" <jb...@basho.com> wrote:
> Sudha,
>
> The issue with adding Marvin tests for S3 is the requirement and
> associated costs for an S3 account. For obvious reasons, I can not
> put my account credentials into a test case checked into a public repo
> used many people. Also, the indeterminate nature of synchronization
> process (i.e. polling every sixty seconds and the bandwidth available
> to S3 for download/upload operations) makes it very difficult to
> programmatically verify. Therefore, S3 is only testable manually at
> this time.
>
> Thanks,
> -John
>
> On Jan 15, 2013, at 6:40 PM, Sudha Ponnaganti
> <su...@citrix.com> wrote:
>
> > Hi Nitin/John,
> >
> > There are two stories listed in JIRA - CLOUDSTACK 509 and CLOUDSTACK
> 714. Looks like 714 is duplicate of 509.
> > Can I mark 714 as duplicate of 509??
> >
> > Also need response from John B/Prasanna, if any tests are added to
> Marvin. If so have they been run??
> >
> >
> > Thanks
> > /Sudha
>
Re: S3 Based secondary storage - Stories 509 and 714
Posted by Prasanna Santhanam <ts...@apache.org>.
On Tue, Jan 15, 2013 at 07:52:15PM -0500, John Burwell wrote:
> Sudha,
>
> The issue with adding Marvin tests for S3 is the requirement and
> associated costs for an S3 account. For obvious reasons, I can not
> put my account credentials into a test case checked into a public repo
> used many people. Also, the indeterminate nature of synchronization
> process (i.e. polling every sixty seconds and the bandwidth available
> to S3 for download/upload operations) makes it very difficult to
> programmatically verify. Therefore, S3 is only testable manually at
> this time.
Hey John -
If there are any illustrative tests with dummy-credentials for this I'd
love to run them with an account I own. Just for understanding the
development of the test.
Thanks,
--
Prasanna.,
Re: S3 Based secondary storage - Stories 509 and 714
Posted by John Burwell <jb...@basho.com>.
Sudha,
The issue with adding Marvin tests for S3 is the requirement and
associated costs for an S3 account. For obvious reasons, I can not
put my account credentials into a test case checked into a public repo
used many people. Also, the indeterminate nature of synchronization
process (i.e. polling every sixty seconds and the bandwidth available
to S3 for download/upload operations) makes it very difficult to
programmatically verify. Therefore, S3 is only testable manually at
this time.
Thanks,
-John
On Jan 15, 2013, at 6:40 PM, Sudha Ponnaganti
<su...@citrix.com> wrote:
> Hi Nitin/John,
>
> There are two stories listed in JIRA - CLOUDSTACK 509 and CLOUDSTACK 714. Looks like 714 is duplicate of 509.
> Can I mark 714 as duplicate of 509??
>
> Also need response from John B/Prasanna, if any tests are added to Marvin. If so have they been run??
>
>
> Thanks
> /Sudha