[Discuss] API name alias

[Kishan Kavala <Ki...@citrix.com>]

APICommand annotation in API Cmd object has a name parameter. Currently name parameter takes only one value. I plan to enhance this to support comma separated values. This will allow multiple API names for the same API Cmd object.

Current:
@APICommand(name = "apiName1", ..

Proposed:
@APICommand(name = "apiName1, apiAlias2, apiAlias3", ..

Requirement:
As part of CLOUDSTACK-763, I'll be introducing NetworkACLList (grouping of NetworkACLItems).  Current APIs use *NetworkACL (create NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related APIs. These APIs have to be changed to *NetworkACL Item(create NetworkACLItem/deleteNetworkACLItem etc..) to get the terminology right. We also need to support old API names for backward compatibility. Hence the need for API name alias.

Terminology:
NetworkACLItem - Individual ACL Entry (was NetworlACL earlier). 
NetworkACL - Group of Network ACL Items. API will use the term NetworkACLList to differentiate from the existing NetworkACL APIs.

Re: [Discuss] API name alias

[John Burwell <jb...@basho.com>]

Kishan,

Annotations support the specification of arrays as annotation values which allows the compiler to verify the form of the value (avoiding a potential runtime exception due to a typo) and a more expressive value.  Annotation array values are treated as a varargs -- therefore, single names do not require brackets.  For these reasons, I recommend using a array value for the annotation rather than a comma-separated string.

Thanks,
-John


On Apr 8, 2013, at 9:03 AM, Kishan Kavala <Ki...@citrix.com> wrote:

> APICommand annotation in API Cmd object has a name parameter. Currently name parameter takes only one value. I plan to enhance this to support comma separated values. This will allow multiple API names for the same API Cmd object.
> 
> Current:
> @APICommand(name = "apiName1", ..
> 
> Proposed:
> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
> 
> Requirement:
> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList (grouping of NetworkACLItems).  Current APIs use *NetworkACL (create NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related APIs. These APIs have to be changed to *NetworkACL Item(create NetworkACLItem/deleteNetworkACLItem etc..) to get the terminology right. We also need to support old API names for backward compatibility. Hence the need for API name alias.
> 
> Terminology:
> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier). 
> NetworkACL - Group of Network ACL Items. API will use the term NetworkACLList to differentiate from the existing NetworkACL APIs.
> 
> 

RE: [Discuss] API name alias

[Rajesh Battala <ra...@citrix.com>]

> -----Original Message-----
> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
> Sent: Tuesday, April 9, 2013 3:19 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [Discuss] API name alias
> 
> 
> 
> On 4/8/13 11:18 AM, "Rohit Yadav" <bh...@apache.org> wrote:
> 
> >On Mon, Apr 8, 2013 at 7:10 PM, John Burwell <jb...@basho.com> wrote:
> >
> >> Rohit,
> >>
> >> Why would we need to rewrite any Java code to switch to an array?  As
> >>I  mentioned, array annotation values are treated as varargs.
> >>Therefore,  annotations with single values can be left alone.  Only
> >>cases where  multiple values are required would require the addition of curly
> braces.
> >>  For example, for an APICommand with one name, the following would
> >>continue  to work:
> >>
> >
> >Hey John, you're right but it's a matter of code styling, I prefer
> >writing;
> >
> >@APICommand(name = {'name1'}) even if it's only one name, this way any
> >subsequent api author would know that name is an array and they can
> >have aliases which is what Kishan wants.
> >
> >
> >>
> >>         @APICommand(name="apiName1")
> >>
> >> While for an APICommand with multiple names, the following form would
> >>now  be supported:
> >>
> >>         @APICommand(name={"apiName1", "apiName2", "apiName3"})
> >>
> >> I am not familiar with the Marvin code, so I can not speak to the
> >>impact  of this change to it.  However, the only changes to the Java
> >>code should be  the API commands with multiple names and the classes
> >>that actually consume  the value of this annotation.
> >>
> >
> >Yes, you're right we can skip the code styling I referred then it won't
> >consume much coding, we just need to fix these;
> >
> >- Fix APICommand annotation interface definition to accept name as an
> >array
> >- Fix method in ApiServer which creates apiname->cmdclass map to care
> >for aliases
> >- Fix ApiDiscovery to take care of the aliases (create different
> >response objs for the same cmd class)
> >- Fix apidocs XmlWriter class to take of the aliases and the api html
> >generator to process the same.
> >- Fix Marvin's codegenerator.py to take care of the aliases (which
> >means create apiname related modules which would contain cmd and
> >response boilterplate fields/class)
> >
> >Cheers.
> >
> >
> >>
> >> Thanks,
> >> -John
> >>
> >> On Apr 8, 2013, at 9:22 AM, Rohit Yadav <bh...@apache.org> wrote:
> >>
> >> > On Mon, Apr 8, 2013 at 6:33 PM, Kishan Kavala
> >><Kishan.Kavala@citrix.com
> >> >wrote:
> >> >
> >> >> APICommand annotation in API Cmd object has a name parameter.
> >>Currently
> >> >> name parameter takes only one value. I plan to enhance this to
> >>support
> >> >> comma separated values. This will allow multiple API names for the
> >>same
> >> API
> >> >> Cmd object.
> >> >>
> >> >> Current:
> >> >> @APICommand(name = "apiName1", ..
> >> >>
> >> >> Proposed:
> >> >> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
> >> >>
> >> >
> >> > It's a hack, but doable. While not elegant as John suggests,
> >> > parsing
> >> comma
> >> > separated value can lead to some issues.
> >> >
> >> > Changing the name field to array would require a lot of rewriting
> >> > the present apis annotation name fields, again doable by a small
> >> > python
> >> program
> >> > (like the one I used in the last name field refactoring).
> >> >
> >> > This would really help remove duplicate cmd classes for apis such
> >> > as
> >>copy
> >> > iso and volume which does the same job but requires two different
> >> > cmd
> >> class
> >> > implementations.
> >> >
> >> > Whatever way you decide, pl. make sure to fix ApiDiscovery,
> >> > ApiServer cmdclass-apiname map generator method and apidocs
> >> > generation
> >>accordingly.
> >> >
> >> > Cheers.
> >> >
> >> >
> >> >>
> >> >> Requirement:
> >> >> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList
> >>(grouping
> >> of
> >> >> NetworkACLItems).  Current APIs use *NetworkACL (create
> >> >> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related APIs.
> >> These
> >> >> APIs have to be changed to *NetworkACL Item(create
> >> >> NetworkACLItem/deleteNetworkACLItem etc..) to get the terminology
> >> right. We
> >> >> also need to support old API names for backward compatibility.
> >> >> Hence
> >>the
> >> >> need for API name alias.
> >> >>
> >> >> Terminology:
> >> >> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
> >> >> NetworkACL - Group of Network ACL Items. API will use the term
> >> >> NetworkACLList to differentiate from the existing NetworkACL APIs.
> >> >>
> 
> How about an additional attribute 'preferred' or 'deprecated' ?
> 
> @APICommand(name={'Foo', 'Bar'}, preferred='Foo', deprecated='Bar')

The above convention looks good. In future release we can remove the deprecated api's.  preferred/deprecated will be list type?

Thanks
Rajesh Battala

Re: [Discuss] API name alias

[Nitin Mehta <Ni...@citrix.com>]

In case we need different response tags we can also extend the existing
api class
 


Thanks,
-Nitin

On 09/04/13 2:29 PM, "Kishan Kavala" <Ki...@citrix.com> wrote:

>Thanks everyone for your inputs.
>We should avoid making changes to the APIs which have only one name. So
>they'll continue to have the annotation:
>
>@APICommand(name="apiName1")
>
>For APIs with multiple names, we should support String[] as suggested by
>John. Preferred and deprecated would be required in case of multiple API
>names. Preferred and deprecated params should also support String[].
>@APICommand(name={'Foo', 'Bar', 'FooBar'}, preferred='Foo',
>deprecated={'Bar','FooBar'})
>
>
>> -----Original Message-----
>> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
>> Sent: Tuesday, 9 April 2013 12:05 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [Discuss] API name alias
>> 
>> Rohit, that's like saying we won't fix a leaky faucet today because
>>we're
>> moving in a couple of years anyway. It isn't a big deal to add this and
>>it
>> clarifies to the end user what is the correct alias to use.
>> 
>> On 4/8/13 9:52 PM, "Rohit Yadav" <bh...@apache.org> wrote:
>> 
>> >If we are still on our previously discussed plan (search a wiki shared
>> >by
>> >Min) on deprecating the whole query based API (Server), it's just
>> >unnecessary work on present APIs. I think for now just changing the
>> >name attribute type from String to String[] would just work, as John
>> suggested.
>> >
>> >Adding other fields, may not hold much use for now. Except for few of
>> >them the rest of 300 apis will have preferred == current apiname and
>> >deprecated as blank string.
>> >
>> >I think we should just deprecated all cmd classes (query based) over a
>> >long period say 1 or 2 years, and/or find a way to rewrite/refactor
>> >them so as to reuse them with our new rest based API server for future.
>> >
>> >Cheers.
>> >
>> >On Tue, Apr 9, 2013 at 3:18 AM, Chiradeep Vittal <
>> >Chiradeep.Vittal@citrix.com> wrote:
>> >
>> >>
>> >>
>> >> On 4/8/13 11:18 AM, "Rohit Yadav" <bh...@apache.org> wrote:
>> >>
>> >> >On Mon, Apr 8, 2013 at 7:10 PM, John Burwell <jb...@basho.com>
>> >>wrote:
>> >> >
>> >> >> Rohit,
>> >> >>
>> >> >> Why would we need to rewrite any Java code to switch to an array?
>> >>As I
>> >> >> mentioned, array annotation values are treated as varargs.
>> >>Therefore,
>> >> >> annotations with single values can be left alone.  Only cases
>> >> >> where multiple values are required would require the addition of
>> >> >> curly
>> >>braces.
>> >> >>  For example, for an APICommand with one name, the following would
>> >> >>continue  to work:
>> >> >>
>> >> >
>> >> >Hey John, you're right but it's a matter of code styling, I prefer
>> >> >writing;
>> >> >
>> >> >@APICommand(name = {'name1'}) even if it's only one name, this way
>> >> >any subsequent api author would know that name is an array and they
>> >> >can
>> >>have
>> >> >aliases which is what Kishan wants.
>> >> >
>> >> >
>> >> >>
>> >> >>         @APICommand(name="apiName1")
>> >> >>
>> >> >> While for an APICommand with multiple names, the following form
>> >> >>would now  be supported:
>> >> >>
>> >> >>         @APICommand(name={"apiName1", "apiName2",
>>"apiName3"})@APICommand(name={"apiName1", "apiName2",
>>"apiName3"})@APICommand(name={"apiName1", "apiName2", "apiName3"})
>> >> >>
>> >> >> I am not familiar with the Marvin code, so I can not speak to the
>> >>impact
>> >> >> of this change to it.  However, the only changes to the Java code
>> >> >>should be  the API commands with multiple names and the classes
>> >> >>that actually consume  the value of this annotation.
>> >> >>
>> >> >
>> >> >Yes, you're right we can skip the code styling I referred then it
>> >> >won't consume much coding, we just need to fix these;
>> >> >
>> >> >- Fix APICommand annotation interface definition to accept name as
>> >> >an array
>> >> >- Fix method in ApiServer which creates apiname->cmdclass map to
>> >> >care
>> >>for
>> >> >aliases
>> >> >- Fix ApiDiscovery to take care of the aliases (create different
>> >>response
>> >> >objs for the same cmd class)
>> >> >- Fix apidocs XmlWriter class to take of the aliases and the api
>> >> >html generator to process the same.
>> >> >- Fix Marvin's codegenerator.py to take care of the aliases (which
>> >>means
>> >> >create apiname related modules which would contain cmd and response
>> >> >boilterplate fields/class)
>> >> >
>> >> >Cheers.
>> >> >
>> >> >
>> >> >>
>> >> >> Thanks,
>> >> >> -John
>> >> >>
>> >> >> On Apr 8, 2013, at 9:22 AM, Rohit Yadav <bh...@apache.org>
>>wrote:
>> >> >>
>> >> >> > On Mon, Apr 8, 2013 at 6:33 PM, Kishan Kavala
>> >> >><Kishan.Kavala@citrix.com
>> >> >> >wrote:
>> >> >> >
>> >> >> >> APICommand annotation in API Cmd object has a name parameter.
>> >> >>Currently
>> >> >> >> name parameter takes only one value. I plan to enhance this to
>> >> >>support
>> >> >> >> comma separated values. This will allow multiple API names for
>> >> >> >> the
>> >> >>same
>> >> >> API
>> >> >> >> Cmd object.
>> >> >> >>
>> >> >> >> Current:
>> >> >> >> @APICommand(name = "apiName1", ..
>> >> >> >>
>> >> >> >> Proposed:
>> >> >> >> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
>> >> >> >>
>> >> >> >
>> >> >> > It's a hack, but doable. While not elegant as John suggests,
>> >>parsing
>> >> >> comma
>> >> >> > separated value can lead to some issues.
>> >> >> >
>> >> >> > Changing the name field to array would require a lot of
>> >> >> > rewriting
>> >>the
>> >> >> > present apis annotation name fields, again doable by a small
>> >> >> > python
>> >> >> program
>> >> >> > (like the one I used in the last name field refactoring).
>> >> >> >
>> >> >> > This would really help remove duplicate cmd classes for apis
>> >> >> > such
>> >>as
>> >> >>copy
>> >> >> > iso and volume which does the same job but requires two
>> >> >> > different
>> >>cmd
>> >> >> class
>> >> >> > implementations.
>> >> >> >
>> >> >> > Whatever way you decide, pl. make sure to fix ApiDiscovery,
>> >>ApiServer
>> >> >> > cmdclass-apiname map generator method and apidocs generation
>> >> >>accordingly.
>> >> >> >
>> >> >> > Cheers.
>> >> >> >
>> >> >> >
>> >> >> >>
>> >> >> >> Requirement:
>> >> >> >> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList
>> >> >>(grouping
>> >> >> of
>> >> >> >> NetworkACLItems).  Current APIs use *NetworkACL (create
>> >> >> >> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related
>> >>APIs.
>> >> >> These
>> >> >> >> APIs have to be changed to *NetworkACL Item(create
>> >> >> >> NetworkACLItem/deleteNetworkACLItem etc..) to get the
>> >> >> >> terminology
>> >> >> right. We
>> >> >> >> also need to support old API names for backward compatibility.
>> >>Hence
>> >> >>the
>> >> >> >> need for API name alias.
>> >> >> >>
>> >> >> >> Terminology:
>> >> >> >> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
>> >> >> >> NetworkACL - Group of Network ACL Items. API will use the term
>> >> >> >> NetworkACLList to differentiate from the existing NetworkACL
>>APIs.
>> >> >> >>
>> >>
>> >> How about an additional attribute 'preferred' or 'deprecated' ?
>> >>
>> >> @APICommand(name={'Foo', 'Bar'}, preferred='Foo', deprecated='Bar')
>> >>
>> >>
>

RE: [Discuss] API name alias

[Kishan Kavala <Ki...@citrix.com>]

Thanks everyone for your inputs. 
We should avoid making changes to the APIs which have only one name. So they'll continue to have the annotation:

@APICommand(name="apiName1")

For APIs with multiple names, we should support String[] as suggested by John. Preferred and deprecated would be required in case of multiple API names. Preferred and deprecated params should also support String[].
@APICommand(name={'Foo', 'Bar', 'FooBar'}, preferred='Foo', deprecated={'Bar','FooBar'})


> -----Original Message-----
> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
> Sent: Tuesday, 9 April 2013 12:05 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [Discuss] API name alias
> 
> Rohit, that's like saying we won't fix a leaky faucet today because we're
> moving in a couple of years anyway. It isn't a big deal to add this and it
> clarifies to the end user what is the correct alias to use.
> 
> On 4/8/13 9:52 PM, "Rohit Yadav" <bh...@apache.org> wrote:
> 
> >If we are still on our previously discussed plan (search a wiki shared
> >by
> >Min) on deprecating the whole query based API (Server), it's just
> >unnecessary work on present APIs. I think for now just changing the
> >name attribute type from String to String[] would just work, as John
> suggested.
> >
> >Adding other fields, may not hold much use for now. Except for few of
> >them the rest of 300 apis will have preferred == current apiname and
> >deprecated as blank string.
> >
> >I think we should just deprecated all cmd classes (query based) over a
> >long period say 1 or 2 years, and/or find a way to rewrite/refactor
> >them so as to reuse them with our new rest based API server for future.
> >
> >Cheers.
> >
> >On Tue, Apr 9, 2013 at 3:18 AM, Chiradeep Vittal <
> >Chiradeep.Vittal@citrix.com> wrote:
> >
> >>
> >>
> >> On 4/8/13 11:18 AM, "Rohit Yadav" <bh...@apache.org> wrote:
> >>
> >> >On Mon, Apr 8, 2013 at 7:10 PM, John Burwell <jb...@basho.com>
> >>wrote:
> >> >
> >> >> Rohit,
> >> >>
> >> >> Why would we need to rewrite any Java code to switch to an array?
> >>As I
> >> >> mentioned, array annotation values are treated as varargs.
> >>Therefore,
> >> >> annotations with single values can be left alone.  Only cases
> >> >> where multiple values are required would require the addition of
> >> >> curly
> >>braces.
> >> >>  For example, for an APICommand with one name, the following would
> >> >>continue  to work:
> >> >>
> >> >
> >> >Hey John, you're right but it's a matter of code styling, I prefer
> >> >writing;
> >> >
> >> >@APICommand(name = {'name1'}) even if it's only one name, this way
> >> >any subsequent api author would know that name is an array and they
> >> >can
> >>have
> >> >aliases which is what Kishan wants.
> >> >
> >> >
> >> >>
> >> >>         @APICommand(name="apiName1")
> >> >>
> >> >> While for an APICommand with multiple names, the following form
> >> >>would now  be supported:
> >> >>
> >> >>         @APICommand(name={"apiName1", "apiName2", "apiName3"})@APICommand(name={"apiName1", "apiName2", "apiName3"})@APICommand(name={"apiName1", "apiName2", "apiName3"})
> >> >>
> >> >> I am not familiar with the Marvin code, so I can not speak to the
> >>impact
> >> >> of this change to it.  However, the only changes to the Java code
> >> >>should be  the API commands with multiple names and the classes
> >> >>that actually consume  the value of this annotation.
> >> >>
> >> >
> >> >Yes, you're right we can skip the code styling I referred then it
> >> >won't consume much coding, we just need to fix these;
> >> >
> >> >- Fix APICommand annotation interface definition to accept name as
> >> >an array
> >> >- Fix method in ApiServer which creates apiname->cmdclass map to
> >> >care
> >>for
> >> >aliases
> >> >- Fix ApiDiscovery to take care of the aliases (create different
> >>response
> >> >objs for the same cmd class)
> >> >- Fix apidocs XmlWriter class to take of the aliases and the api
> >> >html generator to process the same.
> >> >- Fix Marvin's codegenerator.py to take care of the aliases (which
> >>means
> >> >create apiname related modules which would contain cmd and response
> >> >boilterplate fields/class)
> >> >
> >> >Cheers.
> >> >
> >> >
> >> >>
> >> >> Thanks,
> >> >> -John
> >> >>
> >> >> On Apr 8, 2013, at 9:22 AM, Rohit Yadav <bh...@apache.org> wrote:
> >> >>
> >> >> > On Mon, Apr 8, 2013 at 6:33 PM, Kishan Kavala
> >> >><Kishan.Kavala@citrix.com
> >> >> >wrote:
> >> >> >
> >> >> >> APICommand annotation in API Cmd object has a name parameter.
> >> >>Currently
> >> >> >> name parameter takes only one value. I plan to enhance this to
> >> >>support
> >> >> >> comma separated values. This will allow multiple API names for
> >> >> >> the
> >> >>same
> >> >> API
> >> >> >> Cmd object.
> >> >> >>
> >> >> >> Current:
> >> >> >> @APICommand(name = "apiName1", ..
> >> >> >>
> >> >> >> Proposed:
> >> >> >> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
> >> >> >>
> >> >> >
> >> >> > It's a hack, but doable. While not elegant as John suggests,
> >>parsing
> >> >> comma
> >> >> > separated value can lead to some issues.
> >> >> >
> >> >> > Changing the name field to array would require a lot of
> >> >> > rewriting
> >>the
> >> >> > present apis annotation name fields, again doable by a small
> >> >> > python
> >> >> program
> >> >> > (like the one I used in the last name field refactoring).
> >> >> >
> >> >> > This would really help remove duplicate cmd classes for apis
> >> >> > such
> >>as
> >> >>copy
> >> >> > iso and volume which does the same job but requires two
> >> >> > different
> >>cmd
> >> >> class
> >> >> > implementations.
> >> >> >
> >> >> > Whatever way you decide, pl. make sure to fix ApiDiscovery,
> >>ApiServer
> >> >> > cmdclass-apiname map generator method and apidocs generation
> >> >>accordingly.
> >> >> >
> >> >> > Cheers.
> >> >> >
> >> >> >
> >> >> >>
> >> >> >> Requirement:
> >> >> >> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList
> >> >>(grouping
> >> >> of
> >> >> >> NetworkACLItems).  Current APIs use *NetworkACL (create
> >> >> >> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related
> >>APIs.
> >> >> These
> >> >> >> APIs have to be changed to *NetworkACL Item(create
> >> >> >> NetworkACLItem/deleteNetworkACLItem etc..) to get the
> >> >> >> terminology
> >> >> right. We
> >> >> >> also need to support old API names for backward compatibility.
> >>Hence
> >> >>the
> >> >> >> need for API name alias.
> >> >> >>
> >> >> >> Terminology:
> >> >> >> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
> >> >> >> NetworkACL - Group of Network ACL Items. API will use the term
> >> >> >> NetworkACLList to differentiate from the existing NetworkACL APIs.
> >> >> >>
> >>
> >> How about an additional attribute 'preferred' or 'deprecated' ?
> >>
> >> @APICommand(name={'Foo', 'Bar'}, preferred='Foo', deprecated='Bar')
> >>
> >>

Re: [Discuss] API name alias

[Rohit Yadav <bh...@apache.org>]

On Tue, Apr 9, 2013 at 11:34 PM, Chiradeep Vittal <
Chiradeep.Vittal@citrix.com> wrote:

> In Kishan's case he is adding a better, accurate name for the API. The API
> was misnamed in the previous version (ACL = Access Control List, but ACL
> was referring to an individual item within the list). Similarly, there are
> other poorly worded APIs (createIpForwardingRule instead of
> createStaticNat comes to mind).
>

Okay I did not see that I thought putting them in the apiname array of name
fields would work. In such a case, we can just keep name = "current or
preferred or new api name" and another field "deprecated" can have the old
name. Whenever someone queries by that name, we can message them somehow to
avoid using the deprecated api name.

Cheers.


>
> On 4/9/13 2:19 AM, "Rohit Yadav" <bh...@apache.org> wrote:
>
> >On Tue, Apr 9, 2013 at 12:04 PM, Chiradeep Vittal <
> >Chiradeep.Vittal@citrix.com> wrote:
> >
> >> Rohit, that's like saying we won't fix a leaky faucet today because
> >>we're
> >> moving in a couple of years anyway. It isn't a big deal to add this and
> >>it
> >> clarifies to the end user what is the correct alias to use.
> >>
> >
> >Chiradeep, if there are multiple apinames for the same operation (cmd
> >class) where does the question of "correct alias to use" come?
> >
> >Excuse my ignorance, I don't understand why we need "preferred" and
> >"deprecated" fields in that respect. IMO it won't solve any problem but
> >add
> >confusion and ambiguity, unless there is some use you may explain?
> >
> >As Kishan shares, he would use the array styled approach which John
> >suggested and I think it should work, why we want preferred and deprecated
> >Strings or String[].
> >
> >Cheers.
> >
> >
> >> On 4/8/13 9:52 PM, "Rohit Yadav" <bh...@apache.org> wrote:
> >>
> >> >If we are still on our previously discussed plan (search a wiki shared
> >>by
> >> >Min) on deprecating the whole query based API (Server), it's just
> >> >unnecessary work on present APIs. I think for now just changing the
> >>name
> >> >attribute type from String to String[] would just work, as John
> >>suggested.
> >> >
> >> >Adding other fields, may not hold much use for now. Except for few of
> >>them
> >> >the rest of 300 apis will have preferred == current apiname and
> >>deprecated
> >> >as blank string.
> >> >
> >> >I think we should just deprecated all cmd classes (query based) over a
> >> >long
> >> >period say 1 or 2 years, and/or find a way to rewrite/refactor them so
> >>as
> >> >to reuse them with our new rest based API server for future.
> >> >
> >> >Cheers.
> >> >
> >> >On Tue, Apr 9, 2013 at 3:18 AM, Chiradeep Vittal <
> >> >Chiradeep.Vittal@citrix.com> wrote:
> >> >
> >> >>
> >> >>
> >> >> On 4/8/13 11:18 AM, "Rohit Yadav" <bh...@apache.org> wrote:
> >> >>
> >> >> >On Mon, Apr 8, 2013 at 7:10 PM, John Burwell <jb...@basho.com>
> >> >>wrote:
> >> >> >
> >> >> >> Rohit,
> >> >> >>
> >> >> >> Why would we need to rewrite any Java code to switch to an array?
> >> >>As I
> >> >> >> mentioned, array annotation values are treated as varargs.
> >> >>Therefore,
> >> >> >> annotations with single values can be left alone.  Only cases
> >>where
> >> >> >> multiple values are required would require the addition of curly
> >> >>braces.
> >> >> >>  For example, for an APICommand with one name, the following would
> >> >> >>continue
> >> >> >> to work:
> >> >> >>
> >> >> >
> >> >> >Hey John, you're right but it's a matter of code styling, I prefer
> >> >> >writing;
> >> >> >
> >> >> >@APICommand(name = {'name1'}) even if it's only one name, this way
> >>any
> >> >> >subsequent api author would know that name is an array and they can
> >> >>have
> >> >> >aliases which is what Kishan wants.
> >> >> >
> >> >> >
> >> >> >>
> >> >> >>         @APICommand(name="apiName1")
> >> >> >>
> >> >> >> While for an APICommand with multiple names, the following form
> >>would
> >> >> >>now
> >> >> >> be supported:
> >> >> >>
> >> >> >>         @APICommand(name={"apiName1", "apiName2", "apiName3"})
> >> >> >>
> >> >> >> I am not familiar with the Marvin code, so I can not speak to the
> >> >>impact
> >> >> >> of this change to it.  However, the only changes to the Java code
> >> >> >>should be
> >> >> >> the API commands with multiple names and the classes that actually
> >> >> >>consume
> >> >> >> the value of this annotation.
> >> >> >>
> >> >> >
> >> >> >Yes, you're right we can skip the code styling I referred then it
> >>won't
> >> >> >consume much coding, we just need to fix these;
> >> >> >
> >> >> >- Fix APICommand annotation interface definition to accept name as
> >>an
> >> >> >array
> >> >> >- Fix method in ApiServer which creates apiname->cmdclass map to
> >>care
> >> >>for
> >> >> >aliases
> >> >> >- Fix ApiDiscovery to take care of the aliases (create different
> >> >>response
> >> >> >objs for the same cmd class)
> >> >> >- Fix apidocs XmlWriter class to take of the aliases and the api
> >>html
> >> >> >generator to process the same.
> >> >> >- Fix Marvin's codegenerator.py to take care of the aliases (which
> >> >>means
> >> >> >create apiname related modules which would contain cmd and response
> >> >> >boilterplate fields/class)
> >> >> >
> >> >> >Cheers.
> >> >> >
> >> >> >
> >> >> >>
> >> >> >> Thanks,
> >> >> >> -John
> >> >> >>
> >> >> >> On Apr 8, 2013, at 9:22 AM, Rohit Yadav <bh...@apache.org>
> >>wrote:
> >> >> >>
> >> >> >> > On Mon, Apr 8, 2013 at 6:33 PM, Kishan Kavala
> >> >> >><Kishan.Kavala@citrix.com
> >> >> >> >wrote:
> >> >> >> >
> >> >> >> >> APICommand annotation in API Cmd object has a name parameter.
> >> >> >>Currently
> >> >> >> >> name parameter takes only one value. I plan to enhance this to
> >> >> >>support
> >> >> >> >> comma separated values. This will allow multiple API names for
> >>the
> >> >> >>same
> >> >> >> API
> >> >> >> >> Cmd object.
> >> >> >> >>
> >> >> >> >> Current:
> >> >> >> >> @APICommand(name = "apiName1", ..
> >> >> >> >>
> >> >> >> >> Proposed:
> >> >> >> >> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
> >> >> >> >>
> >> >> >> >
> >> >> >> > It's a hack, but doable. While not elegant as John suggests,
> >> >>parsing
> >> >> >> comma
> >> >> >> > separated value can lead to some issues.
> >> >> >> >
> >> >> >> > Changing the name field to array would require a lot of
> >>rewriting
> >> >>the
> >> >> >> > present apis annotation name fields, again doable by a small
> >>python
> >> >> >> program
> >> >> >> > (like the one I used in the last name field refactoring).
> >> >> >> >
> >> >> >> > This would really help remove duplicate cmd classes for apis
> >>such
> >> >>as
> >> >> >>copy
> >> >> >> > iso and volume which does the same job but requires two
> >>different
> >> >>cmd
> >> >> >> class
> >> >> >> > implementations.
> >> >> >> >
> >> >> >> > Whatever way you decide, pl. make sure to fix ApiDiscovery,
> >> >>ApiServer
> >> >> >> > cmdclass-apiname map generator method and apidocs generation
> >> >> >>accordingly.
> >> >> >> >
> >> >> >> > Cheers.
> >> >> >> >
> >> >> >> >
> >> >> >> >>
> >> >> >> >> Requirement:
> >> >> >> >> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList
> >> >> >>(grouping
> >> >> >> of
> >> >> >> >> NetworkACLItems).  Current APIs use *NetworkACL (create
> >> >> >> >> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related
> >> >>APIs.
> >> >> >> These
> >> >> >> >> APIs have to be changed to *NetworkACL Item(create
> >> >> >> >> NetworkACLItem/deleteNetworkACLItem etc..) to get the
> >>terminology
> >> >> >> right. We
> >> >> >> >> also need to support old API names for backward compatibility.
> >> >>Hence
> >> >> >>the
> >> >> >> >> need for API name alias.
> >> >> >> >>
> >> >> >> >> Terminology:
> >> >> >> >> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
> >> >> >> >> NetworkACL - Group of Network ACL Items. API will use the term
> >> >> >> >> NetworkACLList to differentiate from the existing NetworkACL
> >>APIs.
> >> >> >> >>
> >> >>
> >> >> How about an additional attribute 'preferred' or 'deprecated' ?
> >> >>
> >> >> @APICommand(name={'Foo', 'Bar'}, preferred='Foo', deprecated='Bar')
> >> >>
> >> >>
> >>
> >>
>
>

Re: [Discuss] API name alias

[Chiradeep Vittal <Ch...@citrix.com>]

In Kishan's case he is adding a better, accurate name for the API. The API
was misnamed in the previous version (ACL = Access Control List, but ACL
was referring to an individual item within the list). Similarly, there are
other poorly worded APIs (createIpForwardingRule instead of
createStaticNat comes to mind).

On 4/9/13 2:19 AM, "Rohit Yadav" <bh...@apache.org> wrote:

>On Tue, Apr 9, 2013 at 12:04 PM, Chiradeep Vittal <
>Chiradeep.Vittal@citrix.com> wrote:
>
>> Rohit, that's like saying we won't fix a leaky faucet today because
>>we're
>> moving in a couple of years anyway. It isn't a big deal to add this and
>>it
>> clarifies to the end user what is the correct alias to use.
>>
>
>Chiradeep, if there are multiple apinames for the same operation (cmd
>class) where does the question of "correct alias to use" come?
>
>Excuse my ignorance, I don't understand why we need "preferred" and
>"deprecated" fields in that respect. IMO it won't solve any problem but
>add
>confusion and ambiguity, unless there is some use you may explain?
>
>As Kishan shares, he would use the array styled approach which John
>suggested and I think it should work, why we want preferred and deprecated
>Strings or String[].
>
>Cheers.
>
>
>> On 4/8/13 9:52 PM, "Rohit Yadav" <bh...@apache.org> wrote:
>>
>> >If we are still on our previously discussed plan (search a wiki shared
>>by
>> >Min) on deprecating the whole query based API (Server), it's just
>> >unnecessary work on present APIs. I think for now just changing the
>>name
>> >attribute type from String to String[] would just work, as John
>>suggested.
>> >
>> >Adding other fields, may not hold much use for now. Except for few of
>>them
>> >the rest of 300 apis will have preferred == current apiname and
>>deprecated
>> >as blank string.
>> >
>> >I think we should just deprecated all cmd classes (query based) over a
>> >long
>> >period say 1 or 2 years, and/or find a way to rewrite/refactor them so
>>as
>> >to reuse them with our new rest based API server for future.
>> >
>> >Cheers.
>> >
>> >On Tue, Apr 9, 2013 at 3:18 AM, Chiradeep Vittal <
>> >Chiradeep.Vittal@citrix.com> wrote:
>> >
>> >>
>> >>
>> >> On 4/8/13 11:18 AM, "Rohit Yadav" <bh...@apache.org> wrote:
>> >>
>> >> >On Mon, Apr 8, 2013 at 7:10 PM, John Burwell <jb...@basho.com>
>> >>wrote:
>> >> >
>> >> >> Rohit,
>> >> >>
>> >> >> Why would we need to rewrite any Java code to switch to an array?
>> >>As I
>> >> >> mentioned, array annotation values are treated as varargs.
>> >>Therefore,
>> >> >> annotations with single values can be left alone.  Only cases
>>where
>> >> >> multiple values are required would require the addition of curly
>> >>braces.
>> >> >>  For example, for an APICommand with one name, the following would
>> >> >>continue
>> >> >> to work:
>> >> >>
>> >> >
>> >> >Hey John, you're right but it's a matter of code styling, I prefer
>> >> >writing;
>> >> >
>> >> >@APICommand(name = {'name1'}) even if it's only one name, this way
>>any
>> >> >subsequent api author would know that name is an array and they can
>> >>have
>> >> >aliases which is what Kishan wants.
>> >> >
>> >> >
>> >> >>
>> >> >>         @APICommand(name="apiName1")
>> >> >>
>> >> >> While for an APICommand with multiple names, the following form
>>would
>> >> >>now
>> >> >> be supported:
>> >> >>
>> >> >>         @APICommand(name={"apiName1", "apiName2", "apiName3"})
>> >> >>
>> >> >> I am not familiar with the Marvin code, so I can not speak to the
>> >>impact
>> >> >> of this change to it.  However, the only changes to the Java code
>> >> >>should be
>> >> >> the API commands with multiple names and the classes that actually
>> >> >>consume
>> >> >> the value of this annotation.
>> >> >>
>> >> >
>> >> >Yes, you're right we can skip the code styling I referred then it
>>won't
>> >> >consume much coding, we just need to fix these;
>> >> >
>> >> >- Fix APICommand annotation interface definition to accept name as
>>an
>> >> >array
>> >> >- Fix method in ApiServer which creates apiname->cmdclass map to
>>care
>> >>for
>> >> >aliases
>> >> >- Fix ApiDiscovery to take care of the aliases (create different
>> >>response
>> >> >objs for the same cmd class)
>> >> >- Fix apidocs XmlWriter class to take of the aliases and the api
>>html
>> >> >generator to process the same.
>> >> >- Fix Marvin's codegenerator.py to take care of the aliases (which
>> >>means
>> >> >create apiname related modules which would contain cmd and response
>> >> >boilterplate fields/class)
>> >> >
>> >> >Cheers.
>> >> >
>> >> >
>> >> >>
>> >> >> Thanks,
>> >> >> -John
>> >> >>
>> >> >> On Apr 8, 2013, at 9:22 AM, Rohit Yadav <bh...@apache.org>
>>wrote:
>> >> >>
>> >> >> > On Mon, Apr 8, 2013 at 6:33 PM, Kishan Kavala
>> >> >><Kishan.Kavala@citrix.com
>> >> >> >wrote:
>> >> >> >
>> >> >> >> APICommand annotation in API Cmd object has a name parameter.
>> >> >>Currently
>> >> >> >> name parameter takes only one value. I plan to enhance this to
>> >> >>support
>> >> >> >> comma separated values. This will allow multiple API names for
>>the
>> >> >>same
>> >> >> API
>> >> >> >> Cmd object.
>> >> >> >>
>> >> >> >> Current:
>> >> >> >> @APICommand(name = "apiName1", ..
>> >> >> >>
>> >> >> >> Proposed:
>> >> >> >> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
>> >> >> >>
>> >> >> >
>> >> >> > It's a hack, but doable. While not elegant as John suggests,
>> >>parsing
>> >> >> comma
>> >> >> > separated value can lead to some issues.
>> >> >> >
>> >> >> > Changing the name field to array would require a lot of
>>rewriting
>> >>the
>> >> >> > present apis annotation name fields, again doable by a small
>>python
>> >> >> program
>> >> >> > (like the one I used in the last name field refactoring).
>> >> >> >
>> >> >> > This would really help remove duplicate cmd classes for apis
>>such
>> >>as
>> >> >>copy
>> >> >> > iso and volume which does the same job but requires two
>>different
>> >>cmd
>> >> >> class
>> >> >> > implementations.
>> >> >> >
>> >> >> > Whatever way you decide, pl. make sure to fix ApiDiscovery,
>> >>ApiServer
>> >> >> > cmdclass-apiname map generator method and apidocs generation
>> >> >>accordingly.
>> >> >> >
>> >> >> > Cheers.
>> >> >> >
>> >> >> >
>> >> >> >>
>> >> >> >> Requirement:
>> >> >> >> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList
>> >> >>(grouping
>> >> >> of
>> >> >> >> NetworkACLItems).  Current APIs use *NetworkACL (create
>> >> >> >> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related
>> >>APIs.
>> >> >> These
>> >> >> >> APIs have to be changed to *NetworkACL Item(create
>> >> >> >> NetworkACLItem/deleteNetworkACLItem etc..) to get the
>>terminology
>> >> >> right. We
>> >> >> >> also need to support old API names for backward compatibility.
>> >>Hence
>> >> >>the
>> >> >> >> need for API name alias.
>> >> >> >>
>> >> >> >> Terminology:
>> >> >> >> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
>> >> >> >> NetworkACL - Group of Network ACL Items. API will use the term
>> >> >> >> NetworkACLList to differentiate from the existing NetworkACL
>>APIs.
>> >> >> >>
>> >>
>> >> How about an additional attribute 'preferred' or 'deprecated' ?
>> >>
>> >> @APICommand(name={'Foo', 'Bar'}, preferred='Foo', deprecated='Bar')
>> >>
>> >>
>>
>>

Re: [Discuss] API name alias

[Rohit Yadav <bh...@apache.org>]

On Tue, Apr 9, 2013 at 12:04 PM, Chiradeep Vittal <
Chiradeep.Vittal@citrix.com> wrote:

> Rohit, that's like saying we won't fix a leaky faucet today because we're
> moving in a couple of years anyway. It isn't a big deal to add this and it
> clarifies to the end user what is the correct alias to use.
>

Chiradeep, if there are multiple apinames for the same operation (cmd
class) where does the question of "correct alias to use" come?

Excuse my ignorance, I don't understand why we need "preferred" and
"deprecated" fields in that respect. IMO it won't solve any problem but add
confusion and ambiguity, unless there is some use you may explain?

As Kishan shares, he would use the array styled approach which John
suggested and I think it should work, why we want preferred and deprecated
Strings or String[].

Cheers.


> On 4/8/13 9:52 PM, "Rohit Yadav" <bh...@apache.org> wrote:
>
> >If we are still on our previously discussed plan (search a wiki shared by
> >Min) on deprecating the whole query based API (Server), it's just
> >unnecessary work on present APIs. I think for now just changing the name
> >attribute type from String to String[] would just work, as John suggested.
> >
> >Adding other fields, may not hold much use for now. Except for few of them
> >the rest of 300 apis will have preferred == current apiname and deprecated
> >as blank string.
> >
> >I think we should just deprecated all cmd classes (query based) over a
> >long
> >period say 1 or 2 years, and/or find a way to rewrite/refactor them so as
> >to reuse them with our new rest based API server for future.
> >
> >Cheers.
> >
> >On Tue, Apr 9, 2013 at 3:18 AM, Chiradeep Vittal <
> >Chiradeep.Vittal@citrix.com> wrote:
> >
> >>
> >>
> >> On 4/8/13 11:18 AM, "Rohit Yadav" <bh...@apache.org> wrote:
> >>
> >> >On Mon, Apr 8, 2013 at 7:10 PM, John Burwell <jb...@basho.com>
> >>wrote:
> >> >
> >> >> Rohit,
> >> >>
> >> >> Why would we need to rewrite any Java code to switch to an array?
> >>As I
> >> >> mentioned, array annotation values are treated as varargs.
> >>Therefore,
> >> >> annotations with single values can be left alone.  Only cases where
> >> >> multiple values are required would require the addition of curly
> >>braces.
> >> >>  For example, for an APICommand with one name, the following would
> >> >>continue
> >> >> to work:
> >> >>
> >> >
> >> >Hey John, you're right but it's a matter of code styling, I prefer
> >> >writing;
> >> >
> >> >@APICommand(name = {'name1'}) even if it's only one name, this way any
> >> >subsequent api author would know that name is an array and they can
> >>have
> >> >aliases which is what Kishan wants.
> >> >
> >> >
> >> >>
> >> >>         @APICommand(name="apiName1")
> >> >>
> >> >> While for an APICommand with multiple names, the following form would
> >> >>now
> >> >> be supported:
> >> >>
> >> >>         @APICommand(name={"apiName1", "apiName2", "apiName3"})
> >> >>
> >> >> I am not familiar with the Marvin code, so I can not speak to the
> >>impact
> >> >> of this change to it.  However, the only changes to the Java code
> >> >>should be
> >> >> the API commands with multiple names and the classes that actually
> >> >>consume
> >> >> the value of this annotation.
> >> >>
> >> >
> >> >Yes, you're right we can skip the code styling I referred then it won't
> >> >consume much coding, we just need to fix these;
> >> >
> >> >- Fix APICommand annotation interface definition to accept name as an
> >> >array
> >> >- Fix method in ApiServer which creates apiname->cmdclass map to care
> >>for
> >> >aliases
> >> >- Fix ApiDiscovery to take care of the aliases (create different
> >>response
> >> >objs for the same cmd class)
> >> >- Fix apidocs XmlWriter class to take of the aliases and the api html
> >> >generator to process the same.
> >> >- Fix Marvin's codegenerator.py to take care of the aliases (which
> >>means
> >> >create apiname related modules which would contain cmd and response
> >> >boilterplate fields/class)
> >> >
> >> >Cheers.
> >> >
> >> >
> >> >>
> >> >> Thanks,
> >> >> -John
> >> >>
> >> >> On Apr 8, 2013, at 9:22 AM, Rohit Yadav <bh...@apache.org> wrote:
> >> >>
> >> >> > On Mon, Apr 8, 2013 at 6:33 PM, Kishan Kavala
> >> >><Kishan.Kavala@citrix.com
> >> >> >wrote:
> >> >> >
> >> >> >> APICommand annotation in API Cmd object has a name parameter.
> >> >>Currently
> >> >> >> name parameter takes only one value. I plan to enhance this to
> >> >>support
> >> >> >> comma separated values. This will allow multiple API names for the
> >> >>same
> >> >> API
> >> >> >> Cmd object.
> >> >> >>
> >> >> >> Current:
> >> >> >> @APICommand(name = "apiName1", ..
> >> >> >>
> >> >> >> Proposed:
> >> >> >> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
> >> >> >>
> >> >> >
> >> >> > It's a hack, but doable. While not elegant as John suggests,
> >>parsing
> >> >> comma
> >> >> > separated value can lead to some issues.
> >> >> >
> >> >> > Changing the name field to array would require a lot of rewriting
> >>the
> >> >> > present apis annotation name fields, again doable by a small python
> >> >> program
> >> >> > (like the one I used in the last name field refactoring).
> >> >> >
> >> >> > This would really help remove duplicate cmd classes for apis such
> >>as
> >> >>copy
> >> >> > iso and volume which does the same job but requires two different
> >>cmd
> >> >> class
> >> >> > implementations.
> >> >> >
> >> >> > Whatever way you decide, pl. make sure to fix ApiDiscovery,
> >>ApiServer
> >> >> > cmdclass-apiname map generator method and apidocs generation
> >> >>accordingly.
> >> >> >
> >> >> > Cheers.
> >> >> >
> >> >> >
> >> >> >>
> >> >> >> Requirement:
> >> >> >> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList
> >> >>(grouping
> >> >> of
> >> >> >> NetworkACLItems).  Current APIs use *NetworkACL (create
> >> >> >> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related
> >>APIs.
> >> >> These
> >> >> >> APIs have to be changed to *NetworkACL Item(create
> >> >> >> NetworkACLItem/deleteNetworkACLItem etc..) to get the terminology
> >> >> right. We
> >> >> >> also need to support old API names for backward compatibility.
> >>Hence
> >> >>the
> >> >> >> need for API name alias.
> >> >> >>
> >> >> >> Terminology:
> >> >> >> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
> >> >> >> NetworkACL - Group of Network ACL Items. API will use the term
> >> >> >> NetworkACLList to differentiate from the existing NetworkACL APIs.
> >> >> >>
> >>
> >> How about an additional attribute 'preferred' or 'deprecated' ?
> >>
> >> @APICommand(name={'Foo', 'Bar'}, preferred='Foo', deprecated='Bar')
> >>
> >>
>
>

Re: [Discuss] API name alias

[Chiradeep Vittal <Ch...@citrix.com>]

Rohit, that's like saying we won't fix a leaky faucet today because we're
moving in a couple of years anyway. It isn't a big deal to add this and it
clarifies to the end user what is the correct alias to use.

On 4/8/13 9:52 PM, "Rohit Yadav" <bh...@apache.org> wrote:

>If we are still on our previously discussed plan (search a wiki shared by
>Min) on deprecating the whole query based API (Server), it's just
>unnecessary work on present APIs. I think for now just changing the name
>attribute type from String to String[] would just work, as John suggested.
>
>Adding other fields, may not hold much use for now. Except for few of them
>the rest of 300 apis will have preferred == current apiname and deprecated
>as blank string.
>
>I think we should just deprecated all cmd classes (query based) over a
>long
>period say 1 or 2 years, and/or find a way to rewrite/refactor them so as
>to reuse them with our new rest based API server for future.
>
>Cheers.
>
>On Tue, Apr 9, 2013 at 3:18 AM, Chiradeep Vittal <
>Chiradeep.Vittal@citrix.com> wrote:
>
>>
>>
>> On 4/8/13 11:18 AM, "Rohit Yadav" <bh...@apache.org> wrote:
>>
>> >On Mon, Apr 8, 2013 at 7:10 PM, John Burwell <jb...@basho.com>
>>wrote:
>> >
>> >> Rohit,
>> >>
>> >> Why would we need to rewrite any Java code to switch to an array?
>>As I
>> >> mentioned, array annotation values are treated as varargs.
>>Therefore,
>> >> annotations with single values can be left alone.  Only cases where
>> >> multiple values are required would require the addition of curly
>>braces.
>> >>  For example, for an APICommand with one name, the following would
>> >>continue
>> >> to work:
>> >>
>> >
>> >Hey John, you're right but it's a matter of code styling, I prefer
>> >writing;
>> >
>> >@APICommand(name = {'name1'}) even if it's only one name, this way any
>> >subsequent api author would know that name is an array and they can
>>have
>> >aliases which is what Kishan wants.
>> >
>> >
>> >>
>> >>         @APICommand(name="apiName1")
>> >>
>> >> While for an APICommand with multiple names, the following form would
>> >>now
>> >> be supported:
>> >>
>> >>         @APICommand(name={"apiName1", "apiName2", "apiName3"})
>> >>
>> >> I am not familiar with the Marvin code, so I can not speak to the
>>impact
>> >> of this change to it.  However, the only changes to the Java code
>> >>should be
>> >> the API commands with multiple names and the classes that actually
>> >>consume
>> >> the value of this annotation.
>> >>
>> >
>> >Yes, you're right we can skip the code styling I referred then it won't
>> >consume much coding, we just need to fix these;
>> >
>> >- Fix APICommand annotation interface definition to accept name as an
>> >array
>> >- Fix method in ApiServer which creates apiname->cmdclass map to care
>>for
>> >aliases
>> >- Fix ApiDiscovery to take care of the aliases (create different
>>response
>> >objs for the same cmd class)
>> >- Fix apidocs XmlWriter class to take of the aliases and the api html
>> >generator to process the same.
>> >- Fix Marvin's codegenerator.py to take care of the aliases (which
>>means
>> >create apiname related modules which would contain cmd and response
>> >boilterplate fields/class)
>> >
>> >Cheers.
>> >
>> >
>> >>
>> >> Thanks,
>> >> -John
>> >>
>> >> On Apr 8, 2013, at 9:22 AM, Rohit Yadav <bh...@apache.org> wrote:
>> >>
>> >> > On Mon, Apr 8, 2013 at 6:33 PM, Kishan Kavala
>> >><Kishan.Kavala@citrix.com
>> >> >wrote:
>> >> >
>> >> >> APICommand annotation in API Cmd object has a name parameter.
>> >>Currently
>> >> >> name parameter takes only one value. I plan to enhance this to
>> >>support
>> >> >> comma separated values. This will allow multiple API names for the
>> >>same
>> >> API
>> >> >> Cmd object.
>> >> >>
>> >> >> Current:
>> >> >> @APICommand(name = "apiName1", ..
>> >> >>
>> >> >> Proposed:
>> >> >> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
>> >> >>
>> >> >
>> >> > It's a hack, but doable. While not elegant as John suggests,
>>parsing
>> >> comma
>> >> > separated value can lead to some issues.
>> >> >
>> >> > Changing the name field to array would require a lot of rewriting
>>the
>> >> > present apis annotation name fields, again doable by a small python
>> >> program
>> >> > (like the one I used in the last name field refactoring).
>> >> >
>> >> > This would really help remove duplicate cmd classes for apis such
>>as
>> >>copy
>> >> > iso and volume which does the same job but requires two different
>>cmd
>> >> class
>> >> > implementations.
>> >> >
>> >> > Whatever way you decide, pl. make sure to fix ApiDiscovery,
>>ApiServer
>> >> > cmdclass-apiname map generator method and apidocs generation
>> >>accordingly.
>> >> >
>> >> > Cheers.
>> >> >
>> >> >
>> >> >>
>> >> >> Requirement:
>> >> >> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList
>> >>(grouping
>> >> of
>> >> >> NetworkACLItems).  Current APIs use *NetworkACL (create
>> >> >> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related
>>APIs.
>> >> These
>> >> >> APIs have to be changed to *NetworkACL Item(create
>> >> >> NetworkACLItem/deleteNetworkACLItem etc..) to get the terminology
>> >> right. We
>> >> >> also need to support old API names for backward compatibility.
>>Hence
>> >>the
>> >> >> need for API name alias.
>> >> >>
>> >> >> Terminology:
>> >> >> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
>> >> >> NetworkACL - Group of Network ACL Items. API will use the term
>> >> >> NetworkACLList to differentiate from the existing NetworkACL APIs.
>> >> >>
>>
>> How about an additional attribute 'preferred' or 'deprecated' ?
>>
>> @APICommand(name={'Foo', 'Bar'}, preferred='Foo', deprecated='Bar')
>>
>>

Re: [Discuss] API name alias

[Rohit Yadav <bh...@apache.org>]

If we are still on our previously discussed plan (search a wiki shared by
Min) on deprecating the whole query based API (Server), it's just
unnecessary work on present APIs. I think for now just changing the name
attribute type from String to String[] would just work, as John suggested.

Adding other fields, may not hold much use for now. Except for few of them
the rest of 300 apis will have preferred == current apiname and deprecated
as blank string.

I think we should just deprecated all cmd classes (query based) over a long
period say 1 or 2 years, and/or find a way to rewrite/refactor them so as
to reuse them with our new rest based API server for future.

Cheers.

On Tue, Apr 9, 2013 at 3:18 AM, Chiradeep Vittal <
Chiradeep.Vittal@citrix.com> wrote:

>
>
> On 4/8/13 11:18 AM, "Rohit Yadav" <bh...@apache.org> wrote:
>
> >On Mon, Apr 8, 2013 at 7:10 PM, John Burwell <jb...@basho.com> wrote:
> >
> >> Rohit,
> >>
> >> Why would we need to rewrite any Java code to switch to an array?  As I
> >> mentioned, array annotation values are treated as varargs.  Therefore,
> >> annotations with single values can be left alone.  Only cases where
> >> multiple values are required would require the addition of curly braces.
> >>  For example, for an APICommand with one name, the following would
> >>continue
> >> to work:
> >>
> >
> >Hey John, you're right but it's a matter of code styling, I prefer
> >writing;
> >
> >@APICommand(name = {'name1'}) even if it's only one name, this way any
> >subsequent api author would know that name is an array and they can have
> >aliases which is what Kishan wants.
> >
> >
> >>
> >>         @APICommand(name="apiName1")
> >>
> >> While for an APICommand with multiple names, the following form would
> >>now
> >> be supported:
> >>
> >>         @APICommand(name={"apiName1", "apiName2", "apiName3"})
> >>
> >> I am not familiar with the Marvin code, so I can not speak to the impact
> >> of this change to it.  However, the only changes to the Java code
> >>should be
> >> the API commands with multiple names and the classes that actually
> >>consume
> >> the value of this annotation.
> >>
> >
> >Yes, you're right we can skip the code styling I referred then it won't
> >consume much coding, we just need to fix these;
> >
> >- Fix APICommand annotation interface definition to accept name as an
> >array
> >- Fix method in ApiServer which creates apiname->cmdclass map to care for
> >aliases
> >- Fix ApiDiscovery to take care of the aliases (create different response
> >objs for the same cmd class)
> >- Fix apidocs XmlWriter class to take of the aliases and the api html
> >generator to process the same.
> >- Fix Marvin's codegenerator.py to take care of the aliases (which means
> >create apiname related modules which would contain cmd and response
> >boilterplate fields/class)
> >
> >Cheers.
> >
> >
> >>
> >> Thanks,
> >> -John
> >>
> >> On Apr 8, 2013, at 9:22 AM, Rohit Yadav <bh...@apache.org> wrote:
> >>
> >> > On Mon, Apr 8, 2013 at 6:33 PM, Kishan Kavala
> >><Kishan.Kavala@citrix.com
> >> >wrote:
> >> >
> >> >> APICommand annotation in API Cmd object has a name parameter.
> >>Currently
> >> >> name parameter takes only one value. I plan to enhance this to
> >>support
> >> >> comma separated values. This will allow multiple API names for the
> >>same
> >> API
> >> >> Cmd object.
> >> >>
> >> >> Current:
> >> >> @APICommand(name = "apiName1", ..
> >> >>
> >> >> Proposed:
> >> >> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
> >> >>
> >> >
> >> > It's a hack, but doable. While not elegant as John suggests, parsing
> >> comma
> >> > separated value can lead to some issues.
> >> >
> >> > Changing the name field to array would require a lot of rewriting the
> >> > present apis annotation name fields, again doable by a small python
> >> program
> >> > (like the one I used in the last name field refactoring).
> >> >
> >> > This would really help remove duplicate cmd classes for apis such as
> >>copy
> >> > iso and volume which does the same job but requires two different cmd
> >> class
> >> > implementations.
> >> >
> >> > Whatever way you decide, pl. make sure to fix ApiDiscovery, ApiServer
> >> > cmdclass-apiname map generator method and apidocs generation
> >>accordingly.
> >> >
> >> > Cheers.
> >> >
> >> >
> >> >>
> >> >> Requirement:
> >> >> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList
> >>(grouping
> >> of
> >> >> NetworkACLItems).  Current APIs use *NetworkACL (create
> >> >> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related APIs.
> >> These
> >> >> APIs have to be changed to *NetworkACL Item(create
> >> >> NetworkACLItem/deleteNetworkACLItem etc..) to get the terminology
> >> right. We
> >> >> also need to support old API names for backward compatibility. Hence
> >>the
> >> >> need for API name alias.
> >> >>
> >> >> Terminology:
> >> >> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
> >> >> NetworkACL - Group of Network ACL Items. API will use the term
> >> >> NetworkACLList to differentiate from the existing NetworkACL APIs.
> >> >>
>
> How about an additional attribute 'preferred' or 'deprecated' ?
>
> @APICommand(name={'Foo', 'Bar'}, preferred='Foo', deprecated='Bar')
>
>

RE: [DIscuss]Storage image store plugin framework refactoring

[Edison Su <Ed...@citrix.com>]

> -----Original Message-----
> From: Min Chen
> Sent: Monday, April 22, 2013 10:15 AM
> To: dev@cloudstack.apache.org
> Cc: Edison Su
> Subject: Re: [DIscuss]Storage image store plugin framework refactoring
> 
> See my comments inline.
> 
> -min
> 
> On 4/16/13 11:41 PM, "Sanjeev Neelarapu" <sa...@citrix.com>
> wrote:
> 
> >Hi,
> >
> >Few more review comments:
> >1.In a fresh installed environment will SSVM continue to be there? If
> >yes will it be auto launched or admin should launch it?
> [Min] ssvm will still be there and auto-launched as before.
> >2.How do we seed system vm templates in case of object storage?
> [Min] We will trigger system vm template download while adding an S3
> object storage into cloudstack. The download is happening through MS host.
> >3.Is there any difference in seeding system vm templates with NFS
> >storage compared to what we do as of today?
> [Min] We are not planning to change system vm templates seeding with NFS
> storage at this point, still assume that you have pre-seeded system vm
> template on NFS.
> >4.In upgraded environment, can we have both NFS and object storage? If
> >not what is the procedure to migrate from nfs to object storage ?
> [Min] In our FS, we have made an assumption that "you can add multiple
> image stores from the same provider, but we prevent you from adding
> multiple image stores from different providers." This implied that we cannot
> have both NFS and S3 co-existing for this release. You raised a very good
> point, honestly I don't have an answer at this point. Edison, any thoughts on
> this?

The possible migration solution is that, treat existing NFS secondary storage as cache storage, then sync all the templates/snapshots on NFS to S3/swift object storage.

> >
> >Thanks,
> >Sanjeev
> >
> >-----Original Message-----
> >From: Min Chen [mailto:min.chen@citrix.com]
> >Sent: Wednesday, April 10, 2013 3:23 AM
> >To: dev@cloudstack.apache.org
> >Subject: Re: [DIscuss]Storage image store plugin framework refactoring
> >
> >Hi Sangeetha,
> >
> >	See my answers inline.
> >
> >	Thanks
> >	-min
> >
> >On 4/9/13 2:48 PM, "Sangeetha Hariharan"
> ><Sa...@citrix.com>
> >wrote:
> >
> >>Can image stores from different providers (NFS,SWIFT,S3) coexist in
> >>the same deployment ?
> >>Is the scope for NFS always Zone-wide and SWIFT and S3 region-wide?
> >[Min] Currently this is our assumption: you can add multiple image
> >stores from the same provider, but we prevent you from adding multiple
> >image stores from different providers. For NFS image store providers,
> >it is always zone-wide. And For S3/Swift, it is always region-wide.
> >>
> >>There is an api for "enableImageStoreCmd" . Can multiple image stores
> >>be enabled at the same time?
> >>I don't see any api for disabling image store? What does it mean to
> >>disable an already enabled image store? Will all the resources like
> >>templates, iso and snapshot residing in this image store not be
> >>available to the users anymore?
> >[Min] On second thought, we decide to drop this api. addImageStore api
> >will automatically make the image store added active to be consistent
> >with current secondary storage behavior.
> >>
> >>
> >>deleteImageStoreCmd - Will we allow for deletion of an image store
> >>when it has resources like  templates, iso and snapshots residing in it?
> >[Min] Deletion of an image store will check if it has resources like
> >template, snapshots, and volumes. If there are resources residing in
> >it, we will throw error.
> >>
> >>-Thanks
> >>Sangeetha
> >>
> >>-----Original Message-----
> >>From: Edison Su [mailto:Edison.su@citrix.com]
> >>Sent: Tuesday, April 09, 2013 2:01 PM
> >>To: dev@cloudstack.apache.org
> >>Subject: RE: [DIscuss]Storage image store plugin framework refactoring
> >>
> >>
> >>
> >>> -----Original Message-----
> >>> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
> >>> Sent: Monday, April 08, 2013 11:33 PM
> >>> To: dev@cloudstack.apache.org
> >>> Subject: Re: [DIscuss]Storage image store plugin framework
> >>> refactoring
> >>>
> >>> We can't deprecate APIs unless we are changing to 5.0 AFAIK the next
> >>> release in plan is 4.2
> >>For back compatibility, these APIs:
> >>addSecondaryStorageCmd,addS3Cmd,addSwiftCmd,listS3Cmd,listSwiftCm
> d,
> >>can still be wired to new code, so this APIs will still work, but we
> >>can mark them as deprecated in the API document.
> >>
> >>>
> >>> How does an upgrade from 4.0/4.1 to 4.2 work with this proposal?
> >>> - what happens to existing SSVMs?
> >>> - if there are multiple SSVMs?
> >>> - current cache-like deployments for S3 and Swift
> >>
> >>The existing deployment model will still work.
> >>There are following combinations currently supported by CloudStack:
> >>1. Only NFS as secondary storage. In the new framework, it's NFS as a
> >>backup storage.
> >>2. NFS + swift/S3. In the new framework, it's swift/s3 as backup
> >>storage, while, NFS as cache storage.
> >>So the upgrade code from pre-4.2 to 4.2 needs to handle above cases,
> >>migrate DB properly.
> >>
> >>>
> >>> The SSVM code also does duty for VMWare deployments to aid in data
> >>> movement.
> >>
> >>
> >>The existing SSVM will still work, it will be renamed to
> >>transportation VM in the code(maybe on the UI, it still be called
> >>SSVM). The main functionality of these VMs are to help transfer data
> >>from one place to another.  As you said, Vmware deployment needs these
> >>VMs(as Vmware can't directly download template into primary storage),
> >>and old NFS based secondary storage deployment model needs
> them(data
> >>copy when cross zone).
> >>
> >>
> >>> How does this change?
> >>>
> >>> On 4/8/13 6:34 PM, "Sangeetha Hariharan"
> >>> <Sa...@citrix.com>
> >>> wrote:
> >>>
> >>> >Min,
> >>> >
> >>> >Could you also include the details of the API changes (new
> >>> >parameters) that will be proposed as part of this feature?
> >>> >Also it would be helpful if you list the request and response
> >>> >parameters for the new API calls.
> >>> >For all the API calls that are being deprecated , is there any
> >>> >specific error message that will be returned?
> >>> >
> >>> >-Thanks
> >>> >Sangeetha
> >>> >
> >>> >-----Original Message-----
> >>> >From: Min Chen [mailto:min.chen@citrix.com]
> >>> >Sent: Monday, April 08, 2013 4:45 PM
> >>> >To: dev@cloudstack.apache.org
> >>> >Subject: [DIscuss]Storage image store plugin framework refactoring
> >>> >
> >>> >Hi All,
> >>> >
> >>> >Currently CloudStack does not offer a flexible pluggable framework
> >>> >for users to easily integrate and configure any 3rd-party object
> >>> >stores for such backup services as registering templates, taking
> >>>snapshots, etc.
> >>> >Along with Edison's recent refactored storage subsystem 2.0 that
> >>> >mainly refactored current CloudStack primary storage
> >>> >implementation, we are proposing to develop a storage backup object
> >>> >store plugin framework to allow CloudStack to systematically manage
> >>> >and configure various types of backup data stores from different
> >>> >vendors, like NFS,
> >>>S3, Swift, etc.
> >>> >With this new plugin framework, we would like to achieve following
> >>> >functionalities:
> >>> >1. Support different object store providers in a uniform and
> >>> >pluggable fashion.
> >>> >2. Enable region wide object backup using S3-like object store.
> >>> >3. Provide pluggable data motion strategies to handle data transfer
> >>> >from one data store to another data store.
> >>> >4. Provide a scalable cache storage framework while moving data
> >>> >between primary storage and backup storage for certain hypervisor
> >>>needs.
> >>> >5. Support flexible combinations of primary storage, secondary
> >>> >storage and hypervisors, such as (NFS, NFS, Xen), (NF3, S3,
> >>> >Vmware), (ISCSI, Swift, KVM), ...., etc.
> >>> >The proposed ImageStore plugin framework architecture is detailed
> >>> >in our FS here:
> >>> >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+B
> ack
> >>> >u
> >>> p+O
> >>> >bje
> >>> >ct+Store+Plugin+Framework.
> >>> >The JIRA ticket to track this feature is:
> >>> >https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is
> >>> >currently carried out in feature branch  "object_store".
> >>> >Please let me know your comments and suggestions.
> >>> >
> >>> >Thanks
> >>> >-min
> >>> >
> >>> >
> >>
> >

Re: [DIscuss]Storage image store plugin framework refactoring

[Min Chen <mi...@citrix.com>]

See my comments inline.

-min

On 4/16/13 11:41 PM, "Sanjeev Neelarapu" <sa...@citrix.com>
wrote:

>Hi,
>
>Few more review comments:
>1.In a fresh installed environment will SSVM continue to be there? If yes
>will it be auto launched or admin should launch it?
[Min] ssvm will still be there and auto-launched as before.
>2.How do we seed system vm templates in case of object storage?
[Min] We will trigger system vm template download while adding an S3
object storage into cloudstack. The download is happening through MS host.
>3.Is there any difference in seeding system vm templates with NFS storage
>compared to what we do as of today?
[Min] We are not planning to change system vm templates seeding with NFS
storage at this point, still assume that you have pre-seeded system vm
template on NFS.
>4.In upgraded environment, can we have both NFS and object storage? If
>not what is the procedure to migrate from nfs to object storage ?
[Min] In our FS, we have made an assumption that "you can add multiple
image stores from the same provider, but we prevent you from adding
multiple image
stores from different providers." This implied that we cannot have both
NFS and S3 co-existing for this release. You raised a very good point,
honestly I don't have an answer at this point. Edison, any thoughts on
this?
>
>Thanks,
>Sanjeev
>
>-----Original Message-----
>From: Min Chen [mailto:min.chen@citrix.com]
>Sent: Wednesday, April 10, 2013 3:23 AM
>To: dev@cloudstack.apache.org
>Subject: Re: [DIscuss]Storage image store plugin framework refactoring
>
>Hi Sangeetha, 
>
>	See my answers inline.
>
>	Thanks
>	-min
>
>On 4/9/13 2:48 PM, "Sangeetha Hariharan" <Sa...@citrix.com>
>wrote:
>
>>Can image stores from different providers (NFS,SWIFT,S3) coexist in the
>>same deployment ?
>>Is the scope for NFS always Zone-wide and SWIFT and S3 region-wide?
>[Min] Currently this is our assumption: you can add multiple image stores
>from the same provider, but we prevent you from adding multiple image
>stores from different providers. For NFS image store providers, it is
>always zone-wide. And For S3/Swift, it is always region-wide.
>>
>>There is an api for "enableImageStoreCmd" . Can multiple image stores
>>be enabled at the same time?
>>I don't see any api for disabling image store? What does it mean to
>>disable an already enabled image store? Will all the resources like
>>templates, iso and snapshot residing in this image store not be
>>available to the users anymore?
>[Min] On second thought, we decide to drop this api. addImageStore api
>will automatically make the image store added active to be consistent
>with current secondary storage behavior.
>> 
>>
>>deleteImageStoreCmd - Will we allow for deletion of an image store when
>>it has resources like  templates, iso and snapshots residing in it?
>[Min] Deletion of an image store will check if it has resources like
>template, snapshots, and volumes. If there are resources residing in it,
>we will throw error.
>>
>>-Thanks
>>Sangeetha
>>
>>-----Original Message-----
>>From: Edison Su [mailto:Edison.su@citrix.com]
>>Sent: Tuesday, April 09, 2013 2:01 PM
>>To: dev@cloudstack.apache.org
>>Subject: RE: [DIscuss]Storage image store plugin framework refactoring
>>
>>
>>
>>> -----Original Message-----
>>> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
>>> Sent: Monday, April 08, 2013 11:33 PM
>>> To: dev@cloudstack.apache.org
>>> Subject: Re: [DIscuss]Storage image store plugin framework
>>> refactoring
>>> 
>>> We can't deprecate APIs unless we are changing to 5.0 AFAIK the next
>>> release in plan is 4.2
>>For back compatibility, these APIs:
>>addSecondaryStorageCmd,addS3Cmd,addSwiftCmd,listS3Cmd,listSwiftCmd, can
>>still be wired to new code, so this APIs will still work, but we can
>>mark them as deprecated in the API document.
>>
>>> 
>>> How does an upgrade from 4.0/4.1 to 4.2 work with this proposal?
>>> - what happens to existing SSVMs?
>>> - if there are multiple SSVMs?
>>> - current cache-like deployments for S3 and Swift
>>
>>The existing deployment model will still work.
>>There are following combinations currently supported by CloudStack:
>>1. Only NFS as secondary storage. In the new framework, it's NFS as a
>>backup storage.
>>2. NFS + swift/S3. In the new framework, it's swift/s3 as backup
>>storage, while, NFS as cache storage.
>>So the upgrade code from pre-4.2 to 4.2 needs to handle above cases,
>>migrate DB properly.
>>
>>> 
>>> The SSVM code also does duty for VMWare deployments to aid in data
>>> movement.
>>
>>
>>The existing SSVM will still work, it will be renamed to transportation
>>VM in the code(maybe on the UI, it still be called SSVM). The main
>>functionality of these VMs are to help transfer data from one place to
>>another.  As you said, Vmware deployment needs these VMs(as Vmware
>>can't directly download template into primary storage), and old NFS
>>based secondary storage deployment model needs them(data copy when cross
>>zone).
>>
>>
>>> How does this change?
>>> 
>>> On 4/8/13 6:34 PM, "Sangeetha Hariharan"
>>> <Sa...@citrix.com>
>>> wrote:
>>> 
>>> >Min,
>>> >
>>> >Could you also include the details of the API changes (new
>>> >parameters) that will be proposed as part of this feature?
>>> >Also it would be helpful if you list the request and response
>>> >parameters for the new API calls.
>>> >For all the API calls that are being deprecated , is there any
>>> >specific error message that will be returned?
>>> >
>>> >-Thanks
>>> >Sangeetha
>>> >
>>> >-----Original Message-----
>>> >From: Min Chen [mailto:min.chen@citrix.com]
>>> >Sent: Monday, April 08, 2013 4:45 PM
>>> >To: dev@cloudstack.apache.org
>>> >Subject: [DIscuss]Storage image store plugin framework refactoring
>>> >
>>> >Hi All,
>>> >
>>> >Currently CloudStack does not offer a flexible pluggable framework
>>> >for users to easily integrate and configure any 3rd-party object
>>> >stores for such backup services as registering templates, taking
>>>snapshots, etc.
>>> >Along with Edison's recent refactored storage subsystem 2.0 that
>>> >mainly refactored current CloudStack primary storage implementation,
>>> >we are proposing to develop a storage backup object store plugin
>>> >framework to allow CloudStack to systematically manage and configure
>>> >various types of backup data stores from different vendors, like
>>> >NFS,
>>>S3, Swift, etc.
>>> >With this new plugin framework, we would like to achieve following
>>> >functionalities:
>>> >1. Support different object store providers in a uniform and
>>> >pluggable fashion.
>>> >2. Enable region wide object backup using S3-like object store.
>>> >3. Provide pluggable data motion strategies to handle data transfer
>>> >from one data store to another data store.
>>> >4. Provide a scalable cache storage framework while moving data
>>> >between primary storage and backup storage for certain hypervisor
>>>needs.
>>> >5. Support flexible combinations of primary storage, secondary
>>> >storage and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware),
>>> >(ISCSI, Swift, KVM), ...., etc.
>>> >The proposed ImageStore plugin framework architecture is detailed in
>>> >our FS here:
>>> >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backu
>>> p+O
>>> >bje
>>> >ct+Store+Plugin+Framework.
>>> >The JIRA ticket to track this feature is:
>>> >https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is
>>> >currently carried out in feature branch  "object_store".
>>> >Please let me know your comments and suggestions.
>>> >
>>> >Thanks
>>> >-min
>>> >
>>> >
>>
>

RE: [DIscuss]Storage image store plugin framework refactoring

[Sanjeev Neelarapu <sa...@citrix.com>]

Hi,

Few more review comments:
1.In a fresh installed environment will SSVM continue to be there? If yes will it be auto launched or admin should launch it?
2.How do we seed system vm templates in case of object storage?
3.Is there any difference in seeding system vm templates with NFS storage compared to what we do as of today?
4.In upgraded environment, can we have both NFS and object storage? If not what is the procedure to migrate from nfs to object storage ?

Thanks,
Sanjeev

-----Original Message-----
From: Min Chen [mailto:min.chen@citrix.com] 
Sent: Wednesday, April 10, 2013 3:23 AM
To: dev@cloudstack.apache.org
Subject: Re: [DIscuss]Storage image store plugin framework refactoring

Hi Sangeetha, 

	See my answers inline.

	Thanks
	-min

On 4/9/13 2:48 PM, "Sangeetha Hariharan" <Sa...@citrix.com>
wrote:

>Can image stores from different providers (NFS,SWIFT,S3) coexist in the 
>same deployment ?
>Is the scope for NFS always Zone-wide and SWIFT and S3 region-wide?
[Min] Currently this is our assumption: you can add multiple image stores from the same provider, but we prevent you from adding multiple image stores from different providers. For NFS image store providers, it is always zone-wide. And For S3/Swift, it is always region-wide.
>
>There is an api for "enableImageStoreCmd" . Can multiple image stores 
>be enabled at the same time?
>I don't see any api for disabling image store? What does it mean to 
>disable an already enabled image store? Will all the resources like 
>templates, iso and snapshot residing in this image store not be 
>available to the users anymore?
[Min] On second thought, we decide to drop this api. addImageStore api will automatically make the image store added active to be consistent with current secondary storage behavior.
> 
>
>deleteImageStoreCmd - Will we allow for deletion of an image store when 
>it has resources like  templates, iso and snapshots residing in it?
[Min] Deletion of an image store will check if it has resources like template, snapshots, and volumes. If there are resources residing in it, we will throw error.
>
>-Thanks
>Sangeetha
>
>-----Original Message-----
>From: Edison Su [mailto:Edison.su@citrix.com]
>Sent: Tuesday, April 09, 2013 2:01 PM
>To: dev@cloudstack.apache.org
>Subject: RE: [DIscuss]Storage image store plugin framework refactoring
>
>
>
>> -----Original Message-----
>> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
>> Sent: Monday, April 08, 2013 11:33 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [DIscuss]Storage image store plugin framework 
>> refactoring
>> 
>> We can't deprecate APIs unless we are changing to 5.0 AFAIK the next 
>> release in plan is 4.2
>For back compatibility, these APIs:
>addSecondaryStorageCmd,addS3Cmd,addSwiftCmd,listS3Cmd,listSwiftCmd, can 
>still be wired to new code, so this APIs will still work, but we can 
>mark them as deprecated in the API document.
>
>> 
>> How does an upgrade from 4.0/4.1 to 4.2 work with this proposal?
>> - what happens to existing SSVMs?
>> - if there are multiple SSVMs?
>> - current cache-like deployments for S3 and Swift
>
>The existing deployment model will still work.
>There are following combinations currently supported by CloudStack:
>1. Only NFS as secondary storage. In the new framework, it's NFS as a 
>backup storage.
>2. NFS + swift/S3. In the new framework, it's swift/s3 as backup 
>storage, while, NFS as cache storage.
>So the upgrade code from pre-4.2 to 4.2 needs to handle above cases, 
>migrate DB properly.
>
>> 
>> The SSVM code also does duty for VMWare deployments to aid in data 
>> movement.
>
>
>The existing SSVM will still work, it will be renamed to transportation 
>VM in the code(maybe on the UI, it still be called SSVM). The main 
>functionality of these VMs are to help transfer data from one place to 
>another.  As you said, Vmware deployment needs these VMs(as Vmware 
>can't directly download template into primary storage), and old NFS 
>based secondary storage deployment model needs them(data copy when cross zone).
>
>
>> How does this change?
>> 
>> On 4/8/13 6:34 PM, "Sangeetha Hariharan"
>> <Sa...@citrix.com>
>> wrote:
>> 
>> >Min,
>> >
>> >Could you also include the details of the API changes (new
>> >parameters) that will be proposed as part of this feature?
>> >Also it would be helpful if you list the request and response 
>> >parameters for the new API calls.
>> >For all the API calls that are being deprecated , is there any 
>> >specific error message that will be returned?
>> >
>> >-Thanks
>> >Sangeetha
>> >
>> >-----Original Message-----
>> >From: Min Chen [mailto:min.chen@citrix.com]
>> >Sent: Monday, April 08, 2013 4:45 PM
>> >To: dev@cloudstack.apache.org
>> >Subject: [DIscuss]Storage image store plugin framework refactoring
>> >
>> >Hi All,
>> >
>> >Currently CloudStack does not offer a flexible pluggable framework 
>> >for users to easily integrate and configure any 3rd-party object 
>> >stores for such backup services as registering templates, taking
>>snapshots, etc.
>> >Along with Edison's recent refactored storage subsystem 2.0 that 
>> >mainly refactored current CloudStack primary storage implementation, 
>> >we are proposing to develop a storage backup object store plugin 
>> >framework to allow CloudStack to systematically manage and configure 
>> >various types of backup data stores from different vendors, like 
>> >NFS,
>>S3, Swift, etc.
>> >With this new plugin framework, we would like to achieve following
>> >functionalities:
>> >1. Support different object store providers in a uniform and 
>> >pluggable fashion.
>> >2. Enable region wide object backup using S3-like object store.
>> >3. Provide pluggable data motion strategies to handle data transfer 
>> >from one data store to another data store.
>> >4. Provide a scalable cache storage framework while moving data 
>> >between primary storage and backup storage for certain hypervisor
>>needs.
>> >5. Support flexible combinations of primary storage, secondary 
>> >storage and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware), 
>> >(ISCSI, Swift, KVM), ...., etc.
>> >The proposed ImageStore plugin framework architecture is detailed in 
>> >our FS here:
>> >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backu
>> p+O
>> >bje
>> >ct+Store+Plugin+Framework.
>> >The JIRA ticket to track this feature is:
>> >https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is 
>> >currently carried out in feature branch  "object_store".
>> >Please let me know your comments and suggestions.
>> >
>> >Thanks
>> >-min
>> >
>> >
>

Re: [DIscuss]Storage image store plugin framework refactoring

[Min Chen <mi...@citrix.com>]

Hi Sangeetha, 

	See my answers inline.

	Thanks
	-min

On 4/9/13 2:48 PM, "Sangeetha Hariharan" <Sa...@citrix.com>
wrote:

>Can image stores from different providers (NFS,SWIFT,S3) coexist in the
>same deployment ?
>Is the scope for NFS always Zone-wide and SWIFT and S3 region-wide?
[Min] Currently this is our assumption: you can add multiple image stores
from the same provider, but we prevent you from adding multiple image
stores from different providers. For NFS image store providers, it is
always zone-wide. And For S3/Swift, it is always region-wide.
>
>There is an api for "enableImageStoreCmd" . Can multiple image stores be
>enabled at the same time?
>I don't see any api for disabling image store? What does it mean to
>disable an already enabled image store? Will all the resources like
>templates, iso and snapshot residing in this image store not be available
>to the users anymore?
[Min] On second thought, we decide to drop this api. addImageStore api
will automatically make the image store added active to be consistent with
current secondary storage behavior.
> 
>
>deleteImageStoreCmd - Will we allow for deletion of an image store when
>it has resources like  templates, iso and snapshots residing in it?
[Min] Deletion of an image store will check if it has resources like
template, snapshots, and volumes. If there are resources residing in it,
we will throw error.
>
>-Thanks
>Sangeetha 
>
>-----Original Message-----
>From: Edison Su [mailto:Edison.su@citrix.com]
>Sent: Tuesday, April 09, 2013 2:01 PM
>To: dev@cloudstack.apache.org
>Subject: RE: [DIscuss]Storage image store plugin framework refactoring
>
>
>
>> -----Original Message-----
>> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
>> Sent: Monday, April 08, 2013 11:33 PM
>> To: dev@cloudstack.apache.org
>> Subject: Re: [DIscuss]Storage image store plugin framework refactoring
>> 
>> We can't deprecate APIs unless we are changing to 5.0 AFAIK the next
>> release in plan is 4.2
>For back compatibility, these APIs:
>addSecondaryStorageCmd,addS3Cmd,addSwiftCmd,listS3Cmd,listSwiftCmd, can
>still be wired to new code, so this APIs will still work, but we can mark
>them as deprecated in the API document.
>
>> 
>> How does an upgrade from 4.0/4.1 to 4.2 work with this proposal?
>> - what happens to existing SSVMs?
>> - if there are multiple SSVMs?
>> - current cache-like deployments for S3 and Swift
>
>The existing deployment model will still work.
>There are following combinations currently supported by CloudStack:
>1. Only NFS as secondary storage. In the new framework, it's NFS as a
>backup storage.
>2. NFS + swift/S3. In the new framework, it's swift/s3 as backup storage,
>while, NFS as cache storage.
>So the upgrade code from pre-4.2 to 4.2 needs to handle above cases,
>migrate DB properly.
>
>> 
>> The SSVM code also does duty for VMWare deployments to aid in data
>> movement.
>
>
>The existing SSVM will still work, it will be renamed to transportation
>VM in the code(maybe on the UI, it still be called SSVM). The main
>functionality of these VMs are to help transfer data from one place to
>another.  As you said, Vmware deployment needs these VMs(as Vmware can't
>directly download template into primary storage), and old NFS based
>secondary storage deployment model needs them(data copy when cross zone).
>
>
>> How does this change?
>> 
>> On 4/8/13 6:34 PM, "Sangeetha Hariharan"
>> <Sa...@citrix.com>
>> wrote:
>> 
>> >Min,
>> >
>> >Could you also include the details of the API changes (new
>> >parameters) that will be proposed as part of this feature?
>> >Also it would be helpful if you list the request and response
>> >parameters for the new API calls.
>> >For all the API calls that are being deprecated , is there any
>> >specific error message that will be returned?
>> >
>> >-Thanks
>> >Sangeetha
>> >
>> >-----Original Message-----
>> >From: Min Chen [mailto:min.chen@citrix.com]
>> >Sent: Monday, April 08, 2013 4:45 PM
>> >To: dev@cloudstack.apache.org
>> >Subject: [DIscuss]Storage image store plugin framework refactoring
>> >
>> >Hi All,
>> >
>> >Currently CloudStack does not offer a flexible pluggable framework
>> >for users to easily integrate and configure any 3rd-party object
>> >stores for such backup services as registering templates, taking
>>snapshots, etc.
>> >Along with Edison's recent refactored storage subsystem 2.0 that
>> >mainly refactored current CloudStack primary storage implementation,
>> >we are proposing to develop a storage backup object store plugin
>> >framework to allow CloudStack to systematically manage and configure
>> >various types of backup data stores from different vendors, like NFS,
>>S3, Swift, etc.
>> >With this new plugin framework, we would like to achieve following
>> >functionalities:
>> >1. Support different object store providers in a uniform and
>> >pluggable fashion.
>> >2. Enable region wide object backup using S3-like object store.
>> >3. Provide pluggable data motion strategies to handle data transfer
>> >from one data store to another data store.
>> >4. Provide a scalable cache storage framework while moving data
>> >between primary storage and backup storage for certain hypervisor
>>needs.
>> >5. Support flexible combinations of primary storage, secondary
>> >storage and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware),
>> >(ISCSI, Swift, KVM), ...., etc.
>> >The proposed ImageStore plugin framework architecture is detailed in
>> >our FS here:
>> >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backu
>> p+O
>> >bje
>> >ct+Store+Plugin+Framework.
>> >The JIRA ticket to track this feature is:
>> >https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is
>> >currently carried out in feature branch  "object_store".
>> >Please let me know your comments and suggestions.
>> >
>> >Thanks
>> >-min
>> >
>> >
>

RE: [DIscuss]Storage image store plugin framework refactoring

[Sangeetha Hariharan <Sa...@citrix.com>]

Can image stores from different providers (NFS,SWIFT,S3) coexist in the same deployment ?
Is the scope for NFS always Zone-wide and SWIFT and S3 region-wide?

There is an api for "enableImageStoreCmd" . Can multiple image stores be enabled at the same time?
I don't see any api for disabling image store? What does it mean to disable an already enabled image store? Will all the resources like templates, iso and snapshot residing in this image store not be available to the users anymore? 

deleteImageStoreCmd - Will we allow for deletion of an image store when it has resources like  templates, iso and snapshots residing in it?

-Thanks
Sangeetha 

-----Original Message-----
From: Edison Su [mailto:Edison.su@citrix.com] 
Sent: Tuesday, April 09, 2013 2:01 PM
To: dev@cloudstack.apache.org
Subject: RE: [DIscuss]Storage image store plugin framework refactoring



> -----Original Message-----
> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
> Sent: Monday, April 08, 2013 11:33 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DIscuss]Storage image store plugin framework refactoring
> 
> We can't deprecate APIs unless we are changing to 5.0 AFAIK the next 
> release in plan is 4.2
For back compatibility, these APIs: addSecondaryStorageCmd,addS3Cmd,addSwiftCmd,listS3Cmd,listSwiftCmd, can still be wired to new code, so this APIs will still work, but we can mark them as deprecated in the API document.

> 
> How does an upgrade from 4.0/4.1 to 4.2 work with this proposal?
> - what happens to existing SSVMs?
> - if there are multiple SSVMs?
> - current cache-like deployments for S3 and Swift

The existing deployment model will still work.
There are following combinations currently supported by CloudStack:
1. Only NFS as secondary storage. In the new framework, it's NFS as a backup storage.
2. NFS + swift/S3. In the new framework, it's swift/s3 as backup storage, while, NFS as cache storage.
So the upgrade code from pre-4.2 to 4.2 needs to handle above cases, migrate DB properly.

> 
> The SSVM code also does duty for VMWare deployments to aid in data 
> movement.


The existing SSVM will still work, it will be renamed to transportation VM in the code(maybe on the UI, it still be called SSVM). The main functionality of these VMs are to help transfer data from one place to another.  As you said, Vmware deployment needs these VMs(as Vmware can't directly download template into primary storage), and old NFS based secondary storage deployment model needs them(data copy when cross zone).


> How does this change?
> 
> On 4/8/13 6:34 PM, "Sangeetha Hariharan" 
> <Sa...@citrix.com>
> wrote:
> 
> >Min,
> >
> >Could you also include the details of the API changes (new 
> >parameters) that will be proposed as part of this feature?
> >Also it would be helpful if you list the request and response 
> >parameters for the new API calls.
> >For all the API calls that are being deprecated , is there any 
> >specific error message that will be returned?
> >
> >-Thanks
> >Sangeetha
> >
> >-----Original Message-----
> >From: Min Chen [mailto:min.chen@citrix.com]
> >Sent: Monday, April 08, 2013 4:45 PM
> >To: dev@cloudstack.apache.org
> >Subject: [DIscuss]Storage image store plugin framework refactoring
> >
> >Hi All,
> >
> >Currently CloudStack does not offer a flexible pluggable framework 
> >for users to easily integrate and configure any 3rd-party object 
> >stores for such backup services as registering templates, taking snapshots, etc.
> >Along with Edison's recent refactored storage subsystem 2.0 that 
> >mainly refactored current CloudStack primary storage implementation,  
> >we are proposing to develop a storage backup object store plugin 
> >framework to allow CloudStack to systematically manage and configure 
> >various types of backup data stores from different vendors, like NFS, S3, Swift, etc.
> >With this new plugin framework, we would like to achieve following
> >functionalities:
> >1. Support different object store providers in a uniform and 
> >pluggable fashion.
> >2. Enable region wide object backup using S3-like object store.
> >3. Provide pluggable data motion strategies to handle data transfer 
> >from one data store to another data store.
> >4. Provide a scalable cache storage framework while moving data 
> >between primary storage and backup storage for certain hypervisor needs.
> >5. Support flexible combinations of primary storage, secondary 
> >storage and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware), 
> >(ISCSI, Swift, KVM), ...., etc.
> >The proposed ImageStore plugin framework architecture is detailed in 
> >our FS here:
> >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backu
> p+O
> >bje
> >ct+Store+Plugin+Framework.
> >The JIRA ticket to track this feature is:
> >https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is 
> >currently carried out in feature branch  "object_store".
> >Please let me know your comments and suggestions.
> >
> >Thanks
> >-min
> >
> >

RE: [DIscuss]Storage image store plugin framework refactoring

[Edison Su <Ed...@citrix.com>]

> -----Original Message-----
> From: Chiradeep Vittal [mailto:Chiradeep.Vittal@citrix.com]
> Sent: Monday, April 08, 2013 11:33 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DIscuss]Storage image store plugin framework refactoring
> 
> We can't deprecate APIs unless we are changing to 5.0 AFAIK the next
> release in plan is 4.2
For back compatibility, these APIs: addSecondaryStorageCmd,addS3Cmd,addSwiftCmd,listS3Cmd,listSwiftCmd, can still be wired to new code, so this APIs will still work, but we can mark them as deprecated in the API document.

> 
> How does an upgrade from 4.0/4.1 to 4.2 work with this proposal?
> - what happens to existing SSVMs?
> - if there are multiple SSVMs?
> - current cache-like deployments for S3 and Swift

The existing deployment model will still work.
There are following combinations currently supported by CloudStack:
1. Only NFS as secondary storage. In the new framework, it's NFS as a backup storage.
2. NFS + swift/S3. In the new framework, it's swift/s3 as backup storage, while, NFS as cache storage.
So the upgrade code from pre-4.2 to 4.2 needs to handle above cases, migrate DB properly.

> 
> The SSVM code also does duty for VMWare deployments to aid in data
> movement.


The existing SSVM will still work, it will be renamed to transportation VM in the code(maybe on the UI, it still be called SSVM). The main functionality of these VMs are to help transfer data from one place to another.  As you said, Vmware deployment needs these VMs(as Vmware can't directly download template into primary storage), and old NFS based secondary storage deployment model needs them(data copy when cross zone).


> How does this change?
> 
> On 4/8/13 6:34 PM, "Sangeetha Hariharan" <Sa...@citrix.com>
> wrote:
> 
> >Min,
> >
> >Could you also include the details of the API changes (new parameters)
> >that will be proposed as part of this feature?
> >Also it would be helpful if you list the request and response
> >parameters for the new API calls.
> >For all the API calls that are being deprecated , is there any specific
> >error message that will be returned?
> >
> >-Thanks
> >Sangeetha
> >
> >-----Original Message-----
> >From: Min Chen [mailto:min.chen@citrix.com]
> >Sent: Monday, April 08, 2013 4:45 PM
> >To: dev@cloudstack.apache.org
> >Subject: [DIscuss]Storage image store plugin framework refactoring
> >
> >Hi All,
> >
> >Currently CloudStack does not offer a flexible pluggable framework for
> >users to easily integrate and configure any 3rd-party object stores for
> >such backup services as registering templates, taking snapshots, etc.
> >Along with Edison's recent refactored storage subsystem 2.0 that mainly
> >refactored current CloudStack primary storage implementation,  we are
> >proposing to develop a storage backup object store plugin framework to
> >allow CloudStack to systematically manage and configure various types
> >of backup data stores from different vendors, like NFS, S3, Swift, etc.
> >With this new plugin framework, we would like to achieve following
> >functionalities:
> >1. Support different object store providers in a uniform and pluggable
> >fashion.
> >2. Enable region wide object backup using S3-like object store.
> >3. Provide pluggable data motion strategies to handle data transfer
> >from one data store to another data store.
> >4. Provide a scalable cache storage framework while moving data between
> >primary storage and backup storage for certain hypervisor needs.
> >5. Support flexible combinations of primary storage, secondary storage
> >and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware), (ISCSI,
> >Swift, KVM), ...., etc.
> >The proposed ImageStore plugin framework architecture is detailed in
> >our FS here:
> >https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backu
> p+O
> >bje
> >ct+Store+Plugin+Framework.
> >The JIRA ticket to track this feature is:
> >https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is
> >currently carried out in feature branch  "object_store".
> >Please let me know your comments and suggestions.
> >
> >Thanks
> >-min
> >
> >

Re: [DIscuss]Storage image store plugin framework refactoring

[Chiradeep Vittal <Ch...@citrix.com>]

We can't deprecate APIs unless we are changing to 5.0
AFAIK the next release in plan is 4.2

How does an upgrade from 4.0/4.1 to 4.2 work with this proposal?
- what happens to existing SSVMs?
- if there are multiple SSVMs?
- current cache-like deployments for S3 and Swift

The SSVM code also does duty for VMWare deployments to aid in data
movement.
How does this change?

On 4/8/13 6:34 PM, "Sangeetha Hariharan" <Sa...@citrix.com>
wrote:

>Min,
>
>Could you also include the details of the API changes (new parameters)
>that will be proposed as part of this feature?
>Also it would be helpful if you list the request and response parameters
>for the new API calls.
>For all the API calls that are being deprecated , is there any specific
>error message that will be returned?
>
>-Thanks
>Sangeetha 
>
>-----Original Message-----
>From: Min Chen [mailto:min.chen@citrix.com]
>Sent: Monday, April 08, 2013 4:45 PM
>To: dev@cloudstack.apache.org
>Subject: [DIscuss]Storage image store plugin framework refactoring
>
>Hi All,
>
>Currently CloudStack does not offer a flexible pluggable framework for
>users to easily integrate and configure any 3rd-party object stores for
>such backup services as registering templates, taking snapshots, etc.
>Along with Edison's recent refactored storage subsystem 2.0 that mainly
>refactored current CloudStack primary storage implementation,  we are
>proposing to develop a storage backup object store plugin framework to
>allow CloudStack to systematically manage and configure various types of
>backup data stores from different vendors, like NFS, S3, Swift, etc. With
>this new plugin framework, we would like to achieve following
>functionalities:
>1. Support different object store providers in a uniform and pluggable
>fashion.
>2. Enable region wide object backup using S3-like object store.
>3. Provide pluggable data motion strategies to handle data transfer from
>one data store to another data store.
>4. Provide a scalable cache storage framework while moving data between
>primary storage and backup storage for certain hypervisor needs.
>5. Support flexible combinations of primary storage, secondary storage
>and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware), (ISCSI,
>Swift, KVM), ...., etc.
>The proposed ImageStore plugin framework architecture is detailed in our
>FS here: 
>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backup+Obje
>ct+Store+Plugin+Framework.
>The JIRA ticket to track this feature is:
>https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is
>currently carried out in feature branch  "object_store".
>Please let me know your comments and suggestions.
>
>Thanks
>-min
>
>

Re: [DIscuss]Storage image store plugin framework refactoring

[Nitin Mehta <Ni...@citrix.com>]

On 09/04/13 11:03 PM, "Min Chen" <mi...@citrix.com> wrote:

>It may not be that simple to just use api name alias. For example, we
>currently have listS3sCmd, with new image store plugin framework, we
>introduced a new API "listImageStoresCmd" to cover image stores from
>various providers, not only S3. We also introduced a new response
>ImageStoreResponse for this new listImageStoreCmd api. So you are
>suggesting to use the following naming alias to direct old apis to the new
>apis, right?
>
>@APICommand(name = "listImageStores, listS3s, listSwift", ..
>
>
>But there are two issues here:
>1. Previous listS3sCmd api is corresponding to new API
>listImageStores&provider=S3, so this will not be a simple redirect.
>2. Previous listS3sCmd response is S3Response, which is different from new
>ImageStoreResponse, although its information can be found in new
>ImageStoreResponse. Will this break back compatibility as well?

My $.02
Yes, it will break the backward compatibility. For now keep both the
api's, but the UI should start using the new api with
listImageStores&provider=S3.
We should use deprecated annotation to mark that the api is deprecated
just like in the Java Docs and finally remove the api in 5.0.

>
>Thanks
>-min
>
>
>On 4/9/13 10:14 AM, "Chiradeep Vittal" <Ch...@citrix.com>
>wrote:
>
>>We can have alias for an existing API. See the other ML discussion.
>>
>>On 4/9/13 9:27 AM, "Min Chen" <mi...@citrix.com> wrote:
>>
>>>Yes, I will include more api change details in FS in next few days.
>>>According to Chiradeep, it seems that we cannot simply deprecate old API
>>>in 4.2, Edison and I will discuss this and update FS with details on how
>>>to handle these old APIs.
>>>
>>>Thanks
>>>-min
>>>
>>>On 4/8/13 6:34 PM, "Sangeetha Hariharan"
>>><Sa...@citrix.com>
>>>wrote:
>>>
>>>>Min,
>>>>
>>>>Could you also include the details of the API changes (new parameters)
>>>>that will be proposed as part of this feature?
>>>>Also it would be helpful if you list the request and response
>>>>parameters
>>>>for the new API calls.
>>>>For all the API calls that are being deprecated , is there any specific
>>>>error message that will be returned?
>>>>
>>>>-Thanks
>>>>Sangeetha 
>>>>
>>>>-----Original Message-----
>>>>From: Min Chen [mailto:min.chen@citrix.com]
>>>>Sent: Monday, April 08, 2013 4:45 PM
>>>>To: dev@cloudstack.apache.org
>>>>Subject: [DIscuss]Storage image store plugin framework refactoring
>>>>
>>>>Hi All,
>>>>
>>>>Currently CloudStack does not offer a flexible pluggable framework for
>>>>users to easily integrate and configure any 3rd-party object stores for
>>>>such backup services as registering templates, taking snapshots, etc.
>>>>Along with Edison's recent refactored storage subsystem 2.0 that mainly
>>>>refactored current CloudStack primary storage implementation,  we are
>>>>proposing to develop a storage backup object store plugin framework to
>>>>allow CloudStack to systematically manage and configure various types
>>>>of
>>>>backup data stores from different vendors, like NFS, S3, Swift, etc.
>>>>With
>>>>this new plugin framework, we would like to achieve following
>>>>functionalities:
>>>>1. Support different object store providers in a uniform and pluggable
>>>>fashion.
>>>>2. Enable region wide object backup using S3-like object store.
>>>>3. Provide pluggable data motion strategies to handle data transfer
>>>>from
>>>>one data store to another data store.
>>>>4. Provide a scalable cache storage framework while moving data between
>>>>primary storage and backup storage for certain hypervisor needs.
>>>>5. Support flexible combinations of primary storage, secondary storage
>>>>and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware), (ISCSI,
>>>>Swift, KVM), ...., etc.
>>>>The proposed ImageStore plugin framework architecture is detailed in
>>>>our
>>>>FS here: 
>>>>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backup+O
>>>>b
>>>>j
>>>>e
>>>>ct+Store+Plugin+Framework.
>>>>The JIRA ticket to track this feature is:
>>>>https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is
>>>>currently carried out in feature branch  "object_store".
>>>>Please let me know your comments and suggestions.
>>>>
>>>>Thanks
>>>>-min
>>>>
>>>>
>>>
>>
>

Re: [DIscuss]Storage image store plugin framework refactoring

[Min Chen <mi...@citrix.com>]

It may not be that simple to just use api name alias. For example, we
currently have listS3sCmd, with new image store plugin framework, we
introduced a new API "listImageStoresCmd" to cover image stores from
various providers, not only S3. We also introduced a new response
ImageStoreResponse for this new listImageStoreCmd api. So you are
suggesting to use the following naming alias to direct old apis to the new
apis, right?

@APICommand(name = "listImageStores, listS3s, listSwift", ..


But there are two issues here:
1. Previous listS3sCmd api is corresponding to new API
listImageStores&provider=S3, so this will not be a simple redirect.
2. Previous listS3sCmd response is S3Response, which is different from new
ImageStoreResponse, although its information can be found in new
ImageStoreResponse. Will this break back compatibility as well?

Thanks
-min


On 4/9/13 10:14 AM, "Chiradeep Vittal" <Ch...@citrix.com> wrote:

>We can have alias for an existing API. See the other ML discussion.
>
>On 4/9/13 9:27 AM, "Min Chen" <mi...@citrix.com> wrote:
>
>>Yes, I will include more api change details in FS in next few days.
>>According to Chiradeep, it seems that we cannot simply deprecate old API
>>in 4.2, Edison and I will discuss this and update FS with details on how
>>to handle these old APIs.
>>
>>Thanks
>>-min
>>
>>On 4/8/13 6:34 PM, "Sangeetha Hariharan" <Sa...@citrix.com>
>>wrote:
>>
>>>Min,
>>>
>>>Could you also include the details of the API changes (new parameters)
>>>that will be proposed as part of this feature?
>>>Also it would be helpful if you list the request and response parameters
>>>for the new API calls.
>>>For all the API calls that are being deprecated , is there any specific
>>>error message that will be returned?
>>>
>>>-Thanks
>>>Sangeetha 
>>>
>>>-----Original Message-----
>>>From: Min Chen [mailto:min.chen@citrix.com]
>>>Sent: Monday, April 08, 2013 4:45 PM
>>>To: dev@cloudstack.apache.org
>>>Subject: [DIscuss]Storage image store plugin framework refactoring
>>>
>>>Hi All,
>>>
>>>Currently CloudStack does not offer a flexible pluggable framework for
>>>users to easily integrate and configure any 3rd-party object stores for
>>>such backup services as registering templates, taking snapshots, etc.
>>>Along with Edison's recent refactored storage subsystem 2.0 that mainly
>>>refactored current CloudStack primary storage implementation,  we are
>>>proposing to develop a storage backup object store plugin framework to
>>>allow CloudStack to systematically manage and configure various types of
>>>backup data stores from different vendors, like NFS, S3, Swift, etc.
>>>With
>>>this new plugin framework, we would like to achieve following
>>>functionalities:
>>>1. Support different object store providers in a uniform and pluggable
>>>fashion.
>>>2. Enable region wide object backup using S3-like object store.
>>>3. Provide pluggable data motion strategies to handle data transfer from
>>>one data store to another data store.
>>>4. Provide a scalable cache storage framework while moving data between
>>>primary storage and backup storage for certain hypervisor needs.
>>>5. Support flexible combinations of primary storage, secondary storage
>>>and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware), (ISCSI,
>>>Swift, KVM), ...., etc.
>>>The proposed ImageStore plugin framework architecture is detailed in our
>>>FS here: 
>>>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backup+Ob
>>>j
>>>e
>>>ct+Store+Plugin+Framework.
>>>The JIRA ticket to track this feature is:
>>>https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is
>>>currently carried out in feature branch  "object_store".
>>>Please let me know your comments and suggestions.
>>>
>>>Thanks
>>>-min
>>>
>>>
>>
>

Re: [DIscuss]Storage image store plugin framework refactoring

[Chiradeep Vittal <Ch...@citrix.com>]

We can have alias for an existing API. See the other ML discussion.

On 4/9/13 9:27 AM, "Min Chen" <mi...@citrix.com> wrote:

>Yes, I will include more api change details in FS in next few days.
>According to Chiradeep, it seems that we cannot simply deprecate old API
>in 4.2, Edison and I will discuss this and update FS with details on how
>to handle these old APIs.
>
>Thanks
>-min
>
>On 4/8/13 6:34 PM, "Sangeetha Hariharan" <Sa...@citrix.com>
>wrote:
>
>>Min,
>>
>>Could you also include the details of the API changes (new parameters)
>>that will be proposed as part of this feature?
>>Also it would be helpful if you list the request and response parameters
>>for the new API calls.
>>For all the API calls that are being deprecated , is there any specific
>>error message that will be returned?
>>
>>-Thanks
>>Sangeetha 
>>
>>-----Original Message-----
>>From: Min Chen [mailto:min.chen@citrix.com]
>>Sent: Monday, April 08, 2013 4:45 PM
>>To: dev@cloudstack.apache.org
>>Subject: [DIscuss]Storage image store plugin framework refactoring
>>
>>Hi All,
>>
>>Currently CloudStack does not offer a flexible pluggable framework for
>>users to easily integrate and configure any 3rd-party object stores for
>>such backup services as registering templates, taking snapshots, etc.
>>Along with Edison's recent refactored storage subsystem 2.0 that mainly
>>refactored current CloudStack primary storage implementation,  we are
>>proposing to develop a storage backup object store plugin framework to
>>allow CloudStack to systematically manage and configure various types of
>>backup data stores from different vendors, like NFS, S3, Swift, etc. With
>>this new plugin framework, we would like to achieve following
>>functionalities:
>>1. Support different object store providers in a uniform and pluggable
>>fashion.
>>2. Enable region wide object backup using S3-like object store.
>>3. Provide pluggable data motion strategies to handle data transfer from
>>one data store to another data store.
>>4. Provide a scalable cache storage framework while moving data between
>>primary storage and backup storage for certain hypervisor needs.
>>5. Support flexible combinations of primary storage, secondary storage
>>and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware), (ISCSI,
>>Swift, KVM), ...., etc.
>>The proposed ImageStore plugin framework architecture is detailed in our
>>FS here: 
>>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backup+Obj
>>e
>>ct+Store+Plugin+Framework.
>>The JIRA ticket to track this feature is:
>>https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is
>>currently carried out in feature branch  "object_store".
>>Please let me know your comments and suggestions.
>>
>>Thanks
>>-min
>>
>>
>

Re: [DIscuss]Storage image store plugin framework refactoring

[Min Chen <mi...@citrix.com>]

Yes, I will include more api change details in FS in next few days.
According to Chiradeep, it seems that we cannot simply deprecate old API
in 4.2, Edison and I will discuss this and update FS with details on how
to handle these old APIs.

Thanks
-min

On 4/8/13 6:34 PM, "Sangeetha Hariharan" <Sa...@citrix.com>
wrote:

>Min,
>
>Could you also include the details of the API changes (new parameters)
>that will be proposed as part of this feature?
>Also it would be helpful if you list the request and response parameters
>for the new API calls.
>For all the API calls that are being deprecated , is there any specific
>error message that will be returned?
>
>-Thanks
>Sangeetha 
>
>-----Original Message-----
>From: Min Chen [mailto:min.chen@citrix.com]
>Sent: Monday, April 08, 2013 4:45 PM
>To: dev@cloudstack.apache.org
>Subject: [DIscuss]Storage image store plugin framework refactoring
>
>Hi All,
>
>Currently CloudStack does not offer a flexible pluggable framework for
>users to easily integrate and configure any 3rd-party object stores for
>such backup services as registering templates, taking snapshots, etc.
>Along with Edison's recent refactored storage subsystem 2.0 that mainly
>refactored current CloudStack primary storage implementation,  we are
>proposing to develop a storage backup object store plugin framework to
>allow CloudStack to systematically manage and configure various types of
>backup data stores from different vendors, like NFS, S3, Swift, etc. With
>this new plugin framework, we would like to achieve following
>functionalities:
>1. Support different object store providers in a uniform and pluggable
>fashion.
>2. Enable region wide object backup using S3-like object store.
>3. Provide pluggable data motion strategies to handle data transfer from
>one data store to another data store.
>4. Provide a scalable cache storage framework while moving data between
>primary storage and backup storage for certain hypervisor needs.
>5. Support flexible combinations of primary storage, secondary storage
>and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware), (ISCSI,
>Swift, KVM), ...., etc.
>The proposed ImageStore plugin framework architecture is detailed in our
>FS here: 
>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backup+Obje
>ct+Store+Plugin+Framework.
>The JIRA ticket to track this feature is:
>https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is
>currently carried out in feature branch  "object_store".
>Please let me know your comments and suggestions.
>
>Thanks
>-min
>
>

RE: [DIscuss]Storage image store plugin framework refactoring

[Sangeetha Hariharan <Sa...@citrix.com>]

Min,

Could you also include the details of the API changes (new parameters)  that will be proposed as part of this feature? 
Also it would be helpful if you list the request and response parameters for the new API calls.
For all the API calls that are being deprecated , is there any specific error message that will be returned?

-Thanks
Sangeetha 

-----Original Message-----
From: Min Chen [mailto:min.chen@citrix.com] 
Sent: Monday, April 08, 2013 4:45 PM
To: dev@cloudstack.apache.org
Subject: [DIscuss]Storage image store plugin framework refactoring

Hi All,

Currently CloudStack does not offer a flexible pluggable framework for users to easily integrate and configure any 3rd-party object stores for such backup services as registering templates, taking snapshots, etc. Along with Edison's recent refactored storage subsystem 2.0 that mainly refactored current CloudStack primary storage implementation,  we are proposing to develop a storage backup object store plugin framework to allow CloudStack to systematically manage and configure various types of backup data stores from different vendors, like NFS, S3, Swift, etc. With this new plugin framework, we would like to achieve following functionalities:
1. Support different object store providers in a uniform and pluggable fashion.
2. Enable region wide object backup using S3-like object store.
3. Provide pluggable data motion strategies to handle data transfer from one data store to another data store.
4. Provide a scalable cache storage framework while moving data between primary storage and backup storage for certain hypervisor needs.
5. Support flexible combinations of primary storage, secondary storage and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware), (ISCSI, Swift, KVM), ...., etc.
The proposed ImageStore plugin framework architecture is detailed in our FS here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backup+Object+Store+Plugin+Framework.
The JIRA ticket to track this feature is: https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is currently carried out in feature branch  "object_store".
Please let me know your comments and suggestions.

Thanks
-min

Re: [DIscuss]Storage image store plugin framework refactoring

[Min Chen <mi...@citrix.com>]

Yes. You should be able to use it to test S3 object store.

THanks
-min

On 4/22/13 4:53 AM, "Sanjeev Neelarapu" <sa...@citrix.com>
wrote:

>Hi,
>
>I have prepared RIAK CS object store in our lab environment and planning
>to use it for object store feature testing .
>As per my understanding RIAK CS supports S3 APIs. So let me know whether
>I can use it as an alternative to s3 based object store.
>
>Thanks,
>Sanjeev
>
>-----Original Message-----
>From: prasanna [mailto:srivatsav.prasanna@gmail.com] On Behalf Of
>Prasanna Santhanam
>Sent: Wednesday, April 10, 2013 12:42 PM
>To: dev@cloudstack.apache.org
>Subject: Re: [DIscuss]Storage image store plugin framework refactoring
>
>On Tue, Apr 09, 2013 at 09:19:11AM -0700, Min Chen wrote:
>> Not yet at this moment. But we are planning to provide a sample plugin
>> implementation for this effort. If you are looking at object_store
>> branch, you will see a plugin called
>> "cloud-plugin-storage-image-sample", which will be the placeholder for
>>a sample image store implementation.
>> 
>
>Hmmm - more frameworks and no reference implementation? :(
>
>I'm keeping an eye on the branch and was hoping there'd be an
>implementation of an objectstore because I'm looking to span this
>solution across some qa labs. RiakCS, Swift are all opensource now.
>
>--
>Prasanna.,

RE: [DIscuss]Storage image store plugin framework refactoring

[Sanjeev Neelarapu <sa...@citrix.com>]

Hi,

I have prepared RIAK CS object store in our lab environment and planning to use it for object store feature testing . 
As per my understanding RIAK CS supports S3 APIs. So let me know whether I can use it as an alternative to s3 based object store.

Thanks,
Sanjeev

-----Original Message-----
From: prasanna [mailto:srivatsav.prasanna@gmail.com] On Behalf Of Prasanna Santhanam
Sent: Wednesday, April 10, 2013 12:42 PM
To: dev@cloudstack.apache.org
Subject: Re: [DIscuss]Storage image store plugin framework refactoring

On Tue, Apr 09, 2013 at 09:19:11AM -0700, Min Chen wrote:
> Not yet at this moment. But we are planning to provide a sample plugin 
> implementation for this effort. If you are looking at object_store 
> branch, you will see a plugin called 
> "cloud-plugin-storage-image-sample", which will be the placeholder for a sample image store implementation.
> 

Hmmm - more frameworks and no reference implementation? :(

I'm keeping an eye on the branch and was hoping there'd be an implementation of an objectstore because I'm looking to span this solution across some qa labs. RiakCS, Swift are all opensource now.

--
Prasanna.,

Re: [DIscuss]Storage image store plugin framework refactoring

[Prasanna Santhanam <ts...@apache.org>]

On Tue, Apr 09, 2013 at 09:19:11AM -0700, Min Chen wrote:
> Not yet at this moment. But we are planning to provide a sample plugin
> implementation for this effort. If you are looking at object_store branch,
> you will see a plugin called "cloud-plugin-storage-image-sample", which
> will be the placeholder for a sample image store implementation.
> 

Hmmm - more frameworks and no reference implementation? :(

I'm keeping an eye on the branch and was hoping there'd be an
implementation of an objectstore because I'm looking to span this
solution across some qa labs. RiakCS, Swift are all opensource
now.

-- 
Prasanna.,

Re: [DIscuss]Storage image store plugin framework refactoring

[Min Chen <mi...@citrix.com>]

Not yet at this moment. But we are planning to provide a sample plugin
implementation for this effort. If you are looking at object_store branch,
you will see a plugin called "cloud-plugin-storage-image-sample", which
will be the placeholder for a sample image store implementation.

Thanks
-min

On 4/9/13 1:59 AM, "Prasanna Santhanam" <ts...@apache.org> wrote:

>On Mon, Apr 08, 2013 at 04:45:23PM -0700, Min Chen wrote:
>> Hi All,
>> 
>> Currently CloudStack does not offer a flexible pluggable framework
>> for users to easily integrate and configure any 3rd-party object
>> stores for such backup services as registering templates, taking
>> snapshots, etc. Along with Edison's recent refactored storage
>> subsystem 2.0 that mainly refactored current CloudStack primary
>> storage implementation,  we are proposing to develop a storage
>> backup object store plugin framework to allow CloudStack to
>> systematically manage and configure various types of backup data
>> stores from different vendors, like NFS, S3, Swift, etc. With this
>> new plugin framework, we would like to achieve following
>> functionalities:
>
>> 1. Support different object store providers in a uniform and
>> pluggable fashion.
>> 2. Enable region wide object backup using S3-like object store.
>> 3. Provide pluggable data motion strategies to handle data transfer
>> from one data store to another data store.
>> 4. Provide a scalable cache storage framework while moving data
>> between primary storage and backup storage for certain hypervisor
>> needs.
>> 5. Support flexible combinations of primary storage, secondary
>> storage and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware),
>> (ISCSI, Swift, KVM), ?., etc.
>> The proposed ImageStore plugin framework architecture is detailed in
>> our FS here:
>> 
>>https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backup+Obj
>>ect+Store+Plugin+Framework.
>> The JIRA ticket to track this feature is:
>> https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is
>> currently carried out in feature branch  "object_store".
>> Please let me know your comments and suggestions.
>
>Perhaps it is too early to ask, will there be a reference
>implementation done for any object store solutions as part of the
>refactoring?
>
>-- 
>Prasanna.,

Re: [DIscuss]Storage image store plugin framework refactoring

[Prasanna Santhanam <ts...@apache.org>]

On Mon, Apr 08, 2013 at 04:45:23PM -0700, Min Chen wrote:
> Hi All,
> 
> Currently CloudStack does not offer a flexible pluggable framework
> for users to easily integrate and configure any 3rd-party object
> stores for such backup services as registering templates, taking
> snapshots, etc. Along with Edison's recent refactored storage
> subsystem 2.0 that mainly refactored current CloudStack primary
> storage implementation,  we are proposing to develop a storage
> backup object store plugin framework to allow CloudStack to
> systematically manage and configure various types of backup data
> stores from different vendors, like NFS, S3, Swift, etc. With this
> new plugin framework, we would like to achieve following
> functionalities:

> 1. Support different object store providers in a uniform and
> pluggable fashion.
> 2. Enable region wide object backup using S3-like object store.
> 3. Provide pluggable data motion strategies to handle data transfer
> from one data store to another data store.
> 4. Provide a scalable cache storage framework while moving data
> between primary storage and backup storage for certain hypervisor
> needs.
> 5. Support flexible combinations of primary storage, secondary
> storage and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware),
> (ISCSI, Swift, KVM), ?., etc.
> The proposed ImageStore plugin framework architecture is detailed in
> our FS here:
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backup+Object+Store+Plugin+Framework.
> The JIRA ticket to track this feature is:
> https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is
> currently carried out in feature branch  "object_store".
> Please let me know your comments and suggestions.

Perhaps it is too early to ask, will there be a reference
implementation done for any object store solutions as part of the
refactoring?

-- 
Prasanna.,

[DIscuss]Storage image store plugin framework refactoring

[Min Chen <mi...@citrix.com>]

Hi All,

Currently CloudStack does not offer a flexible pluggable framework for users to easily integrate and configure any 3rd-party object stores for such backup services as registering templates, taking snapshots, etc. Along with Edison's recent refactored storage subsystem 2.0 that mainly refactored current CloudStack primary storage implementation,  we are proposing to develop a storage backup object store plugin framework to allow CloudStack to systematically manage and configure various types of backup data stores from different vendors, like NFS, S3, Swift, etc. With this new plugin framework, we would like to achieve following functionalities:
1. Support different object store providers in a uniform and pluggable fashion.
2. Enable region wide object backup using S3-like object store.
3. Provide pluggable data motion strategies to handle data transfer from one data store to another data store.
4. Provide a scalable cache storage framework while moving data between primary storage and backup storage for certain hypervisor needs.
5. Support flexible combinations of primary storage, secondary storage and hypervisors, such as (NFS, NFS, Xen), (NF3, S3, Vmware), (ISCSI, Swift, KVM), …., etc.
The proposed ImageStore plugin framework architecture is detailed in our FS here: https://cwiki.apache.org/confluence/display/CLOUDSTACK/Storage+Backup+Object+Store+Plugin+Framework.
The JIRA ticket to track this feature is: https://issues.apache.org/jira/browse/CLOUDSTACK-1975. The work is currently carried out in feature branch  "object_store".
Please let me know your comments and suggestions.

Thanks
-min

Re: [Discuss] API name alias

[Chiradeep Vittal <Ch...@citrix.com>]

On 4/8/13 11:18 AM, "Rohit Yadav" <bh...@apache.org> wrote:

>On Mon, Apr 8, 2013 at 7:10 PM, John Burwell <jb...@basho.com> wrote:
>
>> Rohit,
>>
>> Why would we need to rewrite any Java code to switch to an array?  As I
>> mentioned, array annotation values are treated as varargs.  Therefore,
>> annotations with single values can be left alone.  Only cases where
>> multiple values are required would require the addition of curly braces.
>>  For example, for an APICommand with one name, the following would
>>continue
>> to work:
>>
>
>Hey John, you're right but it's a matter of code styling, I prefer
>writing;
>
>@APICommand(name = {'name1'}) even if it's only one name, this way any
>subsequent api author would know that name is an array and they can have
>aliases which is what Kishan wants.
>
>
>>
>>         @APICommand(name="apiName1")
>>
>> While for an APICommand with multiple names, the following form would
>>now
>> be supported:
>>
>>         @APICommand(name={"apiName1", "apiName2", "apiName3"})
>>
>> I am not familiar with the Marvin code, so I can not speak to the impact
>> of this change to it.  However, the only changes to the Java code
>>should be
>> the API commands with multiple names and the classes that actually
>>consume
>> the value of this annotation.
>>
>
>Yes, you're right we can skip the code styling I referred then it won't
>consume much coding, we just need to fix these;
>
>- Fix APICommand annotation interface definition to accept name as an
>array
>- Fix method in ApiServer which creates apiname->cmdclass map to care for
>aliases
>- Fix ApiDiscovery to take care of the aliases (create different response
>objs for the same cmd class)
>- Fix apidocs XmlWriter class to take of the aliases and the api html
>generator to process the same.
>- Fix Marvin's codegenerator.py to take care of the aliases (which means
>create apiname related modules which would contain cmd and response
>boilterplate fields/class)
>
>Cheers.
>
>
>>
>> Thanks,
>> -John
>>
>> On Apr 8, 2013, at 9:22 AM, Rohit Yadav <bh...@apache.org> wrote:
>>
>> > On Mon, Apr 8, 2013 at 6:33 PM, Kishan Kavala
>><Kishan.Kavala@citrix.com
>> >wrote:
>> >
>> >> APICommand annotation in API Cmd object has a name parameter.
>>Currently
>> >> name parameter takes only one value. I plan to enhance this to
>>support
>> >> comma separated values. This will allow multiple API names for the
>>same
>> API
>> >> Cmd object.
>> >>
>> >> Current:
>> >> @APICommand(name = "apiName1", ..
>> >>
>> >> Proposed:
>> >> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
>> >>
>> >
>> > It's a hack, but doable. While not elegant as John suggests, parsing
>> comma
>> > separated value can lead to some issues.
>> >
>> > Changing the name field to array would require a lot of rewriting the
>> > present apis annotation name fields, again doable by a small python
>> program
>> > (like the one I used in the last name field refactoring).
>> >
>> > This would really help remove duplicate cmd classes for apis such as
>>copy
>> > iso and volume which does the same job but requires two different cmd
>> class
>> > implementations.
>> >
>> > Whatever way you decide, pl. make sure to fix ApiDiscovery, ApiServer
>> > cmdclass-apiname map generator method and apidocs generation
>>accordingly.
>> >
>> > Cheers.
>> >
>> >
>> >>
>> >> Requirement:
>> >> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList
>>(grouping
>> of
>> >> NetworkACLItems).  Current APIs use *NetworkACL (create
>> >> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related APIs.
>> These
>> >> APIs have to be changed to *NetworkACL Item(create
>> >> NetworkACLItem/deleteNetworkACLItem etc..) to get the terminology
>> right. We
>> >> also need to support old API names for backward compatibility. Hence
>>the
>> >> need for API name alias.
>> >>
>> >> Terminology:
>> >> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
>> >> NetworkACL - Group of Network ACL Items. API will use the term
>> >> NetworkACLList to differentiate from the existing NetworkACL APIs.
>> >>

How about an additional attribute 'preferred' or 'deprecated' ?

@APICommand(name={'Foo', 'Bar'}, preferred='Foo', deprecated='Bar')

Re: [Discuss] API name alias

[Rohit Yadav <bh...@apache.org>]

On Mon, Apr 8, 2013 at 7:10 PM, John Burwell <jb...@basho.com> wrote:

> Rohit,
>
> Why would we need to rewrite any Java code to switch to an array?  As I
> mentioned, array annotation values are treated as varargs.  Therefore,
> annotations with single values can be left alone.  Only cases where
> multiple values are required would require the addition of curly braces.
>  For example, for an APICommand with one name, the following would continue
> to work:
>

Hey John, you're right but it's a matter of code styling, I prefer writing;

@APICommand(name = {'name1'}) even if it's only one name, this way any
subsequent api author would know that name is an array and they can have
aliases which is what Kishan wants.


>
>         @APICommand(name="apiName1")
>
> While for an APICommand with multiple names, the following form would now
> be supported:
>
>         @APICommand(name={"apiName1", "apiName2", "apiName3"})
>
> I am not familiar with the Marvin code, so I can not speak to the impact
> of this change to it.  However, the only changes to the Java code should be
> the API commands with multiple names and the classes that actually consume
> the value of this annotation.
>

Yes, you're right we can skip the code styling I referred then it won't
consume much coding, we just need to fix these;

- Fix APICommand annotation interface definition to accept name as an array
- Fix method in ApiServer which creates apiname->cmdclass map to care for
aliases
- Fix ApiDiscovery to take care of the aliases (create different response
objs for the same cmd class)
- Fix apidocs XmlWriter class to take of the aliases and the api html
generator to process the same.
- Fix Marvin's codegenerator.py to take care of the aliases (which means
create apiname related modules which would contain cmd and response
boilterplate fields/class)

Cheers.


>
> Thanks,
> -John
>
> On Apr 8, 2013, at 9:22 AM, Rohit Yadav <bh...@apache.org> wrote:
>
> > On Mon, Apr 8, 2013 at 6:33 PM, Kishan Kavala <Kishan.Kavala@citrix.com
> >wrote:
> >
> >> APICommand annotation in API Cmd object has a name parameter. Currently
> >> name parameter takes only one value. I plan to enhance this to support
> >> comma separated values. This will allow multiple API names for the same
> API
> >> Cmd object.
> >>
> >> Current:
> >> @APICommand(name = "apiName1", ..
> >>
> >> Proposed:
> >> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
> >>
> >
> > It's a hack, but doable. While not elegant as John suggests, parsing
> comma
> > separated value can lead to some issues.
> >
> > Changing the name field to array would require a lot of rewriting the
> > present apis annotation name fields, again doable by a small python
> program
> > (like the one I used in the last name field refactoring).
> >
> > This would really help remove duplicate cmd classes for apis such as copy
> > iso and volume which does the same job but requires two different cmd
> class
> > implementations.
> >
> > Whatever way you decide, pl. make sure to fix ApiDiscovery, ApiServer
> > cmdclass-apiname map generator method and apidocs generation accordingly.
> >
> > Cheers.
> >
> >
> >>
> >> Requirement:
> >> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList (grouping
> of
> >> NetworkACLItems).  Current APIs use *NetworkACL (create
> >> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related APIs.
> These
> >> APIs have to be changed to *NetworkACL Item(create
> >> NetworkACLItem/deleteNetworkACLItem etc..) to get the terminology
> right. We
> >> also need to support old API names for backward compatibility. Hence the
> >> need for API name alias.
> >>
> >> Terminology:
> >> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
> >> NetworkACL - Group of Network ACL Items. API will use the term
> >> NetworkACLList to differentiate from the existing NetworkACL APIs.
> >>
> >>
> >>
>
>

Re: [Discuss] API name alias

[John Burwell <jb...@basho.com>]

Rohit,

Why would we need to rewrite any Java code to switch to an array?  As I mentioned, array annotation values are treated as varargs.  Therefore, annotations with single values can be left alone.  Only cases where multiple values are required would require the addition of curly braces.  For example, for an APICommand with one name, the following would continue to work:

	@APICommand(name="apiName1")

While for an APICommand with multiple names, the following form would now be supported:

	@APICommand(name={"apiName1", "apiName2", "apiName3"})

I am not familiar with the Marvin code, so I can not speak to the impact of this change to it.  However, the only changes to the Java code should be the API commands with multiple names and the classes that actually consume the value of this annotation.

Thanks,
-John

On Apr 8, 2013, at 9:22 AM, Rohit Yadav <bh...@apache.org> wrote:

> On Mon, Apr 8, 2013 at 6:33 PM, Kishan Kavala <Ki...@citrix.com>wrote:
> 
>> APICommand annotation in API Cmd object has a name parameter. Currently
>> name parameter takes only one value. I plan to enhance this to support
>> comma separated values. This will allow multiple API names for the same API
>> Cmd object.
>> 
>> Current:
>> @APICommand(name = "apiName1", ..
>> 
>> Proposed:
>> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
>> 
> 
> It's a hack, but doable. While not elegant as John suggests, parsing comma
> separated value can lead to some issues.
> 
> Changing the name field to array would require a lot of rewriting the
> present apis annotation name fields, again doable by a small python program
> (like the one I used in the last name field refactoring).
> 
> This would really help remove duplicate cmd classes for apis such as copy
> iso and volume which does the same job but requires two different cmd class
> implementations.
> 
> Whatever way you decide, pl. make sure to fix ApiDiscovery, ApiServer
> cmdclass-apiname map generator method and apidocs generation accordingly.
> 
> Cheers.
> 
> 
>> 
>> Requirement:
>> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList (grouping of
>> NetworkACLItems).  Current APIs use *NetworkACL (create
>> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related APIs. These
>> APIs have to be changed to *NetworkACL Item(create
>> NetworkACLItem/deleteNetworkACLItem etc..) to get the terminology right. We
>> also need to support old API names for backward compatibility. Hence the
>> need for API name alias.
>> 
>> Terminology:
>> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
>> NetworkACL - Group of Network ACL Items. API will use the term
>> NetworkACLList to differentiate from the existing NetworkACL APIs.
>> 
>> 
>> 

Re: [Discuss] API name alias

[Rohit Yadav <bh...@apache.org>]

On Mon, Apr 8, 2013 at 6:33 PM, Kishan Kavala <Ki...@citrix.com>wrote:

> APICommand annotation in API Cmd object has a name parameter. Currently
> name parameter takes only one value. I plan to enhance this to support
> comma separated values. This will allow multiple API names for the same API
> Cmd object.
>
> Current:
> @APICommand(name = "apiName1", ..
>
> Proposed:
> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
>

It's a hack, but doable. While not elegant as John suggests, parsing comma
separated value can lead to some issues.

Changing the name field to array would require a lot of rewriting the
present apis annotation name fields, again doable by a small python program
(like the one I used in the last name field refactoring).

This would really help remove duplicate cmd classes for apis such as copy
iso and volume which does the same job but requires two different cmd class
implementations.

Whatever way you decide, pl. make sure to fix ApiDiscovery, ApiServer
cmdclass-apiname map generator method and apidocs generation accordingly.

Cheers.


>
> Requirement:
> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList (grouping of
> NetworkACLItems).  Current APIs use *NetworkACL (create
> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related APIs. These
> APIs have to be changed to *NetworkACL Item(create
> NetworkACLItem/deleteNetworkACLItem etc..) to get the terminology right. We
> also need to support old API names for backward compatibility. Hence the
> need for API name alias.
>
> Terminology:
> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
> NetworkACL - Group of Network ACL Items. API will use the term
> NetworkACLList to differentiate from the existing NetworkACL APIs.
>
>
>

RE: [Discuss] API name alias

[Koushik Das <ko...@citrix.com>]

What happens when same alias gets used for different commands? I think even with the current mechanism for command name the problem exists i.e. two different commands can have the same name. And in that case how is it handled?


> -----Original Message-----
> From: Kishan Kavala [mailto:Kishan.Kavala@citrix.com]
> Sent: Monday, April 08, 2013 6:34 PM
> To: dev@cloudstack.apache.org
> Subject: [Discuss] API name alias
> 
> APICommand annotation in API Cmd object has a name parameter. Currently
> name parameter takes only one value. I plan to enhance this to support
> comma separated values. This will allow multiple API names for the same API
> Cmd object.
> 
> Current:
> @APICommand(name = "apiName1", ..
> 
> Proposed:
> @APICommand(name = "apiName1, apiAlias2, apiAlias3", ..
> 
> Requirement:
> As part of CLOUDSTACK-763, I'll be introducing NetworkACLList (grouping of
> NetworkACLItems).  Current APIs use *NetworkACL (create
> NetworkACL/deleteNetworkACL etc..) for NetworkACLItem related APIs.
> These APIs have to be changed to *NetworkACL Item(create
> NetworkACLItem/deleteNetworkACLItem etc..) to get the terminology right.
> We also need to support old API names for backward compatibility. Hence
> the need for API name alias.
> 
> Terminology:
> NetworkACLItem - Individual ACL Entry (was NetworlACL earlier).
> NetworkACL - Group of Network ACL Items. API will use the term
> NetworkACLList to differentiate from the existing NetworkACL APIs.
>