Arch Rework RFC: API refactoring updates

[Rohit Yadav <ro...@citrix.com>]

Few updates;

1. Moved and classified all apis (except the anomalies, see other email) into namespace org.apache.cloudstack.api
2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
3. Further in each org.apache.cloudstack.api.{admin,user}.command pkg, the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
4. Few usage related APIs could not be moved from cloud-server to cloud-api, as they are tightly coupled with classes available in cloud-server.
5. commands.properties is fixed.
6. Checked, the api_refactoring merges fine with a minor merge conflicts. The build on the branch is broken, would be fixed soon.

I request that if you're adding any new class, please use the namespace org.apache.cloudstack.

Road ahead:
- Because of 1,2,3; there would be two classes of API server. The first one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;

user API server would use only the org.apache.cloudstack.api.user artifact
admin API server would use both org.apache.cloudstack.api.admin and org.apache.cloudstack.api.user
Based on the arch. rework docs, ppts, talks, Alex's idea was to separate api based on security level would give ultimate isolation.

- Separate out cloud-api, and make it run as a separate web app like awsapi.
- Annotations and separation of API based on api affinity would help automate apidoc generation, api discovery over an api endpoint, so clients (UI or cloudmonkey etc.) would be able to discover 
- The commands.properties syntax is horrible, I want to get rid of the evil syntax by having an apiname annotation for all API Cmd classes and the API server would be able to load this info during runtime. In commands.properties, we should be just able to set policy for user role for each api, if apiname is not declared it's blacklisted. It becomes tricky with plugins. I don't know how to get it right the first time, but let's try.
- ACL and security checking at API layer, the requests would be just passed to (multiple) cloud-engine which won't handle them, as that will only orchestrate.
- Roles would be decided from commands.properties. Multiple API servers can be run with different combination of rules in commands.properties. This would allow separation of policy from mechanism, and multiple roles and end points. For example, an admin can create http routers for these api server, so: <url>/hr, <url>/finance, <url>/marketting can be proxies to different user api servers with different set of whitelisted apis.

Comments, suggestions, flames?

Regards.

Re: Arch Rework RFC: API refactoring updates

[Rohit Yadav <ro...@citrix.com>]

On 06-Dec-2012, at 9:59 PM, Koushik Das <ko...@citrix.com> wrote:

> Rohit,
> 
> Can you explain the "admin" and "user" security level? I don't see that in https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+API+refactoring.

Koushik,
That was part of Alex's talk on "The future of CloudStack Architecture", will ask him to share somewhere, I'm unable to find it myself.
The FS document on that wiki page refers to this work, "packaging the commands per entity category".

I'm not sure if we should do separate cmds per security level, but the idea is that certain apis should not be shipped with the user API server, so that would be the ultimate isolation or restriction of APIs.

>From a sysadmins perspective, if we get it right the storyboard would be like this: they would install CloudStack and it would only have cloud-engine and nothing else, they can go ahead and install the mgmt API server which can be controlled via the UI or cloudmonkey, using which they can configure their hosts, zones, etc. If they want to have API servers for users, they can start a VM which would run user API server. The idea is that unless you want a service, CS should not start it, for ex. for their own consumption an org may not require an user API server. On each VM the user API server could have different sets of APIs served via different urls end points (like url/hr, url/mgmt, url/engg etc.) for users.

> Is this just a classification in terms of packaging or will there be a separate class hierarchy for admin vs. user commands?

Yes, that is finished now, about 300 apis.
You may checkout the code from api_refactoring. But, Mice raised an interesting question about domain admins, which I'm not sure. Will discuss and get back on this. The assumption I made was domain and system admins are within the datacenter. If not, one solution is to run multiple mgmt servers (in VMs as a service) with different sets of whitelisted apis served on different end points.

> Also will there be more stringent input validations/additional checks in user level APIs compared to admin ones?

All the validations should be dealt by the API layer, cloud-engine should orchestrate and not do any checking.

Regards.

> 
> Thanks,
> Koushik
> 
> -----Original Message-----
> From: Rohit Yadav [mailto:rohit.yadav@citrix.com] 
> Sent: Friday, December 07, 2012 1:37 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: Arch Rework RFC: API refactoring updates
> 
> Few updates;
> 
> 1. Moved and classified all apis (except the anomalies, see other email) into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
> 3. Further in each org.apache.cloudstack.api.{admin,user}.command pkg, the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
> 4. Few usage related APIs could not be moved from cloud-server to cloud-api, as they are tightly coupled with classes available in cloud-server.
> 5. commands.properties is fixed.
> 6. Checked, the api_refactoring merges fine with a minor merge conflicts. The build on the branch is broken, would be fixed soon.
> 
> I request that if you're adding any new class, please use the namespace org.apache.cloudstack.
> 
> Road ahead:
> - Because of 1,2,3; there would be two classes of API server. The first one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;
> 
> user API server would use only the org.apache.cloudstack.api.user artifact admin API server would use both org.apache.cloudstack.api.admin and org.apache.cloudstack.api.user Based on the arch. rework docs, ppts, talks, Alex's idea was to separate api based on security level would give ultimate isolation.
> 
> - Separate out cloud-api, and make it run as a separate web app like awsapi.
> - Annotations and separation of API based on api affinity would help automate apidoc generation, api discovery over an api endpoint, so clients (UI or cloudmonkey etc.) would be able to discover
> - The commands.properties syntax is horrible, I want to get rid of the evil syntax by having an apiname annotation for all API Cmd classes and the API server would be able to load this info during runtime. In commands.properties, we should be just able to set policy for user role for each api, if apiname is not declared it's blacklisted. It becomes tricky with plugins. I don't know how to get it right the first time, but let's try.
> - ACL and security checking at API layer, the requests would be just passed to (multiple) cloud-engine which won't handle them, as that will only orchestrate.
> - Roles would be decided from commands.properties. Multiple API servers can be run with different combination of rules in commands.properties. This would allow separation of policy from mechanism, and multiple roles and end points. For example, an admin can create http routers for these api server, so: <url>/hr, <url>/finance, <url>/marketting can be proxies to different user api servers with different set of whitelisted apis.
> 
> Comments, suggestions, flames?
> 
> Regards.

RE: Arch Rework RFC: API refactoring updates

[Koushik Das <ko...@citrix.com>]

Rohit,

Can you explain the "admin" and "user" security level? I don't see that in https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+API+refactoring.
Is this just a classification in terms of packaging or will there be a separate class hierarchy for admin vs. user commands? Also will there be more stringent input validations/additional checks in user level APIs compared to admin ones?

Thanks,
Koushik

-----Original Message-----
From: Rohit Yadav [mailto:rohit.yadav@citrix.com] 
Sent: Friday, December 07, 2012 1:37 AM
To: cloudstack-dev@incubator.apache.org
Subject: Arch Rework RFC: API refactoring updates

Few updates;

1. Moved and classified all apis (except the anomalies, see other email) into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
3. Further in each org.apache.cloudstack.api.{admin,user}.command pkg, the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
4. Few usage related APIs could not be moved from cloud-server to cloud-api, as they are tightly coupled with classes available in cloud-server.
5. commands.properties is fixed.
6. Checked, the api_refactoring merges fine with a minor merge conflicts. The build on the branch is broken, would be fixed soon.

I request that if you're adding any new class, please use the namespace org.apache.cloudstack.

Road ahead:
- Because of 1,2,3; there would be two classes of API server. The first one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;

user API server would use only the org.apache.cloudstack.api.user artifact admin API server would use both org.apache.cloudstack.api.admin and org.apache.cloudstack.api.user Based on the arch. rework docs, ppts, talks, Alex's idea was to separate api based on security level would give ultimate isolation.

- Separate out cloud-api, and make it run as a separate web app like awsapi.
- Annotations and separation of API based on api affinity would help automate apidoc generation, api discovery over an api endpoint, so clients (UI or cloudmonkey etc.) would be able to discover
- The commands.properties syntax is horrible, I want to get rid of the evil syntax by having an apiname annotation for all API Cmd classes and the API server would be able to load this info during runtime. In commands.properties, we should be just able to set policy for user role for each api, if apiname is not declared it's blacklisted. It becomes tricky with plugins. I don't know how to get it right the first time, but let's try.
- ACL and security checking at API layer, the requests would be just passed to (multiple) cloud-engine which won't handle them, as that will only orchestrate.
- Roles would be decided from commands.properties. Multiple API servers can be run with different combination of rules in commands.properties. This would allow separation of policy from mechanism, and multiple roles and end points. For example, an admin can create http routers for these api server, so: <url>/hr, <url>/finance, <url>/marketting can be proxies to different user api servers with different set of whitelisted apis.

Comments, suggestions, flames?

Regards.

RE: Arch Rework RFC: API refactoring updates

[Alex Huang <Al...@citrix.com>]

> What's done and plans:
> I've separated out api cmds as per admin, user pkg so that developers would
> know what security level the apis (Cmd classes) are in. Next, I separated
> them based on entity group (like all vpc, vm etc. related apis go together).
> This was done only for developers, so they would know where to look for
> them and contribute. This would also help us generate better automated
> apidocs, the grouping and classification is done manually by hand. Right now
> I've fixed the build and apidocs build as well.

There two additional benefits to breaking into different java packages (not jars).

- It's easier for developers to look for the command related to their functionality.  Because the java files start with action, it is difficult for developers to look for the commands among 300 commands lumped into the same package.  With his change, developers can first look for the package and then their command.
- Documentation tool can automatically put the commands down as related to each other according to the package.  This allow someone when reading the documentation to find related commands and entities. 

Note that this causes no change in the over the wire semantics whatsoever.  What the otw command is and how that is associated with the java class is defined in the commands.properties file and is unrelated to java packages.

We are absolutely open to suggestions as to how commands should be grouped.  Some are obvious (e.g. all commands related to loadbalancing is grouped under loadbalancing).  Some are not.   And we don't expect we'll get everything right on the first pass but we hope to get it started.

--Alex

Re: Arch Rework RFC: API refactoring updates

[Rohit Yadav <ro...@citrix.com>]

Sorry folks, I got it wrong and I re-read my emails and Alex is right. Pl. see his reply.

What's done and plans:
I've separated out api cmds as per admin, user pkg so that developers would know what security level the apis (Cmd classes) are in. Next, I separated them based on entity group (like all vpc, vm etc. related apis go together). This was done only for developers, so they would know where to look for them and contribute. This would also help us generate better automated apidocs, the grouping and classification is done manually by hand. Right now I've fixed the build and apidocs build as well.

The next thing we're going to do is work on @ACL annotation, APIAccessChecker (that checks roles, etc. which is an adapter, so organizations can throw their own role based checkers). We also have @validators which can do db, range/size based (also adapter, so one can diy db validations). What we want to do is for a group of apis, implement them and show that it works from top to bottom (the api refactoring work). And if we get it right, we can replicate that for all the apis, separate out api+service layer as a api web app and make it interact with cloud-engine.

And yes, all the API servers would have access to both org.apache.cloudstack.api…{admin, user} artefacts or classes. So, a sysadmin can decide what kind of apiserver with what set of apis (whitelisted, if not defined, the api would be blacklisted) they want to offer at which end point (running on a VM as a service).

Sorry for the confusion.

Regards,
Rohit
PS. /me slaps himself


On 06-Dec-2012, at 4:29 PM, Mice Xia <mi...@tcloudcomputing.com> wrote:

> [quote]
> 1. Moved and classified all apis (except the anomalies, see other email) into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
> - Because of 1,2,3; there would be two classes of API server. The first one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;
> [/quote]
> 
> I haven’t seen the codes, just one question. 
> If the intention is to deploy API server separately based on the security level, where are APIs enabled for domain-admin located? In the user package or in the admin package?
> 
> Regards
> Mice
> 
> -----Original Message-----
> From: Rohit Yadav [mailto:rohit.yadav@citrix.com] 
> Sent: Friday, December 07, 2012 4:07 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: Arch Rework RFC: API refactoring updates
> 
> Few updates;
> 
> 1. Moved and classified all apis (except the anomalies, see other email) into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
> 3. Further in each org.apache.cloudstack.api.{admin,user}.command pkg, the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
> 4. Few usage related APIs could not be moved from cloud-server to cloud-api, as they are tightly coupled with classes available in cloud-server.
> 5. commands.properties is fixed.
> 6. Checked, the api_refactoring merges fine with a minor merge conflicts. The build on the branch is broken, would be fixed soon.
> 
> I request that if you're adding any new class, please use the namespace org.apache.cloudstack.
> 
> Road ahead:
> - Because of 1,2,3; there would be two classes of API server. The first one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;
> 
> user API server would use only the org.apache.cloudstack.api.user artifact admin API server would use both org.apache.cloudstack.api.admin and org.apache.cloudstack.api.user Based on the arch. rework docs, ppts, talks, Alex's idea was to separate api based on security level would give ultimate isolation.
> 
> - Separate out cloud-api, and make it run as a separate web app like awsapi.
> - Annotations and separation of API based on api affinity would help automate apidoc generation, api discovery over an api endpoint, so clients (UI or cloudmonkey etc.) would be able to discover
> - The commands.properties syntax is horrible, I want to get rid of the evil syntax by having an apiname annotation for all API Cmd classes and the API server would be able to load this info during runtime. In commands.properties, we should be just able to set policy for user role for each api, if apiname is not declared it's blacklisted. It becomes tricky with plugins. I don't know how to get it right the first time, but let's try.
> - ACL and security checking at API layer, the requests would be just passed to (multiple) cloud-engine which won't handle them, as that will only orchestrate.
> - Roles would be decided from commands.properties. Multiple API servers can be run with different combination of rules in commands.properties. This would allow separation of policy from mechanism, and multiple roles and end points. For example, an admin can create http routers for these api server, so: <url>/hr, <url>/finance, <url>/marketting can be proxies to different user api servers with different set of whitelisted apis.
> 
> Comments, suggestions, flames?
> 
> Regards.

RE: Arch Rework RFC: API refactoring updates

[Fang Wang <fa...@citrix.com>]

Mice,

Thanks for the feedback. See Alex's comments, he clarified several points.

The packaging of the commands itself does not provide role-based access control. The API server contains the logic
To do access control.  

Thanks,
-Fang
-----Original Message-----
From: Mice Xia [mailto:mice_xia@tcloudcomputing.com] 
Sent: Friday, December 07, 2012 4:36 PM
To: cloudstack-dev@incubator.apache.org
Subject: RE: Arch Rework RFC: API refactoring updates

Fang,

I vote for the second way, to keep it simple and clear, each API server has the same binary artifacts and contains all API implementations, access control is based on RBAC/ACL, and blacklist(white List) can be defined on a per API server basis. And perhaps we don’t need to explicitly package classes by roles ( or security level), this seems to hardcode APIs' ACL in the code level.

My 2 cent.

-Mice

-----Original Message-----
From: Fang Wang [mailto:fang.wang@citrix.com]
Sent: Saturday, December 08, 2012 7:44 AM
To: cloudstack-dev@incubator.apache.org
Subject: RE: Arch Rework RFC: API refactoring updates

Hi Mice,

For now, we just separate the API into 2 roles: user and admin. 
The admin can be separated further into root admin, domain admin, etc. Further dynamic roles can be added. 
Once the framework is there, we can further separate the admin roles. 

Per your question, there can be 2 ways to handle the domain admin requests:
- one is like you mentioned, sysadmin has separate api servers to handle user and domain api requests separately.
- Another way is to deploy one api server, and the api server does the role base access check. So user cann't access the domain Admin apis., the access would be blocked and denied.  The advantage of this method is the flexibility.
E.g, add another new domain, or a new category admin, we can dynamically deploy and apply the roles. 


Thanks,
-Fang

-----Original Message-----
From: Mice Xia [mailto:weiran.xia1@gmail.com]
Sent: Thursday, December 06, 2012 5:15 PM
To: cloudstack-dev@incubator.apache.org
Subject: Re: Arch Rework RFC: API refactoring updates

rohit,

i asked this question because requests from domain admins are usually from outside of datacenter for a public cloud.

so artifact built from user package can be directly deployed as a user api server and serve user requests. but to serve domain admin request, sysadmin has to prepare another api server with another set of apis, which is a sub set of admin package, is this correct?

sent from phone, sorry for typos.

mice

在 2012年12月7日星期五，Rohit Yadav <ro...@citrix.com> 写道：
>
> On 06-Dec-2012, at 4:29 PM, Mice Xia <mi...@tcloudcomputing.com> wrote:
>
>> [quote]
>> 1. Moved and classified all apis (except the anomalies, see other
>> email)
into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
>> - Because of 1,2,3; there would be two classes of API server. The 
>> first
one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;
>> [/quote]
>>
>> I haven’t seen the codes, just one question.
>> If the intention is to deploy API server separately based on the
security level, where are APIs enabled for domain-admin located? In the user package or in the admin package?
>
> The admin package. For domain-admin etc. the sysadmin would run mgmt
servers with different sets of apis s/he wants to provide for a domain admin. There won't be any pre configured set of apis, this would allow a system admin to decide exactly what level of access s/he wants to provide to a domainadmin role user. This allows him/her to create multiple end points of mgmt server as well, just like the user api server.
>
> Regards.
>
>>
>> Regards
>> Mice
>>
>> -----Original Message-----
>> From: Rohit Yadav [mailto:rohit.yadav@citrix.com]
>> Sent: Friday, December 07, 2012 4:07 AM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Arch Rework RFC: API refactoring updates
>>
>> Few updates;
>>
>> 1. Moved and classified all apis (except the anomalies, see other
>> email)
into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
>> 3. Further in each org.apache.cloudstack.api.{admin,user}.command
>> pkg,
the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
>> 4. Few usage related APIs could not be moved from cloud-server to
cloud-api, as they are tightly coupled with classes available in cloud-server.
>> 5. commands.properties is fixed.
>> 6. Checked, the api_refactoring merges fine with a minor merge
conflicts. The build on the branch is broken, would be fixed soon.
>>
>> I request that if you're adding any new class, please use the 
>> namespace
org.apache.cloudstack.
>>
>> Road ahead:
>> - Because of 1,2,3; there would be two classes of API server. The 
>> first
one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;
>>
>> user API server would use only the org.apache.cloudstack.api.user
artifact admin API server would use both org.apache.cloudstack.api.admin and org.apache.cloudstack.api.user Based on the arch. rework docs, ppts, talks, Alex's idea was to separate api based on security level would give ultimate isolation.
>>
>> - Separate out cloud-api, and make it run as a separate web app like
awsapi.
>> - Annotations and separation of API based on api affinity would help
automate apidoc generation, api discovery over an api endpoint, so clients (UI or cloudmonkey etc.) would be able to discover
>> - The commands.properties syntax is horrible, I want to get rid of 
>> the
evil syntax by having an apiname annotation for all API Cmd classes and the API server would be able to load this info during runtime. In commands.properties, we should be just able to set policy for user role for each api, if apiname is not declared it's blacklisted. It becomes tricky with plugins. I don't know how to get it right the first time, but let's try.
>> - ACL and security checking at API layer, the requests would be just
passed to (multiple) cloud-engine which won't handle them, as that will only orchestrate.
>> - Roles would be decided from commands.properties. Multiple API 
>> servers
can be run with different combination of rules in commands.properties. This would allow separation of policy from mechanism, and multiple roles and end points. For example, an admin can create http routers for these api server,
so: <url>/hr, <url>/finance, <url>/marketting can be proxies to different user api servers with different set of whitelisted apis.
>>
>> Comments, suggestions, flames?
>>
>> Regards.
>
>

--
Sent from Gmail Mobile

Re: Arch Rework RFC: API refactoring updates

[Rohit Yadav <ro...@citrix.com>]

From the last mail on this thread (http://markmail.org/thread/wueimbh6tyas4227), one can clearly see I did not get it the first time, so last week I read a lot of code in cloud-api and cloud-server to understand what's happening, how api server starts, loads stuff, handles requests etc. and what to do about it. I think I can share what I understand now and what I've been dong since Thursday; also so if anyone wants to chip in;

So, why we are doing, what we're doing in api_refactoring:

0. For better apidoc generation. Aggregate apis with src packages around common entity for developers.
1. Decouple role based api call verification using a plugin/adapter.
2. Decouple tightly coupled database and acl validations from service layer to api layer using @ACL, @Validator. They can be adapters too.
3. Using CommandType.UUID to annotate fields that are uuids from over the wire requests and use this annotations to convert them from string to internal ID. entityType helps us know the Response class whose @Entity helps us know the db table (impl. in EntityManagerImpl). Useful for @Validator to validate db/range validations as well.
4. Remove bloat, decouple policy from mechanism (using plugins or adapters), data from code wherever possible and use OOP styled coding, remove iterative/primitive C like code wherever possible.
5. FS: https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+API+refactoring

How it's done: (NOTE: Most of it would be done in all Cmd classes, only @Entity in Response classes)

0. Remove IdentityMapper and do 1.
1. Fix/add entityType to a Response.class, in @Parameter. Goto 2.
2. For every Response class that was added in the entityType field in @Parameter in 1., add @Entity(). The idea is that from this @Entity we would know the interface through which we can know the db table. For. example for UserResponse, we should annotate with @Entity(User.class) and which is an interfaces implemented by UserVO.java through which we know it has @Table(name="user"). This is useful to convert the over-the-wire requests that have uuids to convert them to internal ids.

3. In @Parameter, if the entity would receive a uuid, change it's type to CommandType.UUID (recently introduced) and collectionType (if it's collection of Long, or basically list of uuids for example) too. This annotation will help us generate docs so users/developers would know what params is actually uuid. (See 1-3, to better understand why we're doing this)

4. @ACL annotation for all params in a cmd class whose access control for the caller user has to be checked. If the parameter is a Map (generally would be a Long that is UUID type), a map or dictionary has a key and a value, we need to specify which one needs checking. For example:

@ACL(checkValueAccess=true)
@Parameter(bla bla bla, entityType={class1Response.class,class2Response.class})
private Map IpToNetworkList;

5. Add @Validator (defined as @validate in FS) to annotate which field in Cmd needs to be be validated and using which validators (current we've DBEntityValidator.class and RandgeValidator.class). This is work in progress, won't work as of now. You may leave @Validator for now. Example:
@ACL
@Validator(validtors={DBEntityValidator.class})
@Parameter(bla bla bla, type=CommandType.UUID, entityType={class1Response.class,class2Response.class})
private Long domainId;

At present, all apis which are queried by passing a uuid (for ex. list apis with uuid param) are broken on purpose (so it gets attention :). Work in progress.

Regards.

On 09-Dec-2012, at 5:41 PM, Mice Xia <mi...@tcloudcomputing.com> wrote:

> Alex,
> 
> Got it, very clear now and hope this clarification will be updated on the wiki.
> 
> Regards
> Mice
> -----Original Message-----
> From: Alex Huang [mailto:Alex.Huang@citrix.com] 
> Sent: Saturday, December 08, 2012 8:47 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: Arch Rework RFC: API refactoring updates
> 
> Mice,
> 
> Please take a look at my email reply to Dave.  There's no implied ACL in this separation.
> 
> --Alex
> 
>> -----Original Message-----
>> From: Mice Xia [mailto:mice_xia@tcloudcomputing.com]
>> Sent: Friday, December 07, 2012 4:36 PM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: RE: Arch Rework RFC: API refactoring updates
>> 
>> Fang,
>> 
>> I vote for the second way, to keep it simple and clear, each API 
>> server has the same binary artifacts and contains all API 
>> implementations, access control is based on RBAC/ACL, and 
>> blacklist(white List) can be defined on a per API server basis. And 
>> perhaps we don’t need to explicitly package classes by roles ( or security level), this seems to hardcode APIs' ACL in the code level.
>> 
>> My 2 cent.
>> 
>> -Mice
>> 
>> -----Original Message-----
>> From: Fang Wang [mailto:fang.wang@citrix.com]
>> Sent: Saturday, December 08, 2012 7:44 AM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: RE: Arch Rework RFC: API refactoring updates
>> 
>> Hi Mice,
>> 
>> For now, we just separate the API into 2 roles: user and admin.
>> The admin can be separated further into root admin, domain admin, etc.
>> Further dynamic roles can be added.
>> Once the framework is there, we can further separate the admin roles.
>> 
>> Per your question, there can be 2 ways to handle the domain admin requests:
>> - one is like you mentioned, sysadmin has separate api servers to 
>> handle user and domain api requests separately.
>> - Another way is to deploy one api server, and the api server does the 
>> role base access check. So user cann't access the domain Admin apis., 
>> the access would be blocked and denied.  The advantage of this method is the flexibility.
>> E.g, add another new domain, or a new category admin, we can 
>> dynamically deploy and apply the roles.
>> 
>> 
>> Thanks,
>> -Fang
>> 
>> -----Original Message-----
>> From: Mice Xia [mailto:weiran.xia1@gmail.com]
>> Sent: Thursday, December 06, 2012 5:15 PM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Re: Arch Rework RFC: API refactoring updates
>> 
>> rohit,
>> 
>> i asked this question because requests from domain admins are usually 
>> from outside of datacenter for a public cloud.
>> 
>> so artifact built from user package can be directly deployed as a user 
>> api server and serve user requests. but to serve domain admin request, 
>> sysadmin has to prepare another api server with another set of apis, 
>> which is a sub set of admin package, is this correct?
>> 
>> sent from phone, sorry for typos.
>> 
>> mice
>> 
>> 在 2012年12月7日星期五，Rohit Yadav <ro...@citrix.com> 写道：
>>> 
>>> On 06-Dec-2012, at 4:29 PM, Mice Xia <mi...@tcloudcomputing.com>
>> wrote:
>>> 
>>>> [quote]
>>>> 1. Moved and classified all apis (except the anomalies, see other
>>>> email)
>> into namespace org.apache.cloudstack.api 2. There are two packages, 
>> admin and user in which the APIs are grouped based on security level. 
>> Note; APIs are not grouped based on roles.
>>>> - Because of 1,2,3; there would be two classes of API server. The 
>>>> first
>> one would be available for users, the user API server which would 
>> handle requests from network outside the datacenter which Alex refers 
>> to as "over the wire" requests. The second one would be available for 
>> sysadmins, the mgmt API server which would handle requests from 
>> admins/sysadmins within the datacenter;
>>>> [/quote]
>>>> 
>>>> I haven’t seen the codes, just one question.
>>>> If the intention is to deploy API server separately based on the
>> security level, where are APIs enabled for domain-admin located? In 
>> the user package or in the admin package?
>>> 
>>> The admin package. For domain-admin etc. the sysadmin would run mgmt
>> servers with different sets of apis s/he wants to provide for a domain admin.
>> There won't be any pre configured set of apis, this would allow a 
>> system admin to decide exactly what level of access s/he wants to 
>> provide to a domainadmin role user. This allows him/her to create 
>> multiple end points of mgmt server as well, just like the user api server.
>>> 
>>> Regards.
>>> 
>>>> 
>>>> Regards
>>>> Mice
>>>> 
>>>> -----Original Message-----
>>>> From: Rohit Yadav [mailto:rohit.yadav@citrix.com]
>>>> Sent: Friday, December 07, 2012 4:07 AM
>>>> To: cloudstack-dev@incubator.apache.org
>>>> Subject: Arch Rework RFC: API refactoring updates
>>>> 
>>>> Few updates;
>>>> 
>>>> 1. Moved and classified all apis (except the anomalies, see other
>>>> email)
>> into namespace org.apache.cloudstack.api 2. There are two packages, 
>> admin and user in which the APIs are grouped based on security level. 
>> Note; APIs are not grouped based on roles.
>>>> 3. Further in each org.apache.cloudstack.api.{admin,user}.command
>>>> pkg,
>> the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
>>>> 4. Few usage related APIs could not be moved from cloud-server to
>> cloud-api, as they are tightly coupled with classes available in cloud-server.
>>>> 5. commands.properties is fixed.
>>>> 6. Checked, the api_refactoring merges fine with a minor merge
>> conflicts. The build on the branch is broken, would be fixed soon.
>>>> 
>>>> I request that if you're adding any new class, please use the 
>>>> namespace
>> org.apache.cloudstack.
>>>> 
>>>> Road ahead:
>>>> - Because of 1,2,3; there would be two classes of API server. The 
>>>> first
>> one would be available for users, the user API server which would 
>> handle requests from network outside the datacenter which Alex refers 
>> to as "over the wire" requests. The second one would be available for 
>> sysadmins, the mgmt API server which would handle requests from 
>> admins/sysadmins within the datacenter;
>>>> 
>>>> user API server would use only the org.apache.cloudstack.api.user
>> artifact admin API server would use both 
>> org.apache.cloudstack.api.admin and org.apache.cloudstack.api.user 
>> Based on the arch. rework docs, ppts, talks, Alex's idea was to 
>> separate api based on security level would give ultimate isolation.
>>>> 
>>>> - Separate out cloud-api, and make it run as a separate web app 
>>>> like
>> awsapi.
>>>> - Annotations and separation of API based on api affinity would 
>>>> help
>> automate apidoc generation, api discovery over an api endpoint, so 
>> clients (UI or cloudmonkey etc.) would be able to discover
>>>> - The commands.properties syntax is horrible, I want to get rid of 
>>>> the
>> evil syntax by having an apiname annotation for all API Cmd classes 
>> and the API server would be able to load this info during runtime. In 
>> commands.properties, we should be just able to set policy for user 
>> role for each api, if apiname is not declared it's blacklisted. It 
>> becomes tricky with plugins. I don't know how to get it right the first time, but let's try.
>>>> - ACL and security checking at API layer, the requests would be 
>>>> just
>> passed to (multiple) cloud-engine which won't handle them, as that 
>> will only orchestrate.
>>>> - Roles would be decided from commands.properties. Multiple API 
>>>> servers
>> can be run with different combination of rules in commands.properties. 
>> This would allow separation of policy from mechanism, and multiple 
>> roles and end points. For example, an admin can create http routers 
>> for these api server,
>> so: <url>/hr, <url>/finance, <url>/marketting can be proxies to 
>> different user api servers with different set of whitelisted apis.
>>>> 
>>>> Comments, suggestions, flames?
>>>> 
>>>> Regards.
>>> 
>>> 
>> 
>> --
>> Sent from Gmail Mobile

RE: Arch Rework RFC: API refactoring updates

[Mice Xia <mi...@tcloudcomputing.com>]

Alex,

Got it, very clear now and hope this clarification will be updated on the wiki.

Regards
Mice
-----Original Message-----
From: Alex Huang [mailto:Alex.Huang@citrix.com] 
Sent: Saturday, December 08, 2012 8:47 AM
To: cloudstack-dev@incubator.apache.org
Subject: RE: Arch Rework RFC: API refactoring updates

Mice,

Please take a look at my email reply to Dave.  There's no implied ACL in this separation.

--Alex

> -----Original Message-----
> From: Mice Xia [mailto:mice_xia@tcloudcomputing.com]
> Sent: Friday, December 07, 2012 4:36 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: Arch Rework RFC: API refactoring updates
> 
> Fang,
> 
> I vote for the second way, to keep it simple and clear, each API 
> server has the same binary artifacts and contains all API 
> implementations, access control is based on RBAC/ACL, and 
> blacklist(white List) can be defined on a per API server basis. And 
> perhaps we don’t need to explicitly package classes by roles ( or security level), this seems to hardcode APIs' ACL in the code level.
> 
> My 2 cent.
> 
> -Mice
> 
> -----Original Message-----
> From: Fang Wang [mailto:fang.wang@citrix.com]
> Sent: Saturday, December 08, 2012 7:44 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: Arch Rework RFC: API refactoring updates
> 
> Hi Mice,
> 
> For now, we just separate the API into 2 roles: user and admin.
> The admin can be separated further into root admin, domain admin, etc.
> Further dynamic roles can be added.
> Once the framework is there, we can further separate the admin roles.
> 
> Per your question, there can be 2 ways to handle the domain admin requests:
> - one is like you mentioned, sysadmin has separate api servers to 
> handle user and domain api requests separately.
> - Another way is to deploy one api server, and the api server does the 
> role base access check. So user cann't access the domain Admin apis., 
> the access would be blocked and denied.  The advantage of this method is the flexibility.
> E.g, add another new domain, or a new category admin, we can 
> dynamically deploy and apply the roles.
> 
> 
> Thanks,
> -Fang
> 
> -----Original Message-----
> From: Mice Xia [mailto:weiran.xia1@gmail.com]
> Sent: Thursday, December 06, 2012 5:15 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: Arch Rework RFC: API refactoring updates
> 
> rohit,
> 
> i asked this question because requests from domain admins are usually 
> from outside of datacenter for a public cloud.
> 
> so artifact built from user package can be directly deployed as a user 
> api server and serve user requests. but to serve domain admin request, 
> sysadmin has to prepare another api server with another set of apis, 
> which is a sub set of admin package, is this correct?
> 
> sent from phone, sorry for typos.
> 
> mice
> 
> 在 2012年12月7日星期五，Rohit Yadav <ro...@citrix.com> 写道：
> >
> > On 06-Dec-2012, at 4:29 PM, Mice Xia <mi...@tcloudcomputing.com>
> wrote:
> >
> >> [quote]
> >> 1. Moved and classified all apis (except the anomalies, see other
> >> email)
> into namespace org.apache.cloudstack.api 2. There are two packages, 
> admin and user in which the APIs are grouped based on security level. 
> Note; APIs are not grouped based on roles.
> >> - Because of 1,2,3; there would be two classes of API server. The 
> >> first
> one would be available for users, the user API server which would 
> handle requests from network outside the datacenter which Alex refers 
> to as "over the wire" requests. The second one would be available for 
> sysadmins, the mgmt API server which would handle requests from 
> admins/sysadmins within the datacenter;
> >> [/quote]
> >>
> >> I haven’t seen the codes, just one question.
> >> If the intention is to deploy API server separately based on the
> security level, where are APIs enabled for domain-admin located? In 
> the user package or in the admin package?
> >
> > The admin package. For domain-admin etc. the sysadmin would run mgmt
> servers with different sets of apis s/he wants to provide for a domain admin.
> There won't be any pre configured set of apis, this would allow a 
> system admin to decide exactly what level of access s/he wants to 
> provide to a domainadmin role user. This allows him/her to create 
> multiple end points of mgmt server as well, just like the user api server.
> >
> > Regards.
> >
> >>
> >> Regards
> >> Mice
> >>
> >> -----Original Message-----
> >> From: Rohit Yadav [mailto:rohit.yadav@citrix.com]
> >> Sent: Friday, December 07, 2012 4:07 AM
> >> To: cloudstack-dev@incubator.apache.org
> >> Subject: Arch Rework RFC: API refactoring updates
> >>
> >> Few updates;
> >>
> >> 1. Moved and classified all apis (except the anomalies, see other
> >> email)
> into namespace org.apache.cloudstack.api 2. There are two packages, 
> admin and user in which the APIs are grouped based on security level. 
> Note; APIs are not grouped based on roles.
> >> 3. Further in each org.apache.cloudstack.api.{admin,user}.command
> >> pkg,
> the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
> >> 4. Few usage related APIs could not be moved from cloud-server to
> cloud-api, as they are tightly coupled with classes available in cloud-server.
> >> 5. commands.properties is fixed.
> >> 6. Checked, the api_refactoring merges fine with a minor merge
> conflicts. The build on the branch is broken, would be fixed soon.
> >>
> >> I request that if you're adding any new class, please use the 
> >> namespace
> org.apache.cloudstack.
> >>
> >> Road ahead:
> >> - Because of 1,2,3; there would be two classes of API server. The 
> >> first
> one would be available for users, the user API server which would 
> handle requests from network outside the datacenter which Alex refers 
> to as "over the wire" requests. The second one would be available for 
> sysadmins, the mgmt API server which would handle requests from 
> admins/sysadmins within the datacenter;
> >>
> >> user API server would use only the org.apache.cloudstack.api.user
> artifact admin API server would use both 
> org.apache.cloudstack.api.admin and org.apache.cloudstack.api.user 
> Based on the arch. rework docs, ppts, talks, Alex's idea was to 
> separate api based on security level would give ultimate isolation.
> >>
> >> - Separate out cloud-api, and make it run as a separate web app 
> >> like
> awsapi.
> >> - Annotations and separation of API based on api affinity would 
> >> help
> automate apidoc generation, api discovery over an api endpoint, so 
> clients (UI or cloudmonkey etc.) would be able to discover
> >> - The commands.properties syntax is horrible, I want to get rid of 
> >> the
> evil syntax by having an apiname annotation for all API Cmd classes 
> and the API server would be able to load this info during runtime. In 
> commands.properties, we should be just able to set policy for user 
> role for each api, if apiname is not declared it's blacklisted. It 
> becomes tricky with plugins. I don't know how to get it right the first time, but let's try.
> >> - ACL and security checking at API layer, the requests would be 
> >> just
> passed to (multiple) cloud-engine which won't handle them, as that 
> will only orchestrate.
> >> - Roles would be decided from commands.properties. Multiple API 
> >> servers
> can be run with different combination of rules in commands.properties. 
> This would allow separation of policy from mechanism, and multiple 
> roles and end points. For example, an admin can create http routers 
> for these api server,
> so: <url>/hr, <url>/finance, <url>/marketting can be proxies to 
> different user api servers with different set of whitelisted apis.
> >>
> >> Comments, suggestions, flames?
> >>
> >> Regards.
> >
> >
> 
> --
> Sent from Gmail Mobile

RE: Arch Rework RFC: API refactoring updates

[Alex Huang <Al...@citrix.com>]

Mice,

Please take a look at my email reply to Dave.  There's no implied ACL in this separation.

--Alex

> -----Original Message-----
> From: Mice Xia [mailto:mice_xia@tcloudcomputing.com]
> Sent: Friday, December 07, 2012 4:36 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: Arch Rework RFC: API refactoring updates
> 
> Fang,
> 
> I vote for the second way, to keep it simple and clear, each API server has
> the same binary artifacts and contains all API implementations, access control
> is based on RBAC/ACL, and blacklist(white List) can be defined on a per API
> server basis. And perhaps we don’t need to explicitly package classes by
> roles ( or security level), this seems to hardcode APIs' ACL in the code level.
> 
> My 2 cent.
> 
> -Mice
> 
> -----Original Message-----
> From: Fang Wang [mailto:fang.wang@citrix.com]
> Sent: Saturday, December 08, 2012 7:44 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: Arch Rework RFC: API refactoring updates
> 
> Hi Mice,
> 
> For now, we just separate the API into 2 roles: user and admin.
> The admin can be separated further into root admin, domain admin, etc.
> Further dynamic roles can be added.
> Once the framework is there, we can further separate the admin roles.
> 
> Per your question, there can be 2 ways to handle the domain admin requests:
> - one is like you mentioned, sysadmin has separate api servers to handle user
> and domain api requests separately.
> - Another way is to deploy one api server, and the api server does the role
> base access check. So user cann't access the domain Admin apis., the access
> would be blocked and denied.  The advantage of this method is the flexibility.
> E.g, add another new domain, or a new category admin, we can dynamically
> deploy and apply the roles.
> 
> 
> Thanks,
> -Fang
> 
> -----Original Message-----
> From: Mice Xia [mailto:weiran.xia1@gmail.com]
> Sent: Thursday, December 06, 2012 5:15 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: Arch Rework RFC: API refactoring updates
> 
> rohit,
> 
> i asked this question because requests from domain admins are usually from
> outside of datacenter for a public cloud.
> 
> so artifact built from user package can be directly deployed as a user api
> server and serve user requests. but to serve domain admin request,
> sysadmin has to prepare another api server with another set of apis, which is
> a sub set of admin package, is this correct?
> 
> sent from phone, sorry for typos.
> 
> mice
> 
> 在 2012年12月7日星期五，Rohit Yadav <ro...@citrix.com> 写道：
> >
> > On 06-Dec-2012, at 4:29 PM, Mice Xia <mi...@tcloudcomputing.com>
> wrote:
> >
> >> [quote]
> >> 1. Moved and classified all apis (except the anomalies, see other
> >> email)
> into namespace org.apache.cloudstack.api 2. There are two packages, admin
> and user in which the APIs are grouped based on security level. Note; APIs
> are not grouped based on roles.
> >> - Because of 1,2,3; there would be two classes of API server. The
> >> first
> one would be available for users, the user API server which would handle
> requests from network outside the datacenter which Alex refers to as "over
> the wire" requests. The second one would be available for sysadmins, the
> mgmt API server which would handle requests from admins/sysadmins
> within the datacenter;
> >> [/quote]
> >>
> >> I haven’t seen the codes, just one question.
> >> If the intention is to deploy API server separately based on the
> security level, where are APIs enabled for domain-admin located? In the user
> package or in the admin package?
> >
> > The admin package. For domain-admin etc. the sysadmin would run mgmt
> servers with different sets of apis s/he wants to provide for a domain admin.
> There won't be any pre configured set of apis, this would allow a system
> admin to decide exactly what level of access s/he wants to provide to a
> domainadmin role user. This allows him/her to create multiple end points of
> mgmt server as well, just like the user api server.
> >
> > Regards.
> >
> >>
> >> Regards
> >> Mice
> >>
> >> -----Original Message-----
> >> From: Rohit Yadav [mailto:rohit.yadav@citrix.com]
> >> Sent: Friday, December 07, 2012 4:07 AM
> >> To: cloudstack-dev@incubator.apache.org
> >> Subject: Arch Rework RFC: API refactoring updates
> >>
> >> Few updates;
> >>
> >> 1. Moved and classified all apis (except the anomalies, see other
> >> email)
> into namespace org.apache.cloudstack.api 2. There are two packages, admin
> and user in which the APIs are grouped based on security level. Note; APIs
> are not grouped based on roles.
> >> 3. Further in each org.apache.cloudstack.api.{admin,user}.command
> >> pkg,
> the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
> >> 4. Few usage related APIs could not be moved from cloud-server to
> cloud-api, as they are tightly coupled with classes available in cloud-server.
> >> 5. commands.properties is fixed.
> >> 6. Checked, the api_refactoring merges fine with a minor merge
> conflicts. The build on the branch is broken, would be fixed soon.
> >>
> >> I request that if you're adding any new class, please use the
> >> namespace
> org.apache.cloudstack.
> >>
> >> Road ahead:
> >> - Because of 1,2,3; there would be two classes of API server. The
> >> first
> one would be available for users, the user API server which would handle
> requests from network outside the datacenter which Alex refers to as "over
> the wire" requests. The second one would be available for sysadmins, the
> mgmt API server which would handle requests from admins/sysadmins
> within the datacenter;
> >>
> >> user API server would use only the org.apache.cloudstack.api.user
> artifact admin API server would use both org.apache.cloudstack.api.admin
> and org.apache.cloudstack.api.user Based on the arch. rework docs, ppts,
> talks, Alex's idea was to separate api based on security level would give
> ultimate isolation.
> >>
> >> - Separate out cloud-api, and make it run as a separate web app like
> awsapi.
> >> - Annotations and separation of API based on api affinity would help
> automate apidoc generation, api discovery over an api endpoint, so clients
> (UI or cloudmonkey etc.) would be able to discover
> >> - The commands.properties syntax is horrible, I want to get rid of
> >> the
> evil syntax by having an apiname annotation for all API Cmd classes and the
> API server would be able to load this info during runtime. In
> commands.properties, we should be just able to set policy for user role for
> each api, if apiname is not declared it's blacklisted. It becomes tricky with
> plugins. I don't know how to get it right the first time, but let's try.
> >> - ACL and security checking at API layer, the requests would be just
> passed to (multiple) cloud-engine which won't handle them, as that will only
> orchestrate.
> >> - Roles would be decided from commands.properties. Multiple API
> >> servers
> can be run with different combination of rules in commands.properties. This
> would allow separation of policy from mechanism, and multiple roles and end
> points. For example, an admin can create http routers for these api server,
> so: <url>/hr, <url>/finance, <url>/marketting can be proxies to different user
> api servers with different set of whitelisted apis.
> >>
> >> Comments, suggestions, flames?
> >>
> >> Regards.
> >
> >
> 
> --
> Sent from Gmail Mobile

RE: Arch Rework RFC: API refactoring updates

[Mice Xia <mi...@tcloudcomputing.com>]

Fang,

I vote for the second way, to keep it simple and clear, each API server has the same binary artifacts and contains all API implementations, access control is based on RBAC/ACL, and blacklist(white List) can be defined on a per API server basis. And perhaps we don’t need to explicitly package classes by roles ( or security level), this seems to hardcode APIs' ACL in the code level.

My 2 cent.

-Mice

-----Original Message-----
From: Fang Wang [mailto:fang.wang@citrix.com] 
Sent: Saturday, December 08, 2012 7:44 AM
To: cloudstack-dev@incubator.apache.org
Subject: RE: Arch Rework RFC: API refactoring updates

Hi Mice,

For now, we just separate the API into 2 roles: user and admin. 
The admin can be separated further into root admin, domain admin, etc. Further dynamic roles can be added. 
Once the framework is there, we can further separate the admin roles. 

Per your question, there can be 2 ways to handle the domain admin requests:
- one is like you mentioned, sysadmin has separate api servers to handle user and domain api requests separately.
- Another way is to deploy one api server, and the api server does the role base access check. So user cann't access the domain Admin apis., the access would be blocked and denied.  The advantage of this method is the flexibility.
E.g, add another new domain, or a new category admin, we can dynamically deploy and apply the roles. 


Thanks,
-Fang

-----Original Message-----
From: Mice Xia [mailto:weiran.xia1@gmail.com]
Sent: Thursday, December 06, 2012 5:15 PM
To: cloudstack-dev@incubator.apache.org
Subject: Re: Arch Rework RFC: API refactoring updates

rohit,

i asked this question because requests from domain admins are usually from outside of datacenter for a public cloud.

so artifact built from user package can be directly deployed as a user api server and serve user requests. but to serve domain admin request, sysadmin has to prepare another api server with another set of apis, which is a sub set of admin package, is this correct?

sent from phone, sorry for typos.

mice

在 2012年12月7日星期五，Rohit Yadav <ro...@citrix.com> 写道：
>
> On 06-Dec-2012, at 4:29 PM, Mice Xia <mi...@tcloudcomputing.com> wrote:
>
>> [quote]
>> 1. Moved and classified all apis (except the anomalies, see other
>> email)
into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
>> - Because of 1,2,3; there would be two classes of API server. The 
>> first
one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;
>> [/quote]
>>
>> I haven’t seen the codes, just one question.
>> If the intention is to deploy API server separately based on the
security level, where are APIs enabled for domain-admin located? In the user package or in the admin package?
>
> The admin package. For domain-admin etc. the sysadmin would run mgmt
servers with different sets of apis s/he wants to provide for a domain admin. There won't be any pre configured set of apis, this would allow a system admin to decide exactly what level of access s/he wants to provide to a domainadmin role user. This allows him/her to create multiple end points of mgmt server as well, just like the user api server.
>
> Regards.
>
>>
>> Regards
>> Mice
>>
>> -----Original Message-----
>> From: Rohit Yadav [mailto:rohit.yadav@citrix.com]
>> Sent: Friday, December 07, 2012 4:07 AM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Arch Rework RFC: API refactoring updates
>>
>> Few updates;
>>
>> 1. Moved and classified all apis (except the anomalies, see other
>> email)
into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
>> 3. Further in each org.apache.cloudstack.api.{admin,user}.command
>> pkg,
the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
>> 4. Few usage related APIs could not be moved from cloud-server to
cloud-api, as they are tightly coupled with classes available in cloud-server.
>> 5. commands.properties is fixed.
>> 6. Checked, the api_refactoring merges fine with a minor merge
conflicts. The build on the branch is broken, would be fixed soon.
>>
>> I request that if you're adding any new class, please use the 
>> namespace
org.apache.cloudstack.
>>
>> Road ahead:
>> - Because of 1,2,3; there would be two classes of API server. The 
>> first
one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;
>>
>> user API server would use only the org.apache.cloudstack.api.user
artifact admin API server would use both org.apache.cloudstack.api.admin and org.apache.cloudstack.api.user Based on the arch. rework docs, ppts, talks, Alex's idea was to separate api based on security level would give ultimate isolation.
>>
>> - Separate out cloud-api, and make it run as a separate web app like
awsapi.
>> - Annotations and separation of API based on api affinity would help
automate apidoc generation, api discovery over an api endpoint, so clients (UI or cloudmonkey etc.) would be able to discover
>> - The commands.properties syntax is horrible, I want to get rid of 
>> the
evil syntax by having an apiname annotation for all API Cmd classes and the API server would be able to load this info during runtime. In commands.properties, we should be just able to set policy for user role for each api, if apiname is not declared it's blacklisted. It becomes tricky with plugins. I don't know how to get it right the first time, but let's try.
>> - ACL and security checking at API layer, the requests would be just
passed to (multiple) cloud-engine which won't handle them, as that will only orchestrate.
>> - Roles would be decided from commands.properties. Multiple API 
>> servers
can be run with different combination of rules in commands.properties. This would allow separation of policy from mechanism, and multiple roles and end points. For example, an admin can create http routers for these api server,
so: <url>/hr, <url>/finance, <url>/marketting can be proxies to different user api servers with different set of whitelisted apis.
>>
>> Comments, suggestions, flames?
>>
>> Regards.
>
>

--
Sent from Gmail Mobile

RE: Arch Rework RFC: API refactoring updates

[Fang Wang <fa...@citrix.com>]

Hi Mice,

For now, we just separate the API into 2 roles: user and admin. 
The admin can be separated further into root admin, domain admin, etc. Further dynamic roles can be added. 
Once the framework is there, we can further separate the admin roles. 

Per your question, there can be 2 ways to handle the domain admin requests:
- one is like you mentioned, sysadmin has separate api servers to
handle user and domain api requests separately.
- Another way is to deploy one api server, and the api server does the role base access check. So user cann't access the domain
Admin apis., the access would be blocked and denied.  The advantage of this method is the flexibility.
E.g, add another new domain, or a new category admin, we can dynamically deploy and apply the roles. 


Thanks,
-Fang

-----Original Message-----
From: Mice Xia [mailto:weiran.xia1@gmail.com] 
Sent: Thursday, December 06, 2012 5:15 PM
To: cloudstack-dev@incubator.apache.org
Subject: Re: Arch Rework RFC: API refactoring updates

rohit,

i asked this question because requests from domain admins are usually from outside of datacenter for a public cloud.

so artifact built from user package can be directly deployed as a user api server and serve user requests. but to serve domain admin request, sysadmin has to prepare another api server with another set of apis, which is a sub set of admin package, is this correct?

sent from phone, sorry for typos.

mice

在 2012年12月7日星期五，Rohit Yadav <ro...@citrix.com> 写道：
>
> On 06-Dec-2012, at 4:29 PM, Mice Xia <mi...@tcloudcomputing.com> wrote:
>
>> [quote]
>> 1. Moved and classified all apis (except the anomalies, see other 
>> email)
into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
>> - Because of 1,2,3; there would be two classes of API server. The 
>> first
one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;
>> [/quote]
>>
>> I haven’t seen the codes, just one question.
>> If the intention is to deploy API server separately based on the
security level, where are APIs enabled for domain-admin located? In the user package or in the admin package?
>
> The admin package. For domain-admin etc. the sysadmin would run mgmt
servers with different sets of apis s/he wants to provide for a domain admin. There won't be any pre configured set of apis, this would allow a system admin to decide exactly what level of access s/he wants to provide to a domainadmin role user. This allows him/her to create multiple end points of mgmt server as well, just like the user api server.
>
> Regards.
>
>>
>> Regards
>> Mice
>>
>> -----Original Message-----
>> From: Rohit Yadav [mailto:rohit.yadav@citrix.com]
>> Sent: Friday, December 07, 2012 4:07 AM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Arch Rework RFC: API refactoring updates
>>
>> Few updates;
>>
>> 1. Moved and classified all apis (except the anomalies, see other 
>> email)
into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
>> 3. Further in each org.apache.cloudstack.api.{admin,user}.command 
>> pkg,
the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
>> 4. Few usage related APIs could not be moved from cloud-server to
cloud-api, as they are tightly coupled with classes available in cloud-server.
>> 5. commands.properties is fixed.
>> 6. Checked, the api_refactoring merges fine with a minor merge
conflicts. The build on the branch is broken, would be fixed soon.
>>
>> I request that if you're adding any new class, please use the 
>> namespace
org.apache.cloudstack.
>>
>> Road ahead:
>> - Because of 1,2,3; there would be two classes of API server. The 
>> first
one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;
>>
>> user API server would use only the org.apache.cloudstack.api.user
artifact admin API server would use both org.apache.cloudstack.api.admin and org.apache.cloudstack.api.user Based on the arch. rework docs, ppts, talks, Alex's idea was to separate api based on security level would give ultimate isolation.
>>
>> - Separate out cloud-api, and make it run as a separate web app like
awsapi.
>> - Annotations and separation of API based on api affinity would help
automate apidoc generation, api discovery over an api endpoint, so clients (UI or cloudmonkey etc.) would be able to discover
>> - The commands.properties syntax is horrible, I want to get rid of 
>> the
evil syntax by having an apiname annotation for all API Cmd classes and the API server would be able to load this info during runtime. In commands.properties, we should be just able to set policy for user role for each api, if apiname is not declared it's blacklisted. It becomes tricky with plugins. I don't know how to get it right the first time, but let's try.
>> - ACL and security checking at API layer, the requests would be just
passed to (multiple) cloud-engine which won't handle them, as that will only orchestrate.
>> - Roles would be decided from commands.properties. Multiple API 
>> servers
can be run with different combination of rules in commands.properties. This would allow separation of policy from mechanism, and multiple roles and end points. For example, an admin can create http routers for these api server,
so: <url>/hr, <url>/finance, <url>/marketting can be proxies to different user api servers with different set of whitelisted apis.
>>
>> Comments, suggestions, flames?
>>
>> Regards.
>
>

--
Sent from Gmail Mobile

RE: Arch Rework RFC: API refactoring updates

[Fang Wang <fa...@citrix.com>]

Alex, thanks for the clarification. 

User and admin commands separation is aimed for developers and document generation, for the 
current refactoring  plan. 

There is some benefits to physically separate user and admin API servers in the future, is it something we may consider
It? Because it can grant different levels of service and resource allocation to handle admin API commands.
Otherwise the end users and admin users still have the same end point access. 

> 3. role based access is actually going to be delegated to cloud-access to check. There is no a separate command class per role problem so there's no maintenance problem.
For the access-check, I think the API server will do it, as well as the role based ACL check.  

Thanks,
-Fang


-----Original Message-----
From: Alex Huang [mailto:Alex.Huang@citrix.com] 
Sent: Friday, December 07, 2012 4:45 PM
To: cloudstack-dev@incubator.apache.org
Subject: RE: Arch Rework RFC: API refactoring updates

I think there's misunderstanding here.

1. Separation for user and admin commands is done for developers.  Because current CloudStack mixes the options available only to admins with the options available to end users, we see problems with people accessing things that they shouldn't be able to access.  This is not intended as a role-based command access.  This is to force developers to think about what they are putting in.

2. CloudStack actually allows commands to be packaged as a set.  That's done in a text file in commands.properties.  So domain admin commands mixed in with end user commands in a single end point is not a problem.  It is not determined by the packaging of the java files.

3. role based access is actually going to be delegated to cloud-access to check. There is no a separate command class per role problem so there's no maintenance problem.

--Alex

> -----Original Message-----
> From: David Nalley [mailto:david@gnsa.us]
> Sent: Friday, December 07, 2012 3:51 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: Arch Rework RFC: API refactoring updates
> 
> On Fri, Dec 7, 2012 at 1:41 PM, Rohit Yadav <ro...@citrix.com> wrote:
> >
> > On 06-Dec-2012, at 5:15 PM, Mice Xia <we...@gmail.com> wrote:
> >
> >> rohit,
> >>
> >> i asked this question because requests from domain admins are 
> >> usually
> from
> >> outside of datacenter for a public cloud.
> >>
> >> so artifact built from user package can be directly deployed as a 
> >> user api server and serve user requests. but to serve domain admin 
> >> request,
> sysadmin
> >> has to prepare another api server with another set of apis, which 
> >> is a sub set of admin package, is this correct?
> >
> > Thanks for the question Mice, I've shared one solution. Will get to 
> > you on
> this one.
> >
> 
> Are we sure this move is a good idea in general?
> I mean from a public cloud perspective I can understand the logic. In 
> a private cloud though, things tend to be far less boolean.
> This also seems incredibly inflexible for long term maintenance - 
> unless I am completely missing something.

RE: Arch Rework RFC: API refactoring updates

[Alex Huang <Al...@citrix.com>]

I think there's misunderstanding here.

1. Separation for user and admin commands is done for developers.  Because current CloudStack mixes the options available only to admins with the options available to end users, we see problems with people accessing things that they shouldn't be able to access.  This is not intended as a role-based command access.  This is to force developers to think about what they are putting in.

2. CloudStack actually allows commands to be packaged as a set.  That's done in a text file in commands.properties.  So domain admin commands mixed in with end user commands in a single end point is not a problem.  It is not determined by the packaging of the java files.

3. role based access is actually going to be delegated to cloud-access to check. There is no a separate command class per role problem so there's no maintenance problem.

--Alex

> -----Original Message-----
> From: David Nalley [mailto:david@gnsa.us]
> Sent: Friday, December 07, 2012 3:51 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: Arch Rework RFC: API refactoring updates
> 
> On Fri, Dec 7, 2012 at 1:41 PM, Rohit Yadav <ro...@citrix.com> wrote:
> >
> > On 06-Dec-2012, at 5:15 PM, Mice Xia <we...@gmail.com> wrote:
> >
> >> rohit,
> >>
> >> i asked this question because requests from domain admins are usually
> from
> >> outside of datacenter for a public cloud.
> >>
> >> so artifact built from user package can be directly deployed as a user api
> >> server and serve user requests. but to serve domain admin request,
> sysadmin
> >> has to prepare another api server with another set of apis, which is a sub
> >> set of admin package, is this correct?
> >
> > Thanks for the question Mice, I've shared one solution. Will get to you on
> this one.
> >
> 
> Are we sure this move is a good idea in general?
> I mean from a public cloud perspective I can understand the logic. In
> a private cloud though, things tend to be far less boolean.
> This also seems incredibly inflexible for long term maintenance -
> unless I am completely missing something.

Re: Arch Rework RFC: API refactoring updates

[David Nalley <da...@gnsa.us>]

On Fri, Dec 7, 2012 at 1:41 PM, Rohit Yadav <ro...@citrix.com> wrote:
>
> On 06-Dec-2012, at 5:15 PM, Mice Xia <we...@gmail.com> wrote:
>
>> rohit,
>>
>> i asked this question because requests from domain admins are usually from
>> outside of datacenter for a public cloud.
>>
>> so artifact built from user package can be directly deployed as a user api
>> server and serve user requests. but to serve domain admin request, sysadmin
>> has to prepare another api server with another set of apis, which is a sub
>> set of admin package, is this correct?
>
> Thanks for the question Mice, I've shared one solution. Will get to you on this one.
>

Are we sure this move is a good idea in general?
I mean from a public cloud perspective I can understand the logic. In
a private cloud though, things tend to be far less boolean.
This also seems incredibly inflexible for long term maintenance -
unless I am completely missing something.

Re: Arch Rework RFC: API refactoring updates

[Rohit Yadav <ro...@citrix.com>]

On 06-Dec-2012, at 5:15 PM, Mice Xia <we...@gmail.com> wrote:

> rohit,
> 
> i asked this question because requests from domain admins are usually from
> outside of datacenter for a public cloud.
> 
> so artifact built from user package can be directly deployed as a user api
> server and serve user requests. but to serve domain admin request, sysadmin
> has to prepare another api server with another set of apis, which is a sub
> set of admin package, is this correct?

Thanks for the question Mice, I've shared one solution. Will get to you on this one.

Regards.

> 
> sent from phone, sorry for typos.
> 
> mice
> 
> 在 2012年12月7日星期五，Rohit Yadav <ro...@citrix.com> 写道：
>> 
>> On 06-Dec-2012, at 4:29 PM, Mice Xia <mi...@tcloudcomputing.com> wrote:
>> 
>>> [quote]
>>> 1. Moved and classified all apis (except the anomalies, see other email)
> into namespace org.apache.cloudstack.api 2. There are two packages, admin
> and user in which the APIs are grouped based on security level. Note; APIs
> are not grouped based on roles.
>>> - Because of 1,2,3; there would be two classes of API server. The first
> one would be available for users, the user API server which would handle
> requests from network outside the datacenter which Alex refers to as "over
> the wire" requests. The second one would be available for sysadmins, the
> mgmt API server which would handle requests from admins/sysadmins within
> the datacenter;
>>> [/quote]
>>> 
>>> I haven’t seen the codes, just one question.
>>> If the intention is to deploy API server separately based on the
> security level, where are APIs enabled for domain-admin located? In the
> user package or in the admin package?
>> 
>> The admin package. For domain-admin etc. the sysadmin would run mgmt
> servers with different sets of apis s/he wants to provide for a domain
> admin. There won't be any pre configured set of apis, this would allow a
> system admin to decide exactly what level of access s/he wants to provide
> to a domainadmin role user. This allows him/her to create multiple end
> points of mgmt server as well, just like the user api server.
>> 
>> Regards.
>> 
>>> 
>>> Regards
>>> Mice
>>> 
>>> -----Original Message-----
>>> From: Rohit Yadav [mailto:rohit.yadav@citrix.com]
>>> Sent: Friday, December 07, 2012 4:07 AM
>>> To: cloudstack-dev@incubator.apache.org
>>> Subject: Arch Rework RFC: API refactoring updates
>>> 
>>> Few updates;
>>> 
>>> 1. Moved and classified all apis (except the anomalies, see other email)
> into namespace org.apache.cloudstack.api 2. There are two packages, admin
> and user in which the APIs are grouped based on security level. Note; APIs
> are not grouped based on roles.
>>> 3. Further in each org.apache.cloudstack.api.{admin,user}.command pkg,
> the APIs are grouped as per api affinity (for ex. all vm related ones in vm
> pkg).
>>> 4. Few usage related APIs could not be moved from cloud-server to
> cloud-api, as they are tightly coupled with classes available in
> cloud-server.
>>> 5. commands.properties is fixed.
>>> 6. Checked, the api_refactoring merges fine with a minor merge
> conflicts. The build on the branch is broken, would be fixed soon.
>>> 
>>> I request that if you're adding any new class, please use the namespace
> org.apache.cloudstack.
>>> 
>>> Road ahead:
>>> - Because of 1,2,3; there would be two classes of API server. The first
> one would be available for users, the user API server which would handle
> requests from network outside the datacenter which Alex refers to as "over
> the wire" requests. The second one would be available for sysadmins, the
> mgmt API server which would handle requests from admins/sysadmins within
> the datacenter;
>>> 
>>> user API server would use only the org.apache.cloudstack.api.user
> artifact admin API server would use both org.apache.cloudstack.api.admin
> and org.apache.cloudstack.api.user Based on the arch. rework docs, ppts,
> talks, Alex's idea was to separate api based on security level would give
> ultimate isolation.
>>> 
>>> - Separate out cloud-api, and make it run as a separate web app like
> awsapi.
>>> - Annotations and separation of API based on api affinity would help
> automate apidoc generation, api discovery over an api endpoint, so clients
> (UI or cloudmonkey etc.) would be able to discover
>>> - The commands.properties syntax is horrible, I want to get rid of the
> evil syntax by having an apiname annotation for all API Cmd classes and the
> API server would be able to load this info during runtime. In
> commands.properties, we should be just able to set policy for user role for
> each api, if apiname is not declared it's blacklisted. It becomes tricky
> with plugins. I don't know how to get it right the first time, but let's
> try.
>>> - ACL and security checking at API layer, the requests would be just
> passed to (multiple) cloud-engine which won't handle them, as that will
> only orchestrate.
>>> - Roles would be decided from commands.properties. Multiple API servers
> can be run with different combination of rules in commands.properties. This
> would allow separation of policy from mechanism, and multiple roles and end
> points. For example, an admin can create http routers for these api server,
> so: <url>/hr, <url>/finance, <url>/marketting can be proxies to different
> user api servers with different set of whitelisted apis.
>>> 
>>> Comments, suggestions, flames?
>>> 
>>> Regards.
>> 
>> 
> 
> -- 
> Sent from Gmail Mobile

Re: Arch Rework RFC: API refactoring updates

[Mice Xia <we...@gmail.com>]

rohit,

i asked this question because requests from domain admins are usually from
outside of datacenter for a public cloud.

so artifact built from user package can be directly deployed as a user api
server and serve user requests. but to serve domain admin request, sysadmin
has to prepare another api server with another set of apis, which is a sub
set of admin package, is this correct?

sent from phone, sorry for typos.

mice

在 2012年12月7日星期五，Rohit Yadav <ro...@citrix.com> 写道：
>
> On 06-Dec-2012, at 4:29 PM, Mice Xia <mi...@tcloudcomputing.com> wrote:
>
>> [quote]
>> 1. Moved and classified all apis (except the anomalies, see other email)
into namespace org.apache.cloudstack.api 2. There are two packages, admin
and user in which the APIs are grouped based on security level. Note; APIs
are not grouped based on roles.
>> - Because of 1,2,3; there would be two classes of API server. The first
one would be available for users, the user API server which would handle
requests from network outside the datacenter which Alex refers to as "over
the wire" requests. The second one would be available for sysadmins, the
mgmt API server which would handle requests from admins/sysadmins within
the datacenter;
>> [/quote]
>>
>> I haven’t seen the codes, just one question.
>> If the intention is to deploy API server separately based on the
security level, where are APIs enabled for domain-admin located? In the
user package or in the admin package?
>
> The admin package. For domain-admin etc. the sysadmin would run mgmt
servers with different sets of apis s/he wants to provide for a domain
admin. There won't be any pre configured set of apis, this would allow a
system admin to decide exactly what level of access s/he wants to provide
to a domainadmin role user. This allows him/her to create multiple end
points of mgmt server as well, just like the user api server.
>
> Regards.
>
>>
>> Regards
>> Mice
>>
>> -----Original Message-----
>> From: Rohit Yadav [mailto:rohit.yadav@citrix.com]
>> Sent: Friday, December 07, 2012 4:07 AM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Arch Rework RFC: API refactoring updates
>>
>> Few updates;
>>
>> 1. Moved and classified all apis (except the anomalies, see other email)
into namespace org.apache.cloudstack.api 2. There are two packages, admin
and user in which the APIs are grouped based on security level. Note; APIs
are not grouped based on roles.
>> 3. Further in each org.apache.cloudstack.api.{admin,user}.command pkg,
the APIs are grouped as per api affinity (for ex. all vm related ones in vm
pkg).
>> 4. Few usage related APIs could not be moved from cloud-server to
cloud-api, as they are tightly coupled with classes available in
cloud-server.
>> 5. commands.properties is fixed.
>> 6. Checked, the api_refactoring merges fine with a minor merge
conflicts. The build on the branch is broken, would be fixed soon.
>>
>> I request that if you're adding any new class, please use the namespace
org.apache.cloudstack.
>>
>> Road ahead:
>> - Because of 1,2,3; there would be two classes of API server. The first
one would be available for users, the user API server which would handle
requests from network outside the datacenter which Alex refers to as "over
the wire" requests. The second one would be available for sysadmins, the
mgmt API server which would handle requests from admins/sysadmins within
the datacenter;
>>
>> user API server would use only the org.apache.cloudstack.api.user
artifact admin API server would use both org.apache.cloudstack.api.admin
and org.apache.cloudstack.api.user Based on the arch. rework docs, ppts,
talks, Alex's idea was to separate api based on security level would give
ultimate isolation.
>>
>> - Separate out cloud-api, and make it run as a separate web app like
awsapi.
>> - Annotations and separation of API based on api affinity would help
automate apidoc generation, api discovery over an api endpoint, so clients
(UI or cloudmonkey etc.) would be able to discover
>> - The commands.properties syntax is horrible, I want to get rid of the
evil syntax by having an apiname annotation for all API Cmd classes and the
API server would be able to load this info during runtime. In
commands.properties, we should be just able to set policy for user role for
each api, if apiname is not declared it's blacklisted. It becomes tricky
with plugins. I don't know how to get it right the first time, but let's
try.
>> - ACL and security checking at API layer, the requests would be just
passed to (multiple) cloud-engine which won't handle them, as that will
only orchestrate.
>> - Roles would be decided from commands.properties. Multiple API servers
can be run with different combination of rules in commands.properties. This
would allow separation of policy from mechanism, and multiple roles and end
points. For example, an admin can create http routers for these api server,
so: <url>/hr, <url>/finance, <url>/marketting can be proxies to different
user api servers with different set of whitelisted apis.
>>
>> Comments, suggestions, flames?
>>
>> Regards.
>
>

-- 
Sent from Gmail Mobile

Re: Arch Rework RFC: API refactoring updates

[Rohit Yadav <ro...@citrix.com>]

On 06-Dec-2012, at 4:29 PM, Mice Xia <mi...@tcloudcomputing.com> wrote:

> [quote]
> 1. Moved and classified all apis (except the anomalies, see other email) into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
> - Because of 1,2,3; there would be two classes of API server. The first one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;
> [/quote]
> 
> I haven’t seen the codes, just one question. 
> If the intention is to deploy API server separately based on the security level, where are APIs enabled for domain-admin located? In the user package or in the admin package?

The admin package. For domain-admin etc. the sysadmin would run mgmt servers with different sets of apis s/he wants to provide for a domain admin. There won't be any pre configured set of apis, this would allow a system admin to decide exactly what level of access s/he wants to provide to a domainadmin role user. This allows him/her to create multiple end points of mgmt server as well, just like the user api server.

Regards.

> 
> Regards
> Mice
> 
> -----Original Message-----
> From: Rohit Yadav [mailto:rohit.yadav@citrix.com] 
> Sent: Friday, December 07, 2012 4:07 AM
> To: cloudstack-dev@incubator.apache.org
> Subject: Arch Rework RFC: API refactoring updates
> 
> Few updates;
> 
> 1. Moved and classified all apis (except the anomalies, see other email) into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
> 3. Further in each org.apache.cloudstack.api.{admin,user}.command pkg, the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
> 4. Few usage related APIs could not be moved from cloud-server to cloud-api, as they are tightly coupled with classes available in cloud-server.
> 5. commands.properties is fixed.
> 6. Checked, the api_refactoring merges fine with a minor merge conflicts. The build on the branch is broken, would be fixed soon.
> 
> I request that if you're adding any new class, please use the namespace org.apache.cloudstack.
> 
> Road ahead:
> - Because of 1,2,3; there would be two classes of API server. The first one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;
> 
> user API server would use only the org.apache.cloudstack.api.user artifact admin API server would use both org.apache.cloudstack.api.admin and org.apache.cloudstack.api.user Based on the arch. rework docs, ppts, talks, Alex's idea was to separate api based on security level would give ultimate isolation.
> 
> - Separate out cloud-api, and make it run as a separate web app like awsapi.
> - Annotations and separation of API based on api affinity would help automate apidoc generation, api discovery over an api endpoint, so clients (UI or cloudmonkey etc.) would be able to discover
> - The commands.properties syntax is horrible, I want to get rid of the evil syntax by having an apiname annotation for all API Cmd classes and the API server would be able to load this info during runtime. In commands.properties, we should be just able to set policy for user role for each api, if apiname is not declared it's blacklisted. It becomes tricky with plugins. I don't know how to get it right the first time, but let's try.
> - ACL and security checking at API layer, the requests would be just passed to (multiple) cloud-engine which won't handle them, as that will only orchestrate.
> - Roles would be decided from commands.properties. Multiple API servers can be run with different combination of rules in commands.properties. This would allow separation of policy from mechanism, and multiple roles and end points. For example, an admin can create http routers for these api server, so: <url>/hr, <url>/finance, <url>/marketting can be proxies to different user api servers with different set of whitelisted apis.
> 
> Comments, suggestions, flames?
> 
> Regards.

RE: Arch Rework RFC: API refactoring updates

[Mice Xia <mi...@tcloudcomputing.com>]

[quote]
1. Moved and classified all apis (except the anomalies, see other email) into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
- Because of 1,2,3; there would be two classes of API server. The first one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;
[/quote]

I haven’t seen the codes, just one question. 
If the intention is to deploy API server separately based on the security level, where are APIs enabled for domain-admin located? In the user package or in the admin package?

Regards
Mice

-----Original Message-----
From: Rohit Yadav [mailto:rohit.yadav@citrix.com] 
Sent: Friday, December 07, 2012 4:07 AM
To: cloudstack-dev@incubator.apache.org
Subject: Arch Rework RFC: API refactoring updates

Few updates;

1. Moved and classified all apis (except the anomalies, see other email) into namespace org.apache.cloudstack.api 2. There are two packages, admin and user in which the APIs are grouped based on security level. Note; APIs are not grouped based on roles.
3. Further in each org.apache.cloudstack.api.{admin,user}.command pkg, the APIs are grouped as per api affinity (for ex. all vm related ones in vm pkg).
4. Few usage related APIs could not be moved from cloud-server to cloud-api, as they are tightly coupled with classes available in cloud-server.
5. commands.properties is fixed.
6. Checked, the api_refactoring merges fine with a minor merge conflicts. The build on the branch is broken, would be fixed soon.

I request that if you're adding any new class, please use the namespace org.apache.cloudstack.

Road ahead:
- Because of 1,2,3; there would be two classes of API server. The first one would be available for users, the user API server which would handle requests from network outside the datacenter which Alex refers to as "over the wire" requests. The second one would be available for sysadmins, the mgmt API server which would handle requests from admins/sysadmins within the datacenter;

user API server would use only the org.apache.cloudstack.api.user artifact admin API server would use both org.apache.cloudstack.api.admin and org.apache.cloudstack.api.user Based on the arch. rework docs, ppts, talks, Alex's idea was to separate api based on security level would give ultimate isolation.

- Separate out cloud-api, and make it run as a separate web app like awsapi.
- Annotations and separation of API based on api affinity would help automate apidoc generation, api discovery over an api endpoint, so clients (UI or cloudmonkey etc.) would be able to discover
- The commands.properties syntax is horrible, I want to get rid of the evil syntax by having an apiname annotation for all API Cmd classes and the API server would be able to load this info during runtime. In commands.properties, we should be just able to set policy for user role for each api, if apiname is not declared it's blacklisted. It becomes tricky with plugins. I don't know how to get it right the first time, but let's try.
- ACL and security checking at API layer, the requests would be just passed to (multiple) cloud-engine which won't handle them, as that will only orchestrate.
- Roles would be decided from commands.properties. Multiple API servers can be run with different combination of rules in commands.properties. This would allow separation of policy from mechanism, and multiple roles and end points. For example, an admin can create http routers for these api server, so: <url>/hr, <url>/finance, <url>/marketting can be proxies to different user api servers with different set of whitelisted apis.

Comments, suggestions, flames?

Regards.