You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by Alex Huang <Al...@citrix.com> on 2012/08/15 23:42:18 UTC

[PROPOSAL] Breaking CloudStack into subprojects

After doing the merge over process one time, I realize something is wrong with CS structure.  We should think about fixing this.

The problems I found are 

- I know nothing about the UI but I'm responsible for it.
- Someone who is a committer can actually check into areas they know absolutely nothing about.
- As a release manager, I have no way of knowing whether someone doing a checkin is actually an expert in that part of the code because of the above.

So this reminded me of a conversation I had with Amr Awadallah, Cloudera's CTO, when CS joined Apache incubation.  I was picking his brains about how CS should work in Apache given his experience with Hadoop.  His suggested that we break CS into multiple subprojects.  We admit committers to the subprojects based on their experience with that subproject.  

I like to see what's the community's response to a structure like that.  Through Murali's work, CloudStack has already been broken down into finer set of plugins.  We can start with every jar is a sub project with committers assigned to them.  It will take a little time to sort out but it will make going forward a lot easier.

Please comment.

--Alex

RE: [PROPOSAL] Breaking CloudStack into subprojects

Posted by Will Chan <wi...@citrix.com>.
> -----Original Message-----
> From: Chip Childers [mailto:chip.childers@sungard.com]
> Sent: Wednesday, August 15, 2012 4:35 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [PROPOSAL] Breaking CloudStack into subprojects
> 
> I think we're talking about two different things:  First, the architectural
> approach to the software.  Second, the organizational approach to
> managing the software.
> 

This is correct.  I am only arguing against the organizational approach right now with our already created maintainer list.  The architectural approach is something I am definitely in favor of and hoping that will move full steam ahead after 4.0 is released.


Will

Re: [PROPOSAL] Breaking CloudStack into subprojects

Posted by Chip Childers <ch...@sungard.com>.
On Wed, Aug 15, 2012 at 6:51 PM, Will Chan <wi...@citrix.com> wrote:
>
>
>> -----Original Message-----
>> From: Sheng Liang [mailto:Sheng.Liang@citrix.com]
>> Sent: Wednesday, August 15, 2012 3:41 PM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: RE: [PROPOSAL] Breaking CloudStack into subprojects
>>
>> > Let's take the Netscaler AutoScale feature currently being
>> > implemented.  I'm sure they must be at least touching the API, UI, some
>> Resources layer and DAO layer.  If they were committers, do they have to
>> have been invited to all 4 of those subprojects?
>>
>> This is precisely the type of problem exposed by the separation. We need to
>> address these problems head-on and not try to prolong a large monolithic
>> system.
>>
>> Once we have clean separation of projects developers will be naturally
>> inclined to maintain the separation. Autoscaling should ideally be its own
>> project and not require changes to core CloudStack functionalities.
>>
>> Sheng
>
> What you suggest makes sense but I don't think we are quite there yet.  Alex is proposing we assign committers to each of the jar files (defined by him as a subproject) that are currently being created today.  That seems a bit burdensome to me especially when we already have a maintainer system setup to review changes and commit patches.
>
> Will
>

I think we're talking about two different things:  First, the
architectural approach to the software.  Second, the organizational
approach to managing the software.

For the architectural side of things, I'm a fan of Kelvin's comments
on a separate thread.  We should probably shift that discussion over
to respond to Kelvin.

For the organizational issues, I am very much against making multiple
projects.  I'm using the term project *very* specifically here:  ASF
project / sub-project.  We can certainly revisit this at a later time,
but I don't think we have a large enough community of active
participants to truly break up into different sub-projects.  As it
stands right now, I can see myself only agreeing to sub-projects if
there was an explicit agreement that only the "top level project"
(i.e.: CloudStack) was allowed to perform an official release for the
child components.  (reserving the right to evolve my opinion)

I completely understand Alex's concern (all the more so, knowing how
intimate he is with the product as a whole).  This is  complex system,
and I've experienced a similar issue acting in that role for internal
projects in the past.  One person can't be the expert in all layers /
modules.  Adding to this: regardless of how clean we separate the
layers, many features will still have to have code that slices
vertically through the architecture.

The way to deal with this is to think about the role Alex stepped up
for a bit differently.  It shouldn't be his responsibility to
personally *approve* with all changes.  Instead, being the release
branch "gatekeeper" should be a semi-technical and semi-administrative
function.  Clearly using his own knowledge to verify patches is of
significant value to the project, but the administrative function is
to ensure that (1) the commits have been tested, (2) the appropriate
developers have reviewed the commits, (3) communicate with the
community if there is a reason to keep something out of the release
branch so that we can ensure a consensus driven approach to decisions.

For identifying the appropriate reviewers of any commit (point 2
above), we have already established a "maintainer" list [1] that I
think we have already agreed to. Alex should be leaning heavily on the
people mentioned in that list!

-chip

[1] https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Maintainers+Guide

RE: [PROPOSAL] Breaking CloudStack into subprojects

Posted by Will Chan <wi...@citrix.com>.

> -----Original Message-----
> From: Sheng Liang [mailto:Sheng.Liang@citrix.com]
> Sent: Wednesday, August 15, 2012 3:41 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: RE: [PROPOSAL] Breaking CloudStack into subprojects
> 
> > Let's take the Netscaler AutoScale feature currently being
> > implemented.  I'm sure they must be at least touching the API, UI, some
> Resources layer and DAO layer.  If they were committers, do they have to
> have been invited to all 4 of those subprojects?
> 
> This is precisely the type of problem exposed by the separation. We need to
> address these problems head-on and not try to prolong a large monolithic
> system.
> 
> Once we have clean separation of projects developers will be naturally
> inclined to maintain the separation. Autoscaling should ideally be its own
> project and not require changes to core CloudStack functionalities.
> 
> Sheng

What you suggest makes sense but I don't think we are quite there yet.  Alex is proposing we assign committers to each of the jar files (defined by him as a subproject) that are currently being created today.  That seems a bit burdensome to me especially when we already have a maintainer system setup to review changes and commit patches.  

Will

RE: [PROPOSAL] Breaking CloudStack into subprojects

Posted by Sheng Liang <Sh...@citrix.com>.
> Let's take the Netscaler AutoScale feature currently being implemented.  I'm sure they must be at least touching the API,
> UI, some Resources layer and DAO layer.  If they were committers, do they have to have been invited to all 4 of those subprojects?

This is precisely the type of problem exposed by the separation. We need to address these problems head-on and not try to prolong a large monolithic system.

Once we have clean separation of projects developers will be naturally inclined to maintain the separation. Autoscaling should ideally be its own project and not require changes to core CloudStack functionalities.

Sheng


RE: [PROPOSAL] Breaking CloudStack into subprojects

Posted by Will Chan <wi...@citrix.com>.

> -----Original Message-----
> From: David Nalley [mailto:david@gnsa.us]
> Sent: Wednesday, August 15, 2012 3:13 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [PROPOSAL] Breaking CloudStack into subprojects
> 
> On Wed, Aug 15, 2012 at 5:42 PM, Alex Huang <Al...@citrix.com>
> wrote:
> > After doing the merge over process one time, I realize something is wrong
> with CS structure.  We should think about fixing this.
> >
> > The problems I found are
> >
> > - I know nothing about the UI but I'm responsible for it.
> > - Someone who is a committer can actually check into areas they know
> absolutely nothing about.
> > - As a release manager, I have no way of knowing whether someone doing
> a checkin is actually an expert in that part of the code because of the above.
> >
> > So this reminded me of a conversation I had with Amr Awadallah,
> Cloudera's CTO, when CS joined Apache incubation.  I was picking his brains
> about how CS should work in Apache given his experience with Hadoop.  His
> suggested that we break CS into multiple subprojects.  We admit
> committers to the subprojects based on their experience with that
> subproject.
> >
> > I like to see what's the community's response to a structure like that.
> Through Murali's work, CloudStack has already been broken down into finer
> set of plugins.  We can start with every jar is a sub project with committers
> assigned to them.  It will take a little time to sort out but it will make going
> forward a lot easier.
> >
> > Please comment.
> >
> > --Alex
> 
> 
> My initial gut reaction  is 'please no, not that'
> The idea of a modular, loosely coupled tools is nice, it plays well with my
> sensibilities, and giving that you are proposing it, and Sheng seconds it I
> have no doubt there is technical merit.
> 
> That said I think it ignores some key issues:
> One of CloudStack's strengths is that it is a cohesive whole 'product'
> that works together. Splitting this out too much has me fearing the result.
> We aren't like unix utilities such as grep and bash that can coexist, there are
> plenty of interdependencies.
> 
> As a project we trust our committers. The path to gaining commit privileges
> is already one that requires establishing trust and credibility, having
> multiple hoops of equal height within the same project baffles my mind. Yes
> anyone of those committers could commit changes that are deletirious. But
> in general we expect them not to meddle where they have no
> understanding. (For instance, you are incredibly unlikely to see me munging
> with some of the core internals of CloudStack - I know better and don't
> need to go tweak things, or at least if I do to do them in a feature branch)
> Plus, we have a revision control system, and commit messages, and commit
> then review is a pretty nice way to keep folks who care informed.
> 
> I listen to the hadoop folks talk about the difficulty of setting up Hadoop
> MR + HDFS + Hive + Zookeeper + Pig, etc, etc, and the problems they have
> with version matching these different projects. Splitting CloudStack into n-
> number of subprojects means n-number of releases, and likely independent
> releases.
> 
> Not to mention the additional overhead of splitting up into n-number of
> subprojects. And honestly this sounds like more of administrative concern
> than anything. I personally don't think the community is at the size yet
> where we can safely risk fragmentation.
> 
> --David


I like the idea but it just doesn't seem practical to me right now.  Let's take the Netscaler AutoScale feature currently being implemented.  I'm sure they must be at least touching the API, UI, some Resources layer and DAO layer.  If they were committers, do they have to have been invited to all 4 of those subprojects?   On the other hand, when they submit a patch that touches multiple subprojects, do you need multiple committers?  I would assume previously, you just had a couple of maintainers review this patch but there was only a single committer.  Now, with subproject permissions, that may not be the case?

I do agree Alex, that you are unqualified to review UI changes, but presumably, all you are doing is cherry-picking check-ins that have already been reviewed by someone that had that expertise (i.e. Jessica/Brian/Olga)?  Is that the problem you are facing right now because I thought the current maintainer model seems to work in general but I could be wrong.

Will

Re: [PROPOSAL] Breaking CloudStack into subprojects

Posted by David Nalley <da...@gnsa.us>.
On Wed, Aug 15, 2012 at 6:33 PM, Alex Huang <Al...@citrix.com> wrote:
>
>
>> -----Original Message-----
>> From: David Nalley [mailto:david@gnsa.us]
>> Sent: Wednesday, August 15, 2012 3:13 PM
>> To: cloudstack-dev@incubator.apache.org
>> Subject: Re: [PROPOSAL] Breaking CloudStack into subprojects
>>
>> On Wed, Aug 15, 2012 at 5:42 PM, Alex Huang <Al...@citrix.com>
>> wrote:
>> > After doing the merge over process one time, I realize something is wrong
>> with CS structure.  We should think about fixing this.
>> >
>> > The problems I found are
>> >
>> > - I know nothing about the UI but I'm responsible for it.
>> > - Someone who is a committer can actually check into areas they know
>> absolutely nothing about.
>> > - As a release manager, I have no way of knowing whether someone doing
>> a checkin is actually an expert in that part of the code because of the above.
>> >
>> > So this reminded me of a conversation I had with Amr Awadallah,
>> Cloudera's CTO, when CS joined Apache incubation.  I was picking his brains
>> about how CS should work in Apache given his experience with Hadoop.  His
>> suggested that we break CS into multiple subprojects.  We admit committers
>> to the subprojects based on their experience with that subproject.
>> >
>> > I like to see what's the community's response to a structure like that.
>> Through Murali's work, CloudStack has already been broken down into finer
>> set of plugins.  We can start with every jar is a sub project with committers
>> assigned to them.  It will take a little time to sort out but it will make going
>> forward a lot easier.
>> >
>> > Please comment.
>> >
>> > --Alex
>>
>>
>> My initial gut reaction  is 'please no, not that'
>> The idea of a modular, loosely coupled tools is nice, it plays well with my
>> sensibilities, and giving that you are proposing it, and Sheng seconds it I have
>> no doubt there is technical merit.
>>
>> That said I think it ignores some key issues:
>> One of CloudStack's strengths is that it is a cohesive whole 'product'
>> that works together. Splitting this out too much has me fearing the result.
>> We aren't like unix utilities such as grep and bash that can coexist, there are
>> plenty of interdependencies.
>>
>> As a project we trust our committers. The path to gaining commit privileges is
>> already one that requires establishing trust and credibility, having multiple
>> hoops of equal height within the same project baffles my mind. Yes anyone
>> of those committers could commit changes that are deletirious. But in
>> general we expect them not to meddle where they have no understanding.
>> (For instance, you are incredibly unlikely to see me munging with some of
>> the core internals of CloudStack - I know better and don't need to go tweak
>> things, or at least if I do to do them in a feature branch) Plus, we have a
>> revision control system, and commit messages, and commit then review is a
>> pretty nice way to keep folks who care informed.
>>
>> I listen to the hadoop folks talk about the difficulty of setting up Hadoop MR
>> + HDFS + Hive + Zookeeper + Pig, etc, etc, and the problems they have with
>> version matching these different projects. Splitting CloudStack into n-
>> number of subprojects means n-number of releases, and likely independent
>> releases.
>>
>> Not to mention the additional overhead of splitting up into n-number of
>> subprojects. And honestly this sounds like more of administrative concern
>> than anything. I personally don't think the community is at the size yet where
>> we can safely risk fragmentation.
>>
>
> I don't think different subprojects necessarily mean different releases for each project.  It's more of means to put the experts with their expertise.  For example, before Hugo became a committer, Jessica Wang, a UI writer, was committer for the Nicira code.

Huh? I see Murali as the committer for that code:
http://mail-archives.apache.org/mod_mbox/incubator-cloudstack-commits/201207.mbox/%3C20120712001514.3D69313264%40tyr.zones.apache.org%3E

That gets back to people reviewing what they know, I didn't review
Hugo's patch as I know precious little about Nicira's NVP or the
CloudStack end of the code. I don't think we have an abundance (or
any) of that happening. If we do, we still have means of dealing with
it.

--David

RE: [PROPOSAL] Breaking CloudStack into subprojects

Posted by Alex Huang <Al...@citrix.com>.

> -----Original Message-----
> From: David Nalley [mailto:david@gnsa.us]
> Sent: Wednesday, August 15, 2012 3:13 PM
> To: cloudstack-dev@incubator.apache.org
> Subject: Re: [PROPOSAL] Breaking CloudStack into subprojects
> 
> On Wed, Aug 15, 2012 at 5:42 PM, Alex Huang <Al...@citrix.com>
> wrote:
> > After doing the merge over process one time, I realize something is wrong
> with CS structure.  We should think about fixing this.
> >
> > The problems I found are
> >
> > - I know nothing about the UI but I'm responsible for it.
> > - Someone who is a committer can actually check into areas they know
> absolutely nothing about.
> > - As a release manager, I have no way of knowing whether someone doing
> a checkin is actually an expert in that part of the code because of the above.
> >
> > So this reminded me of a conversation I had with Amr Awadallah,
> Cloudera's CTO, when CS joined Apache incubation.  I was picking his brains
> about how CS should work in Apache given his experience with Hadoop.  His
> suggested that we break CS into multiple subprojects.  We admit committers
> to the subprojects based on their experience with that subproject.
> >
> > I like to see what's the community's response to a structure like that.
> Through Murali's work, CloudStack has already been broken down into finer
> set of plugins.  We can start with every jar is a sub project with committers
> assigned to them.  It will take a little time to sort out but it will make going
> forward a lot easier.
> >
> > Please comment.
> >
> > --Alex
> 
> 
> My initial gut reaction  is 'please no, not that'
> The idea of a modular, loosely coupled tools is nice, it plays well with my
> sensibilities, and giving that you are proposing it, and Sheng seconds it I have
> no doubt there is technical merit.
> 
> That said I think it ignores some key issues:
> One of CloudStack's strengths is that it is a cohesive whole 'product'
> that works together. Splitting this out too much has me fearing the result.
> We aren't like unix utilities such as grep and bash that can coexist, there are
> plenty of interdependencies.
> 
> As a project we trust our committers. The path to gaining commit privileges is
> already one that requires establishing trust and credibility, having multiple
> hoops of equal height within the same project baffles my mind. Yes anyone
> of those committers could commit changes that are deletirious. But in
> general we expect them not to meddle where they have no understanding.
> (For instance, you are incredibly unlikely to see me munging with some of
> the core internals of CloudStack - I know better and don't need to go tweak
> things, or at least if I do to do them in a feature branch) Plus, we have a
> revision control system, and commit messages, and commit then review is a
> pretty nice way to keep folks who care informed.
> 
> I listen to the hadoop folks talk about the difficulty of setting up Hadoop MR
> + HDFS + Hive + Zookeeper + Pig, etc, etc, and the problems they have with
> version matching these different projects. Splitting CloudStack into n-
> number of subprojects means n-number of releases, and likely independent
> releases.
> 
> Not to mention the additional overhead of splitting up into n-number of
> subprojects. And honestly this sounds like more of administrative concern
> than anything. I personally don't think the community is at the size yet where
> we can safely risk fragmentation.
> 

I don't think different subprojects necessarily mean different releases for each project.  It's more of means to put the experts with their expertise.  For example, before Hugo became a committer, Jessica Wang, a UI writer, was committer for the Nicira code.  For now, I can still tell who's good with what and I happen to the release manager for 4.0.  It will be much more difficult for someone else to drive a release. 

Also I'm not proposing to do it in 4.0.  Perhaps the next major release.  4.0 needs to move forward as it is now.

--Alex

Re: [PROPOSAL] Breaking CloudStack into subprojects

Posted by David Nalley <da...@gnsa.us>.
On Wed, Aug 15, 2012 at 5:42 PM, Alex Huang <Al...@citrix.com> wrote:
> After doing the merge over process one time, I realize something is wrong with CS structure.  We should think about fixing this.
>
> The problems I found are
>
> - I know nothing about the UI but I'm responsible for it.
> - Someone who is a committer can actually check into areas they know absolutely nothing about.
> - As a release manager, I have no way of knowing whether someone doing a checkin is actually an expert in that part of the code because of the above.
>
> So this reminded me of a conversation I had with Amr Awadallah, Cloudera's CTO, when CS joined Apache incubation.  I was picking his brains about how CS should work in Apache given his experience with Hadoop.  His suggested that we break CS into multiple subprojects.  We admit committers to the subprojects based on their experience with that subproject.
>
> I like to see what's the community's response to a structure like that.  Through Murali's work, CloudStack has already been broken down into finer set of plugins.  We can start with every jar is a sub project with committers assigned to them.  It will take a little time to sort out but it will make going forward a lot easier.
>
> Please comment.
>
> --Alex


My initial gut reaction  is 'please no, not that'
The idea of a modular, loosely coupled tools is nice, it plays well
with my sensibilities, and giving that you are proposing it, and Sheng
seconds it I have no doubt there is technical merit.

That said I think it ignores some key issues:
One of CloudStack's strengths is that it is a cohesive whole 'product'
that works together. Splitting this out too much has me fearing the
result. We aren't like unix utilities such as grep and bash that can
coexist, there are plenty of interdependencies.

As a project we trust our committers. The path to gaining commit
privileges is already one that requires establishing trust and
credibility, having multiple hoops of equal height within the same
project baffles my mind. Yes anyone of those committers could commit
changes that are deletirious. But in general we expect them not to
meddle where they have no understanding. (For instance, you are
incredibly unlikely to see me munging with some of the core internals
of CloudStack - I know better and don't need to go tweak things, or at
least if I do to do them in a feature branch) Plus, we have a revision
control system, and commit messages, and commit then review is a
pretty nice way to keep folks who care informed.

I listen to the hadoop folks talk about the difficulty of setting up
Hadoop MR + HDFS + Hive + Zookeeper + Pig, etc, etc, and the problems
they have with version matching these different projects. Splitting
CloudStack into n-number of subprojects means n-number of releases,
and likely independent releases.

Not to mention the additional overhead of splitting up into n-number
of subprojects. And honestly this sounds like more of administrative
concern than anything. I personally don't think the community is at
the size yet where we can safely risk fragmentation.

--David

Re: [PROPOSAL] Breaking CloudStack into subprojects

Posted by Jim Jagielski <ji...@jaguNET.com>.
Previous history has shown that such "umbrella" projects (those
with several sub-projects) are usually much more difficult to
keep successful than a large project *or* a group of same-level
top level projects.

So here's my -1

On Aug 15, 2012, at 5:42 PM, Alex Huang <Al...@citrix.com> wrote:

> After doing the merge over process one time, I realize something is wrong with CS structure.  We should think about fixing this.
> 
> The problems I found are 
> 
> - I know nothing about the UI but I'm responsible for it.
> - Someone who is a committer can actually check into areas they know absolutely nothing about.
> - As a release manager, I have no way of knowing whether someone doing a checkin is actually an expert in that part of the code because of the above.
> 
> So this reminded me of a conversation I had with Amr Awadallah, Cloudera's CTO, when CS joined Apache incubation.  I was picking his brains about how CS should work in Apache given his experience with Hadoop.  His suggested that we break CS into multiple subprojects.  We admit committers to the subprojects based on their experience with that subproject.  
> 
> I like to see what's the community's response to a structure like that.  Through Murali's work, CloudStack has already been broken down into finer set of plugins.  We can start with every jar is a sub project with committers assigned to them.  It will take a little time to sort out but it will make going forward a lot easier.
> 
> Please comment.
> 
> --Alex
> 


Re: [PROPOSAL] Breaking CloudStack into subprojects

Posted by Alex Karasulu <ak...@apache.org>.
On Thu, Aug 16, 2012 at 12:42 AM, Alex Huang <Al...@citrix.com> wrote:

> After doing the merge over process one time, I realize something is wrong
> with CS structure.  We should think about fixing this.
>
> The problems I found are
>
> - I know nothing about the UI but I'm responsible for it.
> - Someone who is a committer can actually check into areas they know
> absolutely nothing about.
>

Quite possible, however how many times does this actually occur in
practice? And when commits do occur by a not so savvy developer in that
unfamiliar region, is the outcome negative? If there is a problem will the
problem be detected and is it possible to easily revert back?

Erecting fences to guard against these potential problems incurs an
additional administrative overhead. Not only must you manage the
permissions you have to evaluate the abilities of the developers based on a
per subproject basis.

Before discussing what's common at Apache, let's talk more about the OS
way. Erecting fences for the sake of greater control is dangerous. It
divides the community to a degree, and it prevents natural migrations from
one region of the source to another through the "scratch an itch"
mechanism. CloudStack is not like an Apache Commons where everything is
absolutely separable today. It maybe one day, but you want people working
in one area to eventually move into other areas of the source. These fences
will detract from these natural flows. Furthermore now you have different
kinds of committers and a need to evaluate who can work on what.

What's important here is the initial committer evaluation. You want a good
open source citizen who knows their limits and is not going to start
picking at everything without knowing anything about the code base. You
need this trust. There are not many committers here at Apache that think
they know everything and have the gaul to muck around in areas they have no
clue about. Sure there can be a small degree of this, but it's small,
isolated and you really need good unit and integration tests to protect
against this. Finally you do want people to gradually start looking into
areas they may not be very knowledgable in so they eventually become savvy.

As for Apache, the majority of projects here at the ASF, regardless of the
size and complexity of the code base usually grant all or nothing access to
their TLP scope of sources in whatever version control system they use. We
trust in our committers and don't take on committers that seem to lack the
balance to know their limits. We may occasionally find ourselves having to
go into areas of code we are not so familiar with but transparency,
collaboration and communication on the mailing lists help ameliorate the
possible negative impacts. Again unit and integration tests a key to give
us the confidence to try things in areas that might be our core
proficiency. This bolsters trust.

Finally version control is helps correct any problems that might arise in
seconds. Diligence is everyone's responsibility.


> - As a release manager, I have no way of knowing whether someone doing a
> checkin is actually an expert in that part of the code because of the above.
>
>
Yeah but you're talking about having to determine this apriori. You're
going to have to give out expert badges eventually and do this evaluation:
at least the PMC does.


> So this reminded me of a conversation I had with Amr Awadallah, Cloudera's
> CTO, when CS joined Apache incubation.  I was picking his brains about how
> CS should work in Apache given his experience with Hadoop.  His suggested
> that we break CS into multiple subprojects.  We admit committers to the
> subprojects based on their experience with that subproject.
>
>
The Hadoop ecosystem, is no longer a single TLP project. It naturally grew
out this way and so too might CS. Incubation, IMHO, is the wrong time to be
trying to build these fences.


> I like to see what's the community's response to a structure like that.
>  Through Murali's work, CloudStack has already been broken down into finer
> set of plugins.


This obviously helps, not only from a code configuration perspective, but
from the ability to detect alterations on the code with commits to do the
diligence.


> We can start with every jar is a sub project with committers assigned to
> them.  It will take a little time to sort out but it will make going
> forward a lot easier.
>
>
I agree with Jim Jag, highly recommending against this. It's not going to
pay off and is a dangerous direction. I can understand previous corporate
control mechanisms around release management being tempting, but in open
source these control mechanisms are unnecessary and lethal.

-- 
Best Regards,
-- Alex

RE: [PROPOSAL] Breaking CloudStack into subprojects

Posted by Sheng Liang <Sh...@citrix.com>.
+1

I also feel the project is already too big to have a single list of committers.

Sheng

-----Original Message-----
From: Alex Huang [mailto:Alex.Huang@citrix.com] 
Sent: Wednesday, August 15, 2012 2:42 PM
To: cloudstack-dev@incubator.apache.org
Subject: [PROPOSAL] Breaking CloudStack into subprojects

After doing the merge over process one time, I realize something is wrong with CS structure.  We should think about fixing this.

The problems I found are 

- I know nothing about the UI but I'm responsible for it.
- Someone who is a committer can actually check into areas they know absolutely nothing about.
- As a release manager, I have no way of knowing whether someone doing a checkin is actually an expert in that part of the code because of the above.

So this reminded me of a conversation I had with Amr Awadallah, Cloudera's CTO, when CS joined Apache incubation.  I was picking his brains about how CS should work in Apache given his experience with Hadoop.  His suggested that we break CS into multiple subprojects.  We admit committers to the subprojects based on their experience with that subproject.  

I like to see what's the community's response to a structure like that.  Through Murali's work, CloudStack has already been broken down into finer set of plugins.  We can start with every jar is a sub project with committers assigned to them.  It will take a little time to sort out but it will make going forward a lot easier.

Please comment.

--Alex

Re: [PROPOSAL] Breaking CloudStack into subprojects

Posted by Chiradeep Vittal <Ch...@citrix.com>.
+1. 


On 8/15/12 2:53 PM, "Kelven Yang" <ke...@citrix.com> wrote:

>+1
>
>Kelven
>
>
>On 8/15/12 2:42 PM, "Alex Huang" <Al...@citrix.com> wrote:
>
>>After doing the merge over process one time, I realize something is wrong
>>with CS structure.  We should think about fixing this.
>>
>>The problems I found are
>>
>>- I know nothing about the UI but I'm responsible for it.
>>- Someone who is a committer can actually check into areas they know
>>absolutely nothing about.
>>- As a release manager, I have no way of knowing whether someone doing a
>>checkin is actually an expert in that part of the code because of the
>>above.
>>
>>So this reminded me of a conversation I had with Amr Awadallah,
>>Cloudera's CTO, when CS joined Apache incubation.  I was picking his
>>brains about how CS should work in Apache given his experience with
>>Hadoop.  His suggested that we break CS into multiple subprojects.  We
>>admit committers to the subprojects based on their experience with that
>>subproject.  
>>
>>I like to see what's the community's response to a structure like that.
>>Through Murali's work, CloudStack has already been broken down into finer
>>set of plugins.  We can start with every jar is a sub project with
>>committers assigned to them.  It will take a little time to sort out but
>>it will make going forward a lot easier.
>>
>>Please comment.
>>
>>--Alex
>


Re: [PROPOSAL] Breaking CloudStack into subprojects

Posted by Kelven Yang <ke...@citrix.com>.
+1

Kelven


On 8/15/12 2:42 PM, "Alex Huang" <Al...@citrix.com> wrote:

>After doing the merge over process one time, I realize something is wrong
>with CS structure.  We should think about fixing this.
>
>The problems I found are
>
>- I know nothing about the UI but I'm responsible for it.
>- Someone who is a committer can actually check into areas they know
>absolutely nothing about.
>- As a release manager, I have no way of knowing whether someone doing a
>checkin is actually an expert in that part of the code because of the
>above.
>
>So this reminded me of a conversation I had with Amr Awadallah,
>Cloudera's CTO, when CS joined Apache incubation.  I was picking his
>brains about how CS should work in Apache given his experience with
>Hadoop.  His suggested that we break CS into multiple subprojects.  We
>admit committers to the subprojects based on their experience with that
>subproject.  
>
>I like to see what's the community's response to a structure like that.
>Through Murali's work, CloudStack has already been broken down into finer
>set of plugins.  We can start with every jar is a sub project with
>committers assigned to them.  It will take a little time to sort out but
>it will make going forward a lot easier.
>
>Please comment.
>
>--Alex


Re: [PROPOSAL] Breaking CloudStack into subprojects

Posted by Abhinandan Prateek <Ab...@citrix.com>.
+1

This will keep committers focused and will also accurately define the areas for reviewers.

-abhi


----- Original Message -----
From: Alex Huang [mailto:Alex.Huang@citrix.com]
Sent: Thursday, August 16, 2012 03:12 AM
To: cloudstack-dev@incubator.apache.org <cl...@incubator.apache.org>
Subject: [PROPOSAL] Breaking CloudStack into subprojects

After doing the merge over process one time, I realize something is wrong with CS structure.  We should think about fixing this.

The problems I found are 

- I know nothing about the UI but I'm responsible for it.
- Someone who is a committer can actually check into areas they know absolutely nothing about.
- As a release manager, I have no way of knowing whether someone doing a checkin is actually an expert in that part of the code because of the above.

So this reminded me of a conversation I had with Amr Awadallah, Cloudera's CTO, when CS joined Apache incubation.  I was picking his brains about how CS should work in Apache given his experience with Hadoop.  His suggested that we break CS into multiple subprojects.  We admit committers to the subprojects based on their experience with that subproject.  

I like to see what's the community's response to a structure like that.  Through Murali's work, CloudStack has already been broken down into finer set of plugins.  We can start with every jar is a sub project with committers assigned to them.  It will take a little time to sort out but it will make going forward a lot easier.

Please comment.

--Alex