Proposal: create a separate ZooKeeper release artifact for recipes

[Patrick Hunt <ph...@apache.org>]

During the recent developer meetup we discussed how to more
effectively grow the recipes.
https://cwiki.apache.org/confluence/display/ZOOKEEPER/June2011DeveloperMeeting

The concerns seemed to be around managing changes, community growth,
and releases. We discussed a number of approaches:
* move recipes into apache-extras
* or github
* or incubator
* or sub-project

each of these has it's good and bad points. One alternative that I put
forth was to create a separate release artifact for recipes as part of
the ZooKeeper project itself. What would this look like?

* zookeeper/trunk/src/recipes would move to
zookeeper/recipes/{trunk,branches,tags,...}
* there would be distinct/separate releases of zookeeper and
zookeeper-recipes artifacts
** these releases would not necessarily be synchronized, although we'd
want to ensure that particular versions of each release work together
** these releases would go through our standard release process - ie
creation of release candidate by a committer and approval by the PMC
** recipes would no longer be included in "zookeeper" (core) releases
* continue to use the ZOOKEEPER jira, with "recipes" component
* space in the web and wiki sites for recipes specific pages
* recipes continue to use the std zookeeper mailing lists
* commits to recipes would continue to use our regular commit process
(jira, rtc, etc...)
* we would accept new "ZooKeeper committers" with the intent that they
focus on their area of expertise, whether this be core (zookeeper
trunk) or recipes.

This last item is the more interesting I think, the other issues are
all pretty straightforward. Although moving to sub (or out entirely)
would allow us to have a separate commit list it would fragment the
ZooKeeper community. I also believe that over time committers to
recipes would more likely get experience with core and start
contributing there as well. The only issue really, if you want to call
it that, is that we'd have committers with "guardrails" implied. ie
someone with more expertise in recipes than core. However there are
many other Apache projects that take this approach, and do it
successfully. Regardless RTC allows for review both before and after a
change is committed.

What do you think?

Patrick

Re: Proposal: create a separate ZooKeeper release artifact for recipes

[Ted Dunning <te...@gmail.com>]

Separator committer lists are generally frowned on by Apache (ref
Lucene/Solr and also Mahout).

They also are essentially unnecessary.  It isn't that big a deal to have a
single committer list and depend on reversion to enforce community
standards.  Core ZK is review-then-commit and should continue to be.
 Add-on modules that are relatively independent of the core can quite
reasonably be commit-then-review or whatever level of care is implied by
module's content.  The review can include a determination of who should
actually do the commit.

Moreover, projects can easily have more than one artifact.  Mahout has
three independent artifacts (mahout-math, mahout-collections and the core
stuff with examples and integration code).  That works well.  There are
different sets of committers who are typically the ones who work on
different components.  The overall level of diligence required in Mahout is
much lower than for Zookeeper as you might expect, but the principle that
multiple artifacts are fine is still the same.  Likewise, the idea of
committers with specialized expertise also applies here.

There is a tradition in Zookeeper of having a very high bar for
committership, but experience in some other projects indicates that this
doesn't actually help quality or security all that much and it can be
argued to decrease community involvement a bit.


On Wed, Nov 30, 2011 at 5:11 PM, Patrick Hunt <ph...@apache.org> wrote:

> Curator has been getting a lot of interest and was just officially
> announced by Netflix:
>
> http://techblog.netflix.com/2011/11/introducing-curator-netflix-zookeeper.html
>
> I chatted briefly with @adrianco @chad_walters @randgalt (Jordan
> Zimmerman) on twitter and the idea of moving Curator into
> Apache/ZooKeeper came up:
> https://twitter.com/#!/search/phunt%20curator
>
> I wanted to restart this thread as a way of discussing this idea in
> more depth, and to gauge the interest of both communities. I
> personally think this would be a great thing: both for users and for
> the development community itself. Thoughts?
>
> Re the mechanics. See my original post below. Seems to me we (ZK PMC)
> could sponsor Curator as an incubator project with the intent to bring
> it to ZK as either a subproject (it's own committer list) or as a
> separate release artifact of the ZK TLP itself. Or just shortcut the
> incubator process altogether. My preference would be to go incubator
> first. At some point we would deprecate the existing ZK recipes (ala
> Curator had suitable replacements).
>
> Patrick
>
> On Fri, Jul 22, 2011 at 10:59 AM, Benjamin Reed <br...@apache.org> wrote:
> > while i agree with the sentiment of not fragmenting the zookeeper
> > community and recipe committers also moving into core development, i
> > also think it would be good if a strong community of developers grew
> > up around zookeeper recipes. to do that they need a sense of identity,
> > and i'm not sure if they can get that with this proposal.
> >
> > on the other hand the proposal does address all of the other issues i
> > have with recipe maintenance. the key is to grow a committer base.
> >
> > ben
> >
> > On Wed, Jul 13, 2011 at 11:04 AM, Patrick Hunt <ph...@apache.org> wrote:
> >> During the recent developer meetup we discussed how to more
> >> effectively grow the recipes.
> >>
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/June2011DeveloperMeeting
> >>
> >> The concerns seemed to be around managing changes, community growth,
> >> and releases. We discussed a number of approaches:
> >> * move recipes into apache-extras
> >> * or github
> >> * or incubator
> >> * or sub-project
> >>
> >> each of these has it's good and bad points. One alternative that I put
> >> forth was to create a separate release artifact for recipes as part of
> >> the ZooKeeper project itself. What would this look like?
> >>
> >> * zookeeper/trunk/src/recipes would move to
> >> zookeeper/recipes/{trunk,branches,tags,...}
> >> * there would be distinct/separate releases of zookeeper and
> >> zookeeper-recipes artifacts
> >> ** these releases would not necessarily be synchronized, although we'd
> >> want to ensure that particular versions of each release work together
> >> ** these releases would go through our standard release process - ie
> >> creation of release candidate by a committer and approval by the PMC
> >> ** recipes would no longer be included in "zookeeper" (core) releases
> >> * continue to use the ZOOKEEPER jira, with "recipes" component
> >> * space in the web and wiki sites for recipes specific pages
> >> * recipes continue to use the std zookeeper mailing lists
> >> * commits to recipes would continue to use our regular commit process
> >> (jira, rtc, etc...)
> >> * we would accept new "ZooKeeper committers" with the intent that they
> >> focus on their area of expertise, whether this be core (zookeeper
> >> trunk) or recipes.
> >>
> >> This last item is the more interesting I think, the other issues are
> >> all pretty straightforward. Although moving to sub (or out entirely)
> >> would allow us to have a separate commit list it would fragment the
> >> ZooKeeper community. I also believe that over time committers to
> >> recipes would more likely get experience with core and start
> >> contributing there as well. The only issue really, if you want to call
> >> it that, is that we'd have committers with "guardrails" implied. ie
> >> someone with more expertise in recipes than core. However there are
> >> many other Apache projects that take this approach, and do it
> >> successfully. Regardless RTC allows for review both before and after a
> >> change is committed.
> >>
> >> What do you think?
> >>
> >> Patrick
> >>
>

Re: Proposal: create a separate ZooKeeper release artifact for recipes

[Jordan Zimmerman <jz...@netflix.com>]

My initial opinion would be to make it a separate artifact - something
akin to the blessed implementation. What changes would need to be made to
bring it into the incubator? My main concern is that this is an active
service inside of Netflix. I imagine Netflix would fork it internally so
as not to burden our releases.

-Jordan

On 11/30/11 5:11 PM, "Patrick Hunt" <ph...@apache.org> wrote:

>Curator has been getting a lot of interest and was just officially
>announced by Netflix:
>http://techblog.netflix.com/2011/11/introducing-curator-netflix-zookeeper.
>html
>
>I chatted briefly with @adrianco @chad_walters @randgalt (Jordan
>Zimmerman) on twitter and the idea of moving Curator into
>Apache/ZooKeeper came up:
>https://twitter.com/#!/search/phunt%20curator
>
>I wanted to restart this thread as a way of discussing this idea in
>more depth, and to gauge the interest of both communities. I
>personally think this would be a great thing: both for users and for
>the development community itself. Thoughts?
>
>Re the mechanics. See my original post below. Seems to me we (ZK PMC)
>could sponsor Curator as an incubator project with the intent to bring
>it to ZK as either a subproject (it's own committer list) or as a
>separate release artifact of the ZK TLP itself. Or just shortcut the
>incubator process altogether. My preference would be to go incubator
>first. At some point we would deprecate the existing ZK recipes (ala
>Curator had suitable replacements).
>
>Patrick
>
>On Fri, Jul 22, 2011 at 10:59 AM, Benjamin Reed <br...@apache.org> wrote:
>> while i agree with the sentiment of not fragmenting the zookeeper
>> community and recipe committers also moving into core development, i
>> also think it would be good if a strong community of developers grew
>> up around zookeeper recipes. to do that they need a sense of identity,
>> and i'm not sure if they can get that with this proposal.
>>
>> on the other hand the proposal does address all of the other issues i
>> have with recipe maintenance. the key is to grow a committer base.
>>
>> ben
>>
>> On Wed, Jul 13, 2011 at 11:04 AM, Patrick Hunt <ph...@apache.org> wrote:
>>> During the recent developer meetup we discussed how to more
>>> effectively grow the recipes.
>>> 
>>>https://cwiki.apache.org/confluence/display/ZOOKEEPER/June2011DeveloperM
>>>eeting
>>>
>>> The concerns seemed to be around managing changes, community growth,
>>> and releases. We discussed a number of approaches:
>>> * move recipes into apache-extras
>>> * or github
>>> * or incubator
>>> * or sub-project
>>>
>>> each of these has it's good and bad points. One alternative that I put
>>> forth was to create a separate release artifact for recipes as part of
>>> the ZooKeeper project itself. What would this look like?
>>>
>>> * zookeeper/trunk/src/recipes would move to
>>> zookeeper/recipes/{trunk,branches,tags,...}
>>> * there would be distinct/separate releases of zookeeper and
>>> zookeeper-recipes artifacts
>>> ** these releases would not necessarily be synchronized, although we'd
>>> want to ensure that particular versions of each release work together
>>> ** these releases would go through our standard release process - ie
>>> creation of release candidate by a committer and approval by the PMC
>>> ** recipes would no longer be included in "zookeeper" (core) releases
>>> * continue to use the ZOOKEEPER jira, with "recipes" component
>>> * space in the web and wiki sites for recipes specific pages
>>> * recipes continue to use the std zookeeper mailing lists
>>> * commits to recipes would continue to use our regular commit process
>>> (jira, rtc, etc...)
>>> * we would accept new "ZooKeeper committers" with the intent that they
>>> focus on their area of expertise, whether this be core (zookeeper
>>> trunk) or recipes.
>>>
>>> This last item is the more interesting I think, the other issues are
>>> all pretty straightforward. Although moving to sub (or out entirely)
>>> would allow us to have a separate commit list it would fragment the
>>> ZooKeeper community. I also believe that over time committers to
>>> recipes would more likely get experience with core and start
>>> contributing there as well. The only issue really, if you want to call
>>> it that, is that we'd have committers with "guardrails" implied. ie
>>> someone with more expertise in recipes than core. However there are
>>> many other Apache projects that take this approach, and do it
>>> successfully. Regardless RTC allows for review both before and after a
>>> change is committed.
>>>
>>> What do you think?
>>>
>>> Patrick
>>>
>

Re: Proposal: create a separate ZooKeeper release artifact for recipes

[Patrick Hunt <ph...@apache.org>]

Curator has been getting a lot of interest and was just officially
announced by Netflix:
http://techblog.netflix.com/2011/11/introducing-curator-netflix-zookeeper.html

I chatted briefly with @adrianco @chad_walters @randgalt (Jordan
Zimmerman) on twitter and the idea of moving Curator into
Apache/ZooKeeper came up:
https://twitter.com/#!/search/phunt%20curator

I wanted to restart this thread as a way of discussing this idea in
more depth, and to gauge the interest of both communities. I
personally think this would be a great thing: both for users and for
the development community itself. Thoughts?

Re the mechanics. See my original post below. Seems to me we (ZK PMC)
could sponsor Curator as an incubator project with the intent to bring
it to ZK as either a subproject (it's own committer list) or as a
separate release artifact of the ZK TLP itself. Or just shortcut the
incubator process altogether. My preference would be to go incubator
first. At some point we would deprecate the existing ZK recipes (ala
Curator had suitable replacements).

Patrick

On Fri, Jul 22, 2011 at 10:59 AM, Benjamin Reed <br...@apache.org> wrote:
> while i agree with the sentiment of not fragmenting the zookeeper
> community and recipe committers also moving into core development, i
> also think it would be good if a strong community of developers grew
> up around zookeeper recipes. to do that they need a sense of identity,
> and i'm not sure if they can get that with this proposal.
>
> on the other hand the proposal does address all of the other issues i
> have with recipe maintenance. the key is to grow a committer base.
>
> ben
>
> On Wed, Jul 13, 2011 at 11:04 AM, Patrick Hunt <ph...@apache.org> wrote:
>> During the recent developer meetup we discussed how to more
>> effectively grow the recipes.
>> https://cwiki.apache.org/confluence/display/ZOOKEEPER/June2011DeveloperMeeting
>>
>> The concerns seemed to be around managing changes, community growth,
>> and releases. We discussed a number of approaches:
>> * move recipes into apache-extras
>> * or github
>> * or incubator
>> * or sub-project
>>
>> each of these has it's good and bad points. One alternative that I put
>> forth was to create a separate release artifact for recipes as part of
>> the ZooKeeper project itself. What would this look like?
>>
>> * zookeeper/trunk/src/recipes would move to
>> zookeeper/recipes/{trunk,branches,tags,...}
>> * there would be distinct/separate releases of zookeeper and
>> zookeeper-recipes artifacts
>> ** these releases would not necessarily be synchronized, although we'd
>> want to ensure that particular versions of each release work together
>> ** these releases would go through our standard release process - ie
>> creation of release candidate by a committer and approval by the PMC
>> ** recipes would no longer be included in "zookeeper" (core) releases
>> * continue to use the ZOOKEEPER jira, with "recipes" component
>> * space in the web and wiki sites for recipes specific pages
>> * recipes continue to use the std zookeeper mailing lists
>> * commits to recipes would continue to use our regular commit process
>> (jira, rtc, etc...)
>> * we would accept new "ZooKeeper committers" with the intent that they
>> focus on their area of expertise, whether this be core (zookeeper
>> trunk) or recipes.
>>
>> This last item is the more interesting I think, the other issues are
>> all pretty straightforward. Although moving to sub (or out entirely)
>> would allow us to have a separate commit list it would fragment the
>> ZooKeeper community. I also believe that over time committers to
>> recipes would more likely get experience with core and start
>> contributing there as well. The only issue really, if you want to call
>> it that, is that we'd have committers with "guardrails" implied. ie
>> someone with more expertise in recipes than core. However there are
>> many other Apache projects that take this approach, and do it
>> successfully. Regardless RTC allows for review both before and after a
>> change is committed.
>>
>> What do you think?
>>
>> Patrick
>>

Re: Proposal: create a separate ZooKeeper release artifact for recipes

[Benjamin Reed <br...@apache.org>]

while i agree with the sentiment of not fragmenting the zookeeper
community and recipe committers also moving into core development, i
also think it would be good if a strong community of developers grew
up around zookeeper recipes. to do that they need a sense of identity,
and i'm not sure if they can get that with this proposal.

on the other hand the proposal does address all of the other issues i
have with recipe maintenance. the key is to grow a committer base.

ben

On Wed, Jul 13, 2011 at 11:04 AM, Patrick Hunt <ph...@apache.org> wrote:
> During the recent developer meetup we discussed how to more
> effectively grow the recipes.
> https://cwiki.apache.org/confluence/display/ZOOKEEPER/June2011DeveloperMeeting
>
> The concerns seemed to be around managing changes, community growth,
> and releases. We discussed a number of approaches:
> * move recipes into apache-extras
> * or github
> * or incubator
> * or sub-project
>
> each of these has it's good and bad points. One alternative that I put
> forth was to create a separate release artifact for recipes as part of
> the ZooKeeper project itself. What would this look like?
>
> * zookeeper/trunk/src/recipes would move to
> zookeeper/recipes/{trunk,branches,tags,...}
> * there would be distinct/separate releases of zookeeper and
> zookeeper-recipes artifacts
> ** these releases would not necessarily be synchronized, although we'd
> want to ensure that particular versions of each release work together
> ** these releases would go through our standard release process - ie
> creation of release candidate by a committer and approval by the PMC
> ** recipes would no longer be included in "zookeeper" (core) releases
> * continue to use the ZOOKEEPER jira, with "recipes" component
> * space in the web and wiki sites for recipes specific pages
> * recipes continue to use the std zookeeper mailing lists
> * commits to recipes would continue to use our regular commit process
> (jira, rtc, etc...)
> * we would accept new "ZooKeeper committers" with the intent that they
> focus on their area of expertise, whether this be core (zookeeper
> trunk) or recipes.
>
> This last item is the more interesting I think, the other issues are
> all pretty straightforward. Although moving to sub (or out entirely)
> would allow us to have a separate commit list it would fragment the
> ZooKeeper community. I also believe that over time committers to
> recipes would more likely get experience with core and start
> contributing there as well. The only issue really, if you want to call
> it that, is that we'd have committers with "guardrails" implied. ie
> someone with more expertise in recipes than core. However there are
> many other Apache projects that take this approach, and do it
> successfully. Regardless RTC allows for review both before and after a
> change is committed.
>
> What do you think?
>
> Patrick
>