Re: [PROPOSAL] Curator for the Apache Incubator

[Greg Stein <gs...@gmail.com>]

My concern is that we're looking at two "new" committers, rather than
a Curator community. Following normal Incubator work, Curator would
build a community for itself. But then we'd have a community
*distinct* from that of Zookeeper. And it really looks like this
should be part of Zookeeper itself -- a more capable and easier-to-use
client.

So I question the incubation of this. Why do we want to build a
new/separate project? Why isn't this just part of Zookeeper right from
the start?

I would suggest that this work is placed on a branch within Zookeeper.
That Jordan and Jay become committers on that branch (not necessarily
Zookeeper trunk). Over time, the branch can be folded into trunk,
along with all the various tests, doc, and other artifacts that I see
in the GitHub repository. And hopefully that Jordan and Jay become
regular committers (and PMC members!) of the Zookeeper project itself.

The current Zookeeper client can remain for backwards compat, and the
Curator work can become the next-gen client.

Honestly, in my opnion, incubating this project seems like it would
create a distinct community, and really doesn't seem like it would
serve the Zookeeper community.

All that said, I am not familiar with the Zookeeper or Curator
communities. But from this read, I don't think Incubation is the right
approach. I would rather push for a more direct incorporation of
Curator directly into Zookeeper. (use the short-form IP clearance) ...
so, unless somebody can help me understand the communities and
situation better, I'm -1 (binding) on this incubation. I'd rather see
combined, rather than distinct, communities from the start.

Thanks,
-g

On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
<jo...@jordanzimmerman.com> wrote:
> Hello,
>
> I would like to propose that Curator to be an Apache Incubator project.
>
> The proposal can be found here: http://wiki.apache.org/incubator/CuratorProposal
>
> I have included the contents of the proposal below.
>
> Sincerely,
>
> Jordan Zimmerman
>
> ===================
>
> = Curator - ZooKeeper client wrapper and rich ZooKeeper framework =
>
> == Abstract ==
>
> Curator is a set of Java libraries that make using Apache ZooKeeper much easier. While ZooKeeper comes bundled with a Java client, using the client is non-trivial and error prone.
>
> == Proposal ==
>
> Curator is a set of Java libraries that make using Apache ZooKeeper much easier. While ZooKeeper comes bundled with a Java client, using the client is non-trivial and error prone. It consists of three components that build on each other. Curator Client is a replacement for the bundled ZooKeeper class that takes care of some low-level housekeeping and provides some useful utilities. Curator Framework is a high-level API that greatly simplifies using ZooKeeper. It adds many features that build on ZooKeeper and handles the complexity of managing connections to the ZooKeeper cluster and retrying operations. Curator Recipes consists of implementations of some of the common ZooKeeper “recipes”. Additionally, Curator Test is included which includes utilities to help with unit testing ZooKeeper-based applications.
>
> == Background ==
>
> Curator was initially developed by Netflix to make writing ZooKeeper-based applications easier and more reliable. Curator was open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project in July 2011. During this time Curator has been formally released many times and has gained widespread adoption.
>
> == Rationale ==
>
> New users of ZooKeeper are surprised to learn that a significant amount of connection management must be done manually. For example, when the ZooKeeper client connects to the ensemble it must negotiate a new session, etc. This takes some time. If you use a ZooKeeper client API before the connection process has completed, ZooKeeper will throw an exception. These types of exceptions are referred to as “recoverable” errors.
> Curator automatically handles connection management, greatly simplifying client code. Instead of directly using the ZooKeeper APIs you use Curator APIs that internally check for connection completion and wrap each ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle recoverable errors and automatically retry operations. The method of retry is customizable. Curator comes bundled with several implementations (ExponentialBackoffRetry, etc.) or custom implementations can be written.
>
> The ZooKeeper documentation describes many possible uses for ZooKeeper calling each a “recipe”. While the distribution comes bundled with a few implementations of these recipes, most ZooKeeper users will need to manually implement one or more of the recipes. Implementing a ZooKeeper recipe is not trivial. Besides the connection handling issues, there are numerous edge cases that are not well documented that must be considered. For example, many recipes require that an ephemeral-sequential node be created. New users of ZooKeeper will not know that there is an edge case in ephemeral-sequential node creation that requires you to put a special “marker” in the node’s name so that you can search for the created node if an I/O failure occurs. This is but one of many edge cases that are not well documented but are handled by Curator.
>
> = Current Status =
>
> == Meritocracy ==
>
> Curator was initially developed by Jordan Zimmerman in 2011 at Netflix. Developers external to Netflix provided feedback, suggested features and fixes and implemented extensions of Curator. Netflix's engineering team has since maintained the project and has been dedicated towards its improvement. Contributors to Curator include developers from multiple organizations around the world. Curator will be a meritocracy as it enters the Incubator and beyond.
>
> == Community ==
>
> Curator is currently used by a number of organizations all over the world. Curator has an active and growing user and developer community with active participation in the [[http://groups.google.com/group/curator-users]] mailing list and at its !Github home: [[https://github.com/Netflix/curator]].
>
> Since open sourcing the project, there have been fifteen individuals from various organizations who have contributed code.
>
> == Core Developers ==
>
> The core developers for Curator are:
>  * Jordan Zimmerman
>  * Jay Zarfoss
>
> Jordan has contributed towards Apache ZooKeeper and both Jordan and Jay are familiar with Apache principles and philosophy for community driven software development.
>
> == Alignment ==
>
> Curator is a natural complement for Apache ZooKeeper. Java users of ZooKeeper will naturally want to use Curator. When Curator graduates from Incubator it may be useful to distribute Curator artifacts as part of ZooKeeper releases as the preferred/recommended client side library. Further, at graduation a determination can be made as to whether Curator should become a Top Level Project or be merged into ZooKeeper itself.
>
> = Known Risks =
>
> == Orphaned Products ==
>
> Curator is already deployed in production at multiple companies and they are actively participating in creating new features. Curator is getting traction with developers and thus the risks of it being orphaned are minimal.
>
> == Inexperience with Open Source ==
>
> All code developed for Curator has been open sourced by Netflix under Apache 2.0 license.  All committers to Curator are intimately familiar with the Apache model for open-source development and are experienced with working with new contributors.
>
> == Homogeneous Developers ==
>
> The initial committers are from a single organization. However, we expect that once approved for incubation, the project will attract new contributors from diverse organizations and will thus grow organically. The submission of patches from developers from several different organizations is a strong indication that Curator will be widely adopted.
>
> == Reliance on Salaried Developers ==
>
> It is expected that Curator will be developed on salaried and volunteer time, although all of the initial developers will work on it mainly on salaried time.
>
> == Relationships with Other Apache Products ==
>
> Curator depends upon other Apache Projects: Apache ZooKeeper, Apache Log4J, and multiple Apache Commons components. Its build depends upon Apache Maven. Notably, there is interest from other Apache Projects such as HBase in adopting Curator as the client library for ZooKeeper. Apache James Mailbox has already incorporated Curator.
>
> == An Excessive Fascination with the Apache Brand ==
>
> We would like Curator to become an Apache project to further foster a healthy community of contributors and consumers around the project.  Since Curator directly interacts with Apache ZooKeeper and solves an important problem of many ZooKeeper users, residing in the Apache Software Foundation will increase interaction with the larger community.
>
> = Documentation =
>
>  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
>  * Curator issues at GitHub: https://github.com/Netflix/curator/issues
>  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
>
> = Initial Source =
>
>  * git://github.com/Netflix/curator.git
>
> == Source and Intellectual Property Submission Plan ==
>
>  * The initial source is already licensed under the Apache License, Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
>
> == External Dependencies ==
>
> The required external dependencies are all Apache License or compatible licenses. Following components with non-Apache licenses are enumerated:
>
>  * org.slf4j: MIT-like License
>  * org.mockito: MIT-like License
>
> == Cryptography ==
>
> Curator contains no known cryptography.
>
> = Required Resources =
>
> == Mailing lists ==
>
>  * curator-private (with moderated subscriptions)
>  * curator-dev
>  * curator-commits
>  * curator-user
>
> == Github Repositories ==
>
> http://github.com/apache/curator git://git.apache.org/curator.git
>
> == Issue Tracking ==
>
> JIRA Curator (CURATOR)
>
> == Other Resources ==
>
> The existing code already has unit and integration tests so we would like a Jenkins instance to run them whenever a new patch is submitted. This can be added after project creation.
>
> = Initial Committers =
>
>  * Jordan Zimmerman (jzimmerman at netflix dot com)
>  * Jay Zarfoss (jzarfoss at netflix dot com)
>
> = Affiliations =
>
>  * Jordan Zimmerman, Netflix
>  * Jay Zarfoss, Netflix
>
> = Sponsors =
>
> == Champion ==
>
>  * Patrick Hunt
>
> == Nominated Mentors ==
>
>  * Patrick Hunt
>  * Enis Söztutar
>  * Mahadev Konar
>
> == Sponsoring Entity ==
>
>  * Apache Incubator PMC

Re: [PROPOSAL] Curator for the Apache Incubator

[Eric Charles <er...@apache.org>]

Hi,

Inputs from a current Curator user:

I followed one year ago the public thread Patrick mentionned and was 
surprised to see the lack of interest for an higher level Zk client.

If this discussion happened now, maybe the interest would be different, 
but still, I think Curator should be hosted in its own community for the 
following reasons:

1.- I would love to have a client that support multiple server types. I 
use for example Curator to take distributed locks on Zk servers, but 
other servers (Hazelcast...) also implement this functionality. Simply 
by configuration, I would love Curator to give me access to those 
servers without chaning my code. Ok, This is not in the proposal scope, 
but having a separate community opens to door to those type of 
contributions. Obviously, if Curator is part of Zookeeper, this will be 
more difficult.

2.- Zookeeper is getting more and more functionalities (Bookeeper, 
Hedwig...) but not all of these functionalities may find an client part 
in Curator. This is why (see my first point), Curator should not be 
stricly a full Zk client, but a client providing access in some way to 
some distributed server functions.

3.- Client dev (internal machinery such injection, configuration...) is 
quite different from server dev. Some client architecture decisions 
should not be depende on server dev habits and server developers tend to 
negligate clients. I feel everyday the pain of real 
hadoop/hbase/casssandra/... client (I mean without the server jars). 
This is evolving, but not having a separate project for client does not 
help. HTTP client have never been developer in server projects.

I agree that hosting in the same community can create some convergence 
and ease the communication (API, bug solving...), but I still feel 
Curator deserves a dedicated community, especially for the first reason 
I have listed (client towards multiple servers).

Thx, Eric


On 26/02/2013 06:10, Patrick Hunt wrote:
> On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com> wrote:
>> My concern is that we're looking at two "new" committers, rather than
>> a Curator community. Following normal Incubator work, Curator would
>> build a community for itself. But then we'd have a community
>> *distinct* from that of Zookeeper. And it really looks like this
>> should be part of Zookeeper itself -- a more capable and easier-to-use
>> client.
>>
>> So I question the incubation of this. Why do we want to build a
>> new/separate project? Why isn't this just part of Zookeeper right from
>> the start?
>>
>> I would suggest that this work is placed on a branch within Zookeeper.
>> That Jordan and Jay become committers on that branch (not necessarily
>> Zookeeper trunk). Over time, the branch can be folded into trunk,
>> along with all the various tests, doc, and other artifacts that I see
>> in the GitHub repository. And hopefully that Jordan and Jay become
>> regular committers (and PMC members!) of the Zookeeper project itself.
>>
>> The current Zookeeper client can remain for backwards compat, and the
>> Curator work can become the next-gen client.
>>
>> Honestly, in my opnion, incubating this project seems like it would
>> create a distinct community, and really doesn't seem like it would
>> serve the Zookeeper community.
>>
>> All that said, I am not familiar with the Zookeeper or Curator
>> communities. But from this read, I don't think Incubation is the right
>> approach. I would rather push for a more direct incorporation of
>> Curator directly into Zookeeper. (use the short-form IP clearance) ...
>> so, unless somebody can help me understand the communities and
>> situation better, I'm -1 (binding) on this incubation. I'd rather see
>> combined, rather than distinct, communities from the start.
>
> The Curator library is built upon the current ZooKeeper client
> libraries. They are an extension that implements higher level use
> cases (what we call "recipes" in ZK) whereas the ZK libraries
> implement lower level primitives. An analogy might be Apache Crunch
> (recently graduated to TLP) and MapReduce. Also, not everyone agrees
> with Jordan's assessment that Curator is 'better" than (or a
> replacement for) the existing client libraries.
>
> The ZK community discussed incorporating the Curator code about a year
> ago. At that time there wasn't interest to adopt these libraries into
> ZK itself. My belief is that if Curator were to go through incubation
> the ZK community would be more likely to adopt it. Similar to how
> HCatalog spun off to work on the metastore and recently merged back
> into Hive. If there's significant overlap in the Curator podling
> community and the ZK community that will be a strong indication that
> they should be merged (my belief). If not it would have the
> opportunity to go TLP.
>
> Here's my report (ZooKeeper) to the board in Dec 2011 and the response
> I received. This is one of the reasons I advised Jordan to go the
> incubator approach. There's also the discussion thread on ZooKeeper
> dev if you want more insight.
>
>>> A discussion is currently under way regarding the possibility of> merging the recently open sourced "Curator" source base from> Netflix. These are client implementations of ZooKeeper "recipes"> (e.g. leadership election, group membership, etc...) which simplify> the act of using ZooKeeper in client side applications.>> The authors of Curator are unwilling to join the incubator, this is> based on their past experience as well as some of the ongoing issues> they are seeing wrt projects entering the incubator. They have> expressed a preference to come in either as a subproject or as a> separate release artifact of the TLP.
>>
>> to which the board responded:
>>
>> Doug Cutting (chairman):
>> +               Long-term, do you think that Curator will have an
>> +               indepdendent community from Zookeeper?  If so, then it
>> +               ought to enter through the Incubator.  If not then the
>> +               code might still enter through the incubator for
>> +               resolution of IP issues, but then transfer relatively
>> +               quickly to live under the Zookeeper PMC.  Or the
>> +               Zookeeper PMC could directly adopt the code, although
>> +               you might not immediately add its committers to the
>> +               project but rather have them first contribute patches
>> +               through the normal process until their community merit
>> +               is established.
>>
>> Sam Ruby:
>> + suggested that the "short form" of the incubator's IP clearance
>> process would be appropriate if Zookeeper directly adopts this code.
>>
>> The short form can be found here:
>> http://incubator.apache.org/ip-clearance/index.html
>
>
> Regards,
>
> Patrick
>
>>
>> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>> <jo...@jordanzimmerman.com> wrote:
>>> Hello,
>>>
>>> I would like to propose that Curator to be an Apache Incubator project.
>>>
>>> The proposal can be found here: http://wiki.apache.org/incubator/CuratorProposal
>>>
>>> I have included the contents of the proposal below.
>>>
>>> Sincerely,
>>>
>>> Jordan Zimmerman
>>>
>>> ===================
>>>
>>> = Curator - ZooKeeper client wrapper and rich ZooKeeper framework =
>>>
>>> == Abstract ==
>>>
>>> Curator is a set of Java libraries that make using Apache ZooKeeper much easier. While ZooKeeper comes bundled with a Java client, using the client is non-trivial and error prone.
>>>
>>> == Proposal ==
>>>
>>> Curator is a set of Java libraries that make using Apache ZooKeeper much easier. While ZooKeeper comes bundled with a Java client, using the client is non-trivial and error prone. It consists of three components that build on each other. Curator Client is a replacement for the bundled ZooKeeper class that takes care of some low-level housekeeping and provides some useful utilities. Curator Framework is a high-level API that greatly simplifies using ZooKeeper. It adds many features that build on ZooKeeper and handles the complexity of managing connections to the ZooKeeper cluster and retrying operations. Curator Recipes consists of implementations of some of the common ZooKeeper ?recipes?. Additionally, Curator Test is included which includes utilities to help with unit testing ZooKeeper-based applications.
>>>
>>> == Background ==
>>>
>>> Curator was initially developed by Netflix to make writing ZooKeeper-based applications easier and more reliable. Curator was open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project in July 2011. During this time Curator has been formally released many times and has gained widespread adoption.
>>>
>>> == Rationale ==
>>>
>>> New users of ZooKeeper are surprised to learn that a significant amount of connection management must be done manually. For example, when the ZooKeeper client connects to the ensemble it must negotiate a new session, etc. This takes some time. If you use a ZooKeeper client API before the connection process has completed, ZooKeeper will throw an exception. These types of exceptions are referred to as ?recoverable? errors.
>>> Curator automatically handles connection management, greatly simplifying client code. Instead of directly using the ZooKeeper APIs you use Curator APIs that internally check for connection completion and wrap each ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle recoverable errors and automatically retry operations. The method of retry is customizable. Curator comes bundled with several implementations (ExponentialBackoffRetry, etc.) or custom implementations can be written.
>>>
>>> The ZooKeeper documentation describes many possible uses for ZooKeeper calling each a ?recipe?. While the distribution comes bundled with a few implementations of these recipes, most ZooKeeper users will need to manually implement one or more of the recipes. Implementing a ZooKeeper recipe is not trivial. Besides the connection handling issues, there are numerous edge cases that are not well documented that must be considered. For example, many recipes require that an ephemeral-sequential node be created. New users of ZooKeeper will not know that there is an edge case in ephemeral-sequential node creation that requires you to put a special ?marker? in the node?s name so that you can search for the created node if an I/O failure occurs. This is but one of many edge cases that are not well documented but are handled by Curator.
>>>
>>> = Current Status =
>>>
>>> == Meritocracy ==
>>>
>>> Curator was initially developed by Jordan Zimmerman in 2011 at Netflix. Developers external to Netflix provided feedback, suggested features and fixes and implemented extensions of Curator. Netflix's engineering team has since maintained the project and has been dedicated towards its improvement. Contributors to Curator include developers from multiple organizations around the world. Curator will be a meritocracy as it enters the Incubator and beyond.
>>>
>>> == Community ==
>>>
>>> Curator is currently used by a number of organizations all over the world. Curator has an active and growing user and developer community with active participation in the [[http://groups.google.com/group/curator-users]] mailing list and at its !Github home: [[https://github.com/Netflix/curator]].
>>>
>>> Since open sourcing the project, there have been fifteen individuals from various organizations who have contributed code.
>>>
>>> == Core Developers ==
>>>
>>> The core developers for Curator are:
>>>   * Jordan Zimmerman
>>>   * Jay Zarfoss
>>>
>>> Jordan has contributed towards Apache ZooKeeper and both Jordan and Jay are familiar with Apache principles and philosophy for community driven software development.
>>>
>>> == Alignment ==
>>>
>>> Curator is a natural complement for Apache ZooKeeper. Java users of ZooKeeper will naturally want to use Curator. When Curator graduates from Incubator it may be useful to distribute Curator artifacts as part of ZooKeeper releases as the preferred/recommended client side library. Further, at graduation a determination can be made as to whether Curator should become a Top Level Project or be merged into ZooKeeper itself.
>>>
>>> = Known Risks =
>>>
>>> == Orphaned Products ==
>>>
>>> Curator is already deployed in production at multiple companies and they are actively participating in creating new features. Curator is getting traction with developers and thus the risks of it being orphaned are minimal.
>>>
>>> == Inexperience with Open Source ==
>>>
>>> All code developed for Curator has been open sourced by Netflix under Apache 2.0 license.  All committers to Curator are intimately familiar with the Apache model for open-source development and are experienced with working with new contributors.
>>>
>>> == Homogeneous Developers ==
>>>
>>> The initial committers are from a single organization. However, we expect that once approved for incubation, the project will attract new contributors from diverse organizations and will thus grow organically. The submission of patches from developers from several different organizations is a strong indication that Curator will be widely adopted.
>>>
>>> == Reliance on Salaried Developers ==
>>>
>>> It is expected that Curator will be developed on salaried and volunteer time, although all of the initial developers will work on it mainly on salaried time.
>>>
>>> == Relationships with Other Apache Products ==
>>>
>>> Curator depends upon other Apache Projects: Apache ZooKeeper, Apache Log4J, and multiple Apache Commons components. Its build depends upon Apache Maven. Notably, there is interest from other Apache Projects such as HBase in adopting Curator as the client library for ZooKeeper. Apache James Mailbox has already incorporated Curator.
>>>
>>> == An Excessive Fascination with the Apache Brand ==
>>>
>>> We would like Curator to become an Apache project to further foster a healthy community of contributors and consumers around the project.  Since Curator directly interacts with Apache ZooKeeper and solves an important problem of many ZooKeeper users, residing in the Apache Software Foundation will increase interaction with the larger community.
>>>
>>> = Documentation =
>>>
>>>   * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
>>>   * Curator issues at GitHub: https://github.com/Netflix/curator/issues
>>>   * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
>>>
>>> = Initial Source =
>>>
>>>   * git://github.com/Netflix/curator.git
>>>
>>> == Source and Intellectual Property Submission Plan ==
>>>
>>>   * The initial source is already licensed under the Apache License, Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>>
>>> == External Dependencies ==
>>>
>>> The required external dependencies are all Apache License or compatible licenses. Following components with non-Apache licenses are enumerated:
>>>
>>>   * org.slf4j: MIT-like License
>>>   * org.mockito: MIT-like License
>>>
>>> == Cryptography ==
>>>
>>> Curator contains no known cryptography.
>>>
>>> = Required Resources =
>>>
>>> == Mailing lists ==
>>>
>>>   * curator-private (with moderated subscriptions)
>>>   * curator-dev
>>>   * curator-commits
>>>   * curator-user
>>>
>>> == Github Repositories ==
>>>
>>> http://github.com/apache/curator git://git.apache.org/curator.git
>>>
>>> == Issue Tracking ==
>>>
>>> JIRA Curator (CURATOR)
>>>
>>> == Other Resources ==
>>>
>>> The existing code already has unit and integration tests so we would like a Jenkins instance to run them whenever a new patch is submitted. This can be added after project creation.
>>>
>>> = Initial Committers =
>>>
>>>   * Jordan Zimmerman (jzimmerman at netflix dot com)
>>>   * Jay Zarfoss (jzarfoss at netflix dot com)
>>>
>>> = Affiliations =
>>>
>>>   * Jordan Zimmerman, Netflix
>>>   * Jay Zarfoss, Netflix
>>>
>>> = Sponsors =
>>>
>>> == Champion ==
>>>
>>>   * Patrick Hunt
>>>
>>> == Nominated Mentors ==
>>>
>>>   * Patrick Hunt
>>>   * Enis S�ztutar
>>>   * Mahadev Konar
>>>
>>> == Sponsoring Entity ==
>>>
>>>   * Apache Incubator PMC
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Patrick Hunt <ph...@apache.org>]

On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com> wrote:
> My concern is that we're looking at two "new" committers, rather than
> a Curator community. Following normal Incubator work, Curator would
> build a community for itself. But then we'd have a community
> *distinct* from that of Zookeeper. And it really looks like this
> should be part of Zookeeper itself -- a more capable and easier-to-use
> client.
>
> So I question the incubation of this. Why do we want to build a
> new/separate project? Why isn't this just part of Zookeeper right from
> the start?
>
> I would suggest that this work is placed on a branch within Zookeeper.
> That Jordan and Jay become committers on that branch (not necessarily
> Zookeeper trunk). Over time, the branch can be folded into trunk,
> along with all the various tests, doc, and other artifacts that I see
> in the GitHub repository. And hopefully that Jordan and Jay become
> regular committers (and PMC members!) of the Zookeeper project itself.
>
> The current Zookeeper client can remain for backwards compat, and the
> Curator work can become the next-gen client.
>
> Honestly, in my opnion, incubating this project seems like it would
> create a distinct community, and really doesn't seem like it would
> serve the Zookeeper community.
>
> All that said, I am not familiar with the Zookeeper or Curator
> communities. But from this read, I don't think Incubation is the right
> approach. I would rather push for a more direct incorporation of
> Curator directly into Zookeeper. (use the short-form IP clearance) ...
> so, unless somebody can help me understand the communities and
> situation better, I'm -1 (binding) on this incubation. I'd rather see
> combined, rather than distinct, communities from the start.

The Curator library is built upon the current ZooKeeper client
libraries. They are an extension that implements higher level use
cases (what we call "recipes" in ZK) whereas the ZK libraries
implement lower level primitives. An analogy might be Apache Crunch
(recently graduated to TLP) and MapReduce. Also, not everyone agrees
with Jordan's assessment that Curator is 'better" than (or a
replacement for) the existing client libraries.

The ZK community discussed incorporating the Curator code about a year
ago. At that time there wasn't interest to adopt these libraries into
ZK itself. My belief is that if Curator were to go through incubation
the ZK community would be more likely to adopt it. Similar to how
HCatalog spun off to work on the metastore and recently merged back
into Hive. If there's significant overlap in the Curator podling
community and the ZK community that will be a strong indication that
they should be merged (my belief). If not it would have the
opportunity to go TLP.

Here's my report (ZooKeeper) to the board in Dec 2011 and the response
I received. This is one of the reasons I advised Jordan to go the
incubator approach. There's also the discussion thread on ZooKeeper
dev if you want more insight.

>> A discussion is currently under way regarding the possibility of> merging the recently open sourced "Curator" source base from> Netflix. These are client implementations of ZooKeeper "recipes"> (e.g. leadership election, group membership, etc...) which simplify> the act of using ZooKeeper in client side applications.>> The authors of Curator are unwilling to join the incubator, this is> based on their past experience as well as some of the ongoing issues> they are seeing wrt projects entering the incubator. They have> expressed a preference to come in either as a subproject or as a> separate release artifact of the TLP.
>
> to which the board responded:
>
> Doug Cutting (chairman):
> +               Long-term, do you think that Curator will have an
> +               indepdendent community from Zookeeper?  If so, then it
> +               ought to enter through the Incubator.  If not then the
> +               code might still enter through the incubator for
> +               resolution of IP issues, but then transfer relatively
> +               quickly to live under the Zookeeper PMC.  Or the
> +               Zookeeper PMC could directly adopt the code, although
> +               you might not immediately add its committers to the
> +               project but rather have them first contribute patches
> +               through the normal process until their community merit
> +               is established.
>
> Sam Ruby:
> + suggested that the "short form" of the incubator's IP clearance
> process would be appropriate if Zookeeper directly adopts this code.
>
> The short form can be found here:
> http://incubator.apache.org/ip-clearance/index.html


Regards,

Patrick

>
> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
> <jo...@jordanzimmerman.com> wrote:
>> Hello,
>>
>> I would like to propose that Curator to be an Apache Incubator project.
>>
>> The proposal can be found here: http://wiki.apache.org/incubator/CuratorProposal
>>
>> I have included the contents of the proposal below.
>>
>> Sincerely,
>>
>> Jordan Zimmerman
>>
>> ===================
>>
>> = Curator - ZooKeeper client wrapper and rich ZooKeeper framework =
>>
>> == Abstract ==
>>
>> Curator is a set of Java libraries that make using Apache ZooKeeper much easier. While ZooKeeper comes bundled with a Java client, using the client is non-trivial and error prone.
>>
>> == Proposal ==
>>
>> Curator is a set of Java libraries that make using Apache ZooKeeper much easier. While ZooKeeper comes bundled with a Java client, using the client is non-trivial and error prone. It consists of three components that build on each other. Curator Client is a replacement for the bundled ZooKeeper class that takes care of some low-level housekeeping and provides some useful utilities. Curator Framework is a high-level API that greatly simplifies using ZooKeeper. It adds many features that build on ZooKeeper and handles the complexity of managing connections to the ZooKeeper cluster and retrying operations. Curator Recipes consists of implementations of some of the common ZooKeeper “recipes”. Additionally, Curator Test is included which includes utilities to help with unit testing ZooKeeper-based applications.
>>
>> == Background ==
>>
>> Curator was initially developed by Netflix to make writing ZooKeeper-based applications easier and more reliable. Curator was open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project in July 2011. During this time Curator has been formally released many times and has gained widespread adoption.
>>
>> == Rationale ==
>>
>> New users of ZooKeeper are surprised to learn that a significant amount of connection management must be done manually. For example, when the ZooKeeper client connects to the ensemble it must negotiate a new session, etc. This takes some time. If you use a ZooKeeper client API before the connection process has completed, ZooKeeper will throw an exception. These types of exceptions are referred to as “recoverable” errors.
>> Curator automatically handles connection management, greatly simplifying client code. Instead of directly using the ZooKeeper APIs you use Curator APIs that internally check for connection completion and wrap each ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle recoverable errors and automatically retry operations. The method of retry is customizable. Curator comes bundled with several implementations (ExponentialBackoffRetry, etc.) or custom implementations can be written.
>>
>> The ZooKeeper documentation describes many possible uses for ZooKeeper calling each a “recipe”. While the distribution comes bundled with a few implementations of these recipes, most ZooKeeper users will need to manually implement one or more of the recipes. Implementing a ZooKeeper recipe is not trivial. Besides the connection handling issues, there are numerous edge cases that are not well documented that must be considered. For example, many recipes require that an ephemeral-sequential node be created. New users of ZooKeeper will not know that there is an edge case in ephemeral-sequential node creation that requires you to put a special “marker” in the node’s name so that you can search for the created node if an I/O failure occurs. This is but one of many edge cases that are not well documented but are handled by Curator.
>>
>> = Current Status =
>>
>> == Meritocracy ==
>>
>> Curator was initially developed by Jordan Zimmerman in 2011 at Netflix. Developers external to Netflix provided feedback, suggested features and fixes and implemented extensions of Curator. Netflix's engineering team has since maintained the project and has been dedicated towards its improvement. Contributors to Curator include developers from multiple organizations around the world. Curator will be a meritocracy as it enters the Incubator and beyond.
>>
>> == Community ==
>>
>> Curator is currently used by a number of organizations all over the world. Curator has an active and growing user and developer community with active participation in the [[http://groups.google.com/group/curator-users]] mailing list and at its !Github home: [[https://github.com/Netflix/curator]].
>>
>> Since open sourcing the project, there have been fifteen individuals from various organizations who have contributed code.
>>
>> == Core Developers ==
>>
>> The core developers for Curator are:
>>  * Jordan Zimmerman
>>  * Jay Zarfoss
>>
>> Jordan has contributed towards Apache ZooKeeper and both Jordan and Jay are familiar with Apache principles and philosophy for community driven software development.
>>
>> == Alignment ==
>>
>> Curator is a natural complement for Apache ZooKeeper. Java users of ZooKeeper will naturally want to use Curator. When Curator graduates from Incubator it may be useful to distribute Curator artifacts as part of ZooKeeper releases as the preferred/recommended client side library. Further, at graduation a determination can be made as to whether Curator should become a Top Level Project or be merged into ZooKeeper itself.
>>
>> = Known Risks =
>>
>> == Orphaned Products ==
>>
>> Curator is already deployed in production at multiple companies and they are actively participating in creating new features. Curator is getting traction with developers and thus the risks of it being orphaned are minimal.
>>
>> == Inexperience with Open Source ==
>>
>> All code developed for Curator has been open sourced by Netflix under Apache 2.0 license.  All committers to Curator are intimately familiar with the Apache model for open-source development and are experienced with working with new contributors.
>>
>> == Homogeneous Developers ==
>>
>> The initial committers are from a single organization. However, we expect that once approved for incubation, the project will attract new contributors from diverse organizations and will thus grow organically. The submission of patches from developers from several different organizations is a strong indication that Curator will be widely adopted.
>>
>> == Reliance on Salaried Developers ==
>>
>> It is expected that Curator will be developed on salaried and volunteer time, although all of the initial developers will work on it mainly on salaried time.
>>
>> == Relationships with Other Apache Products ==
>>
>> Curator depends upon other Apache Projects: Apache ZooKeeper, Apache Log4J, and multiple Apache Commons components. Its build depends upon Apache Maven. Notably, there is interest from other Apache Projects such as HBase in adopting Curator as the client library for ZooKeeper. Apache James Mailbox has already incorporated Curator.
>>
>> == An Excessive Fascination with the Apache Brand ==
>>
>> We would like Curator to become an Apache project to further foster a healthy community of contributors and consumers around the project.  Since Curator directly interacts with Apache ZooKeeper and solves an important problem of many ZooKeeper users, residing in the Apache Software Foundation will increase interaction with the larger community.
>>
>> = Documentation =
>>
>>  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
>>  * Curator issues at GitHub: https://github.com/Netflix/curator/issues
>>  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
>>
>> = Initial Source =
>>
>>  * git://github.com/Netflix/curator.git
>>
>> == Source and Intellectual Property Submission Plan ==
>>
>>  * The initial source is already licensed under the Apache License, Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>
>> == External Dependencies ==
>>
>> The required external dependencies are all Apache License or compatible licenses. Following components with non-Apache licenses are enumerated:
>>
>>  * org.slf4j: MIT-like License
>>  * org.mockito: MIT-like License
>>
>> == Cryptography ==
>>
>> Curator contains no known cryptography.
>>
>> = Required Resources =
>>
>> == Mailing lists ==
>>
>>  * curator-private (with moderated subscriptions)
>>  * curator-dev
>>  * curator-commits
>>  * curator-user
>>
>> == Github Repositories ==
>>
>> http://github.com/apache/curator git://git.apache.org/curator.git
>>
>> == Issue Tracking ==
>>
>> JIRA Curator (CURATOR)
>>
>> == Other Resources ==
>>
>> The existing code already has unit and integration tests so we would like a Jenkins instance to run them whenever a new patch is submitted. This can be added after project creation.
>>
>> = Initial Committers =
>>
>>  * Jordan Zimmerman (jzimmerman at netflix dot com)
>>  * Jay Zarfoss (jzarfoss at netflix dot com)
>>
>> = Affiliations =
>>
>>  * Jordan Zimmerman, Netflix
>>  * Jay Zarfoss, Netflix
>>
>> = Sponsors =
>>
>> == Champion ==
>>
>>  * Patrick Hunt
>>
>> == Nominated Mentors ==
>>
>>  * Patrick Hunt
>>  * Enis Söztutar
>>  * Mahadev Konar
>>
>> == Sponsoring Entity ==
>>
>>  * Apache Incubator PMC
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

Re: [PROPOSAL] Curator for the Apache Incubator

[Greg Stein <gs...@gmail.com>]

On Tue, Feb 26, 2013 at 2:23 AM, Patrick Hunt <ph...@apache.org> wrote:
> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra <he...@gmail.com> wrote:
>> So isnt this similar to HCatalog which relying on Hive metadata service
>> that ends up as sub project of Apache Hive?
>>
>
> I was against having Curator as a sub when it came up on the original
> discussion thread, I still am.

My point exactly. Stop talking about "sub-projects" because the Board
does not like them.

Integrate Curator into Zookeeper, rather than creating an entirely
separate community. I see no explanation for why Curator is/will be a
separate community.

All that said, if ZK feels that the direction espoused by Curator is
something they do not feel is appropriate, and that they reject...
then, okay. Let Curator blossom as a contrary example to ZK's clients.

Cheers,
-g

Re: [PROPOSAL] Curator for the Apache Incubator

[Greg Stein <gs...@gmail.com>]

On Tue, Feb 26, 2013 at 2:23 AM, Patrick Hunt <ph...@apache.org> wrote:
> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra <he...@gmail.com> wrote:
>> So isnt this similar to HCatalog which relying on Hive metadata service
>> that ends up as sub project of Apache Hive?
>>
>
> I was against having Curator as a sub when it came up on the original
> discussion thread, I still am.

My point exactly. Stop talking about "sub-projects" because the Board
does not like them.

Integrate Curator into Zookeeper, rather than creating an entirely
separate community. I see no explanation for why Curator is/will be a
separate community.

All that said, if ZK feels that the direction espoused by Curator is
something they do not feel is appropriate, and that they reject...
then, okay. Let Curator blossom as a contrary example to ZK's clients.

Cheers,
-g

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Patrick Hunt <ph...@apache.org>]

On Tue, Feb 26, 2013 at 1:10 PM, Mattmann, Chris A (388J)
<ch...@jpl.nasa.gov> wrote:
> Hey Pat,
>
>
> On 2/26/13 11:39 AM, "Patrick Hunt" <ph...@apache.org> wrote:
>
>>[…snip…]
>>>
>>> Either: (a) define Curator to be its own separate project/community,
>>>with
>>> a goal of TLP;
>>> or (b) nix Incubation and just make these guys part of ZK, on a branch
>>> now.
>>
>>Hi Chris. Unfortunately it's not clear cut to me where Curator should
>>go, perhaps because I'm close to it.
>
> Yep, I think that's part of it, but you're doing good discussing it here,
> and please don't take my comments as criticism. If the answer were to be
> that this project followed the Incubator process, and arrived at TLP, I
> would have *zero* concern. My concern is the implication that this could
> end up as part of Zookeeper (ZK). It's more of a general concern about
> allowing
> that within the Incubator. I'll comment more below.
>

Hey Chris, no worries on my end what so ever. I appreciate your
insight. If you remember you're the reason I'm in this mess in the
first place. ;-) (you pulled me into my first incubation project as a
mentor, now I'm hooked)

>>My hope was that Curator could come into the incubator, work through
>>the IP issues and other incubation activities, learn about Apache
>>process, and then decide based on the new found insight/experience.
>
> I totally get it.
>
> I'm not questioning you, more of the Incubator here.
>

That's cool. However changing incubator policy is outside the scope of
this discussion - Curator's specific proposal. That's for a different
day.

> Let me cut to the chase. I *don't* think the Incubator should be home to
> projects
> whose potential destination is as a product of an existing TLP (or
> sub-module).
> I think that leads to a LOT of potential problems down the road, some of
> which
> include:
>
> 1. The dev that happens in the Incubator on new podling X, whose
> destination is TLP
> Y is rarely monitored or seen by those in TLP Y. If it is closely
> monitored, then
> why isn't the new podling part of TLP Y? And if it's not monitored, then
> TLP Y is
> more likely to end up at some forced agreement or bylaw mumbo jumbo to
> assimilate the
> people and products of podling X into the TLP Y.
>
> 2. Creating the new community in the Incubator means that new people will
> be added
> as members of podling X that TLP Y didn't add. We've seen problems with
> TLP Y then not
> wanting to assimilate those podling X PPMC members into the TLP Y.
>
> 3. Sub-modules tend to hide things like sub-projects. I can count on two
> hands how many
> TLPs have went through that process, and all of them typically have had
> hardship and
> crap along the way that I'm trying to save you guys the hassle of
> ("lessons learned")
>
> Instead, if the home for a potential new podling X is TLP Y, my desire
> would be to
> work as an existing member of TLP Y to shepherd the contributions in that
> way.
>
>>
>>The proposal guide talks about this and makes it clear (at least to
>>me) that leaving graduation open is not only an option, but a
>>requirement "Note that the final destination within the Apache
>>organizational structure will be decided upon graduation." Given this
>>I don't see why we'd artificially constrain things up front as you've
>>suggested.
>
> Well the proposal guide is great, and all but it's not doctrine :)
>

One of my favorites. Either it's not written down, or it is. In either
case its what we want it to be.

> I realize you're in a tough spot here and somewhat in a COI, not exactly
> knowing what the right answer is,
> and I'm not sure I know it either, but my gut instinct based on experience
> for a while
> here indicates that Incubation makes sense here and this is a new
> community and so I won't
> stand in the way of that. I will however, throw up my comments and a fuss
> again if this is another
> situation similar to HCatalog where at the end of this you guys try and
> shove it into ZK
> in any other way of intersect([set of Curator PPMC], [set of ZK PMC]).
>

My own view is that if it goes into incubation and then to ZK or TLP
we'lll straighten it out. A worse situation would be to lose the
opportunity to try.

> That's my ultimate worry, and the Incubator is not a clearinghouse for
> punting on those
> types of situations at least not the Incubator I would like to participate
> in.

Think about it this way. If we fail miserably you'll have a great
example to use the next time it comes up. ;-)

Patrick


>
>
>>
>>Patrick
>>
>>
>>>
>>> On 2/26/13 9:40 AM, "Benson Margulies" <bi...@gmail.com> wrote:
>>>
>>>>On Tue, Feb 26, 2013 at 12:22 PM, Patrick Hunt <ph...@apache.org> wrote:
>>>>> On Tue, Feb 26, 2013 at 8:32 AM, Benson Margulies
>>>>><bi...@gmail.com> wrote:
>>>>>> If you think that the right destination for curator is as part of ZK,
>>>>>> then it would be good to see substantive participation of the ZK PMC
>>>>>> in the incubation. The goal should be to 'graduate' by having the
>>>>>> curator community be granted karma at ZK and the code folded in. This
>>>>>> would require, I think, the ZK community to supply at least one
>>>>>> mentor, and to have a plan in advance for the eventual votes based on
>>>>>> success in the incubator.
>>>>>
>>>>> Hi Benson. What you're suggesting matches my current thinking.
>>>>>
>>>>> The three Curator mentors are as follows: Mahadev and I (champion) are
>>>>> both PMC members on ZK, Enis Söztutar is active on HBase and
>>>>> interested in using Curator for that project (which already uses ZK
>>>>> heavily).
>>>>
>>>>OK, that's lovely.
>>>>
>>>>
>>>>>
>>>>> Patrick
>>>>>
>>>>>> On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra
>>>>>><he...@gmail.com> wrote:
>>>>>>> Ah no, I was not suggesting about Curator to become subproject of
>>>>>>>ZK.
>>>>>>>I
>>>>>>> just afraid that if Curator is going as incubator it will end up as
>>>>>>>sub of
>>>>>>> ZK as merging process.
>>>>>>>
>>>>>>> Like Greg has mentioned in another reply, I would prefer Curator to
>>>>>>>be
>>>>>>> merged as a higher level ZK client. Surely project like HBase and
>>>>>>>others
>>>>>>> that relying on ZK would appreciate simpler client to ZK.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Henry
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <ph...@apache.org>
>>>>>>>wrote:
>>>>>>>
>>>>>>>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra
>>>>>>>><he...@gmail.com>
>>>>>>>> wrote:
>>>>>>>> > So isnt this similar to HCatalog which relying on Hive metadata
>>>>>>>>service
>>>>>>>> > that ends up as sub project of Apache Hive?
>>>>>>>> >
>>>>>>>>
>>>>>>>> I was against having Curator as a sub when it came up on the
>>>>>>>>original
>>>>>>>> discussion thread, I still am.
>>>>>>>>
>>>>>>>> Patrick
>>>>>>>>
>>>>>>>> >
>>>>>>>> >
>>>>>>>> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com>
>>>>>>>>wrote:
>>>>>>>> >
>>>>>>>> >> My concern is that we're looking at two "new" committers, rather
>>>>>>>>than
>>>>>>>> >> a Curator community. Following normal Incubator work, Curator
>>>>>>>>would
>>>>>>>> >> build a community for itself. But then we'd have a community
>>>>>>>> >> *distinct* from that of Zookeeper. And it really looks like this
>>>>>>>> >> should be part of Zookeeper itself -- a more capable and
>>>>>>>>easier-to-use
>>>>>>>> >> client.
>>>>>>>> >>
>>>>>>>> >> So I question the incubation of this. Why do we want to build a
>>>>>>>> >> new/separate project? Why isn't this just part of Zookeeper
>>>>>>>>right
>>>>>>>>from
>>>>>>>> >> the start?
>>>>>>>> >>
>>>>>>>> >> I would suggest that this work is placed on a branch within
>>>>>>>>Zookeeper.
>>>>>>>> >> That Jordan and Jay become committers on that branch (not
>>>>>>>>necessarily
>>>>>>>> >> Zookeeper trunk). Over time, the branch can be folded into
>>>>>>>>trunk,
>>>>>>>> >> along with all the various tests, doc, and other artifacts that
>>>>>>>>I
>>>>>>>>see
>>>>>>>> >> in the GitHub repository. And hopefully that Jordan and Jay
>>>>>>>>become
>>>>>>>> >> regular committers (and PMC members!) of the Zookeeper project
>>>>>>>>itself.
>>>>>>>> >>
>>>>>>>> >> The current Zookeeper client can remain for backwards compat,
>>>>>>>>and
>>>>>>>>the
>>>>>>>> >> Curator work can become the next-gen client.
>>>>>>>> >>
>>>>>>>> >> Honestly, in my opnion, incubating this project seems like it
>>>>>>>>would
>>>>>>>> >> create a distinct community, and really doesn't seem like it
>>>>>>>>would
>>>>>>>> >> serve the Zookeeper community.
>>>>>>>> >>
>>>>>>>> >> All that said, I am not familiar with the Zookeeper or Curator
>>>>>>>> >> communities. But from this read, I don't think Incubation is the
>>>>>>>>right
>>>>>>>> >> approach. I would rather push for a more direct incorporation of
>>>>>>>> >> Curator directly into Zookeeper. (use the short-form IP
>>>>>>>>clearance) ...
>>>>>>>> >> so, unless somebody can help me understand the communities and
>>>>>>>> >> situation better, I'm -1 (binding) on this incubation. I'd
>>>>>>>>rather
>>>>>>>>see
>>>>>>>> >> combined, rather than distinct, communities from the start.
>>>>>>>> >>
>>>>>>>> >> Thanks,
>>>>>>>> >> -g
>>>>>>>> >>
>>>>>>>> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>>>>>>>> >> <jo...@jordanzimmerman.com> wrote:
>>>>>>>> >> > Hello,
>>>>>>>> >> >
>>>>>>>> >> > I would like to propose that Curator to be an Apache Incubator
>>>>>>>> project.
>>>>>>>> >> >
>>>>>>>> >> > The proposal can be found here:
>>>>>>>> >> http://wiki.apache.org/incubator/CuratorProposal
>>>>>>>> >> >
>>>>>>>> >> > I have included the contents of the proposal below.
>>>>>>>> >> >
>>>>>>>> >> > Sincerely,
>>>>>>>> >> >
>>>>>>>> >> > Jordan Zimmerman
>>>>>>>> >> >
>>>>>>>> >> > ===================
>>>>>>>> >> >
>>>>>>>> >> > = Curator - ZooKeeper client wrapper and rich ZooKeeper
>>>>>>>>framework =
>>>>>>>> >> >
>>>>>>>> >> > == Abstract ==
>>>>>>>> >> >
>>>>>>>> >> > Curator is a set of Java libraries that make using Apache
>>>>>>>>ZooKeeper
>>>>>>>> much
>>>>>>>> >> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>>>>the
>>>>>>>> client
>>>>>>>> >> is non-trivial and error prone.
>>>>>>>> >> >
>>>>>>>> >> > == Proposal ==
>>>>>>>> >> >
>>>>>>>> >> > Curator is a set of Java libraries that make using Apache
>>>>>>>>ZooKeeper
>>>>>>>> much
>>>>>>>> >> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>>>>the
>>>>>>>> client
>>>>>>>> >> is non-trivial and error prone. It consists of three components
>>>>>>>>that
>>>>>>>> build
>>>>>>>> >> on each other. Curator Client is a replacement for the bundled
>>>>>>>>ZooKeeper
>>>>>>>> >> class that takes care of some low-level housekeeping and
>>>>>>>>provides
>>>>>>>>some
>>>>>>>> >> useful utilities. Curator Framework is a high-level API that
>>>>>>>>greatly
>>>>>>>> >> simplifies using ZooKeeper. It adds many features that build on
>>>>>>>> ZooKeeper
>>>>>>>> >> and handles the complexity of managing connections to the
>>>>>>>>ZooKeeper
>>>>>>>> cluster
>>>>>>>> >> and retrying operations. Curator Recipes consists of
>>>>>>>>implementations of
>>>>>>>> >> some of the common ZooKeeper ³recipes². Additionally, Curator
>>>>>>>>Test is
>>>>>>>> >> included which includes utilities to help with unit testing
>>>>>>>> ZooKeeper-based
>>>>>>>> >> applications.
>>>>>>>> >> >
>>>>>>>> >> > == Background ==
>>>>>>>> >> >
>>>>>>>> >> > Curator was initially developed by Netflix to make writing
>>>>>>>> >> ZooKeeper-based applications easier and more reliable. Curator
>>>>>>>>was
>>>>>>>> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed
>>>>>>>>project in
>>>>>>>> >> July 2011. During this time Curator has been formally released
>>>>>>>>many
>>>>>>>> times
>>>>>>>> >> and has gained widespread adoption.
>>>>>>>> >> >
>>>>>>>> >> > == Rationale ==
>>>>>>>> >> >
>>>>>>>> >> > New users of ZooKeeper are surprised to learn that a
>>>>>>>>significant
>>>>>>>> amount
>>>>>>>> >> of connection management must be done manually. For example,
>>>>>>>>when
>>>>>>>>the
>>>>>>>> >> ZooKeeper client connects to the ensemble it must negotiate a
>>>>>>>>new
>>>>>>>> session,
>>>>>>>> >> etc. This takes some time. If you use a ZooKeeper client API
>>>>>>>>before the
>>>>>>>> >> connection process has completed, ZooKeeper will throw an
>>>>>>>>exception.
>>>>>>>> These
>>>>>>>> >> types of exceptions are referred to as ³recoverable² errors.
>>>>>>>> >> > Curator automatically handles connection management, greatly
>>>>>>>> simplifying
>>>>>>>> >> client code. Instead of directly using the ZooKeeper APIs you
>>>>>>>>use
>>>>>>>> Curator
>>>>>>>> >> APIs that internally check for connection completion and wrap
>>>>>>>>each
>>>>>>>> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism to
>>>>>>>>handle
>>>>>>>> >> recoverable errors and automatically retry operations. The
>>>>>>>>method
>>>>>>>>of
>>>>>>>> retry
>>>>>>>> >> is customizable. Curator comes bundled with several
>>>>>>>>implementations
>>>>>>>> >> (ExponentialBackoffRetry, etc.) or custom implementations can be
>>>>>>>> written.
>>>>>>>> >> >
>>>>>>>> >> > The ZooKeeper documentation describes many possible uses for
>>>>>>>>ZooKeeper
>>>>>>>> >> calling each a ³recipe². While the distribution comes bundled
>>>>>>>>with a few
>>>>>>>> >> implementations of these recipes, most ZooKeeper users will need
>>>>>>>>to
>>>>>>>> >> manually implement one or more of the recipes. Implementing a
>>>>>>>>ZooKeeper
>>>>>>>> >> recipe is not trivial. Besides the connection handling issues,
>>>>>>>>there are
>>>>>>>> >> numerous edge cases that are not well documented that must be
>>>>>>>> considered.
>>>>>>>> >> For example, many recipes require that an ephemeral-sequential
>>>>>>>>node be
>>>>>>>> >> created. New users of ZooKeeper will not know that there is an
>>>>>>>>edge
>>>>>>>> case in
>>>>>>>> >> ephemeral-sequential node creation that requires you to put a
>>>>>>>>special
>>>>>>>> >> ³marker² in the node¹s name so that you can search for the
>>>>>>>>created node
>>>>>>>> if
>>>>>>>> >> an I/O failure occurs. This is but one of many edge cases that
>>>>>>>>are not
>>>>>>>> well
>>>>>>>> >> documented but are handled by Curator.
>>>>>>>> >> >
>>>>>>>> >> > = Current Status =
>>>>>>>> >> >
>>>>>>>> >> > == Meritocracy ==
>>>>>>>> >> >
>>>>>>>> >> > Curator was initially developed by Jordan Zimmerman in 2011 at
>>>>>>>> Netflix.
>>>>>>>> >> Developers external to Netflix provided feedback, suggested
>>>>>>>>features and
>>>>>>>> >> fixes and implemented extensions of Curator. Netflix's
>>>>>>>>engineering team
>>>>>>>> has
>>>>>>>> >> since maintained the project and has been dedicated towards its
>>>>>>>> >> improvement. Contributors to Curator include developers from
>>>>>>>>multiple
>>>>>>>> >> organizations around the world. Curator will be a meritocracy as
>>>>>>>>it
>>>>>>>> enters
>>>>>>>> >> the Incubator and beyond.
>>>>>>>> >> >
>>>>>>>> >> > == Community ==
>>>>>>>> >> >
>>>>>>>> >> > Curator is currently used by a number of organizations all
>>>>>>>>over
>>>>>>>>the
>>>>>>>> >> world. Curator has an active and growing user and developer
>>>>>>>>community
>>>>>>>> with
>>>>>>>> >> active participation in the [[
>>>>>>>> http://groups.google.com/group/curator-users]]
>>>>>>>> >> mailing list and at its !Github home: [[
>>>>>>>> >> https://github.com/Netflix/curator]].
>>>>>>>> >> >
>>>>>>>> >> > Since open sourcing the project, there have been fifteen
>>>>>>>>individuals
>>>>>>>> >> from various organizations who have contributed code.
>>>>>>>> >> >
>>>>>>>> >> > == Core Developers ==
>>>>>>>> >> >
>>>>>>>> >> > The core developers for Curator are:
>>>>>>>> >> >  * Jordan Zimmerman
>>>>>>>> >> >  * Jay Zarfoss
>>>>>>>> >> >
>>>>>>>> >> > Jordan has contributed towards Apache ZooKeeper and both
>>>>>>>>Jordan
>>>>>>>>and
>>>>>>>> Jay
>>>>>>>> >> are familiar with Apache principles and philosophy for community
>>>>>>>>driven
>>>>>>>> >> software development.
>>>>>>>> >> >
>>>>>>>> >> > == Alignment ==
>>>>>>>> >> >
>>>>>>>> >> > Curator is a natural complement for Apache ZooKeeper. Java
>>>>>>>>users of
>>>>>>>> >> ZooKeeper will naturally want to use Curator. When Curator
>>>>>>>>graduates
>>>>>>>> from
>>>>>>>> >> Incubator it may be useful to distribute Curator artifacts as
>>>>>>>>part of
>>>>>>>> >> ZooKeeper releases as the preferred/recommended client side
>>>>>>>>library.
>>>>>>>> >> Further, at graduation a determination can be made as to whether
>>>>>>>>Curator
>>>>>>>> >> should become a Top Level Project or be merged into ZooKeeper
>>>>>>>>itself.
>>>>>>>> >> >
>>>>>>>> >> > = Known Risks =
>>>>>>>> >> >
>>>>>>>> >> > == Orphaned Products ==
>>>>>>>> >> >
>>>>>>>> >> > Curator is already deployed in production at multiple
>>>>>>>>companies
>>>>>>>>and
>>>>>>>> they
>>>>>>>> >> are actively participating in creating new features. Curator is
>>>>>>>>getting
>>>>>>>> >> traction with developers and thus the risks of it being orphaned
>>>>>>>>are
>>>>>>>> >> minimal.
>>>>>>>> >> >
>>>>>>>> >> > == Inexperience with Open Source ==
>>>>>>>> >> >
>>>>>>>> >> > All code developed for Curator has been open sourced by
>>>>>>>>Netflix
>>>>>>>>under
>>>>>>>> >> Apache 2.0 license.  All committers to Curator are intimately
>>>>>>>>familiar
>>>>>>>> with
>>>>>>>> >> the Apache model for open-source development and are experienced
>>>>>>>>with
>>>>>>>> >> working with new contributors.
>>>>>>>> >> >
>>>>>>>> >> > == Homogeneous Developers ==
>>>>>>>> >> >
>>>>>>>> >> > The initial committers are from a single organization.
>>>>>>>>However,
>>>>>>>>we
>>>>>>>> >> expect that once approved for incubation, the project will
>>>>>>>>attract new
>>>>>>>> >> contributors from diverse organizations and will thus grow
>>>>>>>>organically.
>>>>>>>> The
>>>>>>>> >> submission of patches from developers from several different
>>>>>>>> organizations
>>>>>>>> >> is a strong indication that Curator will be widely adopted.
>>>>>>>> >> >
>>>>>>>> >> > == Reliance on Salaried Developers ==
>>>>>>>> >> >
>>>>>>>> >> > It is expected that Curator will be developed on salaried and
>>>>>>>> volunteer
>>>>>>>> >> time, although all of the initial developers will work on it
>>>>>>>>mainly on
>>>>>>>> >> salaried time.
>>>>>>>> >> >
>>>>>>>> >> > == Relationships with Other Apache Products ==
>>>>>>>> >> >
>>>>>>>> >> > Curator depends upon other Apache Projects: Apache ZooKeeper,
>>>>>>>>Apache
>>>>>>>> >> Log4J, and multiple Apache Commons components. Its build depends
>>>>>>>>upon
>>>>>>>> >> Apache Maven. Notably, there is interest from other Apache
>>>>>>>>Projects
>>>>>>>> such as
>>>>>>>> >> HBase in adopting Curator as the client library for ZooKeeper.
>>>>>>>>Apache
>>>>>>>> James
>>>>>>>> >> Mailbox has already incorporated Curator.
>>>>>>>> >> >
>>>>>>>> >> > == An Excessive Fascination with the Apache Brand ==
>>>>>>>> >> >
>>>>>>>> >> > We would like Curator to become an Apache project to further
>>>>>>>>foster a
>>>>>>>> >> healthy community of contributors and consumers around the
>>>>>>>>project.
>>>>>>>>  Since
>>>>>>>> >> Curator directly interacts with Apache ZooKeeper and solves an
>>>>>>>>important
>>>>>>>> >> problem of many ZooKeeper users, residing in the Apache Software
>>>>>>>> Foundation
>>>>>>>> >> will increase interaction with the larger community.
>>>>>>>> >> >
>>>>>>>> >> > = Documentation =
>>>>>>>> >> >
>>>>>>>> >> >  * Curator wiki at GitHub:
>>>>>>>>https://github.com/Netflix/curator/wiki
>>>>>>>> >> >  * Curator issues at GitHub:
>>>>>>>> https://github.com/Netflix/curator/issues
>>>>>>>> >> >  * Curator javadoc at GitHub:
>>>>>>>>http://netflix.github.com/curator/doc/
>>>>>>>> >> >
>>>>>>>> >> > = Initial Source =
>>>>>>>> >> >
>>>>>>>> >> >  * git://github.com/Netflix/curator.git
>>>>>>>> >> >
>>>>>>>> >> > == Source and Intellectual Property Submission Plan ==
>>>>>>>> >> >
>>>>>>>> >> >  * The initial source is already licensed under the Apache
>>>>>>>>License,
>>>>>>>> >> Version 2.0.
>>>>>>>>https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>>>>>>> >> >
>>>>>>>> >> > == External Dependencies ==
>>>>>>>> >> >
>>>>>>>> >> > The required external dependencies are all Apache License or
>>>>>>>> compatible
>>>>>>>> >> licenses. Following components with non-Apache licenses are
>>>>>>>>enumerated:
>>>>>>>> >> >
>>>>>>>> >> >  * org.slf4j: MIT-like License
>>>>>>>> >> >  * org.mockito: MIT-like License
>>>>>>>> >> >
>>>>>>>> >> > == Cryptography ==
>>>>>>>> >> >
>>>>>>>> >> > Curator contains no known cryptography.
>>>>>>>> >> >
>>>>>>>> >> > = Required Resources =
>>>>>>>> >> >
>>>>>>>> >> > == Mailing lists ==
>>>>>>>> >> >
>>>>>>>> >> >  * curator-private (with moderated subscriptions)
>>>>>>>> >> >  * curator-dev
>>>>>>>> >> >  * curator-commits
>>>>>>>> >> >  * curator-user
>>>>>>>> >> >
>>>>>>>> >> > == Github Repositories ==
>>>>>>>> >> >
>>>>>>>> >> > http://github.com/apache/curator
>>>>>>>>git://git.apache.org/curator.git
>>>>>>>> >> >
>>>>>>>> >> > == Issue Tracking ==
>>>>>>>> >> >
>>>>>>>> >> > JIRA Curator (CURATOR)
>>>>>>>> >> >
>>>>>>>> >> > == Other Resources ==
>>>>>>>> >> >
>>>>>>>> >> > The existing code already has unit and integration tests so we
>>>>>>>>would
>>>>>>>> >> like a Jenkins instance to run them whenever a new patch is
>>>>>>>>submitted.
>>>>>>>> This
>>>>>>>> >> can be added after project creation.
>>>>>>>> >> >
>>>>>>>> >> > = Initial Committers =
>>>>>>>> >> >
>>>>>>>> >> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>>>>>>>> >> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>>>>>>>> >> >
>>>>>>>> >> > = Affiliations =
>>>>>>>> >> >
>>>>>>>> >> >  * Jordan Zimmerman, Netflix
>>>>>>>> >> >  * Jay Zarfoss, Netflix
>>>>>>>> >> >
>>>>>>>> >> > = Sponsors =
>>>>>>>> >> >
>>>>>>>> >> > == Champion ==
>>>>>>>> >> >
>>>>>>>> >> >  * Patrick Hunt
>>>>>>>> >> >
>>>>>>>> >> > == Nominated Mentors ==
>>>>>>>> >> >
>>>>>>>> >> >  * Patrick Hunt
>>>>>>>> >> >  * Enis Söztutar
>>>>>>>> >> >  * Mahadev Konar
>>>>>>>> >> >
>>>>>>>> >> > == Sponsoring Entity ==
>>>>>>>> >> >
>>>>>>>> >> >  * Apache Incubator PMC
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>>--------------------------------------------------------------------
>>>>>>>>-
>>>>>>>> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>>>> >> For additional commands, e-mail:
>>>>>>>>general-help@incubator.apache.org
>>>>>>>> >>
>>>>>>>> >>
>>>>>>>>
>>>>>>>>
>>>>>>>>--------------------------------------------------------------------
>>>>>>>>-
>>>>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>
>>>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>For additional commands, e-mail: general-help@incubator.apache.org
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>For additional commands, e-mail: general-help@incubator.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

["Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>]

Hey Pat,


On 2/26/13 11:39 AM, "Patrick Hunt" <ph...@apache.org> wrote:

>[…snip…]
>>
>> Either: (a) define Curator to be its own separate project/community,
>>with
>> a goal of TLP;
>> or (b) nix Incubation and just make these guys part of ZK, on a branch
>> now.
>
>Hi Chris. Unfortunately it's not clear cut to me where Curator should
>go, perhaps because I'm close to it.

Yep, I think that's part of it, but you're doing good discussing it here,
and please don't take my comments as criticism. If the answer were to be
that this project followed the Incubator process, and arrived at TLP, I
would have *zero* concern. My concern is the implication that this could
end up as part of Zookeeper (ZK). It's more of a general concern about
allowing
that within the Incubator. I'll comment more below.


>For example the "Curator as TLP"
>argument might sound like this:
>
>1) curator has been around for a while, people seem to like it
>2) it shares no code with ZK today, in the sense that it's a consumer
>of the ZK APIs, what we call "recipes". No code duplication. Granted
>it does try to make writing clients easier.

Gotcha. That makes a big difference and suggests it should be its own
community and TLP.

>3) (afaik) none of the current ZK committers have contributed to
>Curator, when this proposal came up previously for discussion on the
>ZK dev list there wasn't strong interest to pull it into ZK itself
>4) the initial committers on curator don't have experience at Apache

Got it.

>
>On the contrary I personally think it might make sense to have a
>combined community with Curator and ZooKeeper together, but I'm not
>sure.

If you think that is a good idea, you as an individual on the ZK PMC
simply need to shepherd the contributions into ZK. You have an ICLA
on file. Not sure if Jordan or Jay have an ICLA, but they could easily
file one. Commit some of their patches yourself (i.e., bring Curator
in as ZK code with ZK provenance, and stewardship and mentorship),
and then nominate the Curator contributors for committership/PMC in ZK.

Nothing stopping you from doing that today and nothing the Incubator needs
to muddy up.

>
>My hope was that Curator could come into the incubator, work through
>the IP issues and other incubation activities, learn about Apache
>process, and then decide based on the new found insight/experience.

I totally get it.

I'm not questioning you, more of the Incubator here.

Let me cut to the chase. I *don't* think the Incubator should be home to
projects
whose potential destination is as a product of an existing TLP (or
sub-module).
I think that leads to a LOT of potential problems down the road, some of
which
include:

1. The dev that happens in the Incubator on new podling X, whose
destination is TLP
Y is rarely monitored or seen by those in TLP Y. If it is closely
monitored, then
why isn't the new podling part of TLP Y? And if it's not monitored, then
TLP Y is 
more likely to end up at some forced agreement or bylaw mumbo jumbo to
assimilate the
people and products of podling X into the TLP Y.

2. Creating the new community in the Incubator means that new people will
be added
as members of podling X that TLP Y didn't add. We've seen problems with
TLP Y then not
wanting to assimilate those podling X PPMC members into the TLP Y.

3. Sub-modules tend to hide things like sub-projects. I can count on two
hands how many
TLPs have went through that process, and all of them typically have had
hardship and 
crap along the way that I'm trying to save you guys the hassle of
("lessons learned")

Instead, if the home for a potential new podling X is TLP Y, my desire
would be to 
work as an existing member of TLP Y to shepherd the contributions in that
way. 

>
>The proposal guide talks about this and makes it clear (at least to
>me) that leaving graduation open is not only an option, but a
>requirement "Note that the final destination within the Apache
>organizational structure will be decided upon graduation." Given this
>I don't see why we'd artificially constrain things up front as you've
>suggested.

Well the proposal guide is great, and all but it's not doctrine :)

I realize you're in a tough spot here and somewhat in a COI, not exactly
knowing what the right answer is,
and I'm not sure I know it either, but my gut instinct based on experience
for a while
here indicates that Incubation makes sense here and this is a new
community and so I won't
stand in the way of that. I will however, throw up my comments and a fuss
again if this is another
situation similar to HCatalog where at the end of this you guys try and
shove it into ZK
in any other way of intersect([set of Curator PPMC], [set of ZK PMC]).

That's my ultimate worry, and the Incubator is not a clearinghouse for
punting on those
types of situations at least not the Incubator I would like to participate
in.

Chris


>
>Patrick
>
>
>>
>> On 2/26/13 9:40 AM, "Benson Margulies" <bi...@gmail.com> wrote:
>>
>>>On Tue, Feb 26, 2013 at 12:22 PM, Patrick Hunt <ph...@apache.org> wrote:
>>>> On Tue, Feb 26, 2013 at 8:32 AM, Benson Margulies
>>>><bi...@gmail.com> wrote:
>>>>> If you think that the right destination for curator is as part of ZK,
>>>>> then it would be good to see substantive participation of the ZK PMC
>>>>> in the incubation. The goal should be to 'graduate' by having the
>>>>> curator community be granted karma at ZK and the code folded in. This
>>>>> would require, I think, the ZK community to supply at least one
>>>>> mentor, and to have a plan in advance for the eventual votes based on
>>>>> success in the incubator.
>>>>
>>>> Hi Benson. What you're suggesting matches my current thinking.
>>>>
>>>> The three Curator mentors are as follows: Mahadev and I (champion) are
>>>> both PMC members on ZK, Enis Söztutar is active on HBase and
>>>> interested in using Curator for that project (which already uses ZK
>>>> heavily).
>>>
>>>OK, that's lovely.
>>>
>>>
>>>>
>>>> Patrick
>>>>
>>>>> On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra
>>>>><he...@gmail.com> wrote:
>>>>>> Ah no, I was not suggesting about Curator to become subproject of
>>>>>>ZK.
>>>>>>I
>>>>>> just afraid that if Curator is going as incubator it will end up as
>>>>>>sub of
>>>>>> ZK as merging process.
>>>>>>
>>>>>> Like Greg has mentioned in another reply, I would prefer Curator to
>>>>>>be
>>>>>> merged as a higher level ZK client. Surely project like HBase and
>>>>>>others
>>>>>> that relying on ZK would appreciate simpler client to ZK.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Henry
>>>>>>
>>>>>>
>>>>>> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <ph...@apache.org>
>>>>>>wrote:
>>>>>>
>>>>>>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra
>>>>>>><he...@gmail.com>
>>>>>>> wrote:
>>>>>>> > So isnt this similar to HCatalog which relying on Hive metadata
>>>>>>>service
>>>>>>> > that ends up as sub project of Apache Hive?
>>>>>>> >
>>>>>>>
>>>>>>> I was against having Curator as a sub when it came up on the
>>>>>>>original
>>>>>>> discussion thread, I still am.
>>>>>>>
>>>>>>> Patrick
>>>>>>>
>>>>>>> >
>>>>>>> >
>>>>>>> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com>
>>>>>>>wrote:
>>>>>>> >
>>>>>>> >> My concern is that we're looking at two "new" committers, rather
>>>>>>>than
>>>>>>> >> a Curator community. Following normal Incubator work, Curator
>>>>>>>would
>>>>>>> >> build a community for itself. But then we'd have a community
>>>>>>> >> *distinct* from that of Zookeeper. And it really looks like this
>>>>>>> >> should be part of Zookeeper itself -- a more capable and
>>>>>>>easier-to-use
>>>>>>> >> client.
>>>>>>> >>
>>>>>>> >> So I question the incubation of this. Why do we want to build a
>>>>>>> >> new/separate project? Why isn't this just part of Zookeeper
>>>>>>>right
>>>>>>>from
>>>>>>> >> the start?
>>>>>>> >>
>>>>>>> >> I would suggest that this work is placed on a branch within
>>>>>>>Zookeeper.
>>>>>>> >> That Jordan and Jay become committers on that branch (not
>>>>>>>necessarily
>>>>>>> >> Zookeeper trunk). Over time, the branch can be folded into
>>>>>>>trunk,
>>>>>>> >> along with all the various tests, doc, and other artifacts that
>>>>>>>I
>>>>>>>see
>>>>>>> >> in the GitHub repository. And hopefully that Jordan and Jay
>>>>>>>become
>>>>>>> >> regular committers (and PMC members!) of the Zookeeper project
>>>>>>>itself.
>>>>>>> >>
>>>>>>> >> The current Zookeeper client can remain for backwards compat,
>>>>>>>and
>>>>>>>the
>>>>>>> >> Curator work can become the next-gen client.
>>>>>>> >>
>>>>>>> >> Honestly, in my opnion, incubating this project seems like it
>>>>>>>would
>>>>>>> >> create a distinct community, and really doesn't seem like it
>>>>>>>would
>>>>>>> >> serve the Zookeeper community.
>>>>>>> >>
>>>>>>> >> All that said, I am not familiar with the Zookeeper or Curator
>>>>>>> >> communities. But from this read, I don't think Incubation is the
>>>>>>>right
>>>>>>> >> approach. I would rather push for a more direct incorporation of
>>>>>>> >> Curator directly into Zookeeper. (use the short-form IP
>>>>>>>clearance) ...
>>>>>>> >> so, unless somebody can help me understand the communities and
>>>>>>> >> situation better, I'm -1 (binding) on this incubation. I'd
>>>>>>>rather
>>>>>>>see
>>>>>>> >> combined, rather than distinct, communities from the start.
>>>>>>> >>
>>>>>>> >> Thanks,
>>>>>>> >> -g
>>>>>>> >>
>>>>>>> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>>>>>>> >> <jo...@jordanzimmerman.com> wrote:
>>>>>>> >> > Hello,
>>>>>>> >> >
>>>>>>> >> > I would like to propose that Curator to be an Apache Incubator
>>>>>>> project.
>>>>>>> >> >
>>>>>>> >> > The proposal can be found here:
>>>>>>> >> http://wiki.apache.org/incubator/CuratorProposal
>>>>>>> >> >
>>>>>>> >> > I have included the contents of the proposal below.
>>>>>>> >> >
>>>>>>> >> > Sincerely,
>>>>>>> >> >
>>>>>>> >> > Jordan Zimmerman
>>>>>>> >> >
>>>>>>> >> > ===================
>>>>>>> >> >
>>>>>>> >> > = Curator - ZooKeeper client wrapper and rich ZooKeeper
>>>>>>>framework =
>>>>>>> >> >
>>>>>>> >> > == Abstract ==
>>>>>>> >> >
>>>>>>> >> > Curator is a set of Java libraries that make using Apache
>>>>>>>ZooKeeper
>>>>>>> much
>>>>>>> >> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>>>the
>>>>>>> client
>>>>>>> >> is non-trivial and error prone.
>>>>>>> >> >
>>>>>>> >> > == Proposal ==
>>>>>>> >> >
>>>>>>> >> > Curator is a set of Java libraries that make using Apache
>>>>>>>ZooKeeper
>>>>>>> much
>>>>>>> >> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>>>the
>>>>>>> client
>>>>>>> >> is non-trivial and error prone. It consists of three components
>>>>>>>that
>>>>>>> build
>>>>>>> >> on each other. Curator Client is a replacement for the bundled
>>>>>>>ZooKeeper
>>>>>>> >> class that takes care of some low-level housekeeping and
>>>>>>>provides
>>>>>>>some
>>>>>>> >> useful utilities. Curator Framework is a high-level API that
>>>>>>>greatly
>>>>>>> >> simplifies using ZooKeeper. It adds many features that build on
>>>>>>> ZooKeeper
>>>>>>> >> and handles the complexity of managing connections to the
>>>>>>>ZooKeeper
>>>>>>> cluster
>>>>>>> >> and retrying operations. Curator Recipes consists of
>>>>>>>implementations of
>>>>>>> >> some of the common ZooKeeper ³recipes². Additionally, Curator
>>>>>>>Test is
>>>>>>> >> included which includes utilities to help with unit testing
>>>>>>> ZooKeeper-based
>>>>>>> >> applications.
>>>>>>> >> >
>>>>>>> >> > == Background ==
>>>>>>> >> >
>>>>>>> >> > Curator was initially developed by Netflix to make writing
>>>>>>> >> ZooKeeper-based applications easier and more reliable. Curator
>>>>>>>was
>>>>>>> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed
>>>>>>>project in
>>>>>>> >> July 2011. During this time Curator has been formally released
>>>>>>>many
>>>>>>> times
>>>>>>> >> and has gained widespread adoption.
>>>>>>> >> >
>>>>>>> >> > == Rationale ==
>>>>>>> >> >
>>>>>>> >> > New users of ZooKeeper are surprised to learn that a
>>>>>>>significant
>>>>>>> amount
>>>>>>> >> of connection management must be done manually. For example,
>>>>>>>when
>>>>>>>the
>>>>>>> >> ZooKeeper client connects to the ensemble it must negotiate a
>>>>>>>new
>>>>>>> session,
>>>>>>> >> etc. This takes some time. If you use a ZooKeeper client API
>>>>>>>before the
>>>>>>> >> connection process has completed, ZooKeeper will throw an
>>>>>>>exception.
>>>>>>> These
>>>>>>> >> types of exceptions are referred to as ³recoverable² errors.
>>>>>>> >> > Curator automatically handles connection management, greatly
>>>>>>> simplifying
>>>>>>> >> client code. Instead of directly using the ZooKeeper APIs you
>>>>>>>use
>>>>>>> Curator
>>>>>>> >> APIs that internally check for connection completion and wrap
>>>>>>>each
>>>>>>> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism to
>>>>>>>handle
>>>>>>> >> recoverable errors and automatically retry operations. The
>>>>>>>method
>>>>>>>of
>>>>>>> retry
>>>>>>> >> is customizable. Curator comes bundled with several
>>>>>>>implementations
>>>>>>> >> (ExponentialBackoffRetry, etc.) or custom implementations can be
>>>>>>> written.
>>>>>>> >> >
>>>>>>> >> > The ZooKeeper documentation describes many possible uses for
>>>>>>>ZooKeeper
>>>>>>> >> calling each a ³recipe². While the distribution comes bundled
>>>>>>>with a few
>>>>>>> >> implementations of these recipes, most ZooKeeper users will need
>>>>>>>to
>>>>>>> >> manually implement one or more of the recipes. Implementing a
>>>>>>>ZooKeeper
>>>>>>> >> recipe is not trivial. Besides the connection handling issues,
>>>>>>>there are
>>>>>>> >> numerous edge cases that are not well documented that must be
>>>>>>> considered.
>>>>>>> >> For example, many recipes require that an ephemeral-sequential
>>>>>>>node be
>>>>>>> >> created. New users of ZooKeeper will not know that there is an
>>>>>>>edge
>>>>>>> case in
>>>>>>> >> ephemeral-sequential node creation that requires you to put a
>>>>>>>special
>>>>>>> >> ³marker² in the node¹s name so that you can search for the
>>>>>>>created node
>>>>>>> if
>>>>>>> >> an I/O failure occurs. This is but one of many edge cases that
>>>>>>>are not
>>>>>>> well
>>>>>>> >> documented but are handled by Curator.
>>>>>>> >> >
>>>>>>> >> > = Current Status =
>>>>>>> >> >
>>>>>>> >> > == Meritocracy ==
>>>>>>> >> >
>>>>>>> >> > Curator was initially developed by Jordan Zimmerman in 2011 at
>>>>>>> Netflix.
>>>>>>> >> Developers external to Netflix provided feedback, suggested
>>>>>>>features and
>>>>>>> >> fixes and implemented extensions of Curator. Netflix's
>>>>>>>engineering team
>>>>>>> has
>>>>>>> >> since maintained the project and has been dedicated towards its
>>>>>>> >> improvement. Contributors to Curator include developers from
>>>>>>>multiple
>>>>>>> >> organizations around the world. Curator will be a meritocracy as
>>>>>>>it
>>>>>>> enters
>>>>>>> >> the Incubator and beyond.
>>>>>>> >> >
>>>>>>> >> > == Community ==
>>>>>>> >> >
>>>>>>> >> > Curator is currently used by a number of organizations all
>>>>>>>over
>>>>>>>the
>>>>>>> >> world. Curator has an active and growing user and developer
>>>>>>>community
>>>>>>> with
>>>>>>> >> active participation in the [[
>>>>>>> http://groups.google.com/group/curator-users]]
>>>>>>> >> mailing list and at its !Github home: [[
>>>>>>> >> https://github.com/Netflix/curator]].
>>>>>>> >> >
>>>>>>> >> > Since open sourcing the project, there have been fifteen
>>>>>>>individuals
>>>>>>> >> from various organizations who have contributed code.
>>>>>>> >> >
>>>>>>> >> > == Core Developers ==
>>>>>>> >> >
>>>>>>> >> > The core developers for Curator are:
>>>>>>> >> >  * Jordan Zimmerman
>>>>>>> >> >  * Jay Zarfoss
>>>>>>> >> >
>>>>>>> >> > Jordan has contributed towards Apache ZooKeeper and both
>>>>>>>Jordan
>>>>>>>and
>>>>>>> Jay
>>>>>>> >> are familiar with Apache principles and philosophy for community
>>>>>>>driven
>>>>>>> >> software development.
>>>>>>> >> >
>>>>>>> >> > == Alignment ==
>>>>>>> >> >
>>>>>>> >> > Curator is a natural complement for Apache ZooKeeper. Java
>>>>>>>users of
>>>>>>> >> ZooKeeper will naturally want to use Curator. When Curator
>>>>>>>graduates
>>>>>>> from
>>>>>>> >> Incubator it may be useful to distribute Curator artifacts as
>>>>>>>part of
>>>>>>> >> ZooKeeper releases as the preferred/recommended client side
>>>>>>>library.
>>>>>>> >> Further, at graduation a determination can be made as to whether
>>>>>>>Curator
>>>>>>> >> should become a Top Level Project or be merged into ZooKeeper
>>>>>>>itself.
>>>>>>> >> >
>>>>>>> >> > = Known Risks =
>>>>>>> >> >
>>>>>>> >> > == Orphaned Products ==
>>>>>>> >> >
>>>>>>> >> > Curator is already deployed in production at multiple
>>>>>>>companies
>>>>>>>and
>>>>>>> they
>>>>>>> >> are actively participating in creating new features. Curator is
>>>>>>>getting
>>>>>>> >> traction with developers and thus the risks of it being orphaned
>>>>>>>are
>>>>>>> >> minimal.
>>>>>>> >> >
>>>>>>> >> > == Inexperience with Open Source ==
>>>>>>> >> >
>>>>>>> >> > All code developed for Curator has been open sourced by
>>>>>>>Netflix
>>>>>>>under
>>>>>>> >> Apache 2.0 license.  All committers to Curator are intimately
>>>>>>>familiar
>>>>>>> with
>>>>>>> >> the Apache model for open-source development and are experienced
>>>>>>>with
>>>>>>> >> working with new contributors.
>>>>>>> >> >
>>>>>>> >> > == Homogeneous Developers ==
>>>>>>> >> >
>>>>>>> >> > The initial committers are from a single organization.
>>>>>>>However,
>>>>>>>we
>>>>>>> >> expect that once approved for incubation, the project will
>>>>>>>attract new
>>>>>>> >> contributors from diverse organizations and will thus grow
>>>>>>>organically.
>>>>>>> The
>>>>>>> >> submission of patches from developers from several different
>>>>>>> organizations
>>>>>>> >> is a strong indication that Curator will be widely adopted.
>>>>>>> >> >
>>>>>>> >> > == Reliance on Salaried Developers ==
>>>>>>> >> >
>>>>>>> >> > It is expected that Curator will be developed on salaried and
>>>>>>> volunteer
>>>>>>> >> time, although all of the initial developers will work on it
>>>>>>>mainly on
>>>>>>> >> salaried time.
>>>>>>> >> >
>>>>>>> >> > == Relationships with Other Apache Products ==
>>>>>>> >> >
>>>>>>> >> > Curator depends upon other Apache Projects: Apache ZooKeeper,
>>>>>>>Apache
>>>>>>> >> Log4J, and multiple Apache Commons components. Its build depends
>>>>>>>upon
>>>>>>> >> Apache Maven. Notably, there is interest from other Apache
>>>>>>>Projects
>>>>>>> such as
>>>>>>> >> HBase in adopting Curator as the client library for ZooKeeper.
>>>>>>>Apache
>>>>>>> James
>>>>>>> >> Mailbox has already incorporated Curator.
>>>>>>> >> >
>>>>>>> >> > == An Excessive Fascination with the Apache Brand ==
>>>>>>> >> >
>>>>>>> >> > We would like Curator to become an Apache project to further
>>>>>>>foster a
>>>>>>> >> healthy community of contributors and consumers around the
>>>>>>>project.
>>>>>>>  Since
>>>>>>> >> Curator directly interacts with Apache ZooKeeper and solves an
>>>>>>>important
>>>>>>> >> problem of many ZooKeeper users, residing in the Apache Software
>>>>>>> Foundation
>>>>>>> >> will increase interaction with the larger community.
>>>>>>> >> >
>>>>>>> >> > = Documentation =
>>>>>>> >> >
>>>>>>> >> >  * Curator wiki at GitHub:
>>>>>>>https://github.com/Netflix/curator/wiki
>>>>>>> >> >  * Curator issues at GitHub:
>>>>>>> https://github.com/Netflix/curator/issues
>>>>>>> >> >  * Curator javadoc at GitHub:
>>>>>>>http://netflix.github.com/curator/doc/
>>>>>>> >> >
>>>>>>> >> > = Initial Source =
>>>>>>> >> >
>>>>>>> >> >  * git://github.com/Netflix/curator.git
>>>>>>> >> >
>>>>>>> >> > == Source and Intellectual Property Submission Plan ==
>>>>>>> >> >
>>>>>>> >> >  * The initial source is already licensed under the Apache
>>>>>>>License,
>>>>>>> >> Version 2.0.
>>>>>>>https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>>>>>> >> >
>>>>>>> >> > == External Dependencies ==
>>>>>>> >> >
>>>>>>> >> > The required external dependencies are all Apache License or
>>>>>>> compatible
>>>>>>> >> licenses. Following components with non-Apache licenses are
>>>>>>>enumerated:
>>>>>>> >> >
>>>>>>> >> >  * org.slf4j: MIT-like License
>>>>>>> >> >  * org.mockito: MIT-like License
>>>>>>> >> >
>>>>>>> >> > == Cryptography ==
>>>>>>> >> >
>>>>>>> >> > Curator contains no known cryptography.
>>>>>>> >> >
>>>>>>> >> > = Required Resources =
>>>>>>> >> >
>>>>>>> >> > == Mailing lists ==
>>>>>>> >> >
>>>>>>> >> >  * curator-private (with moderated subscriptions)
>>>>>>> >> >  * curator-dev
>>>>>>> >> >  * curator-commits
>>>>>>> >> >  * curator-user
>>>>>>> >> >
>>>>>>> >> > == Github Repositories ==
>>>>>>> >> >
>>>>>>> >> > http://github.com/apache/curator
>>>>>>>git://git.apache.org/curator.git
>>>>>>> >> >
>>>>>>> >> > == Issue Tracking ==
>>>>>>> >> >
>>>>>>> >> > JIRA Curator (CURATOR)
>>>>>>> >> >
>>>>>>> >> > == Other Resources ==
>>>>>>> >> >
>>>>>>> >> > The existing code already has unit and integration tests so we
>>>>>>>would
>>>>>>> >> like a Jenkins instance to run them whenever a new patch is
>>>>>>>submitted.
>>>>>>> This
>>>>>>> >> can be added after project creation.
>>>>>>> >> >
>>>>>>> >> > = Initial Committers =
>>>>>>> >> >
>>>>>>> >> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>>>>>>> >> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>>>>>>> >> >
>>>>>>> >> > = Affiliations =
>>>>>>> >> >
>>>>>>> >> >  * Jordan Zimmerman, Netflix
>>>>>>> >> >  * Jay Zarfoss, Netflix
>>>>>>> >> >
>>>>>>> >> > = Sponsors =
>>>>>>> >> >
>>>>>>> >> > == Champion ==
>>>>>>> >> >
>>>>>>> >> >  * Patrick Hunt
>>>>>>> >> >
>>>>>>> >> > == Nominated Mentors ==
>>>>>>> >> >
>>>>>>> >> >  * Patrick Hunt
>>>>>>> >> >  * Enis Söztutar
>>>>>>> >> >  * Mahadev Konar
>>>>>>> >> >
>>>>>>> >> > == Sponsoring Entity ==
>>>>>>> >> >
>>>>>>> >> >  * Apache Incubator PMC
>>>>>>> >>
>>>>>>> >>
>>>>>>>--------------------------------------------------------------------
>>>>>>>-
>>>>>>> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>>> >> For additional commands, e-mail:
>>>>>>>general-help@incubator.apache.org
>>>>>>> >>
>>>>>>> >>
>>>>>>>
>>>>>>> 
>>>>>>>--------------------------------------------------------------------
>>>>>>>-
>>>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>>>
>>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>
>>>
>>>---------------------------------------------------------------------
>>>To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>For additional commands, e-mail: general-help@incubator.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>For additional commands, e-mail: general-help@incubator.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Patrick Hunt <ph...@apache.org>]

On Tue, Feb 26, 2013 at 10:30 AM, Mattmann, Chris A (388J)
<ch...@jpl.nasa.gov> wrote:
> Hi Guys,
>
> Sorry I have to ask the question here. If the mentors consist of PMC
> members on ZK
> (at least Pat and Mahadev), what's the problem with creating a branch in
> ZK and just
> having the code be there and getting the Curator proposed committers as
> committers in
> ZK ville, and hopefully PMC members soon thereafter.
>
> I have to agree with Greg here. Seems like Incubation is something that
> might not be
> needed. If the end result of this after N months is that Curator
> "graduates" into another
> set of flipping bylaw updates, and more legislation that makes these
> people unofficial
> PMC members on ZK, then I'm double triple -1 with 50 piles of coal on top.
> Yes, I'm
> still remembering the HCatalog/Hive thing here - I still don't get that.
>
> Either: (a) define Curator to be its own separate project/community, with
> a goal of TLP;
> or (b) nix Incubation and just make these guys part of ZK, on a branch
> now.

Hi Chris. Unfortunately it's not clear cut to me where Curator should
go, perhaps because I'm close to it. For example the "Curator as TLP"
argument might sound like this:

1) curator has been around for a while, people seem to like it
2) it shares no code with ZK today, in the sense that it's a consumer
of the ZK APIs, what we call "recipes". No code duplication. Granted
it does try to make writing clients easier.
3) (afaik) none of the current ZK committers have contributed to
Curator, when this proposal came up previously for discussion on the
ZK dev list there wasn't strong interest to pull it into ZK itself
4) the initial committers on curator don't have experience at Apache

On the contrary I personally think it might make sense to have a
combined community with Curator and ZooKeeper together, but I'm not
sure.

My hope was that Curator could come into the incubator, work through
the IP issues and other incubation activities, learn about Apache
process, and then decide based on the new found insight/experience.

The proposal guide talks about this and makes it clear (at least to
me) that leaving graduation open is not only an option, but a
requirement "Note that the final destination within the Apache
organizational structure will be decided upon graduation." Given this
I don't see why we'd artificially constrain things up front as you've
suggested.

Patrick


>
> On 2/26/13 9:40 AM, "Benson Margulies" <bi...@gmail.com> wrote:
>
>>On Tue, Feb 26, 2013 at 12:22 PM, Patrick Hunt <ph...@apache.org> wrote:
>>> On Tue, Feb 26, 2013 at 8:32 AM, Benson Margulies
>>><bi...@gmail.com> wrote:
>>>> If you think that the right destination for curator is as part of ZK,
>>>> then it would be good to see substantive participation of the ZK PMC
>>>> in the incubation. The goal should be to 'graduate' by having the
>>>> curator community be granted karma at ZK and the code folded in. This
>>>> would require, I think, the ZK community to supply at least one
>>>> mentor, and to have a plan in advance for the eventual votes based on
>>>> success in the incubator.
>>>
>>> Hi Benson. What you're suggesting matches my current thinking.
>>>
>>> The three Curator mentors are as follows: Mahadev and I (champion) are
>>> both PMC members on ZK, Enis Söztutar is active on HBase and
>>> interested in using Curator for that project (which already uses ZK
>>> heavily).
>>
>>OK, that's lovely.
>>
>>
>>>
>>> Patrick
>>>
>>>> On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra
>>>><he...@gmail.com> wrote:
>>>>> Ah no, I was not suggesting about Curator to become subproject of ZK.
>>>>>I
>>>>> just afraid that if Curator is going as incubator it will end up as
>>>>>sub of
>>>>> ZK as merging process.
>>>>>
>>>>> Like Greg has mentioned in another reply, I would prefer Curator to be
>>>>> merged as a higher level ZK client. Surely project like HBase and
>>>>>others
>>>>> that relying on ZK would appreciate simpler client to ZK.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Henry
>>>>>
>>>>>
>>>>> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <ph...@apache.org>
>>>>>wrote:
>>>>>
>>>>>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra
>>>>>><he...@gmail.com>
>>>>>> wrote:
>>>>>> > So isnt this similar to HCatalog which relying on Hive metadata
>>>>>>service
>>>>>> > that ends up as sub project of Apache Hive?
>>>>>> >
>>>>>>
>>>>>> I was against having Curator as a sub when it came up on the original
>>>>>> discussion thread, I still am.
>>>>>>
>>>>>> Patrick
>>>>>>
>>>>>> >
>>>>>> >
>>>>>> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com>
>>>>>>wrote:
>>>>>> >
>>>>>> >> My concern is that we're looking at two "new" committers, rather
>>>>>>than
>>>>>> >> a Curator community. Following normal Incubator work, Curator
>>>>>>would
>>>>>> >> build a community for itself. But then we'd have a community
>>>>>> >> *distinct* from that of Zookeeper. And it really looks like this
>>>>>> >> should be part of Zookeeper itself -- a more capable and
>>>>>>easier-to-use
>>>>>> >> client.
>>>>>> >>
>>>>>> >> So I question the incubation of this. Why do we want to build a
>>>>>> >> new/separate project? Why isn't this just part of Zookeeper right
>>>>>>from
>>>>>> >> the start?
>>>>>> >>
>>>>>> >> I would suggest that this work is placed on a branch within
>>>>>>Zookeeper.
>>>>>> >> That Jordan and Jay become committers on that branch (not
>>>>>>necessarily
>>>>>> >> Zookeeper trunk). Over time, the branch can be folded into trunk,
>>>>>> >> along with all the various tests, doc, and other artifacts that I
>>>>>>see
>>>>>> >> in the GitHub repository. And hopefully that Jordan and Jay become
>>>>>> >> regular committers (and PMC members!) of the Zookeeper project
>>>>>>itself.
>>>>>> >>
>>>>>> >> The current Zookeeper client can remain for backwards compat, and
>>>>>>the
>>>>>> >> Curator work can become the next-gen client.
>>>>>> >>
>>>>>> >> Honestly, in my opnion, incubating this project seems like it
>>>>>>would
>>>>>> >> create a distinct community, and really doesn't seem like it would
>>>>>> >> serve the Zookeeper community.
>>>>>> >>
>>>>>> >> All that said, I am not familiar with the Zookeeper or Curator
>>>>>> >> communities. But from this read, I don't think Incubation is the
>>>>>>right
>>>>>> >> approach. I would rather push for a more direct incorporation of
>>>>>> >> Curator directly into Zookeeper. (use the short-form IP
>>>>>>clearance) ...
>>>>>> >> so, unless somebody can help me understand the communities and
>>>>>> >> situation better, I'm -1 (binding) on this incubation. I'd rather
>>>>>>see
>>>>>> >> combined, rather than distinct, communities from the start.
>>>>>> >>
>>>>>> >> Thanks,
>>>>>> >> -g
>>>>>> >>
>>>>>> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>>>>>> >> <jo...@jordanzimmerman.com> wrote:
>>>>>> >> > Hello,
>>>>>> >> >
>>>>>> >> > I would like to propose that Curator to be an Apache Incubator
>>>>>> project.
>>>>>> >> >
>>>>>> >> > The proposal can be found here:
>>>>>> >> http://wiki.apache.org/incubator/CuratorProposal
>>>>>> >> >
>>>>>> >> > I have included the contents of the proposal below.
>>>>>> >> >
>>>>>> >> > Sincerely,
>>>>>> >> >
>>>>>> >> > Jordan Zimmerman
>>>>>> >> >
>>>>>> >> > ===================
>>>>>> >> >
>>>>>> >> > = Curator - ZooKeeper client wrapper and rich ZooKeeper
>>>>>>framework =
>>>>>> >> >
>>>>>> >> > == Abstract ==
>>>>>> >> >
>>>>>> >> > Curator is a set of Java libraries that make using Apache
>>>>>>ZooKeeper
>>>>>> much
>>>>>> >> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>>the
>>>>>> client
>>>>>> >> is non-trivial and error prone.
>>>>>> >> >
>>>>>> >> > == Proposal ==
>>>>>> >> >
>>>>>> >> > Curator is a set of Java libraries that make using Apache
>>>>>>ZooKeeper
>>>>>> much
>>>>>> >> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>>the
>>>>>> client
>>>>>> >> is non-trivial and error prone. It consists of three components
>>>>>>that
>>>>>> build
>>>>>> >> on each other. Curator Client is a replacement for the bundled
>>>>>>ZooKeeper
>>>>>> >> class that takes care of some low-level housekeeping and provides
>>>>>>some
>>>>>> >> useful utilities. Curator Framework is a high-level API that
>>>>>>greatly
>>>>>> >> simplifies using ZooKeeper. It adds many features that build on
>>>>>> ZooKeeper
>>>>>> >> and handles the complexity of managing connections to the
>>>>>>ZooKeeper
>>>>>> cluster
>>>>>> >> and retrying operations. Curator Recipes consists of
>>>>>>implementations of
>>>>>> >> some of the common ZooKeeper ³recipes². Additionally, Curator
>>>>>>Test is
>>>>>> >> included which includes utilities to help with unit testing
>>>>>> ZooKeeper-based
>>>>>> >> applications.
>>>>>> >> >
>>>>>> >> > == Background ==
>>>>>> >> >
>>>>>> >> > Curator was initially developed by Netflix to make writing
>>>>>> >> ZooKeeper-based applications easier and more reliable. Curator was
>>>>>> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed
>>>>>>project in
>>>>>> >> July 2011. During this time Curator has been formally released
>>>>>>many
>>>>>> times
>>>>>> >> and has gained widespread adoption.
>>>>>> >> >
>>>>>> >> > == Rationale ==
>>>>>> >> >
>>>>>> >> > New users of ZooKeeper are surprised to learn that a significant
>>>>>> amount
>>>>>> >> of connection management must be done manually. For example, when
>>>>>>the
>>>>>> >> ZooKeeper client connects to the ensemble it must negotiate a new
>>>>>> session,
>>>>>> >> etc. This takes some time. If you use a ZooKeeper client API
>>>>>>before the
>>>>>> >> connection process has completed, ZooKeeper will throw an
>>>>>>exception.
>>>>>> These
>>>>>> >> types of exceptions are referred to as ³recoverable² errors.
>>>>>> >> > Curator automatically handles connection management, greatly
>>>>>> simplifying
>>>>>> >> client code. Instead of directly using the ZooKeeper APIs you use
>>>>>> Curator
>>>>>> >> APIs that internally check for connection completion and wrap each
>>>>>> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism to
>>>>>>handle
>>>>>> >> recoverable errors and automatically retry operations. The method
>>>>>>of
>>>>>> retry
>>>>>> >> is customizable. Curator comes bundled with several
>>>>>>implementations
>>>>>> >> (ExponentialBackoffRetry, etc.) or custom implementations can be
>>>>>> written.
>>>>>> >> >
>>>>>> >> > The ZooKeeper documentation describes many possible uses for
>>>>>>ZooKeeper
>>>>>> >> calling each a ³recipe². While the distribution comes bundled
>>>>>>with a few
>>>>>> >> implementations of these recipes, most ZooKeeper users will need
>>>>>>to
>>>>>> >> manually implement one or more of the recipes. Implementing a
>>>>>>ZooKeeper
>>>>>> >> recipe is not trivial. Besides the connection handling issues,
>>>>>>there are
>>>>>> >> numerous edge cases that are not well documented that must be
>>>>>> considered.
>>>>>> >> For example, many recipes require that an ephemeral-sequential
>>>>>>node be
>>>>>> >> created. New users of ZooKeeper will not know that there is an
>>>>>>edge
>>>>>> case in
>>>>>> >> ephemeral-sequential node creation that requires you to put a
>>>>>>special
>>>>>> >> ³marker² in the node¹s name so that you can search for the
>>>>>>created node
>>>>>> if
>>>>>> >> an I/O failure occurs. This is but one of many edge cases that
>>>>>>are not
>>>>>> well
>>>>>> >> documented but are handled by Curator.
>>>>>> >> >
>>>>>> >> > = Current Status =
>>>>>> >> >
>>>>>> >> > == Meritocracy ==
>>>>>> >> >
>>>>>> >> > Curator was initially developed by Jordan Zimmerman in 2011 at
>>>>>> Netflix.
>>>>>> >> Developers external to Netflix provided feedback, suggested
>>>>>>features and
>>>>>> >> fixes and implemented extensions of Curator. Netflix's
>>>>>>engineering team
>>>>>> has
>>>>>> >> since maintained the project and has been dedicated towards its
>>>>>> >> improvement. Contributors to Curator include developers from
>>>>>>multiple
>>>>>> >> organizations around the world. Curator will be a meritocracy as
>>>>>>it
>>>>>> enters
>>>>>> >> the Incubator and beyond.
>>>>>> >> >
>>>>>> >> > == Community ==
>>>>>> >> >
>>>>>> >> > Curator is currently used by a number of organizations all over
>>>>>>the
>>>>>> >> world. Curator has an active and growing user and developer
>>>>>>community
>>>>>> with
>>>>>> >> active participation in the [[
>>>>>> http://groups.google.com/group/curator-users]]
>>>>>> >> mailing list and at its !Github home: [[
>>>>>> >> https://github.com/Netflix/curator]].
>>>>>> >> >
>>>>>> >> > Since open sourcing the project, there have been fifteen
>>>>>>individuals
>>>>>> >> from various organizations who have contributed code.
>>>>>> >> >
>>>>>> >> > == Core Developers ==
>>>>>> >> >
>>>>>> >> > The core developers for Curator are:
>>>>>> >> >  * Jordan Zimmerman
>>>>>> >> >  * Jay Zarfoss
>>>>>> >> >
>>>>>> >> > Jordan has contributed towards Apache ZooKeeper and both Jordan
>>>>>>and
>>>>>> Jay
>>>>>> >> are familiar with Apache principles and philosophy for community
>>>>>>driven
>>>>>> >> software development.
>>>>>> >> >
>>>>>> >> > == Alignment ==
>>>>>> >> >
>>>>>> >> > Curator is a natural complement for Apache ZooKeeper. Java
>>>>>>users of
>>>>>> >> ZooKeeper will naturally want to use Curator. When Curator
>>>>>>graduates
>>>>>> from
>>>>>> >> Incubator it may be useful to distribute Curator artifacts as
>>>>>>part of
>>>>>> >> ZooKeeper releases as the preferred/recommended client side
>>>>>>library.
>>>>>> >> Further, at graduation a determination can be made as to whether
>>>>>>Curator
>>>>>> >> should become a Top Level Project or be merged into ZooKeeper
>>>>>>itself.
>>>>>> >> >
>>>>>> >> > = Known Risks =
>>>>>> >> >
>>>>>> >> > == Orphaned Products ==
>>>>>> >> >
>>>>>> >> > Curator is already deployed in production at multiple companies
>>>>>>and
>>>>>> they
>>>>>> >> are actively participating in creating new features. Curator is
>>>>>>getting
>>>>>> >> traction with developers and thus the risks of it being orphaned
>>>>>>are
>>>>>> >> minimal.
>>>>>> >> >
>>>>>> >> > == Inexperience with Open Source ==
>>>>>> >> >
>>>>>> >> > All code developed for Curator has been open sourced by Netflix
>>>>>>under
>>>>>> >> Apache 2.0 license.  All committers to Curator are intimately
>>>>>>familiar
>>>>>> with
>>>>>> >> the Apache model for open-source development and are experienced
>>>>>>with
>>>>>> >> working with new contributors.
>>>>>> >> >
>>>>>> >> > == Homogeneous Developers ==
>>>>>> >> >
>>>>>> >> > The initial committers are from a single organization. However,
>>>>>>we
>>>>>> >> expect that once approved for incubation, the project will
>>>>>>attract new
>>>>>> >> contributors from diverse organizations and will thus grow
>>>>>>organically.
>>>>>> The
>>>>>> >> submission of patches from developers from several different
>>>>>> organizations
>>>>>> >> is a strong indication that Curator will be widely adopted.
>>>>>> >> >
>>>>>> >> > == Reliance on Salaried Developers ==
>>>>>> >> >
>>>>>> >> > It is expected that Curator will be developed on salaried and
>>>>>> volunteer
>>>>>> >> time, although all of the initial developers will work on it
>>>>>>mainly on
>>>>>> >> salaried time.
>>>>>> >> >
>>>>>> >> > == Relationships with Other Apache Products ==
>>>>>> >> >
>>>>>> >> > Curator depends upon other Apache Projects: Apache ZooKeeper,
>>>>>>Apache
>>>>>> >> Log4J, and multiple Apache Commons components. Its build depends
>>>>>>upon
>>>>>> >> Apache Maven. Notably, there is interest from other Apache
>>>>>>Projects
>>>>>> such as
>>>>>> >> HBase in adopting Curator as the client library for ZooKeeper.
>>>>>>Apache
>>>>>> James
>>>>>> >> Mailbox has already incorporated Curator.
>>>>>> >> >
>>>>>> >> > == An Excessive Fascination with the Apache Brand ==
>>>>>> >> >
>>>>>> >> > We would like Curator to become an Apache project to further
>>>>>>foster a
>>>>>> >> healthy community of contributors and consumers around the
>>>>>>project.
>>>>>>  Since
>>>>>> >> Curator directly interacts with Apache ZooKeeper and solves an
>>>>>>important
>>>>>> >> problem of many ZooKeeper users, residing in the Apache Software
>>>>>> Foundation
>>>>>> >> will increase interaction with the larger community.
>>>>>> >> >
>>>>>> >> > = Documentation =
>>>>>> >> >
>>>>>> >> >  * Curator wiki at GitHub:
>>>>>>https://github.com/Netflix/curator/wiki
>>>>>> >> >  * Curator issues at GitHub:
>>>>>> https://github.com/Netflix/curator/issues
>>>>>> >> >  * Curator javadoc at GitHub:
>>>>>>http://netflix.github.com/curator/doc/
>>>>>> >> >
>>>>>> >> > = Initial Source =
>>>>>> >> >
>>>>>> >> >  * git://github.com/Netflix/curator.git
>>>>>> >> >
>>>>>> >> > == Source and Intellectual Property Submission Plan ==
>>>>>> >> >
>>>>>> >> >  * The initial source is already licensed under the Apache
>>>>>>License,
>>>>>> >> Version 2.0.
>>>>>>https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>>>>> >> >
>>>>>> >> > == External Dependencies ==
>>>>>> >> >
>>>>>> >> > The required external dependencies are all Apache License or
>>>>>> compatible
>>>>>> >> licenses. Following components with non-Apache licenses are
>>>>>>enumerated:
>>>>>> >> >
>>>>>> >> >  * org.slf4j: MIT-like License
>>>>>> >> >  * org.mockito: MIT-like License
>>>>>> >> >
>>>>>> >> > == Cryptography ==
>>>>>> >> >
>>>>>> >> > Curator contains no known cryptography.
>>>>>> >> >
>>>>>> >> > = Required Resources =
>>>>>> >> >
>>>>>> >> > == Mailing lists ==
>>>>>> >> >
>>>>>> >> >  * curator-private (with moderated subscriptions)
>>>>>> >> >  * curator-dev
>>>>>> >> >  * curator-commits
>>>>>> >> >  * curator-user
>>>>>> >> >
>>>>>> >> > == Github Repositories ==
>>>>>> >> >
>>>>>> >> > http://github.com/apache/curator
>>>>>>git://git.apache.org/curator.git
>>>>>> >> >
>>>>>> >> > == Issue Tracking ==
>>>>>> >> >
>>>>>> >> > JIRA Curator (CURATOR)
>>>>>> >> >
>>>>>> >> > == Other Resources ==
>>>>>> >> >
>>>>>> >> > The existing code already has unit and integration tests so we
>>>>>>would
>>>>>> >> like a Jenkins instance to run them whenever a new patch is
>>>>>>submitted.
>>>>>> This
>>>>>> >> can be added after project creation.
>>>>>> >> >
>>>>>> >> > = Initial Committers =
>>>>>> >> >
>>>>>> >> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>>>>>> >> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>>>>>> >> >
>>>>>> >> > = Affiliations =
>>>>>> >> >
>>>>>> >> >  * Jordan Zimmerman, Netflix
>>>>>> >> >  * Jay Zarfoss, Netflix
>>>>>> >> >
>>>>>> >> > = Sponsors =
>>>>>> >> >
>>>>>> >> > == Champion ==
>>>>>> >> >
>>>>>> >> >  * Patrick Hunt
>>>>>> >> >
>>>>>> >> > == Nominated Mentors ==
>>>>>> >> >
>>>>>> >> >  * Patrick Hunt
>>>>>> >> >  * Enis Söztutar
>>>>>> >> >  * Mahadev Konar
>>>>>> >> >
>>>>>> >> > == Sponsoring Entity ==
>>>>>> >> >
>>>>>> >> >  * Apache Incubator PMC
>>>>>> >>
>>>>>> >>
>>>>>>---------------------------------------------------------------------
>>>>>> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>> >> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>> >>
>>>>>> >>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>For additional commands, e-mail: general-help@incubator.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Jordan Zimmerman <jo...@jordanzimmerman.com>]

NOTE: Luciano Resende has been added as a mentor for the Curator Proposal.

-Jordan

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Luciano Resende <lu...@gmail.com>]

On Tue, Feb 26, 2013 at 1:17 PM, Mattmann, Chris A (388J)
<ch...@jpl.nasa.gov> wrote:
> Hi Luciano,
>
> On 2/26/13 12:03 PM, "Luciano Resende" <lu...@gmail.com> wrote:
>
>>>
>>
>>
>>+1, We don't need to discuss exit criteria before entering incubation.
>
> Actually we do and I can. Else I'm pretty useless on the IPMC.
>
> I just went through an experience where my objection/VOTE didn't really
> mean anything in a situation that I didn't think was correct (see
> HCatalog/Hive). I will be darned sure to discuss exit criteria now as I
> wish I would have paid attention and done so then, would have saved hassle
> all around.
>
>
>>And if the Zookeeper PMC wants to adopt Curator as part of the
>>Zookeeper project (see distinction from sub-project language, which is
>>what the Board does not favor),
>>they can work with the community and
>>do it....
>
> Define "working with the community".
>
> My definition of that doesn't include entering the Incubator.
>
> My definition of that means, Pat or someone else on the ZK PMC starts
> getting Curator patches and/or committing them and Jordan or Jay become
> Committers/PMC members on ZK because of those contributions (and have
> their ICLAs on file, etc.)
>
>

Yes, basically this (probably Netflix still need to follow some
processes to donate name + existing large amount of code)... But,
although I haven't checked the names on this thread against Zookeeper
PMC list, I haven't seen anyone jumping on the thread and suggesting
that Zookeeper PMC is open to this direction.

>>Having said that, the exit criteria should really be an
>>option instead of being dictated.
>
> I'm questioning "graduation" to an existing TLP (and not as a new one) as
> a valid exit criteria of the Incubator. I don't think it makes sense.
>

Here, I'm basically saying that, if Zookeeper community does not want
to "adopt" curator now, this should not become a barrier for the
Curator project to enter incubator, as Greg suggested with his -1
earlier in this thread.


-- 
Luciano Resende
http://people.apache.org/~lresende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

["Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>]

Hi Luciano,

On 2/26/13 12:03 PM, "Luciano Resende" <lu...@gmail.com> wrote:

>>
>
>
>+1, We don't need to discuss exit criteria before entering incubation.

Actually we do and I can. Else I'm pretty useless on the IPMC.

I just went through an experience where my objection/VOTE didn't really
mean anything in a situation that I didn't think was correct (see
HCatalog/Hive). I will be darned sure to discuss exit criteria now as I
wish I would have paid attention and done so then, would have saved hassle
all around.


>And if the Zookeeper PMC wants to adopt Curator as part of the
>Zookeeper project (see distinction from sub-project language, which is
>what the Board does not favor),
>they can work with the community and
>do it.... 

Define "working with the community".

My definition of that doesn't include entering the Incubator.

My definition of that means, Pat or someone else on the ZK PMC starts
getting Curator patches and/or committing them and Jordan or Jay become
Committers/PMC members on ZK because of those contributions (and have
their ICLAs on file, etc.)


>Having said that, the exit criteria should really be an
>option instead of being dictated.

I'm questioning "graduation" to an existing TLP (and not as a new one) as
a valid exit criteria of the Incubator. I don't think it makes sense.

Cheers,
Chris


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Luciano Resende <lu...@gmail.com>]

On Tue, Feb 26, 2013 at 11:48 AM, Jordan Zimmerman
<jo...@jordanzimmerman.com> wrote:
> My viewpoint: I can see benefit to Curator both as a TLP and as part of ZK. One of my hopes of this process is to get an idea of what the community wants. Curator has been doing well outside of Apache. But, it's become clear to me that "limited devs - single company" is a hinderance to wider adoption and growth of Curator. It's because of this that I thought the Incubator would be the perfect place for Curator at this point. From the FAQ:
>
> * "The Incubator, among other things, is where the due-diligence of making sure the licence is correct"
> * "The Incubator is also the place where projects can familiarize themselves with how the ASF works under the guidance of Incubator PMC members"
>
> Also, my reading of the Incubator docs shows that graduation does not necessarily mean TLP. I was hoping that, via Incubation, Curator's future could be explored and defined. If that means a Curator TLP, that's great. If it means something else, that's great too.
>
> -Jordan
>


+1, We don't need to discuss exit criteria before entering incubation.
And if the Zookeeper PMC wants to adopt Curator as part of the
Zookeeper project (see distinction from sub-project language, which is
what the Board does not favor), they can work with the community and
do it.... Having said that, the exit criteria should really be an
option instead of being dictated.

BTW, if you guys are still looking for mentors, please count me in.

-- 
Luciano Resende
http://people.apache.org/~lresende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

["Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>]

Hi Jordan,

Yep, thank you. I will be suggesting at graduation time barring any
epiphanies all around that you guys go the TLP route.

Good luck.

Cheers,
Chris


On 3/1/13 6:29 AM, "Jordan Zimmerman" <jo...@jordanzimmerman.com> wrote:

>Chris, 
>
>I added a line to the Alignment section. Let me know if it's OK.
>
>-JZ
>
>On Feb 26, 2013, at 1:23 PM, "Mattmann, Chris A (388J)"
><ch...@jpl.nasa.gov> wrote:
>
>> I would appreciate at some level a note in your proposal regarding at
>> least the concern by
>> one member of the IPMC that Curator should grow into its own TLP rather
>> than be a part of ZK
>> should it be accepted.
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>For additional commands, e-mail: general-help@incubator.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Jordan Zimmerman <jo...@jordanzimmerman.com>]

Chris, 

I added a line to the Alignment section. Let me know if it's OK.

-JZ

On Feb 26, 2013, at 1:23 PM, "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> wrote:

> I would appreciate at some level a note in your proposal regarding at
> least the concern by
> one member of the IPMC that Curator should grow into its own TLP rather
> than be a part of ZK
> should it be accepted.


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

["Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>]

Hi Jordan,

Thanks. To be clear, I'll be in support of your guys' acceptance into the
Apache Incubator.

I would appreciate at some level a note in your proposal regarding at
least the concern by
one member of the IPMC that Curator should grow into its own TLP rather
than be a part of ZK
should it be accepted.

I'm just trying to share experience here which suggests to me you guys
shouldn't try and graduate
into a part of the ZK PMC if you go the Incubation route. By the #s, most
projects
That go through the Incubator end up as TLPs. I think it's just the way to
go.

Take care and good luck.

Cheers,
Chris

On 2/26/13 1:17 PM, "Jordan Zimmerman" <jo...@jordanzimmerman.com> wrote:

>> Quoting Incubator docs written by one or two or a small handful of
>>people
>> in a committee with 100+ members:
>I'm new to this, so I apologize in advance for oddities in etiquette ;)
>
>> So, if your goal is to eventually get into ZK, I just don't think the
>> Incubator is the right way. It sounds like you guys are a distinct
>>project
>> anyways, so I would recommend going that way.
>
>I honestly don't know what the best ending spot for Curator is. I was
>hoping Incubator would sort that out. Of course, I want to defer to the
>experience of long-time Apache members, though.
>
>-Jordan
>---------------------------------------------------------------------
>To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>For additional commands, e-mail: general-help@incubator.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Jordan Zimmerman <jo...@jordanzimmerman.com>]

> Quoting Incubator docs written by one or two or a small handful of people
> in a committee with 100+ members:
I'm new to this, so I apologize in advance for oddities in etiquette ;)

> So, if your goal is to eventually get into ZK, I just don't think the
> Incubator is the right way. It sounds like you guys are a distinct project
> anyways, so I would recommend going that way.

I honestly don't know what the best ending spot for Curator is. I was hoping Incubator would sort that out. Of course, I want to defer to the experience of long-time Apache members, though. 

-Jordan
---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

["Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>]

Hi Jordan,


On 2/26/13 11:48 AM, "Jordan Zimmerman" <jo...@jordanzimmerman.com> wrote:

>My viewpoint: I can see benefit to Curator both as a TLP and as part of
>ZK. One of my hopes of this process is to get an idea of what the
>community wants. Curator has been doing well outside of Apache. But, it's
>become clear to me that "limited devs - single company" is a hinderance
>to wider adoption and growth of Curator. It's because of this that I
>thought the Incubator would be the perfect place for Curator at this
>point. From the FAQ:
>
>* "The Incubator, among other things, is where the due-diligence of
>making sure the licence is correct"
>* "The Incubator is also the place where projects can familiarize
>themselves with how the ASF works under the guidance of Incubator PMC
>members"
>
>Also, my reading of the Incubator docs shows that graduation does not
>necessarily mean TLP. I was hoping that, via Incubation, Curator's future
>could be explored and defined. If that means a Curator TLP, that's great.
>If it means something else, that's great too.

Quoting Incubator docs written by one or two or a small handful of people
in a committee with 100+ members:

http://people.apache.org/committers-by-project.html#incubator-pmc


Isn't exactly quoting scripture or anything. :) I've been on the committee
for a while and at the ASF since 2005, and thus have seen things that have
or haven't made their way into those docs, that I know to be true. It's
really volunteer driven. Yes, I know, we'd like it to be doctrine but in
an org like this it isn't.

That being said, I get what you guys are trying to do and I support
Incubation. I don't support Graduation->existing TLP.
The Incubator is not a home for projects that end up in an existing TLP in
my mind (and in several other people's that have been hanging around here
for a while).

So, if your goal is to eventually get into ZK, I just don't think the
Incubator is the right way. It sounds like you guys are a distinct project
anyways, so I would recommend going that way.

My 2c.

Cheers,
Chris

>
>-Jordan
>
>On Feb 26, 2013, at 10:30 AM, "Mattmann, Chris A (388J)"
><ch...@jpl.nasa.gov> wrote:
>
>> Hi Guys,
>> 
>> Sorry I have to ask the question here. If the mentors consist of PMC
>> members on ZK
>> (at least Pat and Mahadev), what's the problem with creating a branch in
>> ZK and just
>> having the code be there and getting the Curator proposed committers as
>> committers in
>> ZK ville, and hopefully PMC members soon thereafter.
>> 
>> I have to agree with Greg here. Seems like Incubation is something that
>> might not be
>> needed. If the end result of this after N months is that Curator
>> "graduates" into another
>> set of flipping bylaw updates, and more legislation that makes these
>> people unofficial
>> PMC members on ZK, then I'm double triple -1 with 50 piles of coal on
>>top.
>> Yes, I'm 
>> still remembering the HCatalog/Hive thing here - I still don't get that.
>> 
>> Either: (a) define Curator to be its own separate project/community,
>>with
>> a goal of TLP; 
>> or (b) nix Incubation and just make these guys part of ZK, on a branch
>> now. 
>> 
>> Cheers,
>> Chris
>> 
>> On 2/26/13 9:40 AM, "Benson Margulies" <bi...@gmail.com> wrote:
>> 
>>> On Tue, Feb 26, 2013 at 12:22 PM, Patrick Hunt <ph...@apache.org>
>>>wrote:
>>>> On Tue, Feb 26, 2013 at 8:32 AM, Benson Margulies
>>>> <bi...@gmail.com> wrote:
>>>>> If you think that the right destination for curator is as part of ZK,
>>>>> then it would be good to see substantive participation of the ZK PMC
>>>>> in the incubation. The goal should be to 'graduate' by having the
>>>>> curator community be granted karma at ZK and the code folded in. This
>>>>> would require, I think, the ZK community to supply at least one
>>>>> mentor, and to have a plan in advance for the eventual votes based on
>>>>> success in the incubator.
>>>> 
>>>> Hi Benson. What you're suggesting matches my current thinking.
>>>> 
>>>> The three Curator mentors are as follows: Mahadev and I (champion) are
>>>> both PMC members on ZK, Enis Söztutar is active on HBase and
>>>> interested in using Curator for that project (which already uses ZK
>>>> heavily).
>>> 
>>> OK, that's lovely.
>>> 
>>> 
>>>> 
>>>> Patrick
>>>> 
>>>>> On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra
>>>>> <he...@gmail.com> wrote:
>>>>>> Ah no, I was not suggesting about Curator to become subproject of
>>>>>>ZK.
>>>>>> I
>>>>>> just afraid that if Curator is going as incubator it will end up as
>>>>>> sub of
>>>>>> ZK as merging process.
>>>>>> 
>>>>>> Like Greg has mentioned in another reply, I would prefer Curator to
>>>>>>be
>>>>>> merged as a higher level ZK client. Surely project like HBase and
>>>>>> others
>>>>>> that relying on ZK would appreciate simpler client to ZK.
>>>>>> 
>>>>>> Thanks,
>>>>>> 
>>>>>> Henry
>>>>>> 
>>>>>> 
>>>>>> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <ph...@apache.org>
>>>>>> wrote:
>>>>>> 
>>>>>>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra
>>>>>>> <he...@gmail.com>
>>>>>>> wrote:
>>>>>>>> So isnt this similar to HCatalog which relying on Hive metadata
>>>>>>> service
>>>>>>>> that ends up as sub project of Apache Hive?
>>>>>>>> 
>>>>>>> 
>>>>>>> I was against having Curator as a sub when it came up on the
>>>>>>>original
>>>>>>> discussion thread, I still am.
>>>>>>> 
>>>>>>> Patrick
>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com>
>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> My concern is that we're looking at two "new" committers, rather
>>>>>>> than
>>>>>>>>> a Curator community. Following normal Incubator work, Curator
>>>>>>> would
>>>>>>>>> build a community for itself. But then we'd have a community
>>>>>>>>> *distinct* from that of Zookeeper. And it really looks like this
>>>>>>>>> should be part of Zookeeper itself -- a more capable and
>>>>>>> easier-to-use
>>>>>>>>> client.
>>>>>>>>> 
>>>>>>>>> So I question the incubation of this. Why do we want to build a
>>>>>>>>> new/separate project? Why isn't this just part of Zookeeper right
>>>>>>> from
>>>>>>>>> the start?
>>>>>>>>> 
>>>>>>>>> I would suggest that this work is placed on a branch within
>>>>>>> Zookeeper.
>>>>>>>>> That Jordan and Jay become committers on that branch (not
>>>>>>> necessarily
>>>>>>>>> Zookeeper trunk). Over time, the branch can be folded into trunk,
>>>>>>>>> along with all the various tests, doc, and other artifacts that I
>>>>>>> see
>>>>>>>>> in the GitHub repository. And hopefully that Jordan and Jay
>>>>>>>>>become
>>>>>>>>> regular committers (and PMC members!) of the Zookeeper project
>>>>>>> itself.
>>>>>>>>> 
>>>>>>>>> The current Zookeeper client can remain for backwards compat, and
>>>>>>> the
>>>>>>>>> Curator work can become the next-gen client.
>>>>>>>>> 
>>>>>>>>> Honestly, in my opnion, incubating this project seems like it
>>>>>>> would
>>>>>>>>> create a distinct community, and really doesn't seem like it
>>>>>>>>>would
>>>>>>>>> serve the Zookeeper community.
>>>>>>>>> 
>>>>>>>>> All that said, I am not familiar with the Zookeeper or Curator
>>>>>>>>> communities. But from this read, I don't think Incubation is the
>>>>>>> right
>>>>>>>>> approach. I would rather push for a more direct incorporation of
>>>>>>>>> Curator directly into Zookeeper. (use the short-form IP
>>>>>>> clearance) ...
>>>>>>>>> so, unless somebody can help me understand the communities and
>>>>>>>>> situation better, I'm -1 (binding) on this incubation. I'd rather
>>>>>>> see
>>>>>>>>> combined, rather than distinct, communities from the start.
>>>>>>>>> 
>>>>>>>>> Thanks,
>>>>>>>>> -g
>>>>>>>>> 
>>>>>>>>> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>>>>>>>>> <jo...@jordanzimmerman.com> wrote:
>>>>>>>>>> Hello,
>>>>>>>>>> 
>>>>>>>>>> I would like to propose that Curator to be an Apache Incubator
>>>>>>> project.
>>>>>>>>>> 
>>>>>>>>>> The proposal can be found here:
>>>>>>>>> http://wiki.apache.org/incubator/CuratorProposal
>>>>>>>>>> 
>>>>>>>>>> I have included the contents of the proposal below.
>>>>>>>>>> 
>>>>>>>>>> Sincerely,
>>>>>>>>>> 
>>>>>>>>>> Jordan Zimmerman
>>>>>>>>>> 
>>>>>>>>>> ===================
>>>>>>>>>> 
>>>>>>>>>> = Curator - ZooKeeper client wrapper and rich ZooKeeper
>>>>>>> framework =
>>>>>>>>>> 
>>>>>>>>>> == Abstract ==
>>>>>>>>>> 
>>>>>>>>>> Curator is a set of Java libraries that make using Apache
>>>>>>> ZooKeeper
>>>>>>> much
>>>>>>>>> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>>> the
>>>>>>> client
>>>>>>>>> is non-trivial and error prone.
>>>>>>>>>> 
>>>>>>>>>> == Proposal ==
>>>>>>>>>> 
>>>>>>>>>> Curator is a set of Java libraries that make using Apache
>>>>>>> ZooKeeper
>>>>>>> much
>>>>>>>>> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>>> the
>>>>>>> client
>>>>>>>>> is non-trivial and error prone. It consists of three components
>>>>>>> that
>>>>>>> build
>>>>>>>>> on each other. Curator Client is a replacement for the bundled
>>>>>>> ZooKeeper
>>>>>>>>> class that takes care of some low-level housekeeping and provides
>>>>>>> some
>>>>>>>>> useful utilities. Curator Framework is a high-level API that
>>>>>>> greatly
>>>>>>>>> simplifies using ZooKeeper. It adds many features that build on
>>>>>>> ZooKeeper
>>>>>>>>> and handles the complexity of managing connections to the
>>>>>>> ZooKeeper
>>>>>>> cluster
>>>>>>>>> and retrying operations. Curator Recipes consists of
>>>>>>> implementations of
>>>>>>>>> some of the common ZooKeeper ³recipes². Additionally, Curator
>>>>>>> Test is
>>>>>>>>> included which includes utilities to help with unit testing
>>>>>>> ZooKeeper-based
>>>>>>>>> applications.
>>>>>>>>>> 
>>>>>>>>>> == Background ==
>>>>>>>>>> 
>>>>>>>>>> Curator was initially developed by Netflix to make writing
>>>>>>>>> ZooKeeper-based applications easier and more reliable. Curator
>>>>>>>>>was
>>>>>>>>> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed
>>>>>>> project in
>>>>>>>>> July 2011. During this time Curator has been formally released
>>>>>>> many
>>>>>>> times
>>>>>>>>> and has gained widespread adoption.
>>>>>>>>>> 
>>>>>>>>>> == Rationale ==
>>>>>>>>>> 
>>>>>>>>>> New users of ZooKeeper are surprised to learn that a significant
>>>>>>> amount
>>>>>>>>> of connection management must be done manually. For example, when
>>>>>>> the
>>>>>>>>> ZooKeeper client connects to the ensemble it must negotiate a new
>>>>>>> session,
>>>>>>>>> etc. This takes some time. If you use a ZooKeeper client API
>>>>>>> before the
>>>>>>>>> connection process has completed, ZooKeeper will throw an
>>>>>>> exception.
>>>>>>> These
>>>>>>>>> types of exceptions are referred to as ³recoverable² errors.
>>>>>>>>>> Curator automatically handles connection management, greatly
>>>>>>> simplifying
>>>>>>>>> client code. Instead of directly using the ZooKeeper APIs you use
>>>>>>> Curator
>>>>>>>>> APIs that internally check for connection completion and wrap
>>>>>>>>>each
>>>>>>>>> ZooKeeper API in a retry loop. Curator uses a retry mechanism to
>>>>>>> handle
>>>>>>>>> recoverable errors and automatically retry operations. The method
>>>>>>> of
>>>>>>> retry
>>>>>>>>> is customizable. Curator comes bundled with several
>>>>>>> implementations
>>>>>>>>> (ExponentialBackoffRetry, etc.) or custom implementations can be
>>>>>>> written.
>>>>>>>>>> 
>>>>>>>>>> The ZooKeeper documentation describes many possible uses for
>>>>>>> ZooKeeper
>>>>>>>>> calling each a ³recipe². While the distribution comes bundled
>>>>>>> with a few
>>>>>>>>> implementations of these recipes, most ZooKeeper users will need
>>>>>>> to
>>>>>>>>> manually implement one or more of the recipes. Implementing a
>>>>>>> ZooKeeper
>>>>>>>>> recipe is not trivial. Besides the connection handling issues,
>>>>>>> there are
>>>>>>>>> numerous edge cases that are not well documented that must be
>>>>>>> considered.
>>>>>>>>> For example, many recipes require that an ephemeral-sequential
>>>>>>> node be
>>>>>>>>> created. New users of ZooKeeper will not know that there is an
>>>>>>> edge
>>>>>>> case in
>>>>>>>>> ephemeral-sequential node creation that requires you to put a
>>>>>>> special
>>>>>>>>> ³marker² in the node¹s name so that you can search for the
>>>>>>> created node
>>>>>>> if
>>>>>>>>> an I/O failure occurs. This is but one of many edge cases that
>>>>>>> are not
>>>>>>> well
>>>>>>>>> documented but are handled by Curator.
>>>>>>>>>> 
>>>>>>>>>> = Current Status =
>>>>>>>>>> 
>>>>>>>>>> == Meritocracy ==
>>>>>>>>>> 
>>>>>>>>>> Curator was initially developed by Jordan Zimmerman in 2011 at
>>>>>>> Netflix.
>>>>>>>>> Developers external to Netflix provided feedback, suggested
>>>>>>> features and
>>>>>>>>> fixes and implemented extensions of Curator. Netflix's
>>>>>>> engineering team
>>>>>>> has
>>>>>>>>> since maintained the project and has been dedicated towards its
>>>>>>>>> improvement. Contributors to Curator include developers from
>>>>>>> multiple
>>>>>>>>> organizations around the world. Curator will be a meritocracy as
>>>>>>> it
>>>>>>> enters
>>>>>>>>> the Incubator and beyond.
>>>>>>>>>> 
>>>>>>>>>> == Community ==
>>>>>>>>>> 
>>>>>>>>>> Curator is currently used by a number of organizations all over
>>>>>>> the
>>>>>>>>> world. Curator has an active and growing user and developer
>>>>>>> community
>>>>>>> with
>>>>>>>>> active participation in the [[
>>>>>>> http://groups.google.com/group/curator-users]]
>>>>>>>>> mailing list and at its !Github home: [[
>>>>>>>>> https://github.com/Netflix/curator]].
>>>>>>>>>> 
>>>>>>>>>> Since open sourcing the project, there have been fifteen
>>>>>>> individuals
>>>>>>>>> from various organizations who have contributed code.
>>>>>>>>>> 
>>>>>>>>>> == Core Developers ==
>>>>>>>>>> 
>>>>>>>>>> The core developers for Curator are:
>>>>>>>>>> * Jordan Zimmerman
>>>>>>>>>> * Jay Zarfoss
>>>>>>>>>> 
>>>>>>>>>> Jordan has contributed towards Apache ZooKeeper and both Jordan
>>>>>>> and
>>>>>>> Jay
>>>>>>>>> are familiar with Apache principles and philosophy for community
>>>>>>> driven
>>>>>>>>> software development.
>>>>>>>>>> 
>>>>>>>>>> == Alignment ==
>>>>>>>>>> 
>>>>>>>>>> Curator is a natural complement for Apache ZooKeeper. Java
>>>>>>> users of
>>>>>>>>> ZooKeeper will naturally want to use Curator. When Curator
>>>>>>> graduates
>>>>>>> from
>>>>>>>>> Incubator it may be useful to distribute Curator artifacts as
>>>>>>> part of
>>>>>>>>> ZooKeeper releases as the preferred/recommended client side
>>>>>>> library.
>>>>>>>>> Further, at graduation a determination can be made as to whether
>>>>>>> Curator
>>>>>>>>> should become a Top Level Project or be merged into ZooKeeper
>>>>>>> itself.
>>>>>>>>>> 
>>>>>>>>>> = Known Risks =
>>>>>>>>>> 
>>>>>>>>>> == Orphaned Products ==
>>>>>>>>>> 
>>>>>>>>>> Curator is already deployed in production at multiple companies
>>>>>>> and
>>>>>>> they
>>>>>>>>> are actively participating in creating new features. Curator is
>>>>>>> getting
>>>>>>>>> traction with developers and thus the risks of it being orphaned
>>>>>>> are
>>>>>>>>> minimal.
>>>>>>>>>> 
>>>>>>>>>> == Inexperience with Open Source ==
>>>>>>>>>> 
>>>>>>>>>> All code developed for Curator has been open sourced by Netflix
>>>>>>> under
>>>>>>>>> Apache 2.0 license.  All committers to Curator are intimately
>>>>>>> familiar
>>>>>>> with
>>>>>>>>> the Apache model for open-source development and are experienced
>>>>>>> with
>>>>>>>>> working with new contributors.
>>>>>>>>>> 
>>>>>>>>>> == Homogeneous Developers ==
>>>>>>>>>> 
>>>>>>>>>> The initial committers are from a single organization. However,
>>>>>>> we
>>>>>>>>> expect that once approved for incubation, the project will
>>>>>>> attract new
>>>>>>>>> contributors from diverse organizations and will thus grow
>>>>>>> organically.
>>>>>>> The
>>>>>>>>> submission of patches from developers from several different
>>>>>>> organizations
>>>>>>>>> is a strong indication that Curator will be widely adopted.
>>>>>>>>>> 
>>>>>>>>>> == Reliance on Salaried Developers ==
>>>>>>>>>> 
>>>>>>>>>> It is expected that Curator will be developed on salaried and
>>>>>>> volunteer
>>>>>>>>> time, although all of the initial developers will work on it
>>>>>>> mainly on
>>>>>>>>> salaried time.
>>>>>>>>>> 
>>>>>>>>>> == Relationships with Other Apache Products ==
>>>>>>>>>> 
>>>>>>>>>> Curator depends upon other Apache Projects: Apache ZooKeeper,
>>>>>>> Apache
>>>>>>>>> Log4J, and multiple Apache Commons components. Its build depends
>>>>>>> upon
>>>>>>>>> Apache Maven. Notably, there is interest from other Apache
>>>>>>> Projects
>>>>>>> such as
>>>>>>>>> HBase in adopting Curator as the client library for ZooKeeper.
>>>>>>> Apache
>>>>>>> James
>>>>>>>>> Mailbox has already incorporated Curator.
>>>>>>>>>> 
>>>>>>>>>> == An Excessive Fascination with the Apache Brand ==
>>>>>>>>>> 
>>>>>>>>>> We would like Curator to become an Apache project to further
>>>>>>> foster a
>>>>>>>>> healthy community of contributors and consumers around the
>>>>>>> project.
>>>>>>> Since
>>>>>>>>> Curator directly interacts with Apache ZooKeeper and solves an
>>>>>>> important
>>>>>>>>> problem of many ZooKeeper users, residing in the Apache Software
>>>>>>> Foundation
>>>>>>>>> will increase interaction with the larger community.
>>>>>>>>>> 
>>>>>>>>>> = Documentation =
>>>>>>>>>> 
>>>>>>>>>> * Curator wiki at GitHub:
>>>>>>> https://github.com/Netflix/curator/wiki
>>>>>>>>>> * Curator issues at GitHub:
>>>>>>> https://github.com/Netflix/curator/issues
>>>>>>>>>> * Curator javadoc at GitHub:
>>>>>>> http://netflix.github.com/curator/doc/
>>>>>>>>>> 
>>>>>>>>>> = Initial Source =
>>>>>>>>>> 
>>>>>>>>>> * git://github.com/Netflix/curator.git
>>>>>>>>>> 
>>>>>>>>>> == Source and Intellectual Property Submission Plan ==
>>>>>>>>>> 
>>>>>>>>>> * The initial source is already licensed under the Apache
>>>>>>> License,
>>>>>>>>> Version 2.0.
>>>>>>> https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>>>>>>>>> 
>>>>>>>>>> == External Dependencies ==
>>>>>>>>>> 
>>>>>>>>>> The required external dependencies are all Apache License or
>>>>>>> compatible
>>>>>>>>> licenses. Following components with non-Apache licenses are
>>>>>>> enumerated:
>>>>>>>>>> 
>>>>>>>>>> * org.slf4j: MIT-like License
>>>>>>>>>> * org.mockito: MIT-like License
>>>>>>>>>> 
>>>>>>>>>> == Cryptography ==
>>>>>>>>>> 
>>>>>>>>>> Curator contains no known cryptography.
>>>>>>>>>> 
>>>>>>>>>> = Required Resources =
>>>>>>>>>> 
>>>>>>>>>> == Mailing lists ==
>>>>>>>>>> 
>>>>>>>>>> * curator-private (with moderated subscriptions)
>>>>>>>>>> * curator-dev
>>>>>>>>>> * curator-commits
>>>>>>>>>> * curator-user
>>>>>>>>>> 
>>>>>>>>>> == Github Repositories ==
>>>>>>>>>> 
>>>>>>>>>> http://github.com/apache/curator
>>>>>>> git://git.apache.org/curator.git
>>>>>>>>>> 
>>>>>>>>>> == Issue Tracking ==
>>>>>>>>>> 
>>>>>>>>>> JIRA Curator (CURATOR)
>>>>>>>>>> 
>>>>>>>>>> == Other Resources ==
>>>>>>>>>> 
>>>>>>>>>> The existing code already has unit and integration tests so we
>>>>>>> would
>>>>>>>>> like a Jenkins instance to run them whenever a new patch is
>>>>>>> submitted.
>>>>>>> This
>>>>>>>>> can be added after project creation.
>>>>>>>>>> 
>>>>>>>>>> = Initial Committers =
>>>>>>>>>> 
>>>>>>>>>> * Jordan Zimmerman (jzimmerman at netflix dot com)
>>>>>>>>>> * Jay Zarfoss (jzarfoss at netflix dot com)
>>>>>>>>>> 
>>>>>>>>>> = Affiliations =
>>>>>>>>>> 
>>>>>>>>>> * Jordan Zimmerman, Netflix
>>>>>>>>>> * Jay Zarfoss, Netflix
>>>>>>>>>> 
>>>>>>>>>> = Sponsors =
>>>>>>>>>> 
>>>>>>>>>> == Champion ==
>>>>>>>>>> 
>>>>>>>>>> * Patrick Hunt
>>>>>>>>>> 
>>>>>>>>>> == Nominated Mentors ==
>>>>>>>>>> 
>>>>>>>>>> * Patrick Hunt
>>>>>>>>>> * Enis Söztutar
>>>>>>>>>> * Mahadev Konar
>>>>>>>>>> 
>>>>>>>>>> == Sponsoring Entity ==
>>>>>>>>>> 
>>>>>>>>>> * Apache Incubator PMC
>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>>>--------------------------------------------------------------------
>>>>>>>-
>>>>>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>general-help@incubator.apache.org
>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>--------------------------------------------------------------------
>>>>>>>-
>>>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>>> 
>>>>>>> 
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: general-help@incubator.apache.org
>>> 
>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>> 
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>For additional commands, e-mail: general-help@incubator.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Jordan Zimmerman <jo...@jordanzimmerman.com>]

My viewpoint: I can see benefit to Curator both as a TLP and as part of ZK. One of my hopes of this process is to get an idea of what the community wants. Curator has been doing well outside of Apache. But, it's become clear to me that "limited devs - single company" is a hinderance to wider adoption and growth of Curator. It's because of this that I thought the Incubator would be the perfect place for Curator at this point. From the FAQ:

* "The Incubator, among other things, is where the due-diligence of making sure the licence is correct"
* "The Incubator is also the place where projects can familiarize themselves with how the ASF works under the guidance of Incubator PMC members"

Also, my reading of the Incubator docs shows that graduation does not necessarily mean TLP. I was hoping that, via Incubation, Curator's future could be explored and defined. If that means a Curator TLP, that's great. If it means something else, that's great too.

-Jordan

On Feb 26, 2013, at 10:30 AM, "Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov> wrote:

> Hi Guys,
> 
> Sorry I have to ask the question here. If the mentors consist of PMC
> members on ZK
> (at least Pat and Mahadev), what's the problem with creating a branch in
> ZK and just
> having the code be there and getting the Curator proposed committers as
> committers in
> ZK ville, and hopefully PMC members soon thereafter.
> 
> I have to agree with Greg here. Seems like Incubation is something that
> might not be
> needed. If the end result of this after N months is that Curator
> "graduates" into another
> set of flipping bylaw updates, and more legislation that makes these
> people unofficial 
> PMC members on ZK, then I'm double triple -1 with 50 piles of coal on top.
> Yes, I'm 
> still remembering the HCatalog/Hive thing here - I still don't get that.
> 
> Either: (a) define Curator to be its own separate project/community, with
> a goal of TLP; 
> or (b) nix Incubation and just make these guys part of ZK, on a branch
> now. 
> 
> Cheers,
> Chris
> 
> On 2/26/13 9:40 AM, "Benson Margulies" <bi...@gmail.com> wrote:
> 
>> On Tue, Feb 26, 2013 at 12:22 PM, Patrick Hunt <ph...@apache.org> wrote:
>>> On Tue, Feb 26, 2013 at 8:32 AM, Benson Margulies
>>> <bi...@gmail.com> wrote:
>>>> If you think that the right destination for curator is as part of ZK,
>>>> then it would be good to see substantive participation of the ZK PMC
>>>> in the incubation. The goal should be to 'graduate' by having the
>>>> curator community be granted karma at ZK and the code folded in. This
>>>> would require, I think, the ZK community to supply at least one
>>>> mentor, and to have a plan in advance for the eventual votes based on
>>>> success in the incubator.
>>> 
>>> Hi Benson. What you're suggesting matches my current thinking.
>>> 
>>> The three Curator mentors are as follows: Mahadev and I (champion) are
>>> both PMC members on ZK, Enis Söztutar is active on HBase and
>>> interested in using Curator for that project (which already uses ZK
>>> heavily).
>> 
>> OK, that's lovely.
>> 
>> 
>>> 
>>> Patrick
>>> 
>>>> On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra
>>>> <he...@gmail.com> wrote:
>>>>> Ah no, I was not suggesting about Curator to become subproject of ZK.
>>>>> I
>>>>> just afraid that if Curator is going as incubator it will end up as
>>>>> sub of
>>>>> ZK as merging process.
>>>>> 
>>>>> Like Greg has mentioned in another reply, I would prefer Curator to be
>>>>> merged as a higher level ZK client. Surely project like HBase and
>>>>> others
>>>>> that relying on ZK would appreciate simpler client to ZK.
>>>>> 
>>>>> Thanks,
>>>>> 
>>>>> Henry
>>>>> 
>>>>> 
>>>>> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <ph...@apache.org>
>>>>> wrote:
>>>>> 
>>>>>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra
>>>>>> <he...@gmail.com>
>>>>>> wrote:
>>>>>>> So isnt this similar to HCatalog which relying on Hive metadata
>>>>>> service
>>>>>>> that ends up as sub project of Apache Hive?
>>>>>>> 
>>>>>> 
>>>>>> I was against having Curator as a sub when it came up on the original
>>>>>> discussion thread, I still am.
>>>>>> 
>>>>>> Patrick
>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com>
>>>>>> wrote:
>>>>>>> 
>>>>>>>> My concern is that we're looking at two "new" committers, rather
>>>>>> than
>>>>>>>> a Curator community. Following normal Incubator work, Curator
>>>>>> would
>>>>>>>> build a community for itself. But then we'd have a community
>>>>>>>> *distinct* from that of Zookeeper. And it really looks like this
>>>>>>>> should be part of Zookeeper itself -- a more capable and
>>>>>> easier-to-use
>>>>>>>> client.
>>>>>>>> 
>>>>>>>> So I question the incubation of this. Why do we want to build a
>>>>>>>> new/separate project? Why isn't this just part of Zookeeper right
>>>>>> from
>>>>>>>> the start?
>>>>>>>> 
>>>>>>>> I would suggest that this work is placed on a branch within
>>>>>> Zookeeper.
>>>>>>>> That Jordan and Jay become committers on that branch (not
>>>>>> necessarily
>>>>>>>> Zookeeper trunk). Over time, the branch can be folded into trunk,
>>>>>>>> along with all the various tests, doc, and other artifacts that I
>>>>>> see
>>>>>>>> in the GitHub repository. And hopefully that Jordan and Jay become
>>>>>>>> regular committers (and PMC members!) of the Zookeeper project
>>>>>> itself.
>>>>>>>> 
>>>>>>>> The current Zookeeper client can remain for backwards compat, and
>>>>>> the
>>>>>>>> Curator work can become the next-gen client.
>>>>>>>> 
>>>>>>>> Honestly, in my opnion, incubating this project seems like it
>>>>>> would
>>>>>>>> create a distinct community, and really doesn't seem like it would
>>>>>>>> serve the Zookeeper community.
>>>>>>>> 
>>>>>>>> All that said, I am not familiar with the Zookeeper or Curator
>>>>>>>> communities. But from this read, I don't think Incubation is the
>>>>>> right
>>>>>>>> approach. I would rather push for a more direct incorporation of
>>>>>>>> Curator directly into Zookeeper. (use the short-form IP
>>>>>> clearance) ...
>>>>>>>> so, unless somebody can help me understand the communities and
>>>>>>>> situation better, I'm -1 (binding) on this incubation. I'd rather
>>>>>> see
>>>>>>>> combined, rather than distinct, communities from the start.
>>>>>>>> 
>>>>>>>> Thanks,
>>>>>>>> -g
>>>>>>>> 
>>>>>>>> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>>>>>>>> <jo...@jordanzimmerman.com> wrote:
>>>>>>>>> Hello,
>>>>>>>>> 
>>>>>>>>> I would like to propose that Curator to be an Apache Incubator
>>>>>> project.
>>>>>>>>> 
>>>>>>>>> The proposal can be found here:
>>>>>>>> http://wiki.apache.org/incubator/CuratorProposal
>>>>>>>>> 
>>>>>>>>> I have included the contents of the proposal below.
>>>>>>>>> 
>>>>>>>>> Sincerely,
>>>>>>>>> 
>>>>>>>>> Jordan Zimmerman
>>>>>>>>> 
>>>>>>>>> ===================
>>>>>>>>> 
>>>>>>>>> = Curator - ZooKeeper client wrapper and rich ZooKeeper
>>>>>> framework =
>>>>>>>>> 
>>>>>>>>> == Abstract ==
>>>>>>>>> 
>>>>>>>>> Curator is a set of Java libraries that make using Apache
>>>>>> ZooKeeper
>>>>>> much
>>>>>>>> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>> the
>>>>>> client
>>>>>>>> is non-trivial and error prone.
>>>>>>>>> 
>>>>>>>>> == Proposal ==
>>>>>>>>> 
>>>>>>>>> Curator is a set of Java libraries that make using Apache
>>>>>> ZooKeeper
>>>>>> much
>>>>>>>> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>> the
>>>>>> client
>>>>>>>> is non-trivial and error prone. It consists of three components
>>>>>> that
>>>>>> build
>>>>>>>> on each other. Curator Client is a replacement for the bundled
>>>>>> ZooKeeper
>>>>>>>> class that takes care of some low-level housekeeping and provides
>>>>>> some
>>>>>>>> useful utilities. Curator Framework is a high-level API that
>>>>>> greatly
>>>>>>>> simplifies using ZooKeeper. It adds many features that build on
>>>>>> ZooKeeper
>>>>>>>> and handles the complexity of managing connections to the
>>>>>> ZooKeeper
>>>>>> cluster
>>>>>>>> and retrying operations. Curator Recipes consists of
>>>>>> implementations of
>>>>>>>> some of the common ZooKeeper ³recipes². Additionally, Curator
>>>>>> Test is
>>>>>>>> included which includes utilities to help with unit testing
>>>>>> ZooKeeper-based
>>>>>>>> applications.
>>>>>>>>> 
>>>>>>>>> == Background ==
>>>>>>>>> 
>>>>>>>>> Curator was initially developed by Netflix to make writing
>>>>>>>> ZooKeeper-based applications easier and more reliable. Curator was
>>>>>>>> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed
>>>>>> project in
>>>>>>>> July 2011. During this time Curator has been formally released
>>>>>> many
>>>>>> times
>>>>>>>> and has gained widespread adoption.
>>>>>>>>> 
>>>>>>>>> == Rationale ==
>>>>>>>>> 
>>>>>>>>> New users of ZooKeeper are surprised to learn that a significant
>>>>>> amount
>>>>>>>> of connection management must be done manually. For example, when
>>>>>> the
>>>>>>>> ZooKeeper client connects to the ensemble it must negotiate a new
>>>>>> session,
>>>>>>>> etc. This takes some time. If you use a ZooKeeper client API
>>>>>> before the
>>>>>>>> connection process has completed, ZooKeeper will throw an
>>>>>> exception.
>>>>>> These
>>>>>>>> types of exceptions are referred to as ³recoverable² errors.
>>>>>>>>> Curator automatically handles connection management, greatly
>>>>>> simplifying
>>>>>>>> client code. Instead of directly using the ZooKeeper APIs you use
>>>>>> Curator
>>>>>>>> APIs that internally check for connection completion and wrap each
>>>>>>>> ZooKeeper API in a retry loop. Curator uses a retry mechanism to
>>>>>> handle
>>>>>>>> recoverable errors and automatically retry operations. The method
>>>>>> of
>>>>>> retry
>>>>>>>> is customizable. Curator comes bundled with several
>>>>>> implementations
>>>>>>>> (ExponentialBackoffRetry, etc.) or custom implementations can be
>>>>>> written.
>>>>>>>>> 
>>>>>>>>> The ZooKeeper documentation describes many possible uses for
>>>>>> ZooKeeper
>>>>>>>> calling each a ³recipe². While the distribution comes bundled
>>>>>> with a few
>>>>>>>> implementations of these recipes, most ZooKeeper users will need
>>>>>> to
>>>>>>>> manually implement one or more of the recipes. Implementing a
>>>>>> ZooKeeper
>>>>>>>> recipe is not trivial. Besides the connection handling issues,
>>>>>> there are
>>>>>>>> numerous edge cases that are not well documented that must be
>>>>>> considered.
>>>>>>>> For example, many recipes require that an ephemeral-sequential
>>>>>> node be
>>>>>>>> created. New users of ZooKeeper will not know that there is an
>>>>>> edge
>>>>>> case in
>>>>>>>> ephemeral-sequential node creation that requires you to put a
>>>>>> special
>>>>>>>> ³marker² in the node¹s name so that you can search for the
>>>>>> created node
>>>>>> if
>>>>>>>> an I/O failure occurs. This is but one of many edge cases that
>>>>>> are not
>>>>>> well
>>>>>>>> documented but are handled by Curator.
>>>>>>>>> 
>>>>>>>>> = Current Status =
>>>>>>>>> 
>>>>>>>>> == Meritocracy ==
>>>>>>>>> 
>>>>>>>>> Curator was initially developed by Jordan Zimmerman in 2011 at
>>>>>> Netflix.
>>>>>>>> Developers external to Netflix provided feedback, suggested
>>>>>> features and
>>>>>>>> fixes and implemented extensions of Curator. Netflix's
>>>>>> engineering team
>>>>>> has
>>>>>>>> since maintained the project and has been dedicated towards its
>>>>>>>> improvement. Contributors to Curator include developers from
>>>>>> multiple
>>>>>>>> organizations around the world. Curator will be a meritocracy as
>>>>>> it
>>>>>> enters
>>>>>>>> the Incubator and beyond.
>>>>>>>>> 
>>>>>>>>> == Community ==
>>>>>>>>> 
>>>>>>>>> Curator is currently used by a number of organizations all over
>>>>>> the
>>>>>>>> world. Curator has an active and growing user and developer
>>>>>> community
>>>>>> with
>>>>>>>> active participation in the [[
>>>>>> http://groups.google.com/group/curator-users]]
>>>>>>>> mailing list and at its !Github home: [[
>>>>>>>> https://github.com/Netflix/curator]].
>>>>>>>>> 
>>>>>>>>> Since open sourcing the project, there have been fifteen
>>>>>> individuals
>>>>>>>> from various organizations who have contributed code.
>>>>>>>>> 
>>>>>>>>> == Core Developers ==
>>>>>>>>> 
>>>>>>>>> The core developers for Curator are:
>>>>>>>>> * Jordan Zimmerman
>>>>>>>>> * Jay Zarfoss
>>>>>>>>> 
>>>>>>>>> Jordan has contributed towards Apache ZooKeeper and both Jordan
>>>>>> and
>>>>>> Jay
>>>>>>>> are familiar with Apache principles and philosophy for community
>>>>>> driven
>>>>>>>> software development.
>>>>>>>>> 
>>>>>>>>> == Alignment ==
>>>>>>>>> 
>>>>>>>>> Curator is a natural complement for Apache ZooKeeper. Java
>>>>>> users of
>>>>>>>> ZooKeeper will naturally want to use Curator. When Curator
>>>>>> graduates
>>>>>> from
>>>>>>>> Incubator it may be useful to distribute Curator artifacts as
>>>>>> part of
>>>>>>>> ZooKeeper releases as the preferred/recommended client side
>>>>>> library.
>>>>>>>> Further, at graduation a determination can be made as to whether
>>>>>> Curator
>>>>>>>> should become a Top Level Project or be merged into ZooKeeper
>>>>>> itself.
>>>>>>>>> 
>>>>>>>>> = Known Risks =
>>>>>>>>> 
>>>>>>>>> == Orphaned Products ==
>>>>>>>>> 
>>>>>>>>> Curator is already deployed in production at multiple companies
>>>>>> and
>>>>>> they
>>>>>>>> are actively participating in creating new features. Curator is
>>>>>> getting
>>>>>>>> traction with developers and thus the risks of it being orphaned
>>>>>> are
>>>>>>>> minimal.
>>>>>>>>> 
>>>>>>>>> == Inexperience with Open Source ==
>>>>>>>>> 
>>>>>>>>> All code developed for Curator has been open sourced by Netflix
>>>>>> under
>>>>>>>> Apache 2.0 license.  All committers to Curator are intimately
>>>>>> familiar
>>>>>> with
>>>>>>>> the Apache model for open-source development and are experienced
>>>>>> with
>>>>>>>> working with new contributors.
>>>>>>>>> 
>>>>>>>>> == Homogeneous Developers ==
>>>>>>>>> 
>>>>>>>>> The initial committers are from a single organization. However,
>>>>>> we
>>>>>>>> expect that once approved for incubation, the project will
>>>>>> attract new
>>>>>>>> contributors from diverse organizations and will thus grow
>>>>>> organically.
>>>>>> The
>>>>>>>> submission of patches from developers from several different
>>>>>> organizations
>>>>>>>> is a strong indication that Curator will be widely adopted.
>>>>>>>>> 
>>>>>>>>> == Reliance on Salaried Developers ==
>>>>>>>>> 
>>>>>>>>> It is expected that Curator will be developed on salaried and
>>>>>> volunteer
>>>>>>>> time, although all of the initial developers will work on it
>>>>>> mainly on
>>>>>>>> salaried time.
>>>>>>>>> 
>>>>>>>>> == Relationships with Other Apache Products ==
>>>>>>>>> 
>>>>>>>>> Curator depends upon other Apache Projects: Apache ZooKeeper,
>>>>>> Apache
>>>>>>>> Log4J, and multiple Apache Commons components. Its build depends
>>>>>> upon
>>>>>>>> Apache Maven. Notably, there is interest from other Apache
>>>>>> Projects
>>>>>> such as
>>>>>>>> HBase in adopting Curator as the client library for ZooKeeper.
>>>>>> Apache
>>>>>> James
>>>>>>>> Mailbox has already incorporated Curator.
>>>>>>>>> 
>>>>>>>>> == An Excessive Fascination with the Apache Brand ==
>>>>>>>>> 
>>>>>>>>> We would like Curator to become an Apache project to further
>>>>>> foster a
>>>>>>>> healthy community of contributors and consumers around the
>>>>>> project.
>>>>>> Since
>>>>>>>> Curator directly interacts with Apache ZooKeeper and solves an
>>>>>> important
>>>>>>>> problem of many ZooKeeper users, residing in the Apache Software
>>>>>> Foundation
>>>>>>>> will increase interaction with the larger community.
>>>>>>>>> 
>>>>>>>>> = Documentation =
>>>>>>>>> 
>>>>>>>>> * Curator wiki at GitHub:
>>>>>> https://github.com/Netflix/curator/wiki
>>>>>>>>> * Curator issues at GitHub:
>>>>>> https://github.com/Netflix/curator/issues
>>>>>>>>> * Curator javadoc at GitHub:
>>>>>> http://netflix.github.com/curator/doc/
>>>>>>>>> 
>>>>>>>>> = Initial Source =
>>>>>>>>> 
>>>>>>>>> * git://github.com/Netflix/curator.git
>>>>>>>>> 
>>>>>>>>> == Source and Intellectual Property Submission Plan ==
>>>>>>>>> 
>>>>>>>>> * The initial source is already licensed under the Apache
>>>>>> License,
>>>>>>>> Version 2.0.
>>>>>> https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>>>>>>>> 
>>>>>>>>> == External Dependencies ==
>>>>>>>>> 
>>>>>>>>> The required external dependencies are all Apache License or
>>>>>> compatible
>>>>>>>> licenses. Following components with non-Apache licenses are
>>>>>> enumerated:
>>>>>>>>> 
>>>>>>>>> * org.slf4j: MIT-like License
>>>>>>>>> * org.mockito: MIT-like License
>>>>>>>>> 
>>>>>>>>> == Cryptography ==
>>>>>>>>> 
>>>>>>>>> Curator contains no known cryptography.
>>>>>>>>> 
>>>>>>>>> = Required Resources =
>>>>>>>>> 
>>>>>>>>> == Mailing lists ==
>>>>>>>>> 
>>>>>>>>> * curator-private (with moderated subscriptions)
>>>>>>>>> * curator-dev
>>>>>>>>> * curator-commits
>>>>>>>>> * curator-user
>>>>>>>>> 
>>>>>>>>> == Github Repositories ==
>>>>>>>>> 
>>>>>>>>> http://github.com/apache/curator
>>>>>> git://git.apache.org/curator.git
>>>>>>>>> 
>>>>>>>>> == Issue Tracking ==
>>>>>>>>> 
>>>>>>>>> JIRA Curator (CURATOR)
>>>>>>>>> 
>>>>>>>>> == Other Resources ==
>>>>>>>>> 
>>>>>>>>> The existing code already has unit and integration tests so we
>>>>>> would
>>>>>>>> like a Jenkins instance to run them whenever a new patch is
>>>>>> submitted.
>>>>>> This
>>>>>>>> can be added after project creation.
>>>>>>>>> 
>>>>>>>>> = Initial Committers =
>>>>>>>>> 
>>>>>>>>> * Jordan Zimmerman (jzimmerman at netflix dot com)
>>>>>>>>> * Jay Zarfoss (jzarfoss at netflix dot com)
>>>>>>>>> 
>>>>>>>>> = Affiliations =
>>>>>>>>> 
>>>>>>>>> * Jordan Zimmerman, Netflix
>>>>>>>>> * Jay Zarfoss, Netflix
>>>>>>>>> 
>>>>>>>>> = Sponsors =
>>>>>>>>> 
>>>>>>>>> == Champion ==
>>>>>>>>> 
>>>>>>>>> * Patrick Hunt
>>>>>>>>> 
>>>>>>>>> == Nominated Mentors ==
>>>>>>>>> 
>>>>>>>>> * Patrick Hunt
>>>>>>>>> * Enis Söztutar
>>>>>>>>> * Mahadev Konar
>>>>>>>>> 
>>>>>>>>> == Sponsoring Entity ==
>>>>>>>>> 
>>>>>>>>> * Apache Incubator PMC
>>>>>>>> 
>>>>>>>> 
>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>> 
>>>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: general-help@incubator.apache.org
>>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

["Mattmann, Chris A (388J)" <ch...@jpl.nasa.gov>]

Hi Guys,

Sorry I have to ask the question here. If the mentors consist of PMC
members on ZK
(at least Pat and Mahadev), what's the problem with creating a branch in
ZK and just
having the code be there and getting the Curator proposed committers as
committers in
ZK ville, and hopefully PMC members soon thereafter.

I have to agree with Greg here. Seems like Incubation is something that
might not be
needed. If the end result of this after N months is that Curator
"graduates" into another
set of flipping bylaw updates, and more legislation that makes these
people unofficial 
PMC members on ZK, then I'm double triple -1 with 50 piles of coal on top.
Yes, I'm 
still remembering the HCatalog/Hive thing here - I still don't get that.

Either: (a) define Curator to be its own separate project/community, with
a goal of TLP; 
or (b) nix Incubation and just make these guys part of ZK, on a branch
now. 

Cheers,
Chris

On 2/26/13 9:40 AM, "Benson Margulies" <bi...@gmail.com> wrote:

>On Tue, Feb 26, 2013 at 12:22 PM, Patrick Hunt <ph...@apache.org> wrote:
>> On Tue, Feb 26, 2013 at 8:32 AM, Benson Margulies
>><bi...@gmail.com> wrote:
>>> If you think that the right destination for curator is as part of ZK,
>>> then it would be good to see substantive participation of the ZK PMC
>>> in the incubation. The goal should be to 'graduate' by having the
>>> curator community be granted karma at ZK and the code folded in. This
>>> would require, I think, the ZK community to supply at least one
>>> mentor, and to have a plan in advance for the eventual votes based on
>>> success in the incubator.
>>
>> Hi Benson. What you're suggesting matches my current thinking.
>>
>> The three Curator mentors are as follows: Mahadev and I (champion) are
>> both PMC members on ZK, Enis Söztutar is active on HBase and
>> interested in using Curator for that project (which already uses ZK
>> heavily).
>
>OK, that's lovely.
>
>
>>
>> Patrick
>>
>>> On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra
>>><he...@gmail.com> wrote:
>>>> Ah no, I was not suggesting about Curator to become subproject of ZK.
>>>>I
>>>> just afraid that if Curator is going as incubator it will end up as
>>>>sub of
>>>> ZK as merging process.
>>>>
>>>> Like Greg has mentioned in another reply, I would prefer Curator to be
>>>> merged as a higher level ZK client. Surely project like HBase and
>>>>others
>>>> that relying on ZK would appreciate simpler client to ZK.
>>>>
>>>> Thanks,
>>>>
>>>> Henry
>>>>
>>>>
>>>> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <ph...@apache.org>
>>>>wrote:
>>>>
>>>>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra
>>>>><he...@gmail.com>
>>>>> wrote:
>>>>> > So isnt this similar to HCatalog which relying on Hive metadata
>>>>>service
>>>>> > that ends up as sub project of Apache Hive?
>>>>> >
>>>>>
>>>>> I was against having Curator as a sub when it came up on the original
>>>>> discussion thread, I still am.
>>>>>
>>>>> Patrick
>>>>>
>>>>> >
>>>>> >
>>>>> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com>
>>>>>wrote:
>>>>> >
>>>>> >> My concern is that we're looking at two "new" committers, rather
>>>>>than
>>>>> >> a Curator community. Following normal Incubator work, Curator
>>>>>would
>>>>> >> build a community for itself. But then we'd have a community
>>>>> >> *distinct* from that of Zookeeper. And it really looks like this
>>>>> >> should be part of Zookeeper itself -- a more capable and
>>>>>easier-to-use
>>>>> >> client.
>>>>> >>
>>>>> >> So I question the incubation of this. Why do we want to build a
>>>>> >> new/separate project? Why isn't this just part of Zookeeper right
>>>>>from
>>>>> >> the start?
>>>>> >>
>>>>> >> I would suggest that this work is placed on a branch within
>>>>>Zookeeper.
>>>>> >> That Jordan and Jay become committers on that branch (not
>>>>>necessarily
>>>>> >> Zookeeper trunk). Over time, the branch can be folded into trunk,
>>>>> >> along with all the various tests, doc, and other artifacts that I
>>>>>see
>>>>> >> in the GitHub repository. And hopefully that Jordan and Jay become
>>>>> >> regular committers (and PMC members!) of the Zookeeper project
>>>>>itself.
>>>>> >>
>>>>> >> The current Zookeeper client can remain for backwards compat, and
>>>>>the
>>>>> >> Curator work can become the next-gen client.
>>>>> >>
>>>>> >> Honestly, in my opnion, incubating this project seems like it
>>>>>would
>>>>> >> create a distinct community, and really doesn't seem like it would
>>>>> >> serve the Zookeeper community.
>>>>> >>
>>>>> >> All that said, I am not familiar with the Zookeeper or Curator
>>>>> >> communities. But from this read, I don't think Incubation is the
>>>>>right
>>>>> >> approach. I would rather push for a more direct incorporation of
>>>>> >> Curator directly into Zookeeper. (use the short-form IP
>>>>>clearance) ...
>>>>> >> so, unless somebody can help me understand the communities and
>>>>> >> situation better, I'm -1 (binding) on this incubation. I'd rather
>>>>>see
>>>>> >> combined, rather than distinct, communities from the start.
>>>>> >>
>>>>> >> Thanks,
>>>>> >> -g
>>>>> >>
>>>>> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>>>>> >> <jo...@jordanzimmerman.com> wrote:
>>>>> >> > Hello,
>>>>> >> >
>>>>> >> > I would like to propose that Curator to be an Apache Incubator
>>>>> project.
>>>>> >> >
>>>>> >> > The proposal can be found here:
>>>>> >> http://wiki.apache.org/incubator/CuratorProposal
>>>>> >> >
>>>>> >> > I have included the contents of the proposal below.
>>>>> >> >
>>>>> >> > Sincerely,
>>>>> >> >
>>>>> >> > Jordan Zimmerman
>>>>> >> >
>>>>> >> > ===================
>>>>> >> >
>>>>> >> > = Curator - ZooKeeper client wrapper and rich ZooKeeper
>>>>>framework =
>>>>> >> >
>>>>> >> > == Abstract ==
>>>>> >> >
>>>>> >> > Curator is a set of Java libraries that make using Apache
>>>>>ZooKeeper
>>>>> much
>>>>> >> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>the
>>>>> client
>>>>> >> is non-trivial and error prone.
>>>>> >> >
>>>>> >> > == Proposal ==
>>>>> >> >
>>>>> >> > Curator is a set of Java libraries that make using Apache
>>>>>ZooKeeper
>>>>> much
>>>>> >> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>the
>>>>> client
>>>>> >> is non-trivial and error prone. It consists of three components
>>>>>that
>>>>> build
>>>>> >> on each other. Curator Client is a replacement for the bundled
>>>>>ZooKeeper
>>>>> >> class that takes care of some low-level housekeeping and provides
>>>>>some
>>>>> >> useful utilities. Curator Framework is a high-level API that
>>>>>greatly
>>>>> >> simplifies using ZooKeeper. It adds many features that build on
>>>>> ZooKeeper
>>>>> >> and handles the complexity of managing connections to the
>>>>>ZooKeeper
>>>>> cluster
>>>>> >> and retrying operations. Curator Recipes consists of
>>>>>implementations of
>>>>> >> some of the common ZooKeeper ³recipes². Additionally, Curator
>>>>>Test is
>>>>> >> included which includes utilities to help with unit testing
>>>>> ZooKeeper-based
>>>>> >> applications.
>>>>> >> >
>>>>> >> > == Background ==
>>>>> >> >
>>>>> >> > Curator was initially developed by Netflix to make writing
>>>>> >> ZooKeeper-based applications easier and more reliable. Curator was
>>>>> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed
>>>>>project in
>>>>> >> July 2011. During this time Curator has been formally released
>>>>>many
>>>>> times
>>>>> >> and has gained widespread adoption.
>>>>> >> >
>>>>> >> > == Rationale ==
>>>>> >> >
>>>>> >> > New users of ZooKeeper are surprised to learn that a significant
>>>>> amount
>>>>> >> of connection management must be done manually. For example, when
>>>>>the
>>>>> >> ZooKeeper client connects to the ensemble it must negotiate a new
>>>>> session,
>>>>> >> etc. This takes some time. If you use a ZooKeeper client API
>>>>>before the
>>>>> >> connection process has completed, ZooKeeper will throw an
>>>>>exception.
>>>>> These
>>>>> >> types of exceptions are referred to as ³recoverable² errors.
>>>>> >> > Curator automatically handles connection management, greatly
>>>>> simplifying
>>>>> >> client code. Instead of directly using the ZooKeeper APIs you use
>>>>> Curator
>>>>> >> APIs that internally check for connection completion and wrap each
>>>>> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism to
>>>>>handle
>>>>> >> recoverable errors and automatically retry operations. The method
>>>>>of
>>>>> retry
>>>>> >> is customizable. Curator comes bundled with several
>>>>>implementations
>>>>> >> (ExponentialBackoffRetry, etc.) or custom implementations can be
>>>>> written.
>>>>> >> >
>>>>> >> > The ZooKeeper documentation describes many possible uses for
>>>>>ZooKeeper
>>>>> >> calling each a ³recipe². While the distribution comes bundled
>>>>>with a few
>>>>> >> implementations of these recipes, most ZooKeeper users will need
>>>>>to
>>>>> >> manually implement one or more of the recipes. Implementing a
>>>>>ZooKeeper
>>>>> >> recipe is not trivial. Besides the connection handling issues,
>>>>>there are
>>>>> >> numerous edge cases that are not well documented that must be
>>>>> considered.
>>>>> >> For example, many recipes require that an ephemeral-sequential
>>>>>node be
>>>>> >> created. New users of ZooKeeper will not know that there is an
>>>>>edge
>>>>> case in
>>>>> >> ephemeral-sequential node creation that requires you to put a
>>>>>special
>>>>> >> ³marker² in the node¹s name so that you can search for the
>>>>>created node
>>>>> if
>>>>> >> an I/O failure occurs. This is but one of many edge cases that
>>>>>are not
>>>>> well
>>>>> >> documented but are handled by Curator.
>>>>> >> >
>>>>> >> > = Current Status =
>>>>> >> >
>>>>> >> > == Meritocracy ==
>>>>> >> >
>>>>> >> > Curator was initially developed by Jordan Zimmerman in 2011 at
>>>>> Netflix.
>>>>> >> Developers external to Netflix provided feedback, suggested
>>>>>features and
>>>>> >> fixes and implemented extensions of Curator. Netflix's
>>>>>engineering team
>>>>> has
>>>>> >> since maintained the project and has been dedicated towards its
>>>>> >> improvement. Contributors to Curator include developers from
>>>>>multiple
>>>>> >> organizations around the world. Curator will be a meritocracy as
>>>>>it
>>>>> enters
>>>>> >> the Incubator and beyond.
>>>>> >> >
>>>>> >> > == Community ==
>>>>> >> >
>>>>> >> > Curator is currently used by a number of organizations all over
>>>>>the
>>>>> >> world. Curator has an active and growing user and developer
>>>>>community
>>>>> with
>>>>> >> active participation in the [[
>>>>> http://groups.google.com/group/curator-users]]
>>>>> >> mailing list and at its !Github home: [[
>>>>> >> https://github.com/Netflix/curator]].
>>>>> >> >
>>>>> >> > Since open sourcing the project, there have been fifteen
>>>>>individuals
>>>>> >> from various organizations who have contributed code.
>>>>> >> >
>>>>> >> > == Core Developers ==
>>>>> >> >
>>>>> >> > The core developers for Curator are:
>>>>> >> >  * Jordan Zimmerman
>>>>> >> >  * Jay Zarfoss
>>>>> >> >
>>>>> >> > Jordan has contributed towards Apache ZooKeeper and both Jordan
>>>>>and
>>>>> Jay
>>>>> >> are familiar with Apache principles and philosophy for community
>>>>>driven
>>>>> >> software development.
>>>>> >> >
>>>>> >> > == Alignment ==
>>>>> >> >
>>>>> >> > Curator is a natural complement for Apache ZooKeeper. Java
>>>>>users of
>>>>> >> ZooKeeper will naturally want to use Curator. When Curator
>>>>>graduates
>>>>> from
>>>>> >> Incubator it may be useful to distribute Curator artifacts as
>>>>>part of
>>>>> >> ZooKeeper releases as the preferred/recommended client side
>>>>>library.
>>>>> >> Further, at graduation a determination can be made as to whether
>>>>>Curator
>>>>> >> should become a Top Level Project or be merged into ZooKeeper
>>>>>itself.
>>>>> >> >
>>>>> >> > = Known Risks =
>>>>> >> >
>>>>> >> > == Orphaned Products ==
>>>>> >> >
>>>>> >> > Curator is already deployed in production at multiple companies
>>>>>and
>>>>> they
>>>>> >> are actively participating in creating new features. Curator is
>>>>>getting
>>>>> >> traction with developers and thus the risks of it being orphaned
>>>>>are
>>>>> >> minimal.
>>>>> >> >
>>>>> >> > == Inexperience with Open Source ==
>>>>> >> >
>>>>> >> > All code developed for Curator has been open sourced by Netflix
>>>>>under
>>>>> >> Apache 2.0 license.  All committers to Curator are intimately
>>>>>familiar
>>>>> with
>>>>> >> the Apache model for open-source development and are experienced
>>>>>with
>>>>> >> working with new contributors.
>>>>> >> >
>>>>> >> > == Homogeneous Developers ==
>>>>> >> >
>>>>> >> > The initial committers are from a single organization. However,
>>>>>we
>>>>> >> expect that once approved for incubation, the project will
>>>>>attract new
>>>>> >> contributors from diverse organizations and will thus grow
>>>>>organically.
>>>>> The
>>>>> >> submission of patches from developers from several different
>>>>> organizations
>>>>> >> is a strong indication that Curator will be widely adopted.
>>>>> >> >
>>>>> >> > == Reliance on Salaried Developers ==
>>>>> >> >
>>>>> >> > It is expected that Curator will be developed on salaried and
>>>>> volunteer
>>>>> >> time, although all of the initial developers will work on it
>>>>>mainly on
>>>>> >> salaried time.
>>>>> >> >
>>>>> >> > == Relationships with Other Apache Products ==
>>>>> >> >
>>>>> >> > Curator depends upon other Apache Projects: Apache ZooKeeper,
>>>>>Apache
>>>>> >> Log4J, and multiple Apache Commons components. Its build depends
>>>>>upon
>>>>> >> Apache Maven. Notably, there is interest from other Apache
>>>>>Projects
>>>>> such as
>>>>> >> HBase in adopting Curator as the client library for ZooKeeper.
>>>>>Apache
>>>>> James
>>>>> >> Mailbox has already incorporated Curator.
>>>>> >> >
>>>>> >> > == An Excessive Fascination with the Apache Brand ==
>>>>> >> >
>>>>> >> > We would like Curator to become an Apache project to further
>>>>>foster a
>>>>> >> healthy community of contributors and consumers around the
>>>>>project.
>>>>>  Since
>>>>> >> Curator directly interacts with Apache ZooKeeper and solves an
>>>>>important
>>>>> >> problem of many ZooKeeper users, residing in the Apache Software
>>>>> Foundation
>>>>> >> will increase interaction with the larger community.
>>>>> >> >
>>>>> >> > = Documentation =
>>>>> >> >
>>>>> >> >  * Curator wiki at GitHub:
>>>>>https://github.com/Netflix/curator/wiki
>>>>> >> >  * Curator issues at GitHub:
>>>>> https://github.com/Netflix/curator/issues
>>>>> >> >  * Curator javadoc at GitHub:
>>>>>http://netflix.github.com/curator/doc/
>>>>> >> >
>>>>> >> > = Initial Source =
>>>>> >> >
>>>>> >> >  * git://github.com/Netflix/curator.git
>>>>> >> >
>>>>> >> > == Source and Intellectual Property Submission Plan ==
>>>>> >> >
>>>>> >> >  * The initial source is already licensed under the Apache
>>>>>License,
>>>>> >> Version 2.0.
>>>>>https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>>>> >> >
>>>>> >> > == External Dependencies ==
>>>>> >> >
>>>>> >> > The required external dependencies are all Apache License or
>>>>> compatible
>>>>> >> licenses. Following components with non-Apache licenses are
>>>>>enumerated:
>>>>> >> >
>>>>> >> >  * org.slf4j: MIT-like License
>>>>> >> >  * org.mockito: MIT-like License
>>>>> >> >
>>>>> >> > == Cryptography ==
>>>>> >> >
>>>>> >> > Curator contains no known cryptography.
>>>>> >> >
>>>>> >> > = Required Resources =
>>>>> >> >
>>>>> >> > == Mailing lists ==
>>>>> >> >
>>>>> >> >  * curator-private (with moderated subscriptions)
>>>>> >> >  * curator-dev
>>>>> >> >  * curator-commits
>>>>> >> >  * curator-user
>>>>> >> >
>>>>> >> > == Github Repositories ==
>>>>> >> >
>>>>> >> > http://github.com/apache/curator
>>>>>git://git.apache.org/curator.git
>>>>> >> >
>>>>> >> > == Issue Tracking ==
>>>>> >> >
>>>>> >> > JIRA Curator (CURATOR)
>>>>> >> >
>>>>> >> > == Other Resources ==
>>>>> >> >
>>>>> >> > The existing code already has unit and integration tests so we
>>>>>would
>>>>> >> like a Jenkins instance to run them whenever a new patch is
>>>>>submitted.
>>>>> This
>>>>> >> can be added after project creation.
>>>>> >> >
>>>>> >> > = Initial Committers =
>>>>> >> >
>>>>> >> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>>>>> >> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>>>>> >> >
>>>>> >> > = Affiliations =
>>>>> >> >
>>>>> >> >  * Jordan Zimmerman, Netflix
>>>>> >> >  * Jay Zarfoss, Netflix
>>>>> >> >
>>>>> >> > = Sponsors =
>>>>> >> >
>>>>> >> > == Champion ==
>>>>> >> >
>>>>> >> >  * Patrick Hunt
>>>>> >> >
>>>>> >> > == Nominated Mentors ==
>>>>> >> >
>>>>> >> >  * Patrick Hunt
>>>>> >> >  * Enis Söztutar
>>>>> >> >  * Mahadev Konar
>>>>> >> >
>>>>> >> > == Sponsoring Entity ==
>>>>> >> >
>>>>> >> >  * Apache Incubator PMC
>>>>> >>
>>>>> >> 
>>>>>---------------------------------------------------------------------
>>>>> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>> >> For additional commands, e-mail: general-help@incubator.apache.org
>>>>> >>
>>>>> >>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>
>>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>For additional commands, e-mail: general-help@incubator.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Benson Margulies <bi...@gmail.com>]

On Tue, Feb 26, 2013 at 12:22 PM, Patrick Hunt <ph...@apache.org> wrote:
> On Tue, Feb 26, 2013 at 8:32 AM, Benson Margulies <bi...@gmail.com> wrote:
>> If you think that the right destination for curator is as part of ZK,
>> then it would be good to see substantive participation of the ZK PMC
>> in the incubation. The goal should be to 'graduate' by having the
>> curator community be granted karma at ZK and the code folded in. This
>> would require, I think, the ZK community to supply at least one
>> mentor, and to have a plan in advance for the eventual votes based on
>> success in the incubator.
>
> Hi Benson. What you're suggesting matches my current thinking.
>
> The three Curator mentors are as follows: Mahadev and I (champion) are
> both PMC members on ZK, Enis Söztutar is active on HBase and
> interested in using Curator for that project (which already uses ZK
> heavily).

OK, that's lovely.


>
> Patrick
>
>> On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra <he...@gmail.com> wrote:
>>> Ah no, I was not suggesting about Curator to become subproject of ZK. I
>>> just afraid that if Curator is going as incubator it will end up as sub of
>>> ZK as merging process.
>>>
>>> Like Greg has mentioned in another reply, I would prefer Curator to be
>>> merged as a higher level ZK client. Surely project like HBase and others
>>> that relying on ZK would appreciate simpler client to ZK.
>>>
>>> Thanks,
>>>
>>> Henry
>>>
>>>
>>> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <ph...@apache.org> wrote:
>>>
>>>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra <he...@gmail.com>
>>>> wrote:
>>>> > So isnt this similar to HCatalog which relying on Hive metadata service
>>>> > that ends up as sub project of Apache Hive?
>>>> >
>>>>
>>>> I was against having Curator as a sub when it came up on the original
>>>> discussion thread, I still am.
>>>>
>>>> Patrick
>>>>
>>>> >
>>>> >
>>>> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com> wrote:
>>>> >
>>>> >> My concern is that we're looking at two "new" committers, rather than
>>>> >> a Curator community. Following normal Incubator work, Curator would
>>>> >> build a community for itself. But then we'd have a community
>>>> >> *distinct* from that of Zookeeper. And it really looks like this
>>>> >> should be part of Zookeeper itself -- a more capable and easier-to-use
>>>> >> client.
>>>> >>
>>>> >> So I question the incubation of this. Why do we want to build a
>>>> >> new/separate project? Why isn't this just part of Zookeeper right from
>>>> >> the start?
>>>> >>
>>>> >> I would suggest that this work is placed on a branch within Zookeeper.
>>>> >> That Jordan and Jay become committers on that branch (not necessarily
>>>> >> Zookeeper trunk). Over time, the branch can be folded into trunk,
>>>> >> along with all the various tests, doc, and other artifacts that I see
>>>> >> in the GitHub repository. And hopefully that Jordan and Jay become
>>>> >> regular committers (and PMC members!) of the Zookeeper project itself.
>>>> >>
>>>> >> The current Zookeeper client can remain for backwards compat, and the
>>>> >> Curator work can become the next-gen client.
>>>> >>
>>>> >> Honestly, in my opnion, incubating this project seems like it would
>>>> >> create a distinct community, and really doesn't seem like it would
>>>> >> serve the Zookeeper community.
>>>> >>
>>>> >> All that said, I am not familiar with the Zookeeper or Curator
>>>> >> communities. But from this read, I don't think Incubation is the right
>>>> >> approach. I would rather push for a more direct incorporation of
>>>> >> Curator directly into Zookeeper. (use the short-form IP clearance) ...
>>>> >> so, unless somebody can help me understand the communities and
>>>> >> situation better, I'm -1 (binding) on this incubation. I'd rather see
>>>> >> combined, rather than distinct, communities from the start.
>>>> >>
>>>> >> Thanks,
>>>> >> -g
>>>> >>
>>>> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>>>> >> <jo...@jordanzimmerman.com> wrote:
>>>> >> > Hello,
>>>> >> >
>>>> >> > I would like to propose that Curator to be an Apache Incubator
>>>> project.
>>>> >> >
>>>> >> > The proposal can be found here:
>>>> >> http://wiki.apache.org/incubator/CuratorProposal
>>>> >> >
>>>> >> > I have included the contents of the proposal below.
>>>> >> >
>>>> >> > Sincerely,
>>>> >> >
>>>> >> > Jordan Zimmerman
>>>> >> >
>>>> >> > ===================
>>>> >> >
>>>> >> > = Curator - ZooKeeper client wrapper and rich ZooKeeper framework =
>>>> >> >
>>>> >> > == Abstract ==
>>>> >> >
>>>> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
>>>> much
>>>> >> easier. While ZooKeeper comes bundled with a Java client, using the
>>>> client
>>>> >> is non-trivial and error prone.
>>>> >> >
>>>> >> > == Proposal ==
>>>> >> >
>>>> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
>>>> much
>>>> >> easier. While ZooKeeper comes bundled with a Java client, using the
>>>> client
>>>> >> is non-trivial and error prone. It consists of three components that
>>>> build
>>>> >> on each other. Curator Client is a replacement for the bundled ZooKeeper
>>>> >> class that takes care of some low-level housekeeping and provides some
>>>> >> useful utilities. Curator Framework is a high-level API that greatly
>>>> >> simplifies using ZooKeeper. It adds many features that build on
>>>> ZooKeeper
>>>> >> and handles the complexity of managing connections to the ZooKeeper
>>>> cluster
>>>> >> and retrying operations. Curator Recipes consists of implementations of
>>>> >> some of the common ZooKeeper “recipes”. Additionally, Curator Test is
>>>> >> included which includes utilities to help with unit testing
>>>> ZooKeeper-based
>>>> >> applications.
>>>> >> >
>>>> >> > == Background ==
>>>> >> >
>>>> >> > Curator was initially developed by Netflix to make writing
>>>> >> ZooKeeper-based applications easier and more reliable. Curator was
>>>> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project in
>>>> >> July 2011. During this time Curator has been formally released many
>>>> times
>>>> >> and has gained widespread adoption.
>>>> >> >
>>>> >> > == Rationale ==
>>>> >> >
>>>> >> > New users of ZooKeeper are surprised to learn that a significant
>>>> amount
>>>> >> of connection management must be done manually. For example, when the
>>>> >> ZooKeeper client connects to the ensemble it must negotiate a new
>>>> session,
>>>> >> etc. This takes some time. If you use a ZooKeeper client API before the
>>>> >> connection process has completed, ZooKeeper will throw an exception.
>>>> These
>>>> >> types of exceptions are referred to as “recoverable” errors.
>>>> >> > Curator automatically handles connection management, greatly
>>>> simplifying
>>>> >> client code. Instead of directly using the ZooKeeper APIs you use
>>>> Curator
>>>> >> APIs that internally check for connection completion and wrap each
>>>> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle
>>>> >> recoverable errors and automatically retry operations. The method of
>>>> retry
>>>> >> is customizable. Curator comes bundled with several implementations
>>>> >> (ExponentialBackoffRetry, etc.) or custom implementations can be
>>>> written.
>>>> >> >
>>>> >> > The ZooKeeper documentation describes many possible uses for ZooKeeper
>>>> >> calling each a “recipe”. While the distribution comes bundled with a few
>>>> >> implementations of these recipes, most ZooKeeper users will need to
>>>> >> manually implement one or more of the recipes. Implementing a ZooKeeper
>>>> >> recipe is not trivial. Besides the connection handling issues, there are
>>>> >> numerous edge cases that are not well documented that must be
>>>> considered.
>>>> >> For example, many recipes require that an ephemeral-sequential node be
>>>> >> created. New users of ZooKeeper will not know that there is an edge
>>>> case in
>>>> >> ephemeral-sequential node creation that requires you to put a special
>>>> >> “marker” in the node’s name so that you can search for the created node
>>>> if
>>>> >> an I/O failure occurs. This is but one of many edge cases that are not
>>>> well
>>>> >> documented but are handled by Curator.
>>>> >> >
>>>> >> > = Current Status =
>>>> >> >
>>>> >> > == Meritocracy ==
>>>> >> >
>>>> >> > Curator was initially developed by Jordan Zimmerman in 2011 at
>>>> Netflix.
>>>> >> Developers external to Netflix provided feedback, suggested features and
>>>> >> fixes and implemented extensions of Curator. Netflix's engineering team
>>>> has
>>>> >> since maintained the project and has been dedicated towards its
>>>> >> improvement. Contributors to Curator include developers from multiple
>>>> >> organizations around the world. Curator will be a meritocracy as it
>>>> enters
>>>> >> the Incubator and beyond.
>>>> >> >
>>>> >> > == Community ==
>>>> >> >
>>>> >> > Curator is currently used by a number of organizations all over the
>>>> >> world. Curator has an active and growing user and developer community
>>>> with
>>>> >> active participation in the [[
>>>> http://groups.google.com/group/curator-users]]
>>>> >> mailing list and at its !Github home: [[
>>>> >> https://github.com/Netflix/curator]].
>>>> >> >
>>>> >> > Since open sourcing the project, there have been fifteen individuals
>>>> >> from various organizations who have contributed code.
>>>> >> >
>>>> >> > == Core Developers ==
>>>> >> >
>>>> >> > The core developers for Curator are:
>>>> >> >  * Jordan Zimmerman
>>>> >> >  * Jay Zarfoss
>>>> >> >
>>>> >> > Jordan has contributed towards Apache ZooKeeper and both Jordan and
>>>> Jay
>>>> >> are familiar with Apache principles and philosophy for community driven
>>>> >> software development.
>>>> >> >
>>>> >> > == Alignment ==
>>>> >> >
>>>> >> > Curator is a natural complement for Apache ZooKeeper. Java users of
>>>> >> ZooKeeper will naturally want to use Curator. When Curator graduates
>>>> from
>>>> >> Incubator it may be useful to distribute Curator artifacts as part of
>>>> >> ZooKeeper releases as the preferred/recommended client side library.
>>>> >> Further, at graduation a determination can be made as to whether Curator
>>>> >> should become a Top Level Project or be merged into ZooKeeper itself.
>>>> >> >
>>>> >> > = Known Risks =
>>>> >> >
>>>> >> > == Orphaned Products ==
>>>> >> >
>>>> >> > Curator is already deployed in production at multiple companies and
>>>> they
>>>> >> are actively participating in creating new features. Curator is getting
>>>> >> traction with developers and thus the risks of it being orphaned are
>>>> >> minimal.
>>>> >> >
>>>> >> > == Inexperience with Open Source ==
>>>> >> >
>>>> >> > All code developed for Curator has been open sourced by Netflix under
>>>> >> Apache 2.0 license.  All committers to Curator are intimately familiar
>>>> with
>>>> >> the Apache model for open-source development and are experienced with
>>>> >> working with new contributors.
>>>> >> >
>>>> >> > == Homogeneous Developers ==
>>>> >> >
>>>> >> > The initial committers are from a single organization. However, we
>>>> >> expect that once approved for incubation, the project will attract new
>>>> >> contributors from diverse organizations and will thus grow organically.
>>>> The
>>>> >> submission of patches from developers from several different
>>>> organizations
>>>> >> is a strong indication that Curator will be widely adopted.
>>>> >> >
>>>> >> > == Reliance on Salaried Developers ==
>>>> >> >
>>>> >> > It is expected that Curator will be developed on salaried and
>>>> volunteer
>>>> >> time, although all of the initial developers will work on it mainly on
>>>> >> salaried time.
>>>> >> >
>>>> >> > == Relationships with Other Apache Products ==
>>>> >> >
>>>> >> > Curator depends upon other Apache Projects: Apache ZooKeeper, Apache
>>>> >> Log4J, and multiple Apache Commons components. Its build depends upon
>>>> >> Apache Maven. Notably, there is interest from other Apache Projects
>>>> such as
>>>> >> HBase in adopting Curator as the client library for ZooKeeper. Apache
>>>> James
>>>> >> Mailbox has already incorporated Curator.
>>>> >> >
>>>> >> > == An Excessive Fascination with the Apache Brand ==
>>>> >> >
>>>> >> > We would like Curator to become an Apache project to further foster a
>>>> >> healthy community of contributors and consumers around the project.
>>>>  Since
>>>> >> Curator directly interacts with Apache ZooKeeper and solves an important
>>>> >> problem of many ZooKeeper users, residing in the Apache Software
>>>> Foundation
>>>> >> will increase interaction with the larger community.
>>>> >> >
>>>> >> > = Documentation =
>>>> >> >
>>>> >> >  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
>>>> >> >  * Curator issues at GitHub:
>>>> https://github.com/Netflix/curator/issues
>>>> >> >  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
>>>> >> >
>>>> >> > = Initial Source =
>>>> >> >
>>>> >> >  * git://github.com/Netflix/curator.git
>>>> >> >
>>>> >> > == Source and Intellectual Property Submission Plan ==
>>>> >> >
>>>> >> >  * The initial source is already licensed under the Apache License,
>>>> >> Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>>> >> >
>>>> >> > == External Dependencies ==
>>>> >> >
>>>> >> > The required external dependencies are all Apache License or
>>>> compatible
>>>> >> licenses. Following components with non-Apache licenses are enumerated:
>>>> >> >
>>>> >> >  * org.slf4j: MIT-like License
>>>> >> >  * org.mockito: MIT-like License
>>>> >> >
>>>> >> > == Cryptography ==
>>>> >> >
>>>> >> > Curator contains no known cryptography.
>>>> >> >
>>>> >> > = Required Resources =
>>>> >> >
>>>> >> > == Mailing lists ==
>>>> >> >
>>>> >> >  * curator-private (with moderated subscriptions)
>>>> >> >  * curator-dev
>>>> >> >  * curator-commits
>>>> >> >  * curator-user
>>>> >> >
>>>> >> > == Github Repositories ==
>>>> >> >
>>>> >> > http://github.com/apache/curator git://git.apache.org/curator.git
>>>> >> >
>>>> >> > == Issue Tracking ==
>>>> >> >
>>>> >> > JIRA Curator (CURATOR)
>>>> >> >
>>>> >> > == Other Resources ==
>>>> >> >
>>>> >> > The existing code already has unit and integration tests so we would
>>>> >> like a Jenkins instance to run them whenever a new patch is submitted.
>>>> This
>>>> >> can be added after project creation.
>>>> >> >
>>>> >> > = Initial Committers =
>>>> >> >
>>>> >> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>>>> >> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>>>> >> >
>>>> >> > = Affiliations =
>>>> >> >
>>>> >> >  * Jordan Zimmerman, Netflix
>>>> >> >  * Jay Zarfoss, Netflix
>>>> >> >
>>>> >> > = Sponsors =
>>>> >> >
>>>> >> > == Champion ==
>>>> >> >
>>>> >> >  * Patrick Hunt
>>>> >> >
>>>> >> > == Nominated Mentors ==
>>>> >> >
>>>> >> >  * Patrick Hunt
>>>> >> >  * Enis Söztutar
>>>> >> >  * Mahadev Konar
>>>> >> >
>>>> >> > == Sponsoring Entity ==
>>>> >> >
>>>> >> >  * Apache Incubator PMC
>>>> >>
>>>> >> ---------------------------------------------------------------------
>>>> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>> >> For additional commands, e-mail: general-help@incubator.apache.org
>>>> >>
>>>> >>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Patrick Hunt <ph...@apache.org>]

On Tue, Feb 26, 2013 at 8:32 AM, Benson Margulies <bi...@gmail.com> wrote:
> If you think that the right destination for curator is as part of ZK,
> then it would be good to see substantive participation of the ZK PMC
> in the incubation. The goal should be to 'graduate' by having the
> curator community be granted karma at ZK and the code folded in. This
> would require, I think, the ZK community to supply at least one
> mentor, and to have a plan in advance for the eventual votes based on
> success in the incubator.

Hi Benson. What you're suggesting matches my current thinking.

The three Curator mentors are as follows: Mahadev and I (champion) are
both PMC members on ZK, Enis Söztutar is active on HBase and
interested in using Curator for that project (which already uses ZK
heavily).

Patrick

> On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra <he...@gmail.com> wrote:
>> Ah no, I was not suggesting about Curator to become subproject of ZK. I
>> just afraid that if Curator is going as incubator it will end up as sub of
>> ZK as merging process.
>>
>> Like Greg has mentioned in another reply, I would prefer Curator to be
>> merged as a higher level ZK client. Surely project like HBase and others
>> that relying on ZK would appreciate simpler client to ZK.
>>
>> Thanks,
>>
>> Henry
>>
>>
>> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <ph...@apache.org> wrote:
>>
>>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra <he...@gmail.com>
>>> wrote:
>>> > So isnt this similar to HCatalog which relying on Hive metadata service
>>> > that ends up as sub project of Apache Hive?
>>> >
>>>
>>> I was against having Curator as a sub when it came up on the original
>>> discussion thread, I still am.
>>>
>>> Patrick
>>>
>>> >
>>> >
>>> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com> wrote:
>>> >
>>> >> My concern is that we're looking at two "new" committers, rather than
>>> >> a Curator community. Following normal Incubator work, Curator would
>>> >> build a community for itself. But then we'd have a community
>>> >> *distinct* from that of Zookeeper. And it really looks like this
>>> >> should be part of Zookeeper itself -- a more capable and easier-to-use
>>> >> client.
>>> >>
>>> >> So I question the incubation of this. Why do we want to build a
>>> >> new/separate project? Why isn't this just part of Zookeeper right from
>>> >> the start?
>>> >>
>>> >> I would suggest that this work is placed on a branch within Zookeeper.
>>> >> That Jordan and Jay become committers on that branch (not necessarily
>>> >> Zookeeper trunk). Over time, the branch can be folded into trunk,
>>> >> along with all the various tests, doc, and other artifacts that I see
>>> >> in the GitHub repository. And hopefully that Jordan and Jay become
>>> >> regular committers (and PMC members!) of the Zookeeper project itself.
>>> >>
>>> >> The current Zookeeper client can remain for backwards compat, and the
>>> >> Curator work can become the next-gen client.
>>> >>
>>> >> Honestly, in my opnion, incubating this project seems like it would
>>> >> create a distinct community, and really doesn't seem like it would
>>> >> serve the Zookeeper community.
>>> >>
>>> >> All that said, I am not familiar with the Zookeeper or Curator
>>> >> communities. But from this read, I don't think Incubation is the right
>>> >> approach. I would rather push for a more direct incorporation of
>>> >> Curator directly into Zookeeper. (use the short-form IP clearance) ...
>>> >> so, unless somebody can help me understand the communities and
>>> >> situation better, I'm -1 (binding) on this incubation. I'd rather see
>>> >> combined, rather than distinct, communities from the start.
>>> >>
>>> >> Thanks,
>>> >> -g
>>> >>
>>> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>>> >> <jo...@jordanzimmerman.com> wrote:
>>> >> > Hello,
>>> >> >
>>> >> > I would like to propose that Curator to be an Apache Incubator
>>> project.
>>> >> >
>>> >> > The proposal can be found here:
>>> >> http://wiki.apache.org/incubator/CuratorProposal
>>> >> >
>>> >> > I have included the contents of the proposal below.
>>> >> >
>>> >> > Sincerely,
>>> >> >
>>> >> > Jordan Zimmerman
>>> >> >
>>> >> > ===================
>>> >> >
>>> >> > = Curator - ZooKeeper client wrapper and rich ZooKeeper framework =
>>> >> >
>>> >> > == Abstract ==
>>> >> >
>>> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
>>> much
>>> >> easier. While ZooKeeper comes bundled with a Java client, using the
>>> client
>>> >> is non-trivial and error prone.
>>> >> >
>>> >> > == Proposal ==
>>> >> >
>>> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
>>> much
>>> >> easier. While ZooKeeper comes bundled with a Java client, using the
>>> client
>>> >> is non-trivial and error prone. It consists of three components that
>>> build
>>> >> on each other. Curator Client is a replacement for the bundled ZooKeeper
>>> >> class that takes care of some low-level housekeeping and provides some
>>> >> useful utilities. Curator Framework is a high-level API that greatly
>>> >> simplifies using ZooKeeper. It adds many features that build on
>>> ZooKeeper
>>> >> and handles the complexity of managing connections to the ZooKeeper
>>> cluster
>>> >> and retrying operations. Curator Recipes consists of implementations of
>>> >> some of the common ZooKeeper “recipes”. Additionally, Curator Test is
>>> >> included which includes utilities to help with unit testing
>>> ZooKeeper-based
>>> >> applications.
>>> >> >
>>> >> > == Background ==
>>> >> >
>>> >> > Curator was initially developed by Netflix to make writing
>>> >> ZooKeeper-based applications easier and more reliable. Curator was
>>> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project in
>>> >> July 2011. During this time Curator has been formally released many
>>> times
>>> >> and has gained widespread adoption.
>>> >> >
>>> >> > == Rationale ==
>>> >> >
>>> >> > New users of ZooKeeper are surprised to learn that a significant
>>> amount
>>> >> of connection management must be done manually. For example, when the
>>> >> ZooKeeper client connects to the ensemble it must negotiate a new
>>> session,
>>> >> etc. This takes some time. If you use a ZooKeeper client API before the
>>> >> connection process has completed, ZooKeeper will throw an exception.
>>> These
>>> >> types of exceptions are referred to as “recoverable” errors.
>>> >> > Curator automatically handles connection management, greatly
>>> simplifying
>>> >> client code. Instead of directly using the ZooKeeper APIs you use
>>> Curator
>>> >> APIs that internally check for connection completion and wrap each
>>> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle
>>> >> recoverable errors and automatically retry operations. The method of
>>> retry
>>> >> is customizable. Curator comes bundled with several implementations
>>> >> (ExponentialBackoffRetry, etc.) or custom implementations can be
>>> written.
>>> >> >
>>> >> > The ZooKeeper documentation describes many possible uses for ZooKeeper
>>> >> calling each a “recipe”. While the distribution comes bundled with a few
>>> >> implementations of these recipes, most ZooKeeper users will need to
>>> >> manually implement one or more of the recipes. Implementing a ZooKeeper
>>> >> recipe is not trivial. Besides the connection handling issues, there are
>>> >> numerous edge cases that are not well documented that must be
>>> considered.
>>> >> For example, many recipes require that an ephemeral-sequential node be
>>> >> created. New users of ZooKeeper will not know that there is an edge
>>> case in
>>> >> ephemeral-sequential node creation that requires you to put a special
>>> >> “marker” in the node’s name so that you can search for the created node
>>> if
>>> >> an I/O failure occurs. This is but one of many edge cases that are not
>>> well
>>> >> documented but are handled by Curator.
>>> >> >
>>> >> > = Current Status =
>>> >> >
>>> >> > == Meritocracy ==
>>> >> >
>>> >> > Curator was initially developed by Jordan Zimmerman in 2011 at
>>> Netflix.
>>> >> Developers external to Netflix provided feedback, suggested features and
>>> >> fixes and implemented extensions of Curator. Netflix's engineering team
>>> has
>>> >> since maintained the project and has been dedicated towards its
>>> >> improvement. Contributors to Curator include developers from multiple
>>> >> organizations around the world. Curator will be a meritocracy as it
>>> enters
>>> >> the Incubator and beyond.
>>> >> >
>>> >> > == Community ==
>>> >> >
>>> >> > Curator is currently used by a number of organizations all over the
>>> >> world. Curator has an active and growing user and developer community
>>> with
>>> >> active participation in the [[
>>> http://groups.google.com/group/curator-users]]
>>> >> mailing list and at its !Github home: [[
>>> >> https://github.com/Netflix/curator]].
>>> >> >
>>> >> > Since open sourcing the project, there have been fifteen individuals
>>> >> from various organizations who have contributed code.
>>> >> >
>>> >> > == Core Developers ==
>>> >> >
>>> >> > The core developers for Curator are:
>>> >> >  * Jordan Zimmerman
>>> >> >  * Jay Zarfoss
>>> >> >
>>> >> > Jordan has contributed towards Apache ZooKeeper and both Jordan and
>>> Jay
>>> >> are familiar with Apache principles and philosophy for community driven
>>> >> software development.
>>> >> >
>>> >> > == Alignment ==
>>> >> >
>>> >> > Curator is a natural complement for Apache ZooKeeper. Java users of
>>> >> ZooKeeper will naturally want to use Curator. When Curator graduates
>>> from
>>> >> Incubator it may be useful to distribute Curator artifacts as part of
>>> >> ZooKeeper releases as the preferred/recommended client side library.
>>> >> Further, at graduation a determination can be made as to whether Curator
>>> >> should become a Top Level Project or be merged into ZooKeeper itself.
>>> >> >
>>> >> > = Known Risks =
>>> >> >
>>> >> > == Orphaned Products ==
>>> >> >
>>> >> > Curator is already deployed in production at multiple companies and
>>> they
>>> >> are actively participating in creating new features. Curator is getting
>>> >> traction with developers and thus the risks of it being orphaned are
>>> >> minimal.
>>> >> >
>>> >> > == Inexperience with Open Source ==
>>> >> >
>>> >> > All code developed for Curator has been open sourced by Netflix under
>>> >> Apache 2.0 license.  All committers to Curator are intimately familiar
>>> with
>>> >> the Apache model for open-source development and are experienced with
>>> >> working with new contributors.
>>> >> >
>>> >> > == Homogeneous Developers ==
>>> >> >
>>> >> > The initial committers are from a single organization. However, we
>>> >> expect that once approved for incubation, the project will attract new
>>> >> contributors from diverse organizations and will thus grow organically.
>>> The
>>> >> submission of patches from developers from several different
>>> organizations
>>> >> is a strong indication that Curator will be widely adopted.
>>> >> >
>>> >> > == Reliance on Salaried Developers ==
>>> >> >
>>> >> > It is expected that Curator will be developed on salaried and
>>> volunteer
>>> >> time, although all of the initial developers will work on it mainly on
>>> >> salaried time.
>>> >> >
>>> >> > == Relationships with Other Apache Products ==
>>> >> >
>>> >> > Curator depends upon other Apache Projects: Apache ZooKeeper, Apache
>>> >> Log4J, and multiple Apache Commons components. Its build depends upon
>>> >> Apache Maven. Notably, there is interest from other Apache Projects
>>> such as
>>> >> HBase in adopting Curator as the client library for ZooKeeper. Apache
>>> James
>>> >> Mailbox has already incorporated Curator.
>>> >> >
>>> >> > == An Excessive Fascination with the Apache Brand ==
>>> >> >
>>> >> > We would like Curator to become an Apache project to further foster a
>>> >> healthy community of contributors and consumers around the project.
>>>  Since
>>> >> Curator directly interacts with Apache ZooKeeper and solves an important
>>> >> problem of many ZooKeeper users, residing in the Apache Software
>>> Foundation
>>> >> will increase interaction with the larger community.
>>> >> >
>>> >> > = Documentation =
>>> >> >
>>> >> >  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
>>> >> >  * Curator issues at GitHub:
>>> https://github.com/Netflix/curator/issues
>>> >> >  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
>>> >> >
>>> >> > = Initial Source =
>>> >> >
>>> >> >  * git://github.com/Netflix/curator.git
>>> >> >
>>> >> > == Source and Intellectual Property Submission Plan ==
>>> >> >
>>> >> >  * The initial source is already licensed under the Apache License,
>>> >> Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>> >> >
>>> >> > == External Dependencies ==
>>> >> >
>>> >> > The required external dependencies are all Apache License or
>>> compatible
>>> >> licenses. Following components with non-Apache licenses are enumerated:
>>> >> >
>>> >> >  * org.slf4j: MIT-like License
>>> >> >  * org.mockito: MIT-like License
>>> >> >
>>> >> > == Cryptography ==
>>> >> >
>>> >> > Curator contains no known cryptography.
>>> >> >
>>> >> > = Required Resources =
>>> >> >
>>> >> > == Mailing lists ==
>>> >> >
>>> >> >  * curator-private (with moderated subscriptions)
>>> >> >  * curator-dev
>>> >> >  * curator-commits
>>> >> >  * curator-user
>>> >> >
>>> >> > == Github Repositories ==
>>> >> >
>>> >> > http://github.com/apache/curator git://git.apache.org/curator.git
>>> >> >
>>> >> > == Issue Tracking ==
>>> >> >
>>> >> > JIRA Curator (CURATOR)
>>> >> >
>>> >> > == Other Resources ==
>>> >> >
>>> >> > The existing code already has unit and integration tests so we would
>>> >> like a Jenkins instance to run them whenever a new patch is submitted.
>>> This
>>> >> can be added after project creation.
>>> >> >
>>> >> > = Initial Committers =
>>> >> >
>>> >> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>>> >> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>>> >> >
>>> >> > = Affiliations =
>>> >> >
>>> >> >  * Jordan Zimmerman, Netflix
>>> >> >  * Jay Zarfoss, Netflix
>>> >> >
>>> >> > = Sponsors =
>>> >> >
>>> >> > == Champion ==
>>> >> >
>>> >> >  * Patrick Hunt
>>> >> >
>>> >> > == Nominated Mentors ==
>>> >> >
>>> >> >  * Patrick Hunt
>>> >> >  * Enis Söztutar
>>> >> >  * Mahadev Konar
>>> >> >
>>> >> > == Sponsoring Entity ==
>>> >> >
>>> >> >  * Apache Incubator PMC
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> >> For additional commands, e-mail: general-help@incubator.apache.org
>>> >>
>>> >>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Benson Margulies <bi...@gmail.com>]

If you think that the right destination for curator is as part of ZK,
then it would be good to see substantive participation of the ZK PMC
in the incubation. The goal should be to 'graduate' by having the
curator community be granted karma at ZK and the code folded in. This
would require, I think, the ZK community to supply at least one
mentor, and to have a plan in advance for the eventual votes based on
success in the incubator.

On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra <he...@gmail.com> wrote:
> Ah no, I was not suggesting about Curator to become subproject of ZK. I
> just afraid that if Curator is going as incubator it will end up as sub of
> ZK as merging process.
>
> Like Greg has mentioned in another reply, I would prefer Curator to be
> merged as a higher level ZK client. Surely project like HBase and others
> that relying on ZK would appreciate simpler client to ZK.
>
> Thanks,
>
> Henry
>
>
> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <ph...@apache.org> wrote:
>
>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra <he...@gmail.com>
>> wrote:
>> > So isnt this similar to HCatalog which relying on Hive metadata service
>> > that ends up as sub project of Apache Hive?
>> >
>>
>> I was against having Curator as a sub when it came up on the original
>> discussion thread, I still am.
>>
>> Patrick
>>
>> >
>> >
>> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com> wrote:
>> >
>> >> My concern is that we're looking at two "new" committers, rather than
>> >> a Curator community. Following normal Incubator work, Curator would
>> >> build a community for itself. But then we'd have a community
>> >> *distinct* from that of Zookeeper. And it really looks like this
>> >> should be part of Zookeeper itself -- a more capable and easier-to-use
>> >> client.
>> >>
>> >> So I question the incubation of this. Why do we want to build a
>> >> new/separate project? Why isn't this just part of Zookeeper right from
>> >> the start?
>> >>
>> >> I would suggest that this work is placed on a branch within Zookeeper.
>> >> That Jordan and Jay become committers on that branch (not necessarily
>> >> Zookeeper trunk). Over time, the branch can be folded into trunk,
>> >> along with all the various tests, doc, and other artifacts that I see
>> >> in the GitHub repository. And hopefully that Jordan and Jay become
>> >> regular committers (and PMC members!) of the Zookeeper project itself.
>> >>
>> >> The current Zookeeper client can remain for backwards compat, and the
>> >> Curator work can become the next-gen client.
>> >>
>> >> Honestly, in my opnion, incubating this project seems like it would
>> >> create a distinct community, and really doesn't seem like it would
>> >> serve the Zookeeper community.
>> >>
>> >> All that said, I am not familiar with the Zookeeper or Curator
>> >> communities. But from this read, I don't think Incubation is the right
>> >> approach. I would rather push for a more direct incorporation of
>> >> Curator directly into Zookeeper. (use the short-form IP clearance) ...
>> >> so, unless somebody can help me understand the communities and
>> >> situation better, I'm -1 (binding) on this incubation. I'd rather see
>> >> combined, rather than distinct, communities from the start.
>> >>
>> >> Thanks,
>> >> -g
>> >>
>> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>> >> <jo...@jordanzimmerman.com> wrote:
>> >> > Hello,
>> >> >
>> >> > I would like to propose that Curator to be an Apache Incubator
>> project.
>> >> >
>> >> > The proposal can be found here:
>> >> http://wiki.apache.org/incubator/CuratorProposal
>> >> >
>> >> > I have included the contents of the proposal below.
>> >> >
>> >> > Sincerely,
>> >> >
>> >> > Jordan Zimmerman
>> >> >
>> >> > ===================
>> >> >
>> >> > = Curator - ZooKeeper client wrapper and rich ZooKeeper framework =
>> >> >
>> >> > == Abstract ==
>> >> >
>> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
>> much
>> >> easier. While ZooKeeper comes bundled with a Java client, using the
>> client
>> >> is non-trivial and error prone.
>> >> >
>> >> > == Proposal ==
>> >> >
>> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
>> much
>> >> easier. While ZooKeeper comes bundled with a Java client, using the
>> client
>> >> is non-trivial and error prone. It consists of three components that
>> build
>> >> on each other. Curator Client is a replacement for the bundled ZooKeeper
>> >> class that takes care of some low-level housekeeping and provides some
>> >> useful utilities. Curator Framework is a high-level API that greatly
>> >> simplifies using ZooKeeper. It adds many features that build on
>> ZooKeeper
>> >> and handles the complexity of managing connections to the ZooKeeper
>> cluster
>> >> and retrying operations. Curator Recipes consists of implementations of
>> >> some of the common ZooKeeper “recipes”. Additionally, Curator Test is
>> >> included which includes utilities to help with unit testing
>> ZooKeeper-based
>> >> applications.
>> >> >
>> >> > == Background ==
>> >> >
>> >> > Curator was initially developed by Netflix to make writing
>> >> ZooKeeper-based applications easier and more reliable. Curator was
>> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project in
>> >> July 2011. During this time Curator has been formally released many
>> times
>> >> and has gained widespread adoption.
>> >> >
>> >> > == Rationale ==
>> >> >
>> >> > New users of ZooKeeper are surprised to learn that a significant
>> amount
>> >> of connection management must be done manually. For example, when the
>> >> ZooKeeper client connects to the ensemble it must negotiate a new
>> session,
>> >> etc. This takes some time. If you use a ZooKeeper client API before the
>> >> connection process has completed, ZooKeeper will throw an exception.
>> These
>> >> types of exceptions are referred to as “recoverable” errors.
>> >> > Curator automatically handles connection management, greatly
>> simplifying
>> >> client code. Instead of directly using the ZooKeeper APIs you use
>> Curator
>> >> APIs that internally check for connection completion and wrap each
>> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle
>> >> recoverable errors and automatically retry operations. The method of
>> retry
>> >> is customizable. Curator comes bundled with several implementations
>> >> (ExponentialBackoffRetry, etc.) or custom implementations can be
>> written.
>> >> >
>> >> > The ZooKeeper documentation describes many possible uses for ZooKeeper
>> >> calling each a “recipe”. While the distribution comes bundled with a few
>> >> implementations of these recipes, most ZooKeeper users will need to
>> >> manually implement one or more of the recipes. Implementing a ZooKeeper
>> >> recipe is not trivial. Besides the connection handling issues, there are
>> >> numerous edge cases that are not well documented that must be
>> considered.
>> >> For example, many recipes require that an ephemeral-sequential node be
>> >> created. New users of ZooKeeper will not know that there is an edge
>> case in
>> >> ephemeral-sequential node creation that requires you to put a special
>> >> “marker” in the node’s name so that you can search for the created node
>> if
>> >> an I/O failure occurs. This is but one of many edge cases that are not
>> well
>> >> documented but are handled by Curator.
>> >> >
>> >> > = Current Status =
>> >> >
>> >> > == Meritocracy ==
>> >> >
>> >> > Curator was initially developed by Jordan Zimmerman in 2011 at
>> Netflix.
>> >> Developers external to Netflix provided feedback, suggested features and
>> >> fixes and implemented extensions of Curator. Netflix's engineering team
>> has
>> >> since maintained the project and has been dedicated towards its
>> >> improvement. Contributors to Curator include developers from multiple
>> >> organizations around the world. Curator will be a meritocracy as it
>> enters
>> >> the Incubator and beyond.
>> >> >
>> >> > == Community ==
>> >> >
>> >> > Curator is currently used by a number of organizations all over the
>> >> world. Curator has an active and growing user and developer community
>> with
>> >> active participation in the [[
>> http://groups.google.com/group/curator-users]]
>> >> mailing list and at its !Github home: [[
>> >> https://github.com/Netflix/curator]].
>> >> >
>> >> > Since open sourcing the project, there have been fifteen individuals
>> >> from various organizations who have contributed code.
>> >> >
>> >> > == Core Developers ==
>> >> >
>> >> > The core developers for Curator are:
>> >> >  * Jordan Zimmerman
>> >> >  * Jay Zarfoss
>> >> >
>> >> > Jordan has contributed towards Apache ZooKeeper and both Jordan and
>> Jay
>> >> are familiar with Apache principles and philosophy for community driven
>> >> software development.
>> >> >
>> >> > == Alignment ==
>> >> >
>> >> > Curator is a natural complement for Apache ZooKeeper. Java users of
>> >> ZooKeeper will naturally want to use Curator. When Curator graduates
>> from
>> >> Incubator it may be useful to distribute Curator artifacts as part of
>> >> ZooKeeper releases as the preferred/recommended client side library.
>> >> Further, at graduation a determination can be made as to whether Curator
>> >> should become a Top Level Project or be merged into ZooKeeper itself.
>> >> >
>> >> > = Known Risks =
>> >> >
>> >> > == Orphaned Products ==
>> >> >
>> >> > Curator is already deployed in production at multiple companies and
>> they
>> >> are actively participating in creating new features. Curator is getting
>> >> traction with developers and thus the risks of it being orphaned are
>> >> minimal.
>> >> >
>> >> > == Inexperience with Open Source ==
>> >> >
>> >> > All code developed for Curator has been open sourced by Netflix under
>> >> Apache 2.0 license.  All committers to Curator are intimately familiar
>> with
>> >> the Apache model for open-source development and are experienced with
>> >> working with new contributors.
>> >> >
>> >> > == Homogeneous Developers ==
>> >> >
>> >> > The initial committers are from a single organization. However, we
>> >> expect that once approved for incubation, the project will attract new
>> >> contributors from diverse organizations and will thus grow organically.
>> The
>> >> submission of patches from developers from several different
>> organizations
>> >> is a strong indication that Curator will be widely adopted.
>> >> >
>> >> > == Reliance on Salaried Developers ==
>> >> >
>> >> > It is expected that Curator will be developed on salaried and
>> volunteer
>> >> time, although all of the initial developers will work on it mainly on
>> >> salaried time.
>> >> >
>> >> > == Relationships with Other Apache Products ==
>> >> >
>> >> > Curator depends upon other Apache Projects: Apache ZooKeeper, Apache
>> >> Log4J, and multiple Apache Commons components. Its build depends upon
>> >> Apache Maven. Notably, there is interest from other Apache Projects
>> such as
>> >> HBase in adopting Curator as the client library for ZooKeeper. Apache
>> James
>> >> Mailbox has already incorporated Curator.
>> >> >
>> >> > == An Excessive Fascination with the Apache Brand ==
>> >> >
>> >> > We would like Curator to become an Apache project to further foster a
>> >> healthy community of contributors and consumers around the project.
>>  Since
>> >> Curator directly interacts with Apache ZooKeeper and solves an important
>> >> problem of many ZooKeeper users, residing in the Apache Software
>> Foundation
>> >> will increase interaction with the larger community.
>> >> >
>> >> > = Documentation =
>> >> >
>> >> >  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
>> >> >  * Curator issues at GitHub:
>> https://github.com/Netflix/curator/issues
>> >> >  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
>> >> >
>> >> > = Initial Source =
>> >> >
>> >> >  * git://github.com/Netflix/curator.git
>> >> >
>> >> > == Source and Intellectual Property Submission Plan ==
>> >> >
>> >> >  * The initial source is already licensed under the Apache License,
>> >> Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
>> >> >
>> >> > == External Dependencies ==
>> >> >
>> >> > The required external dependencies are all Apache License or
>> compatible
>> >> licenses. Following components with non-Apache licenses are enumerated:
>> >> >
>> >> >  * org.slf4j: MIT-like License
>> >> >  * org.mockito: MIT-like License
>> >> >
>> >> > == Cryptography ==
>> >> >
>> >> > Curator contains no known cryptography.
>> >> >
>> >> > = Required Resources =
>> >> >
>> >> > == Mailing lists ==
>> >> >
>> >> >  * curator-private (with moderated subscriptions)
>> >> >  * curator-dev
>> >> >  * curator-commits
>> >> >  * curator-user
>> >> >
>> >> > == Github Repositories ==
>> >> >
>> >> > http://github.com/apache/curator git://git.apache.org/curator.git
>> >> >
>> >> > == Issue Tracking ==
>> >> >
>> >> > JIRA Curator (CURATOR)
>> >> >
>> >> > == Other Resources ==
>> >> >
>> >> > The existing code already has unit and integration tests so we would
>> >> like a Jenkins instance to run them whenever a new patch is submitted.
>> This
>> >> can be added after project creation.
>> >> >
>> >> > = Initial Committers =
>> >> >
>> >> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>> >> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>> >> >
>> >> > = Affiliations =
>> >> >
>> >> >  * Jordan Zimmerman, Netflix
>> >> >  * Jay Zarfoss, Netflix
>> >> >
>> >> > = Sponsors =
>> >> >
>> >> > == Champion ==
>> >> >
>> >> >  * Patrick Hunt
>> >> >
>> >> > == Nominated Mentors ==
>> >> >
>> >> >  * Patrick Hunt
>> >> >  * Enis Söztutar
>> >> >  * Mahadev Konar
>> >> >
>> >> > == Sponsoring Entity ==
>> >> >
>> >> >  * Apache Incubator PMC
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> >> For additional commands, e-mail: general-help@incubator.apache.org
>> >>
>> >>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Henry Saputra <he...@gmail.com>]

Ah no, I was not suggesting about Curator to become subproject of ZK. I
just afraid that if Curator is going as incubator it will end up as sub of
ZK as merging process.

Like Greg has mentioned in another reply, I would prefer Curator to be
merged as a higher level ZK client. Surely project like HBase and others
that relying on ZK would appreciate simpler client to ZK.

Thanks,

Henry


On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <ph...@apache.org> wrote:

> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra <he...@gmail.com>
> wrote:
> > So isnt this similar to HCatalog which relying on Hive metadata service
> > that ends up as sub project of Apache Hive?
> >
>
> I was against having Curator as a sub when it came up on the original
> discussion thread, I still am.
>
> Patrick
>
> >
> >
> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com> wrote:
> >
> >> My concern is that we're looking at two "new" committers, rather than
> >> a Curator community. Following normal Incubator work, Curator would
> >> build a community for itself. But then we'd have a community
> >> *distinct* from that of Zookeeper. And it really looks like this
> >> should be part of Zookeeper itself -- a more capable and easier-to-use
> >> client.
> >>
> >> So I question the incubation of this. Why do we want to build a
> >> new/separate project? Why isn't this just part of Zookeeper right from
> >> the start?
> >>
> >> I would suggest that this work is placed on a branch within Zookeeper.
> >> That Jordan and Jay become committers on that branch (not necessarily
> >> Zookeeper trunk). Over time, the branch can be folded into trunk,
> >> along with all the various tests, doc, and other artifacts that I see
> >> in the GitHub repository. And hopefully that Jordan and Jay become
> >> regular committers (and PMC members!) of the Zookeeper project itself.
> >>
> >> The current Zookeeper client can remain for backwards compat, and the
> >> Curator work can become the next-gen client.
> >>
> >> Honestly, in my opnion, incubating this project seems like it would
> >> create a distinct community, and really doesn't seem like it would
> >> serve the Zookeeper community.
> >>
> >> All that said, I am not familiar with the Zookeeper or Curator
> >> communities. But from this read, I don't think Incubation is the right
> >> approach. I would rather push for a more direct incorporation of
> >> Curator directly into Zookeeper. (use the short-form IP clearance) ...
> >> so, unless somebody can help me understand the communities and
> >> situation better, I'm -1 (binding) on this incubation. I'd rather see
> >> combined, rather than distinct, communities from the start.
> >>
> >> Thanks,
> >> -g
> >>
> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
> >> <jo...@jordanzimmerman.com> wrote:
> >> > Hello,
> >> >
> >> > I would like to propose that Curator to be an Apache Incubator
> project.
> >> >
> >> > The proposal can be found here:
> >> http://wiki.apache.org/incubator/CuratorProposal
> >> >
> >> > I have included the contents of the proposal below.
> >> >
> >> > Sincerely,
> >> >
> >> > Jordan Zimmerman
> >> >
> >> > ===================
> >> >
> >> > = Curator - ZooKeeper client wrapper and rich ZooKeeper framework =
> >> >
> >> > == Abstract ==
> >> >
> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
> much
> >> easier. While ZooKeeper comes bundled with a Java client, using the
> client
> >> is non-trivial and error prone.
> >> >
> >> > == Proposal ==
> >> >
> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
> much
> >> easier. While ZooKeeper comes bundled with a Java client, using the
> client
> >> is non-trivial and error prone. It consists of three components that
> build
> >> on each other. Curator Client is a replacement for the bundled ZooKeeper
> >> class that takes care of some low-level housekeeping and provides some
> >> useful utilities. Curator Framework is a high-level API that greatly
> >> simplifies using ZooKeeper. It adds many features that build on
> ZooKeeper
> >> and handles the complexity of managing connections to the ZooKeeper
> cluster
> >> and retrying operations. Curator Recipes consists of implementations of
> >> some of the common ZooKeeper “recipes”. Additionally, Curator Test is
> >> included which includes utilities to help with unit testing
> ZooKeeper-based
> >> applications.
> >> >
> >> > == Background ==
> >> >
> >> > Curator was initially developed by Netflix to make writing
> >> ZooKeeper-based applications easier and more reliable. Curator was
> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project in
> >> July 2011. During this time Curator has been formally released many
> times
> >> and has gained widespread adoption.
> >> >
> >> > == Rationale ==
> >> >
> >> > New users of ZooKeeper are surprised to learn that a significant
> amount
> >> of connection management must be done manually. For example, when the
> >> ZooKeeper client connects to the ensemble it must negotiate a new
> session,
> >> etc. This takes some time. If you use a ZooKeeper client API before the
> >> connection process has completed, ZooKeeper will throw an exception.
> These
> >> types of exceptions are referred to as “recoverable” errors.
> >> > Curator automatically handles connection management, greatly
> simplifying
> >> client code. Instead of directly using the ZooKeeper APIs you use
> Curator
> >> APIs that internally check for connection completion and wrap each
> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle
> >> recoverable errors and automatically retry operations. The method of
> retry
> >> is customizable. Curator comes bundled with several implementations
> >> (ExponentialBackoffRetry, etc.) or custom implementations can be
> written.
> >> >
> >> > The ZooKeeper documentation describes many possible uses for ZooKeeper
> >> calling each a “recipe”. While the distribution comes bundled with a few
> >> implementations of these recipes, most ZooKeeper users will need to
> >> manually implement one or more of the recipes. Implementing a ZooKeeper
> >> recipe is not trivial. Besides the connection handling issues, there are
> >> numerous edge cases that are not well documented that must be
> considered.
> >> For example, many recipes require that an ephemeral-sequential node be
> >> created. New users of ZooKeeper will not know that there is an edge
> case in
> >> ephemeral-sequential node creation that requires you to put a special
> >> “marker” in the node’s name so that you can search for the created node
> if
> >> an I/O failure occurs. This is but one of many edge cases that are not
> well
> >> documented but are handled by Curator.
> >> >
> >> > = Current Status =
> >> >
> >> > == Meritocracy ==
> >> >
> >> > Curator was initially developed by Jordan Zimmerman in 2011 at
> Netflix.
> >> Developers external to Netflix provided feedback, suggested features and
> >> fixes and implemented extensions of Curator. Netflix's engineering team
> has
> >> since maintained the project and has been dedicated towards its
> >> improvement. Contributors to Curator include developers from multiple
> >> organizations around the world. Curator will be a meritocracy as it
> enters
> >> the Incubator and beyond.
> >> >
> >> > == Community ==
> >> >
> >> > Curator is currently used by a number of organizations all over the
> >> world. Curator has an active and growing user and developer community
> with
> >> active participation in the [[
> http://groups.google.com/group/curator-users]]
> >> mailing list and at its !Github home: [[
> >> https://github.com/Netflix/curator]].
> >> >
> >> > Since open sourcing the project, there have been fifteen individuals
> >> from various organizations who have contributed code.
> >> >
> >> > == Core Developers ==
> >> >
> >> > The core developers for Curator are:
> >> >  * Jordan Zimmerman
> >> >  * Jay Zarfoss
> >> >
> >> > Jordan has contributed towards Apache ZooKeeper and both Jordan and
> Jay
> >> are familiar with Apache principles and philosophy for community driven
> >> software development.
> >> >
> >> > == Alignment ==
> >> >
> >> > Curator is a natural complement for Apache ZooKeeper. Java users of
> >> ZooKeeper will naturally want to use Curator. When Curator graduates
> from
> >> Incubator it may be useful to distribute Curator artifacts as part of
> >> ZooKeeper releases as the preferred/recommended client side library.
> >> Further, at graduation a determination can be made as to whether Curator
> >> should become a Top Level Project or be merged into ZooKeeper itself.
> >> >
> >> > = Known Risks =
> >> >
> >> > == Orphaned Products ==
> >> >
> >> > Curator is already deployed in production at multiple companies and
> they
> >> are actively participating in creating new features. Curator is getting
> >> traction with developers and thus the risks of it being orphaned are
> >> minimal.
> >> >
> >> > == Inexperience with Open Source ==
> >> >
> >> > All code developed for Curator has been open sourced by Netflix under
> >> Apache 2.0 license.  All committers to Curator are intimately familiar
> with
> >> the Apache model for open-source development and are experienced with
> >> working with new contributors.
> >> >
> >> > == Homogeneous Developers ==
> >> >
> >> > The initial committers are from a single organization. However, we
> >> expect that once approved for incubation, the project will attract new
> >> contributors from diverse organizations and will thus grow organically.
> The
> >> submission of patches from developers from several different
> organizations
> >> is a strong indication that Curator will be widely adopted.
> >> >
> >> > == Reliance on Salaried Developers ==
> >> >
> >> > It is expected that Curator will be developed on salaried and
> volunteer
> >> time, although all of the initial developers will work on it mainly on
> >> salaried time.
> >> >
> >> > == Relationships with Other Apache Products ==
> >> >
> >> > Curator depends upon other Apache Projects: Apache ZooKeeper, Apache
> >> Log4J, and multiple Apache Commons components. Its build depends upon
> >> Apache Maven. Notably, there is interest from other Apache Projects
> such as
> >> HBase in adopting Curator as the client library for ZooKeeper. Apache
> James
> >> Mailbox has already incorporated Curator.
> >> >
> >> > == An Excessive Fascination with the Apache Brand ==
> >> >
> >> > We would like Curator to become an Apache project to further foster a
> >> healthy community of contributors and consumers around the project.
>  Since
> >> Curator directly interacts with Apache ZooKeeper and solves an important
> >> problem of many ZooKeeper users, residing in the Apache Software
> Foundation
> >> will increase interaction with the larger community.
> >> >
> >> > = Documentation =
> >> >
> >> >  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
> >> >  * Curator issues at GitHub:
> https://github.com/Netflix/curator/issues
> >> >  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
> >> >
> >> > = Initial Source =
> >> >
> >> >  * git://github.com/Netflix/curator.git
> >> >
> >> > == Source and Intellectual Property Submission Plan ==
> >> >
> >> >  * The initial source is already licensed under the Apache License,
> >> Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
> >> >
> >> > == External Dependencies ==
> >> >
> >> > The required external dependencies are all Apache License or
> compatible
> >> licenses. Following components with non-Apache licenses are enumerated:
> >> >
> >> >  * org.slf4j: MIT-like License
> >> >  * org.mockito: MIT-like License
> >> >
> >> > == Cryptography ==
> >> >
> >> > Curator contains no known cryptography.
> >> >
> >> > = Required Resources =
> >> >
> >> > == Mailing lists ==
> >> >
> >> >  * curator-private (with moderated subscriptions)
> >> >  * curator-dev
> >> >  * curator-commits
> >> >  * curator-user
> >> >
> >> > == Github Repositories ==
> >> >
> >> > http://github.com/apache/curator git://git.apache.org/curator.git
> >> >
> >> > == Issue Tracking ==
> >> >
> >> > JIRA Curator (CURATOR)
> >> >
> >> > == Other Resources ==
> >> >
> >> > The existing code already has unit and integration tests so we would
> >> like a Jenkins instance to run them whenever a new patch is submitted.
> This
> >> can be added after project creation.
> >> >
> >> > = Initial Committers =
> >> >
> >> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
> >> >  * Jay Zarfoss (jzarfoss at netflix dot com)
> >> >
> >> > = Affiliations =
> >> >
> >> >  * Jordan Zimmerman, Netflix
> >> >  * Jay Zarfoss, Netflix
> >> >
> >> > = Sponsors =
> >> >
> >> > == Champion ==
> >> >
> >> >  * Patrick Hunt
> >> >
> >> > == Nominated Mentors ==
> >> >
> >> >  * Patrick Hunt
> >> >  * Enis Söztutar
> >> >  * Mahadev Konar
> >> >
> >> > == Sponsoring Entity ==
> >> >
> >> >  * Apache Incubator PMC
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> >> For additional commands, e-mail: general-help@incubator.apache.org
> >>
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: [PROPOSAL] Curator for the Apache Incubator

[Patrick Hunt <ph...@apache.org>]

On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra <he...@gmail.com> wrote:
> So isnt this similar to HCatalog which relying on Hive metadata service
> that ends up as sub project of Apache Hive?
>

I was against having Curator as a sub when it came up on the original
discussion thread, I still am.

Patrick

>
>
> On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com> wrote:
>
>> My concern is that we're looking at two "new" committers, rather than
>> a Curator community. Following normal Incubator work, Curator would
>> build a community for itself. But then we'd have a community
>> *distinct* from that of Zookeeper. And it really looks like this
>> should be part of Zookeeper itself -- a more capable and easier-to-use
>> client.
>>
>> So I question the incubation of this. Why do we want to build a
>> new/separate project? Why isn't this just part of Zookeeper right from
>> the start?
>>
>> I would suggest that this work is placed on a branch within Zookeeper.
>> That Jordan and Jay become committers on that branch (not necessarily
>> Zookeeper trunk). Over time, the branch can be folded into trunk,
>> along with all the various tests, doc, and other artifacts that I see
>> in the GitHub repository. And hopefully that Jordan and Jay become
>> regular committers (and PMC members!) of the Zookeeper project itself.
>>
>> The current Zookeeper client can remain for backwards compat, and the
>> Curator work can become the next-gen client.
>>
>> Honestly, in my opnion, incubating this project seems like it would
>> create a distinct community, and really doesn't seem like it would
>> serve the Zookeeper community.
>>
>> All that said, I am not familiar with the Zookeeper or Curator
>> communities. But from this read, I don't think Incubation is the right
>> approach. I would rather push for a more direct incorporation of
>> Curator directly into Zookeeper. (use the short-form IP clearance) ...
>> so, unless somebody can help me understand the communities and
>> situation better, I'm -1 (binding) on this incubation. I'd rather see
>> combined, rather than distinct, communities from the start.
>>
>> Thanks,
>> -g
>>
>> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>> <jo...@jordanzimmerman.com> wrote:
>> > Hello,
>> >
>> > I would like to propose that Curator to be an Apache Incubator project.
>> >
>> > The proposal can be found here:
>> http://wiki.apache.org/incubator/CuratorProposal
>> >
>> > I have included the contents of the proposal below.
>> >
>> > Sincerely,
>> >
>> > Jordan Zimmerman
>> >
>> > ===================
>> >
>> > = Curator - ZooKeeper client wrapper and rich ZooKeeper framework =
>> >
>> > == Abstract ==
>> >
>> > Curator is a set of Java libraries that make using Apache ZooKeeper much
>> easier. While ZooKeeper comes bundled with a Java client, using the client
>> is non-trivial and error prone.
>> >
>> > == Proposal ==
>> >
>> > Curator is a set of Java libraries that make using Apache ZooKeeper much
>> easier. While ZooKeeper comes bundled with a Java client, using the client
>> is non-trivial and error prone. It consists of three components that build
>> on each other. Curator Client is a replacement for the bundled ZooKeeper
>> class that takes care of some low-level housekeeping and provides some
>> useful utilities. Curator Framework is a high-level API that greatly
>> simplifies using ZooKeeper. It adds many features that build on ZooKeeper
>> and handles the complexity of managing connections to the ZooKeeper cluster
>> and retrying operations. Curator Recipes consists of implementations of
>> some of the common ZooKeeper “recipes”. Additionally, Curator Test is
>> included which includes utilities to help with unit testing ZooKeeper-based
>> applications.
>> >
>> > == Background ==
>> >
>> > Curator was initially developed by Netflix to make writing
>> ZooKeeper-based applications easier and more reliable. Curator was
>> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project in
>> July 2011. During this time Curator has been formally released many times
>> and has gained widespread adoption.
>> >
>> > == Rationale ==
>> >
>> > New users of ZooKeeper are surprised to learn that a significant amount
>> of connection management must be done manually. For example, when the
>> ZooKeeper client connects to the ensemble it must negotiate a new session,
>> etc. This takes some time. If you use a ZooKeeper client API before the
>> connection process has completed, ZooKeeper will throw an exception. These
>> types of exceptions are referred to as “recoverable” errors.
>> > Curator automatically handles connection management, greatly simplifying
>> client code. Instead of directly using the ZooKeeper APIs you use Curator
>> APIs that internally check for connection completion and wrap each
>> ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle
>> recoverable errors and automatically retry operations. The method of retry
>> is customizable. Curator comes bundled with several implementations
>> (ExponentialBackoffRetry, etc.) or custom implementations can be written.
>> >
>> > The ZooKeeper documentation describes many possible uses for ZooKeeper
>> calling each a “recipe”. While the distribution comes bundled with a few
>> implementations of these recipes, most ZooKeeper users will need to
>> manually implement one or more of the recipes. Implementing a ZooKeeper
>> recipe is not trivial. Besides the connection handling issues, there are
>> numerous edge cases that are not well documented that must be considered.
>> For example, many recipes require that an ephemeral-sequential node be
>> created. New users of ZooKeeper will not know that there is an edge case in
>> ephemeral-sequential node creation that requires you to put a special
>> “marker” in the node’s name so that you can search for the created node if
>> an I/O failure occurs. This is but one of many edge cases that are not well
>> documented but are handled by Curator.
>> >
>> > = Current Status =
>> >
>> > == Meritocracy ==
>> >
>> > Curator was initially developed by Jordan Zimmerman in 2011 at Netflix.
>> Developers external to Netflix provided feedback, suggested features and
>> fixes and implemented extensions of Curator. Netflix's engineering team has
>> since maintained the project and has been dedicated towards its
>> improvement. Contributors to Curator include developers from multiple
>> organizations around the world. Curator will be a meritocracy as it enters
>> the Incubator and beyond.
>> >
>> > == Community ==
>> >
>> > Curator is currently used by a number of organizations all over the
>> world. Curator has an active and growing user and developer community with
>> active participation in the [[http://groups.google.com/group/curator-users]]
>> mailing list and at its !Github home: [[
>> https://github.com/Netflix/curator]].
>> >
>> > Since open sourcing the project, there have been fifteen individuals
>> from various organizations who have contributed code.
>> >
>> > == Core Developers ==
>> >
>> > The core developers for Curator are:
>> >  * Jordan Zimmerman
>> >  * Jay Zarfoss
>> >
>> > Jordan has contributed towards Apache ZooKeeper and both Jordan and Jay
>> are familiar with Apache principles and philosophy for community driven
>> software development.
>> >
>> > == Alignment ==
>> >
>> > Curator is a natural complement for Apache ZooKeeper. Java users of
>> ZooKeeper will naturally want to use Curator. When Curator graduates from
>> Incubator it may be useful to distribute Curator artifacts as part of
>> ZooKeeper releases as the preferred/recommended client side library.
>> Further, at graduation a determination can be made as to whether Curator
>> should become a Top Level Project or be merged into ZooKeeper itself.
>> >
>> > = Known Risks =
>> >
>> > == Orphaned Products ==
>> >
>> > Curator is already deployed in production at multiple companies and they
>> are actively participating in creating new features. Curator is getting
>> traction with developers and thus the risks of it being orphaned are
>> minimal.
>> >
>> > == Inexperience with Open Source ==
>> >
>> > All code developed for Curator has been open sourced by Netflix under
>> Apache 2.0 license.  All committers to Curator are intimately familiar with
>> the Apache model for open-source development and are experienced with
>> working with new contributors.
>> >
>> > == Homogeneous Developers ==
>> >
>> > The initial committers are from a single organization. However, we
>> expect that once approved for incubation, the project will attract new
>> contributors from diverse organizations and will thus grow organically. The
>> submission of patches from developers from several different organizations
>> is a strong indication that Curator will be widely adopted.
>> >
>> > == Reliance on Salaried Developers ==
>> >
>> > It is expected that Curator will be developed on salaried and volunteer
>> time, although all of the initial developers will work on it mainly on
>> salaried time.
>> >
>> > == Relationships with Other Apache Products ==
>> >
>> > Curator depends upon other Apache Projects: Apache ZooKeeper, Apache
>> Log4J, and multiple Apache Commons components. Its build depends upon
>> Apache Maven. Notably, there is interest from other Apache Projects such as
>> HBase in adopting Curator as the client library for ZooKeeper. Apache James
>> Mailbox has already incorporated Curator.
>> >
>> > == An Excessive Fascination with the Apache Brand ==
>> >
>> > We would like Curator to become an Apache project to further foster a
>> healthy community of contributors and consumers around the project.  Since
>> Curator directly interacts with Apache ZooKeeper and solves an important
>> problem of many ZooKeeper users, residing in the Apache Software Foundation
>> will increase interaction with the larger community.
>> >
>> > = Documentation =
>> >
>> >  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
>> >  * Curator issues at GitHub: https://github.com/Netflix/curator/issues
>> >  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
>> >
>> > = Initial Source =
>> >
>> >  * git://github.com/Netflix/curator.git
>> >
>> > == Source and Intellectual Property Submission Plan ==
>> >
>> >  * The initial source is already licensed under the Apache License,
>> Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
>> >
>> > == External Dependencies ==
>> >
>> > The required external dependencies are all Apache License or compatible
>> licenses. Following components with non-Apache licenses are enumerated:
>> >
>> >  * org.slf4j: MIT-like License
>> >  * org.mockito: MIT-like License
>> >
>> > == Cryptography ==
>> >
>> > Curator contains no known cryptography.
>> >
>> > = Required Resources =
>> >
>> > == Mailing lists ==
>> >
>> >  * curator-private (with moderated subscriptions)
>> >  * curator-dev
>> >  * curator-commits
>> >  * curator-user
>> >
>> > == Github Repositories ==
>> >
>> > http://github.com/apache/curator git://git.apache.org/curator.git
>> >
>> > == Issue Tracking ==
>> >
>> > JIRA Curator (CURATOR)
>> >
>> > == Other Resources ==
>> >
>> > The existing code already has unit and integration tests so we would
>> like a Jenkins instance to run them whenever a new patch is submitted. This
>> can be added after project creation.
>> >
>> > = Initial Committers =
>> >
>> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>> >
>> > = Affiliations =
>> >
>> >  * Jordan Zimmerman, Netflix
>> >  * Jay Zarfoss, Netflix
>> >
>> > = Sponsors =
>> >
>> > == Champion ==
>> >
>> >  * Patrick Hunt
>> >
>> > == Nominated Mentors ==
>> >
>> >  * Patrick Hunt
>> >  * Enis Söztutar
>> >  * Mahadev Konar
>> >
>> > == Sponsoring Entity ==
>> >
>> >  * Apache Incubator PMC
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org

Re: [PROPOSAL] Curator for the Apache Incubator

[Henry Saputra <he...@gmail.com>]

So isnt this similar to HCatalog which relying on Hive metadata service
that ends up as sub project of Apache Hive?

If Apache Zookeeper would like to add Curator as the next gen client I
think it would be easier and integrated with the rest of ZK community.

Just my 2-cents

- Henry



On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com> wrote:

> My concern is that we're looking at two "new" committers, rather than
> a Curator community. Following normal Incubator work, Curator would
> build a community for itself. But then we'd have a community
> *distinct* from that of Zookeeper. And it really looks like this
> should be part of Zookeeper itself -- a more capable and easier-to-use
> client.
>
> So I question the incubation of this. Why do we want to build a
> new/separate project? Why isn't this just part of Zookeeper right from
> the start?
>
> I would suggest that this work is placed on a branch within Zookeeper.
> That Jordan and Jay become committers on that branch (not necessarily
> Zookeeper trunk). Over time, the branch can be folded into trunk,
> along with all the various tests, doc, and other artifacts that I see
> in the GitHub repository. And hopefully that Jordan and Jay become
> regular committers (and PMC members!) of the Zookeeper project itself.
>
> The current Zookeeper client can remain for backwards compat, and the
> Curator work can become the next-gen client.
>
> Honestly, in my opnion, incubating this project seems like it would
> create a distinct community, and really doesn't seem like it would
> serve the Zookeeper community.
>
> All that said, I am not familiar with the Zookeeper or Curator
> communities. But from this read, I don't think Incubation is the right
> approach. I would rather push for a more direct incorporation of
> Curator directly into Zookeeper. (use the short-form IP clearance) ...
> so, unless somebody can help me understand the communities and
> situation better, I'm -1 (binding) on this incubation. I'd rather see
> combined, rather than distinct, communities from the start.
>
> Thanks,
> -g
>
> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
> <jo...@jordanzimmerman.com> wrote:
> > Hello,
> >
> > I would like to propose that Curator to be an Apache Incubator project.
> >
> > The proposal can be found here:
> http://wiki.apache.org/incubator/CuratorProposal
> >
> > I have included the contents of the proposal below.
> >
> > Sincerely,
> >
> > Jordan Zimmerman
> >
> > ===================
> >
> > = Curator - ZooKeeper client wrapper and rich ZooKeeper framework =
> >
> > == Abstract ==
> >
> > Curator is a set of Java libraries that make using Apache ZooKeeper much
> easier. While ZooKeeper comes bundled with a Java client, using the client
> is non-trivial and error prone.
> >
> > == Proposal ==
> >
> > Curator is a set of Java libraries that make using Apache ZooKeeper much
> easier. While ZooKeeper comes bundled with a Java client, using the client
> is non-trivial and error prone. It consists of three components that build
> on each other. Curator Client is a replacement for the bundled ZooKeeper
> class that takes care of some low-level housekeeping and provides some
> useful utilities. Curator Framework is a high-level API that greatly
> simplifies using ZooKeeper. It adds many features that build on ZooKeeper
> and handles the complexity of managing connections to the ZooKeeper cluster
> and retrying operations. Curator Recipes consists of implementations of
> some of the common ZooKeeper “recipes”. Additionally, Curator Test is
> included which includes utilities to help with unit testing ZooKeeper-based
> applications.
> >
> > == Background ==
> >
> > Curator was initially developed by Netflix to make writing
> ZooKeeper-based applications easier and more reliable. Curator was
> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project in
> July 2011. During this time Curator has been formally released many times
> and has gained widespread adoption.
> >
> > == Rationale ==
> >
> > New users of ZooKeeper are surprised to learn that a significant amount
> of connection management must be done manually. For example, when the
> ZooKeeper client connects to the ensemble it must negotiate a new session,
> etc. This takes some time. If you use a ZooKeeper client API before the
> connection process has completed, ZooKeeper will throw an exception. These
> types of exceptions are referred to as “recoverable” errors.
> > Curator automatically handles connection management, greatly simplifying
> client code. Instead of directly using the ZooKeeper APIs you use Curator
> APIs that internally check for connection completion and wrap each
> ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle
> recoverable errors and automatically retry operations. The method of retry
> is customizable. Curator comes bundled with several implementations
> (ExponentialBackoffRetry, etc.) or custom implementations can be written.
> >
> > The ZooKeeper documentation describes many possible uses for ZooKeeper
> calling each a “recipe”. While the distribution comes bundled with a few
> implementations of these recipes, most ZooKeeper users will need to
> manually implement one or more of the recipes. Implementing a ZooKeeper
> recipe is not trivial. Besides the connection handling issues, there are
> numerous edge cases that are not well documented that must be considered.
> For example, many recipes require that an ephemeral-sequential node be
> created. New users of ZooKeeper will not know that there is an edge case in
> ephemeral-sequential node creation that requires you to put a special
> “marker” in the node’s name so that you can search for the created node if
> an I/O failure occurs. This is but one of many edge cases that are not well
> documented but are handled by Curator.
> >
> > = Current Status =
> >
> > == Meritocracy ==
> >
> > Curator was initially developed by Jordan Zimmerman in 2011 at Netflix.
> Developers external to Netflix provided feedback, suggested features and
> fixes and implemented extensions of Curator. Netflix's engineering team has
> since maintained the project and has been dedicated towards its
> improvement. Contributors to Curator include developers from multiple
> organizations around the world. Curator will be a meritocracy as it enters
> the Incubator and beyond.
> >
> > == Community ==
> >
> > Curator is currently used by a number of organizations all over the
> world. Curator has an active and growing user and developer community with
> active participation in the [[http://groups.google.com/group/curator-users]]
> mailing list and at its !Github home: [[
> https://github.com/Netflix/curator]].
> >
> > Since open sourcing the project, there have been fifteen individuals
> from various organizations who have contributed code.
> >
> > == Core Developers ==
> >
> > The core developers for Curator are:
> >  * Jordan Zimmerman
> >  * Jay Zarfoss
> >
> > Jordan has contributed towards Apache ZooKeeper and both Jordan and Jay
> are familiar with Apache principles and philosophy for community driven
> software development.
> >
> > == Alignment ==
> >
> > Curator is a natural complement for Apache ZooKeeper. Java users of
> ZooKeeper will naturally want to use Curator. When Curator graduates from
> Incubator it may be useful to distribute Curator artifacts as part of
> ZooKeeper releases as the preferred/recommended client side library.
> Further, at graduation a determination can be made as to whether Curator
> should become a Top Level Project or be merged into ZooKeeper itself.
> >
> > = Known Risks =
> >
> > == Orphaned Products ==
> >
> > Curator is already deployed in production at multiple companies and they
> are actively participating in creating new features. Curator is getting
> traction with developers and thus the risks of it being orphaned are
> minimal.
> >
> > == Inexperience with Open Source ==
> >
> > All code developed for Curator has been open sourced by Netflix under
> Apache 2.0 license.  All committers to Curator are intimately familiar with
> the Apache model for open-source development and are experienced with
> working with new contributors.
> >
> > == Homogeneous Developers ==
> >
> > The initial committers are from a single organization. However, we
> expect that once approved for incubation, the project will attract new
> contributors from diverse organizations and will thus grow organically. The
> submission of patches from developers from several different organizations
> is a strong indication that Curator will be widely adopted.
> >
> > == Reliance on Salaried Developers ==
> >
> > It is expected that Curator will be developed on salaried and volunteer
> time, although all of the initial developers will work on it mainly on
> salaried time.
> >
> > == Relationships with Other Apache Products ==
> >
> > Curator depends upon other Apache Projects: Apache ZooKeeper, Apache
> Log4J, and multiple Apache Commons components. Its build depends upon
> Apache Maven. Notably, there is interest from other Apache Projects such as
> HBase in adopting Curator as the client library for ZooKeeper. Apache James
> Mailbox has already incorporated Curator.
> >
> > == An Excessive Fascination with the Apache Brand ==
> >
> > We would like Curator to become an Apache project to further foster a
> healthy community of contributors and consumers around the project.  Since
> Curator directly interacts with Apache ZooKeeper and solves an important
> problem of many ZooKeeper users, residing in the Apache Software Foundation
> will increase interaction with the larger community.
> >
> > = Documentation =
> >
> >  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
> >  * Curator issues at GitHub: https://github.com/Netflix/curator/issues
> >  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
> >
> > = Initial Source =
> >
> >  * git://github.com/Netflix/curator.git
> >
> > == Source and Intellectual Property Submission Plan ==
> >
> >  * The initial source is already licensed under the Apache License,
> Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
> >
> > == External Dependencies ==
> >
> > The required external dependencies are all Apache License or compatible
> licenses. Following components with non-Apache licenses are enumerated:
> >
> >  * org.slf4j: MIT-like License
> >  * org.mockito: MIT-like License
> >
> > == Cryptography ==
> >
> > Curator contains no known cryptography.
> >
> > = Required Resources =
> >
> > == Mailing lists ==
> >
> >  * curator-private (with moderated subscriptions)
> >  * curator-dev
> >  * curator-commits
> >  * curator-user
> >
> > == Github Repositories ==
> >
> > http://github.com/apache/curator git://git.apache.org/curator.git
> >
> > == Issue Tracking ==
> >
> > JIRA Curator (CURATOR)
> >
> > == Other Resources ==
> >
> > The existing code already has unit and integration tests so we would
> like a Jenkins instance to run them whenever a new patch is submitted. This
> can be added after project creation.
> >
> > = Initial Committers =
> >
> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
> >  * Jay Zarfoss (jzarfoss at netflix dot com)
> >
> > = Affiliations =
> >
> >  * Jordan Zimmerman, Netflix
> >  * Jay Zarfoss, Netflix
> >
> > = Sponsors =
> >
> > == Champion ==
> >
> >  * Patrick Hunt
> >
> > == Nominated Mentors ==
> >
> >  * Patrick Hunt
> >  * Enis Söztutar
> >  * Mahadev Konar
> >
> > == Sponsoring Entity ==
> >
> >  * Apache Incubator PMC
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>
>

Re: [PROPOSAL] Curator for the Apache Incubator

[Patrick Hunt <ph...@apache.org>]

On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gs...@gmail.com> wrote:
> My concern is that we're looking at two "new" committers, rather than
> a Curator community. Following normal Incubator work, Curator would
> build a community for itself. But then we'd have a community
> *distinct* from that of Zookeeper. And it really looks like this
> should be part of Zookeeper itself -- a more capable and easier-to-use
> client.
>
> So I question the incubation of this. Why do we want to build a
> new/separate project? Why isn't this just part of Zookeeper right from
> the start?
>
> I would suggest that this work is placed on a branch within Zookeeper.
> That Jordan and Jay become committers on that branch (not necessarily
> Zookeeper trunk). Over time, the branch can be folded into trunk,
> along with all the various tests, doc, and other artifacts that I see
> in the GitHub repository. And hopefully that Jordan and Jay become
> regular committers (and PMC members!) of the Zookeeper project itself.
>
> The current Zookeeper client can remain for backwards compat, and the
> Curator work can become the next-gen client.
>
> Honestly, in my opnion, incubating this project seems like it would
> create a distinct community, and really doesn't seem like it would
> serve the Zookeeper community.
>
> All that said, I am not familiar with the Zookeeper or Curator
> communities. But from this read, I don't think Incubation is the right
> approach. I would rather push for a more direct incorporation of
> Curator directly into Zookeeper. (use the short-form IP clearance) ...
> so, unless somebody can help me understand the communities and
> situation better, I'm -1 (binding) on this incubation. I'd rather see
> combined, rather than distinct, communities from the start.

The Curator library is built upon the current ZooKeeper client
libraries. They are an extension that implements higher level use
cases (what we call "recipes" in ZK) whereas the ZK libraries
implement lower level primitives. An analogy might be Apache Crunch
(recently graduated to TLP) and MapReduce. Also, not everyone agrees
with Jordan's assessment that Curator is 'better" than (or a
replacement for) the existing client libraries.

The ZK community discussed incorporating the Curator code about a year
ago. At that time there wasn't interest to adopt these libraries into
ZK itself. My belief is that if Curator were to go through incubation
the ZK community would be more likely to adopt it. Similar to how
HCatalog spun off to work on the metastore and recently merged back
into Hive. If there's significant overlap in the Curator podling
community and the ZK community that will be a strong indication that
they should be merged (my belief). If not it would have the
opportunity to go TLP.

Here's my report (ZooKeeper) to the board in Dec 2011 and the response
I received. This is one of the reasons I advised Jordan to go the
incubator approach. There's also the discussion thread on ZooKeeper
dev if you want more insight.

>> A discussion is currently under way regarding the possibility of> merging the recently open sourced "Curator" source base from> Netflix. These are client implementations of ZooKeeper "recipes"> (e.g. leadership election, group membership, etc...) which simplify> the act of using ZooKeeper in client side applications.>> The authors of Curator are unwilling to join the incubator, this is> based on their past experience as well as some of the ongoing issues> they are seeing wrt projects entering the incubator. They have> expressed a preference to come in either as a subproject or as a> separate release artifact of the TLP.
>
> to which the board responded:
>
> Doug Cutting (chairman):
> +               Long-term, do you think that Curator will have an
> +               indepdendent community from Zookeeper?  If so, then it
> +               ought to enter through the Incubator.  If not then the
> +               code might still enter through the incubator for
> +               resolution of IP issues, but then transfer relatively
> +               quickly to live under the Zookeeper PMC.  Or the
> +               Zookeeper PMC could directly adopt the code, although
> +               you might not immediately add its committers to the
> +               project but rather have them first contribute patches
> +               through the normal process until their community merit
> +               is established.
>
> Sam Ruby:
> + suggested that the "short form" of the incubator's IP clearance
> process would be appropriate if Zookeeper directly adopts this code.
>
> The short form can be found here:
> http://incubator.apache.org/ip-clearance/index.html


Regards,

Patrick

>
> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
> <jo...@jordanzimmerman.com> wrote:
>> Hello,
>>
>> I would like to propose that Curator to be an Apache Incubator project.
>>
>> The proposal can be found here: http://wiki.apache.org/incubator/CuratorProposal
>>
>> I have included the contents of the proposal below.
>>
>> Sincerely,
>>
>> Jordan Zimmerman
>>
>> ===================
>>
>> = Curator - ZooKeeper client wrapper and rich ZooKeeper framework =
>>
>> == Abstract ==
>>
>> Curator is a set of Java libraries that make using Apache ZooKeeper much easier. While ZooKeeper comes bundled with a Java client, using the client is non-trivial and error prone.
>>
>> == Proposal ==
>>
>> Curator is a set of Java libraries that make using Apache ZooKeeper much easier. While ZooKeeper comes bundled with a Java client, using the client is non-trivial and error prone. It consists of three components that build on each other. Curator Client is a replacement for the bundled ZooKeeper class that takes care of some low-level housekeeping and provides some useful utilities. Curator Framework is a high-level API that greatly simplifies using ZooKeeper. It adds many features that build on ZooKeeper and handles the complexity of managing connections to the ZooKeeper cluster and retrying operations. Curator Recipes consists of implementations of some of the common ZooKeeper “recipes”. Additionally, Curator Test is included which includes utilities to help with unit testing ZooKeeper-based applications.
>>
>> == Background ==
>>
>> Curator was initially developed by Netflix to make writing ZooKeeper-based applications easier and more reliable. Curator was open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project in July 2011. During this time Curator has been formally released many times and has gained widespread adoption.
>>
>> == Rationale ==
>>
>> New users of ZooKeeper are surprised to learn that a significant amount of connection management must be done manually. For example, when the ZooKeeper client connects to the ensemble it must negotiate a new session, etc. This takes some time. If you use a ZooKeeper client API before the connection process has completed, ZooKeeper will throw an exception. These types of exceptions are referred to as “recoverable” errors.
>> Curator automatically handles connection management, greatly simplifying client code. Instead of directly using the ZooKeeper APIs you use Curator APIs that internally check for connection completion and wrap each ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle recoverable errors and automatically retry operations. The method of retry is customizable. Curator comes bundled with several implementations (ExponentialBackoffRetry, etc.) or custom implementations can be written.
>>
>> The ZooKeeper documentation describes many possible uses for ZooKeeper calling each a “recipe”. While the distribution comes bundled with a few implementations of these recipes, most ZooKeeper users will need to manually implement one or more of the recipes. Implementing a ZooKeeper recipe is not trivial. Besides the connection handling issues, there are numerous edge cases that are not well documented that must be considered. For example, many recipes require that an ephemeral-sequential node be created. New users of ZooKeeper will not know that there is an edge case in ephemeral-sequential node creation that requires you to put a special “marker” in the node’s name so that you can search for the created node if an I/O failure occurs. This is but one of many edge cases that are not well documented but are handled by Curator.
>>
>> = Current Status =
>>
>> == Meritocracy ==
>>
>> Curator was initially developed by Jordan Zimmerman in 2011 at Netflix. Developers external to Netflix provided feedback, suggested features and fixes and implemented extensions of Curator. Netflix's engineering team has since maintained the project and has been dedicated towards its improvement. Contributors to Curator include developers from multiple organizations around the world. Curator will be a meritocracy as it enters the Incubator and beyond.
>>
>> == Community ==
>>
>> Curator is currently used by a number of organizations all over the world. Curator has an active and growing user and developer community with active participation in the [[http://groups.google.com/group/curator-users]] mailing list and at its !Github home: [[https://github.com/Netflix/curator]].
>>
>> Since open sourcing the project, there have been fifteen individuals from various organizations who have contributed code.
>>
>> == Core Developers ==
>>
>> The core developers for Curator are:
>>  * Jordan Zimmerman
>>  * Jay Zarfoss
>>
>> Jordan has contributed towards Apache ZooKeeper and both Jordan and Jay are familiar with Apache principles and philosophy for community driven software development.
>>
>> == Alignment ==
>>
>> Curator is a natural complement for Apache ZooKeeper. Java users of ZooKeeper will naturally want to use Curator. When Curator graduates from Incubator it may be useful to distribute Curator artifacts as part of ZooKeeper releases as the preferred/recommended client side library. Further, at graduation a determination can be made as to whether Curator should become a Top Level Project or be merged into ZooKeeper itself.
>>
>> = Known Risks =
>>
>> == Orphaned Products ==
>>
>> Curator is already deployed in production at multiple companies and they are actively participating in creating new features. Curator is getting traction with developers and thus the risks of it being orphaned are minimal.
>>
>> == Inexperience with Open Source ==
>>
>> All code developed for Curator has been open sourced by Netflix under Apache 2.0 license.  All committers to Curator are intimately familiar with the Apache model for open-source development and are experienced with working with new contributors.
>>
>> == Homogeneous Developers ==
>>
>> The initial committers are from a single organization. However, we expect that once approved for incubation, the project will attract new contributors from diverse organizations and will thus grow organically. The submission of patches from developers from several different organizations is a strong indication that Curator will be widely adopted.
>>
>> == Reliance on Salaried Developers ==
>>
>> It is expected that Curator will be developed on salaried and volunteer time, although all of the initial developers will work on it mainly on salaried time.
>>
>> == Relationships with Other Apache Products ==
>>
>> Curator depends upon other Apache Projects: Apache ZooKeeper, Apache Log4J, and multiple Apache Commons components. Its build depends upon Apache Maven. Notably, there is interest from other Apache Projects such as HBase in adopting Curator as the client library for ZooKeeper. Apache James Mailbox has already incorporated Curator.
>>
>> == An Excessive Fascination with the Apache Brand ==
>>
>> We would like Curator to become an Apache project to further foster a healthy community of contributors and consumers around the project.  Since Curator directly interacts with Apache ZooKeeper and solves an important problem of many ZooKeeper users, residing in the Apache Software Foundation will increase interaction with the larger community.
>>
>> = Documentation =
>>
>>  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
>>  * Curator issues at GitHub: https://github.com/Netflix/curator/issues
>>  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
>>
>> = Initial Source =
>>
>>  * git://github.com/Netflix/curator.git
>>
>> == Source and Intellectual Property Submission Plan ==
>>
>>  * The initial source is already licensed under the Apache License, Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>
>> == External Dependencies ==
>>
>> The required external dependencies are all Apache License or compatible licenses. Following components with non-Apache licenses are enumerated:
>>
>>  * org.slf4j: MIT-like License
>>  * org.mockito: MIT-like License
>>
>> == Cryptography ==
>>
>> Curator contains no known cryptography.
>>
>> = Required Resources =
>>
>> == Mailing lists ==
>>
>>  * curator-private (with moderated subscriptions)
>>  * curator-dev
>>  * curator-commits
>>  * curator-user
>>
>> == Github Repositories ==
>>
>> http://github.com/apache/curator git://git.apache.org/curator.git
>>
>> == Issue Tracking ==
>>
>> JIRA Curator (CURATOR)
>>
>> == Other Resources ==
>>
>> The existing code already has unit and integration tests so we would like a Jenkins instance to run them whenever a new patch is submitted. This can be added after project creation.
>>
>> = Initial Committers =
>>
>>  * Jordan Zimmerman (jzimmerman at netflix dot com)
>>  * Jay Zarfoss (jzarfoss at netflix dot com)
>>
>> = Affiliations =
>>
>>  * Jordan Zimmerman, Netflix
>>  * Jay Zarfoss, Netflix
>>
>> = Sponsors =
>>
>> == Champion ==
>>
>>  * Patrick Hunt
>>
>> == Nominated Mentors ==
>>
>>  * Patrick Hunt
>>  * Enis Söztutar
>>  * Mahadev Konar
>>
>> == Sponsoring Entity ==
>>
>>  * Apache Incubator PMC
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org