Re: [DISCUSSION] Switching from RTC to CTR

[Andrew Purtell <an...@gmail.com>]

@dev is now CCed

I didn't want to over structure the discussion with too much detail up front. I do think CTR without supporting process or boundaries can be more problematic than helpful. That could take the form of customarily waiting for reviews before commit even under a CTR regime. I think this discussion has been great so far. I would also add that CTR moves 'R' from a gating requirement per commit (which can be hard to overcome for niche areas or when volunteer resources are less available) more to RMs. will be back later with more. 


> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com> wrote:
> 
> I'd also favor having this discussion on dev@.
> 
>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <gh...@gmail.com> wrote:
>> 
>> This is already a really interesting and meaningful discussion, and is
>> obviously important to the community.
>> 
>> Is there any reason not to move this straight to the dev@ list?
>> 
>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <to...@cloudera.com> wrote:
>>> 
>>> I'm not very active in HBase these days, so I won't cast a non-zero vote,
>>> but I'm -0 on this idea, for basically two reasons:
>>> 
>>> 1) In my experience at a past job which used CTR, the reality was that it
>>> was more like "Commit and maybe review" rather than "Commit then review".
>>> It's always more fun (and often easier) to write new code than to review
>>> other people's code, so if there isn't a requirement that all code gets
>>> reviewed before commit, it's easy for the ratio of code written to code
>>> reviewed to tend towards values significantly greater than 1:1 over time.
>>> At my past job, this meant that a lot of code made it into production (it
>>> was a website) that hadn't ever been reviewed, and in many cases we found
>>> bugs (or other issues) that would have definitely been caught by a good
>>> code reviewer.
>>> 
>>> 2) CTR has both the advantage and disadvantage of allowing areas of code
>> to
>>> be evolved quickly by a single person. That could be seen as a plus, in
>>> that there are some areas which tend to get ignored because we don't
>> have a
>>> critical mass of people reviewing patches in the area -- patches languish
>>> in these areas currently. However, that could also be seen as a good
>> thing
>>> from a "community over code" perspective -- it's better to not have any
>>> areas of code with bus-factor 1. Feeling the pain of not getting reviews
>> in
>>> these areas with only a single active committer encourages us to find
>>> solutions - either by deprecating "niche" features (as we once did with
>>> various 'contrib' projects) or by recruiting new committers who have
>>> interest in maintaining that code area. Lifting review restrictions would
>>> allow us to sweep bus-factor issues under the rug.
>>> 
>>> That said, I think CTR can be a valuable tool for "move fast on new
>>> experimentation" type projects -- I've participated in several feature
>>> branches in HDFS where we operated on CTR on the branch. The assumption
>> was
>>> that, prior to merging the branch into trunk, all of the patches (or a
>>> consolidated mega-patch) would be thoroughly reviewed by several other
>>> committers. I found this to work very well during early development,
>> since
>>> I could hack on things and even commit pieces of code that had known
>>> issues/TODOs. For trickier patches on the CTR branch, I still tended to
>>> ping experienced contributors for their opinions and feedback before
>>> committing.
>>> 
>>> Perhaps such a hybrid policy would work well in the context of HBase
>>> feature development as well?
>>> 
>>> -Todd
>>> 
>>> 
>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <ap...@apache.org>
>>> wrote:
>>> 
>>>> Just thought of branch merge votes. Sorry for that omission. I think it
>>>> makes sense to keep those, but let's discuss that too in this context.
>>>> 
>>>> 
>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <ap...@apache.org>
>>>> wrote:
>>>> 
>>>>> As Greg Stein said on a thread over at general@incubator
>>> ("[DISCUSSION]
>>>>> Graduate Ignite from the Apache Incubator"):
>>>>> 
>>>>> I always translate RTC as "we don't trust you, so somebody else must
>>>>> approve anything you do." To me, that is a lousy basis for creating a
>>>>> community. Trust and peer respect should be the basis, which implies
>>> CTR.
>>>>> 
>>>>> I happen to agree with this sentiment and would like to propose
>>> switching
>>>>> our consensus on commit procedure from RTC to CTR. Concurrently, I
>>> would
>>>>> like to propose this consensus also include the following:
>>>>> - Checkins should pass precommit or the committer should explain on
>> the
>>>>> JIRA why precommit failures are in their best judgement not related
>>>>> - The PMC should be willing to, ultimately, revoke committership
>> should
>>>>> trust in any individual committer be discovered to be misplaced. I
>>> would
>>>>> expect such a last resort to be exceedingly rare and likely never to
>>>> happen
>>>>> because of course long before that we would be setting correct public
>>>>> examples in the first place and respectfully counseling well
>>> intentioned
>>>>> committers who might stray.
>>>>> 
>>>>> Depending on how the conversation proceeds here I would like to
>> include
>>>>> dev@ in this conversation at the earliest opportunity as well.
>>>>> 
>>>>> Thoughts?
>>>>> 
>>>>> --
>>>>> Best regards,
>>>>> 
>>>>>   - Andy
>>>>> 
>>>>> Problems worthy of attack prove their worth by hitting back. - Piet
>>> Hein
>>>>> (via Tom White)
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Best regards,
>>>> 
>>>>   - Andy
>>>> 
>>>> Problems worthy of attack prove their worth by hitting back. - Piet
>> Hein
>>>> (via Tom White)
>>> 
>>> 
>>> 
>>> --
>>> Todd Lipcon
>>> Software Engineer, Cloudera
> 
> 
> 
> -- 
> Sean

Re: [DISCUSSION] Switching from RTC to CTR

[lars hofhansl <lh...@yahoo.com.INVALID>]

I don't really agree that RTC means "we do not trust you". It just means that changes should be peer reviewed, with which I heartily agree.CTR can work with small group (for example in a branch). For a big project I think it will lead to lower quality (and we already have issues with constantly failing tests - part due to the infrastructure, but in part because they are flaky).
That said, I like the idea to leave this at discretion of the committer. In that case we do not need the specific week time-line. For a small fix I think a committer can just commit without review at all (null checks, etc). For larger changes or features the committer should naturally request some review.
Not a fan of codifying too much details, it's better to trust the judgment of the committer and state some general guidelines.
So what am I saying?
I think we can state that review is not _required_. Period. Then we could state that committers should use good judgment as to when to request feedback.

-- Lars

     From: Andrew Purtell <an...@gmail.com>
 To: "dev@hbase.apache.org" <de...@hbase.apache.org> 
Cc: "private@hbase.apache.org" <pr...@hbase.apache.org> 
 Sent: Thursday, July 30, 2015 6:15 PM
 Subject: Re: [DISCUSSION] Switching from RTC to CTR
   
I appreciate very much the earlier feedback about switching from RTC to
CTR. It helped me think about the essential thing I was after.

I'm thinking of making a formal proposal to adopt this, with a VOTE:

> After posting a patch to JIRA, after one week if there is no review or
veto, a committer can commit their own work.

It's important we discuss this and have a vote because the default
Foundation decision making process (
http://www.apache.org/foundation/voting.html) does not allow what would
amount to lazy consensus when RTC is in effect. Should my proposal pass, we
would arrive at a hybrid policy that is identical to the default Foundation
one *until* one week has elapsed after a code change is proposed. Then, for
a committer, for that one code change, they would be able to operate using
CTR. I think the HBase PMC is empowered to set this kind of policy for our
own project at our option. If you feel I am mistaken about that, please
speak up. Should the vote pass I will run it by board@ for review to be
sure.

We'd document this in the book:
https://hbase.apache.org/book.html#_decisions

Also, looking at https://hbase.apache.org/book.html#_decisions, I don't
think the "patch +1 policy" should remain because the trial OWNERS concept
hasn't worked out, IMHO. The OWNERS concept requires a set of constantly
present and engaged owners, a resource demand that's hard to square with
the volunteer nature of our community. The amount of time any committer or
PMC member has on this project is highly variable day to day and week to
week.  I'm also thinking of calling a VOTE to significantly revise or
strike this section.

Both of these things have a common root: Volunteer time is a very precious
commodity. Our community's supply of volunteer time fluctuates. I would
like to see committers be able to make progress with their own work even in
periods when volunteer time is in very short supply, or when they are
working on niche concerns that simply do not draw sufficient interest from
other committers. (This is different from work that people think isn't
appropriate - in that case ignoring it so it will go away would no longer
be an option, a veto would be required if you want to stop something.)




On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <an...@gmail.com>
wrote:

> Had this thought after getting back on the road. As an alternative to any
> sweeping change we could do one incremental but very significant thing that
> acknowledges our status as trusted and busy peers: After posting a patch to
> JIRA, after one week if there is no review or veto, a committer can commit
> their own work.
>
>
> > On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <ol...@gmail.com>
> wrote:
> >
> > Just curious, I assume if this change is made, would it only apply to
> > master branch?
> >
> > -Mikhail
> >
> > On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
> > <an...@gmail.com> wrote:
> >> @dev is now CCed
> >>
> >> I didn't want to over structure the discussion with too much detail up
> front. I do think CTR without supporting process or boundaries can be more
> problematic than helpful. That could take the form of customarily waiting
> for reviews before commit even under a CTR regime. I think this discussion
> has been great so far. I would also add that CTR moves 'R' from a gating
> requirement per commit (which can be hard to overcome for niche areas or
> when volunteer resources are less available) more to RMs. will be back
> later with more.
> >>
> >>
> >>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com>
> wrote:
> >>>
> >>> I'd also favor having this discussion on dev@.
> >>>
> >>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <gh...@gmail.com>
> wrote:
> >>>>
> >>>> This is already a really interesting and meaningful discussion, and is
> >>>> obviously important to the community.
> >>>>
> >>>> Is there any reason not to move this straight to the dev@ list?
> >>>>
> >>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <to...@cloudera.com>
> wrote:
> >>>>>
> >>>>> I'm not very active in HBase these days, so I won't cast a non-zero
> vote,
> >>>>> but I'm -0 on this idea, for basically two reasons:
> >>>>>
> >>>>> 1) In my experience at a past job which used CTR, the reality was
> that it
> >>>>> was more like "Commit and maybe review" rather than "Commit then
> review".
> >>>>> It's always more fun (and often easier) to write new code than to
> review
> >>>>> other people's code, so if there isn't a requirement that all code
> gets
> >>>>> reviewed before commit, it's easy for the ratio of code written to
> code
> >>>>> reviewed to tend towards values significantly greater than 1:1 over
> time.
> >>>>> At my past job, this meant that a lot of code made it into
> production (it
> >>>>> was a website) that hadn't ever been reviewed, and in many cases we
> found
> >>>>> bugs (or other issues) that would have definitely been caught by a
> good
> >>>>> code reviewer.
> >>>>>
> >>>>> 2) CTR has both the advantage and disadvantage of allowing areas of
> code
> >>>> to
> >>>>> be evolved quickly by a single person. That could be seen as a plus,
> in
> >>>>> that there are some areas which tend to get ignored because we don't
> >>>> have a
> >>>>> critical mass of people reviewing patches in the area -- patches
> languish
> >>>>> in these areas currently. However, that could also be seen as a good
> >>>> thing
> >>>>> from a "community over code" perspective -- it's better to not have
> any
> >>>>> areas of code with bus-factor 1. Feeling the pain of not getting
> reviews
> >>>> in
> >>>>> these areas with only a single active committer encourages us to find
> >>>>> solutions - either by deprecating "niche" features (as we once did
> with
> >>>>> various 'contrib' projects) or by recruiting new committers who have
> >>>>> interest in maintaining that code area. Lifting review restrictions
> would
> >>>>> allow us to sweep bus-factor issues under the rug.
> >>>>>
> >>>>> That said, I think CTR can be a valuable tool for "move fast on new
> >>>>> experimentation" type projects -- I've participated in several
> feature
> >>>>> branches in HDFS where we operated on CTR on the branch. The
> assumption
> >>>> was
> >>>>> that, prior to merging the branch into trunk, all of the patches (or
> a
> >>>>> consolidated mega-patch) would be thoroughly reviewed by several
> other
> >>>>> committers. I found this to work very well during early development,
> >>>> since
> >>>>> I could hack on things and even commit pieces of code that had known
> >>>>> issues/TODOs. For trickier patches on the CTR branch, I still tended
> to
> >>>>> ping experienced contributors for their opinions and feedback before
> >>>>> committing.
> >>>>>
> >>>>> Perhaps such a hybrid policy would work well in the context of HBase
> >>>>> feature development as well?
> >>>>>
> >>>>> -Todd
> >>>>>
> >>>>>
> >>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
> apurtell@apache.org>
> >>>>> wrote:
> >>>>>
> >>>>>> Just thought of branch merge votes. Sorry for that omission. I
> think it
> >>>>>> makes sense to keep those, but let's discuss that too in this
> context.
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
> apurtell@apache.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> As Greg Stein said on a thread over at general@incubator
> >>>>> ("[DISCUSSION]
> >>>>>>> Graduate Ignite from the Apache Incubator"):
> >>>>>>>
> >>>>>>> I always translate RTC as "we don't trust you, so somebody else
> must
> >>>>>>> approve anything you do." To me, that is a lousy basis for
> creating a
> >>>>>>> community. Trust and peer respect should be the basis, which
> implies
> >>>>> CTR.
> >>>>>>>
> >>>>>>> I happen to agree with this sentiment and would like to propose
> >>>>> switching
> >>>>>>> our consensus on commit procedure from RTC to CTR. Concurrently, I
> >>>>> would
> >>>>>>> like to propose this consensus also include the following:
> >>>>>>> - Checkins should pass precommit or the committer should explain on
> >>>> the
> >>>>>>> JIRA why precommit failures are in their best judgement not related
> >>>>>>> - The PMC should be willing to, ultimately, revoke committership
> >>>> should
> >>>>>>> trust in any individual committer be discovered to be misplaced. I
> >>>>> would
> >>>>>>> expect such a last resort to be exceedingly rare and likely never
> to
> >>>>>> happen
> >>>>>>> because of course long before that we would be setting correct
> public
> >>>>>>> examples in the first place and respectfully counseling well
> >>>>> intentioned
> >>>>>>> committers who might stray.
> >>>>>>>
> >>>>>>> Depending on how the conversation proceeds here I would like to
> >>>> include
> >>>>>>> dev@ in this conversation at the earliest opportunity as well.
> >>>>>>>
> >>>>>>> Thoughts?
> >>>>>>>
> >>>>>>> --
> >>>>>>> Best regards,
> >>>>>>>
> >>>>>>>  - Andy
> >>>>>>>
> >>>>>>> Problems worthy of attack prove their worth by hitting back. - Piet
> >>>>> Hein
> >>>>>>> (via Tom White)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best regards,
> >>>>>>
> >>>>>>  - Andy
> >>>>>>
> >>>>>> Problems worthy of attack prove their worth by hitting back. - Piet
> >>>> Hein
> >>>>>> (via Tom White)
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Todd Lipcon
> >>>>> Software Engineer, Cloudera
> >>>
> >>>
> >>>
> >>> --
> >>> Sean
> >
> >
> >
> > --
> > Thanks,
> > Michael Antonov
>


  

Re: [DISCUSSION] Switching from RTC to CTR

[Sean Busbey <bu...@cloudera.com>]

On Sun, Aug 16, 2015 at 1:40 PM, Andrew Purtell <ap...@apache.org> wrote:

> > As Stack mentioned, having the communal norm of reviews means that folks
> are more likely to see more of the code.
>
> I don't get it, because that norm can exist equally under CTR as RTC.
>
> Let me repeat a question I had earlier that nobody has responded to: What
> would be the practical difference/outcome of a committer immediately
> checking in something today without giving time for review versus doing so
> if we said "CTR" or "CTR after one week" *AND* "good committer practice is
> to get a code review and +1 before commit"? Practically, I mean. In both
> cases we'd revert it and have a discussion. Would the lack of attention to
> good practice be more actionable by the PMC because in the first case it is
> a rule violation and in the latter it's just inattention to community
> norms. Is that it? We need rules? (And is that "just because", or is there
> really a lack of trust?)
>

I only see our rules as useful to the extent that they codify our norms  so
that newcomers have an easier time getting situated. We don't have formal
bylaws (beyond those of the foundation and which I am fond of) so the rules
section are the best we have for those folks who learn best with explicitly
spelled out rules (even if they are a simplification like "RTC" instead of
"RTC except release management or unblocking broken builds, or or or").

While it's possible that a norm of reviewing code is possible in both CTR
or RTC, I've never seen a CTR community that managed to maintain the norm.
That strongly biases my opinion of CTR, though I could easily have the
causation backwards.

-- 
Sean

Re: [DISCUSSION] Switching from RTC to CTR

[Stack <st...@duboce.net>]

On Sun, Aug 16, 2015 at 11:40 AM, Andrew Purtell <ap...@apache.org>
wrote:

> > As Stack mentioned, having the communal norm of reviews means that folks
> are more likely to see more of the code.
>
> I don't get it, because that norm can exist equally under CTR as RTC.
>
>
How? If strictly CTR, yes, but I've been reading CTR in this discussion as
CTOR (Commit, Then, Optionally Review). Correct me if I have it wrong.



> Let me repeat a question I had earlier that nobody has responded to: What
> would be the practical difference/outcome of a committer immediately
> checking in something today without giving time for review versus doing so
> if we said "CTR" or "CTR after one week" *AND* "good committer practice is
> to get a code review and +1 before commit"? Practically, I mean.



Probably none -- former would be 'abnormal' so would likely trigger peer
interjection while the latter would be the norm and more likely to be let
slide.


In both
> cases we'd revert it and have a discussion. Would the lack of attention to
> good practice be more actionable by the PMC because in the first case it is
> a rule violation and in the latter it's just inattention to community
> norms. Is that it? We need rules? (And is that "just because", or is there
> really a lack of trust?)
>
>
Yes. We need a few rules and a bit of process. Having a simple set makes
distributed dev by a bunch of peers run smoother.

Above in this thread are a few arguments on how RTC does not have to be
interpreted as lack of trust.



> Anyway, I like how in this discussion members of the PMC have expressed
> some room for committers to make minor commits on their good judgement
> without _necessarily_ getting a review first in order to make progress. My
> motivation here is we are a community of busy people and some things just
> can't get the level of interest as others. As an RM, things like build
> fixes to unblock release candidates of older branches come to mind. I also
> like that we have plenty of room for experimental work on branches with
> scrutiny coming later at branch commit time with a very reasonable bar of 3
> +1s inviting necessary scrutiny. I still think a policy of "one week for
> review, then proceed as CTR" would be useful for a variety of reasons but
> don't see clear support for that here. After I collect a few instances of
> recent anecdotal evidence supporting it I may revive this discussion.
>
>
Ok.
St.Ack





>
> On Fri, Aug 14, 2015 at 9:00 PM, Sean Busbey <bu...@cloudera.com> wrote:
>
> > My apologies if this thread has run its course and I'm showing up late to
> > rehash things.
> >
> > Here's a short list of the reasons I like RTC:
> >
> > 1) Number One With A Bullet: It puts committers and
> > non-committer-contributors on closer to equal footing. If I'm
> participating
> > in the project and I haven't been blessed with a commit bit, what am I
> > supposed to do after my week of having my patch sit around?
> >
> > 2) Community interaction. As Stack mentioned, having the communal norm of
> > reviews means that folks are more likely to see more of the code.
> >
> > 3) Everyone has a bad day. I totally identify with committership being a
> > sign of "I trust you" to project participants. But everyone has one of
> > those days where you're in a rush either because of work or life. Having
> > even a cursory additional set of eyes on things markedly increases the
> > quality of the overall code base over a long enough time line (at least
> in
> > my experience contributing to open source projects). So for me, the trust
> > is largely "to follow the rules" and "to provide feedback in reviews".
> >
> > If we end up with some part of the code that isn't getting reviews, I'd
> > rather the PMC fix that problem instead of backslide on the three points
> > above.
> >
> > That we don't have this problem right now is wonderful. I have been in /
> am
> > currently in some projects where the community is very near end-of-life
> by
> > the ASF's definition of 3 folks to approve a release. My observation of
> > those communities has the CTR ones much more a loose collection of people
> > scratching their own itch. When an ASF project gets to that point, what
> the
> > advantage over just going to the attic and keeping independent forks for
> > those few remaining folks?
> >
> > I kind of view this like the ASF policy on only distributing PMC approved
> > releases. The advice from the foundation for folks who don't like the
> > limitation of waiting for a release is "make more releases." Similarly, I
> > think the problem of reviewer bandwidth is solved with "make more
> > committers."
> >
> > I don't want to hijack this thread, but maybe we could have another one
> > about expectations for committership and ways the PMC could help get more
> > folks on the path?
> >
> >
> > On Sun, Aug 2, 2015 at 1:59 PM, Andrew Purtell <andrew.purtell@gmail.com
> >
> > wrote:
> >
> > > Agreed, committers should be spending more time reviewing others' work.
> > We
> > > can ask. It may happen. It may work for a short while. It may not.
> Shrug.
> > > People will do what they want.
> > >
> > > I'm looking to make one substantial change that will allow committers
> to
> > > make progress even if there's nobody else around or interested for one
> > > week. It happens sometimes. I've already talked about my concerns on
> > > assuming a certain level of available volunteer bandwidth. Let me just
> > say
> > > that things are great now, it's fantastic.
> > >
> > > We are pretty much on the honor system already. I don't buy the
> argument
> > > we can't trust that CTR or CTR after one week can work because
> > committers,
> > > even if asked to customarily get a review before commit, may decide to
> > > check in unreviewed untested destabilizing changes. At least, In that
> > case
> > > I'd argue we have a different problem. If you go back to my original
> > mail,
> > > I do say we shouldn't undertake any change if we as PMC are unwilling
> to
> > > revoke the commit bit from committers who sadly demonstrate themselves
> > > unworthy of trust through checkins of toxic waste without review. Merit
> > > _can_ be un-earned. Commits can be reverted. Just because something is
> > > checked in does not mean it will go out in a release. We could put a
> > > nightly suite of regression tests in place. Both proposals I have made,
> > > especially the latter, are not a binary release of any attempt at
> quality
> > > control. We would have all of the consensus expectations of good
> > committer
> > > behavior still in place.
> > >
> > > Let's assume someone checks in something without getting a review
> today.
> > > What would happen? Someone else would revert it and we'd have a
> > discussion.
> > > If we were operating under CTR-after-one-week (or plain CTR) but with
> > > documented expectations that someone get a review first, what changes,
> in
> > > terms of quality control and project discipline? Maybe the difference
> is
> > we
> > > could more easily justify revoking commit privileges? Maybe. I think
> > every
> > > discussion like that would be strictly case by case though, a
> > conversation
> > > between that person and the PMC, justification for revoking committer
> > > status wouldn't be rule based.
> > >
> > > What definitely would change is my well tested good faith change
> waiting
> > > for review after one week could go in no matter who's on vacation or
> > > whatever. Not saying this is a problem for me today. Like I said,
> things
> > > are great around here today. Awesome.
> > >
> > >
> > > > On Aug 2, 2015, at 10:52 AM, Stack <st...@duboce.net> wrote:
> > > >
> > > > On Fri, Jul 31, 2015 at 3:15 AM, Andrew Purtell <
> > > andrew.purtell@gmail.com>
> > > > wrote:
> > > >
> > > >> I appreciate very much the earlier feedback about switching from RTC
> > to
> > > >> CTR. It helped me think about the essential thing I was after.
> > > >>
> > > >> I'm thinking of making a formal proposal to adopt this, with a VOTE:
> > > >>
> > > >>> After posting a patch to JIRA, after one week if there is no review
> > or
> > > >> veto, a committer can commit their own work.
> > > >>
> > > >> (On a bit of a lag...)
> > > >
> > > > What is the problem we are trying to solve? Blocked committers unable
> > to
> > > > get a review?
> > > >
> > > > Any commit can destabilize. Commits without peer review will
> > destabilize
> > > > more than those that have been reviewed. As a community, I'd think
> that
> > > > anything we can do to act against entropy in the system would be
> > primary
> > > > above all other considerations especially given our project a mature
> > > > datastore.
> > > >
> > > > In my own case, it seems to take me tens of patch postings to arrive
> at
> > > > something a fellow reviewer thinks worthy of commit when working
> with a
> > > > peer on an issue of substance. Often the final patch is radically
> > > different
> > > > from first posting. If auto-commit, the first version would just go
> in.
> > > > Unless review, I'd be 'done'. If review, each iteration would be in
> the
> > > > code base? Churn would go up. Commits would be more and smaller. Our
> > > > project would be made of less coherent 'change sets'.
> > > >
> > > > I'd be with LarsH suggestion that committers should be allowed go
> ahead
> > > > with one-liners or more substantial, non-controversial changes in
> site
> > or
> > > > doc but beyond this, I'd be against auto-commit.
> > > >
> > > >>
> > > >>
> > > >> Also, looking at https://hbase.apache.org/book.html#_decisions, I
> > don't
> > > >> think the "patch +1 policy" should remain because the trial OWNERS
> > > concept
> > > >> hasn't worked out, IMHO. The OWNERS concept requires a set of
> > constantly
> > > >> present and engaged owners, a resource demand that's hard to square
> > with
> > > >> the volunteer nature of our community. The amount of time any
> > committer
> > > or
> > > >> PMC member has on this project is highly variable day to day and
> week
> > to
> > > >> week.  I'm also thinking of calling a VOTE to significantly revise
> or
> > > >> strike this section.
> > > > Yeah. OWNERS project failed. Its ghost is in effect in that we each
> go
> > to
> > > > the domain expert when its clear (e.g. Matteo or Stephen on pv2).
> > > >
> > > >
> > > >> Both of these things have a common root: Volunteer time is a very
> > > precious
> > > >> commodity. Our community's supply of volunteer time fluctuates. I
> > would
> > > >> like to see committers be able to make progress with their own work
> > > even in
> > > >> periods when volunteer time is in very short supply, or when they
> are
> > > >> working on niche concerns that simply do not draw sufficient
> interest
> > > from
> > > >> other committers. (This is different from work that people think
> isn't
> > > >> appropriate - in that case ignoring it so it will go away would no
> > > longer
> > > >> be an option, a veto would be required if you want to stop
> something.)
> > > > Committers should be spending more time reviewing their peers work
> (and
> > > > just as importantly, the work of contributors)?
> > > > St.Ack
> > > >
> > > >
> > > >
> > > >> On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <
> > > andrew.purtell@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> Had this thought after getting back on the road. As an alternative
> to
> > > any
> > > >>> sweeping change we could do one incremental but very significant
> > thing
> > > >> that
> > > >>> acknowledges our status as trusted and busy peers: After posting a
> > > patch
> > > >> to
> > > >>> JIRA, after one week if there is no review or veto, a committer can
> > > >> commit
> > > >>> their own work.
> > > >>>
> > > >>>
> > > >>>>> On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <
> olorinbant@gmail.com
> > >
> > > >>>> wrote:
> > > >>>>
> > > >>>> Just curious, I assume if this change is made, would it only apply
> > to
> > > >>>> master branch?
> > > >>>>
> > > >>>> -Mikhail
> > > >>>>
> > > >>>> On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
> > > >>>> <an...@gmail.com> wrote:
> > > >>>>> @dev is now CCed
> > > >>>>>
> > > >>>>> I didn't want to over structure the discussion with too much
> detail
> > > up
> > > >>> front. I do think CTR without supporting process or boundaries can
> be
> > > >> more
> > > >>> problematic than helpful. That could take the form of customarily
> > > waiting
> > > >>> for reviews before commit even under a CTR regime. I think this
> > > >> discussion
> > > >>> has been great so far. I would also add that CTR moves 'R' from a
> > > gating
> > > >>> requirement per commit (which can be hard to overcome for niche
> areas
> > > or
> > > >>> when volunteer resources are less available) more to RMs. will be
> > back
> > > >>> later with more.
> > > >>>>>
> > > >>>>>
> > > >>>>>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <sean.busbey@gmail.com
> >
> > > >>> wrote:
> > > >>>>>>
> > > >>>>>> I'd also favor having this discussion on dev@.
> > > >>>>>>
> > > >>>>>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <
> > > ghelmling@gmail.com
> > > >>>
> > > >>> wrote:
> > > >>>>>>>
> > > >>>>>>> This is already a really interesting and meaningful discussion,
> > and
> > > >> is
> > > >>>>>>> obviously important to the community.
> > > >>>>>>>
> > > >>>>>>> Is there any reason not to move this straight to the dev@
> list?
> > > >>>>>>>
> > > >>>>>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <
> todd@cloudera.com
> > >
> > > >>> wrote:
> > > >>>>>>>>
> > > >>>>>>>> I'm not very active in HBase these days, so I won't cast a
> > > non-zero
> > > >>> vote,
> > > >>>>>>>> but I'm -0 on this idea, for basically two reasons:
> > > >>>>>>>>
> > > >>>>>>>> 1) In my experience at a past job which used CTR, the reality
> > was
> > > >>> that it
> > > >>>>>>>> was more like "Commit and maybe review" rather than "Commit
> then
> > > >>> review".
> > > >>>>>>>> It's always more fun (and often easier) to write new code than
> > to
> > > >>> review
> > > >>>>>>>> other people's code, so if there isn't a requirement that all
> > code
> > > >>> gets
> > > >>>>>>>> reviewed before commit, it's easy for the ratio of code
> written
> > to
> > > >>> code
> > > >>>>>>>> reviewed to tend towards values significantly greater than 1:1
> > > over
> > > >>> time.
> > > >>>>>>>> At my past job, this meant that a lot of code made it into
> > > >>> production (it
> > > >>>>>>>> was a website) that hadn't ever been reviewed, and in many
> cases
> > > we
> > > >>> found
> > > >>>>>>>> bugs (or other issues) that would have definitely been caught
> > by a
> > > >>> good
> > > >>>>>>>> code reviewer.
> > > >>>>>>>>
> > > >>>>>>>> 2) CTR has both the advantage and disadvantage of allowing
> areas
> > > of
> > > >>> code
> > > >>>>>>> to
> > > >>>>>>>> be evolved quickly by a single person. That could be seen as a
> > > >> plus,
> > > >>> in
> > > >>>>>>>> that there are some areas which tend to get ignored because we
> > > >> don't
> > > >>>>>>> have a
> > > >>>>>>>> critical mass of people reviewing patches in the area --
> patches
> > > >>> languish
> > > >>>>>>>> in these areas currently. However, that could also be seen as
> a
> > > >> good
> > > >>>>>>> thing
> > > >>>>>>>> from a "community over code" perspective -- it's better to not
> > > have
> > > >>> any
> > > >>>>>>>> areas of code with bus-factor 1. Feeling the pain of not
> getting
> > > >>> reviews
> > > >>>>>>> in
> > > >>>>>>>> these areas with only a single active committer encourages us
> to
> > > >> find
> > > >>>>>>>> solutions - either by deprecating "niche" features (as we once
> > did
> > > >>> with
> > > >>>>>>>> various 'contrib' projects) or by recruiting new committers
> who
> > > >> have
> > > >>>>>>>> interest in maintaining that code area. Lifting review
> > > restrictions
> > > >>> would
> > > >>>>>>>> allow us to sweep bus-factor issues under the rug.
> > > >>>>>>>>
> > > >>>>>>>> That said, I think CTR can be a valuable tool for "move fast
> on
> > > new
> > > >>>>>>>> experimentation" type projects -- I've participated in several
> > > >>> feature
> > > >>>>>>>> branches in HDFS where we operated on CTR on the branch. The
> > > >>> assumption
> > > >>>>>>> was
> > > >>>>>>>> that, prior to merging the branch into trunk, all of the
> patches
> > > >> (or
> > > >>> a
> > > >>>>>>>> consolidated mega-patch) would be thoroughly reviewed by
> several
> > > >>> other
> > > >>>>>>>> committers. I found this to work very well during early
> > > >> development,
> > > >>>>>>> since
> > > >>>>>>>> I could hack on things and even commit pieces of code that had
> > > >> known
> > > >>>>>>>> issues/TODOs. For trickier patches on the CTR branch, I still
> > > >> tended
> > > >>> to
> > > >>>>>>>> ping experienced contributors for their opinions and feedback
> > > >> before
> > > >>>>>>>> committing.
> > > >>>>>>>>
> > > >>>>>>>> Perhaps such a hybrid policy would work well in the context of
> > > >> HBase
> > > >>>>>>>> feature development as well?
> > > >>>>>>>>
> > > >>>>>>>> -Todd
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
> > > >>> apurtell@apache.org>
> > > >>>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Just thought of branch merge votes. Sorry for that omission.
> I
> > > >>> think it
> > > >>>>>>>>> makes sense to keep those, but let's discuss that too in this
> > > >>> context.
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
> > > >>> apurtell@apache.org>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> As Greg Stein said on a thread over at general@incubator
> > > >>>>>>>> ("[DISCUSSION]
> > > >>>>>>>>>> Graduate Ignite from the Apache Incubator"):
> > > >>>>>>>>>>
> > > >>>>>>>>>> I always translate RTC as "we don't trust you, so somebody
> > else
> > > >>> must
> > > >>>>>>>>>> approve anything you do." To me, that is a lousy basis for
> > > >>> creating a
> > > >>>>>>>>>> community. Trust and peer respect should be the basis, which
> > > >>> implies
> > > >>>>>>>> CTR.
> > > >>>>>>>>>>
> > > >>>>>>>>>> I happen to agree with this sentiment and would like to
> > propose
> > > >>>>>>>> switching
> > > >>>>>>>>>> our consensus on commit procedure from RTC to CTR.
> > Concurrently,
> > > >> I
> > > >>>>>>>> would
> > > >>>>>>>>>> like to propose this consensus also include the following:
> > > >>>>>>>>>> - Checkins should pass precommit or the committer should
> > explain
> > > >> on
> > > >>>>>>> the
> > > >>>>>>>>>> JIRA why precommit failures are in their best judgement not
> > > >> related
> > > >>>>>>>>>> - The PMC should be willing to, ultimately, revoke
> > committership
> > > >>>>>>> should
> > > >>>>>>>>>> trust in any individual committer be discovered to be
> > misplaced.
> > > >> I
> > > >>>>>>>> would
> > > >>>>>>>>>> expect such a last resort to be exceedingly rare and likely
> > > never
> > > >>> to
> > > >>>>>>>>> happen
> > > >>>>>>>>>> because of course long before that we would be setting
> correct
> > > >>> public
> > > >>>>>>>>>> examples in the first place and respectfully counseling well
> > > >>>>>>>> intentioned
> > > >>>>>>>>>> committers who might stray.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Depending on how the conversation proceeds here I would like
> > to
> > > >>>>>>> include
> > > >>>>>>>>>> dev@ in this conversation at the earliest opportunity as
> > well.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Thoughts?
> > > >>>>>>>>>>
> > > >>>>>>>>>> --
> > > >>>>>>>>>> Best regards,
> > > >>>>>>>>>>
> > > >>>>>>>>>> - Andy
> > > >>>>>>>>>>
> > > >>>>>>>>>> Problems worthy of attack prove their worth by hitting
> back. -
> > > >> Piet
> > > >>>>>>>> Hein
> > > >>>>>>>>>> (via Tom White)
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> --
> > > >>>>>>>>> Best regards,
> > > >>>>>>>>>
> > > >>>>>>>>> - Andy
> > > >>>>>>>>>
> > > >>>>>>>>> Problems worthy of attack prove their worth by hitting back.
> -
> > > >> Piet
> > > >>>>>>> Hein
> > > >>>>>>>>> (via Tom White)
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> --
> > > >>>>>>>> Todd Lipcon
> > > >>>>>>>> Software Engineer, Cloudera
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> --
> > > >>>>>> Sean
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> --
> > > >>>> Thanks,
> > > >>>> Michael Antonov
> > > >>
> > >
> >
> >
> >
> > --
> > Sean
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: [DISCUSSION] Switching from RTC to CTR

[Andrew Purtell <ap...@apache.org>]

> As Stack mentioned, having the communal norm of reviews means that folks
are more likely to see more of the code.

I don't get it, because that norm can exist equally under CTR as RTC.

Let me repeat a question I had earlier that nobody has responded to: What
would be the practical difference/outcome of a committer immediately
checking in something today without giving time for review versus doing so
if we said "CTR" or "CTR after one week" *AND* "good committer practice is
to get a code review and +1 before commit"? Practically, I mean. In both
cases we'd revert it and have a discussion. Would the lack of attention to
good practice be more actionable by the PMC because in the first case it is
a rule violation and in the latter it's just inattention to community
norms. Is that it? We need rules? (And is that "just because", or is there
really a lack of trust?)

Anyway, I like how in this discussion members of the PMC have expressed
some room for committers to make minor commits on their good judgement
without _necessarily_ getting a review first in order to make progress. My
motivation here is we are a community of busy people and some things just
can't get the level of interest as others. As an RM, things like build
fixes to unblock release candidates of older branches come to mind. I also
like that we have plenty of room for experimental work on branches with
scrutiny coming later at branch commit time with a very reasonable bar of 3
+1s inviting necessary scrutiny. I still think a policy of "one week for
review, then proceed as CTR" would be useful for a variety of reasons but
don't see clear support for that here. After I collect a few instances of
recent anecdotal evidence supporting it I may revive this discussion.


On Fri, Aug 14, 2015 at 9:00 PM, Sean Busbey <bu...@cloudera.com> wrote:

> My apologies if this thread has run its course and I'm showing up late to
> rehash things.
>
> Here's a short list of the reasons I like RTC:
>
> 1) Number One With A Bullet: It puts committers and
> non-committer-contributors on closer to equal footing. If I'm participating
> in the project and I haven't been blessed with a commit bit, what am I
> supposed to do after my week of having my patch sit around?
>
> 2) Community interaction. As Stack mentioned, having the communal norm of
> reviews means that folks are more likely to see more of the code.
>
> 3) Everyone has a bad day. I totally identify with committership being a
> sign of "I trust you" to project participants. But everyone has one of
> those days where you're in a rush either because of work or life. Having
> even a cursory additional set of eyes on things markedly increases the
> quality of the overall code base over a long enough time line (at least in
> my experience contributing to open source projects). So for me, the trust
> is largely "to follow the rules" and "to provide feedback in reviews".
>
> If we end up with some part of the code that isn't getting reviews, I'd
> rather the PMC fix that problem instead of backslide on the three points
> above.
>
> That we don't have this problem right now is wonderful. I have been in / am
> currently in some projects where the community is very near end-of-life by
> the ASF's definition of 3 folks to approve a release. My observation of
> those communities has the CTR ones much more a loose collection of people
> scratching their own itch. When an ASF project gets to that point, what the
> advantage over just going to the attic and keeping independent forks for
> those few remaining folks?
>
> I kind of view this like the ASF policy on only distributing PMC approved
> releases. The advice from the foundation for folks who don't like the
> limitation of waiting for a release is "make more releases." Similarly, I
> think the problem of reviewer bandwidth is solved with "make more
> committers."
>
> I don't want to hijack this thread, but maybe we could have another one
> about expectations for committership and ways the PMC could help get more
> folks on the path?
>
>
> On Sun, Aug 2, 2015 at 1:59 PM, Andrew Purtell <an...@gmail.com>
> wrote:
>
> > Agreed, committers should be spending more time reviewing others' work.
> We
> > can ask. It may happen. It may work for a short while. It may not. Shrug.
> > People will do what they want.
> >
> > I'm looking to make one substantial change that will allow committers to
> > make progress even if there's nobody else around or interested for one
> > week. It happens sometimes. I've already talked about my concerns on
> > assuming a certain level of available volunteer bandwidth. Let me just
> say
> > that things are great now, it's fantastic.
> >
> > We are pretty much on the honor system already. I don't buy the argument
> > we can't trust that CTR or CTR after one week can work because
> committers,
> > even if asked to customarily get a review before commit, may decide to
> > check in unreviewed untested destabilizing changes. At least, In that
> case
> > I'd argue we have a different problem. If you go back to my original
> mail,
> > I do say we shouldn't undertake any change if we as PMC are unwilling to
> > revoke the commit bit from committers who sadly demonstrate themselves
> > unworthy of trust through checkins of toxic waste without review. Merit
> > _can_ be un-earned. Commits can be reverted. Just because something is
> > checked in does not mean it will go out in a release. We could put a
> > nightly suite of regression tests in place. Both proposals I have made,
> > especially the latter, are not a binary release of any attempt at quality
> > control. We would have all of the consensus expectations of good
> committer
> > behavior still in place.
> >
> > Let's assume someone checks in something without getting a review today.
> > What would happen? Someone else would revert it and we'd have a
> discussion.
> > If we were operating under CTR-after-one-week (or plain CTR) but with
> > documented expectations that someone get a review first, what changes, in
> > terms of quality control and project discipline? Maybe the difference is
> we
> > could more easily justify revoking commit privileges? Maybe. I think
> every
> > discussion like that would be strictly case by case though, a
> conversation
> > between that person and the PMC, justification for revoking committer
> > status wouldn't be rule based.
> >
> > What definitely would change is my well tested good faith change waiting
> > for review after one week could go in no matter who's on vacation or
> > whatever. Not saying this is a problem for me today. Like I said, things
> > are great around here today. Awesome.
> >
> >
> > > On Aug 2, 2015, at 10:52 AM, Stack <st...@duboce.net> wrote:
> > >
> > > On Fri, Jul 31, 2015 at 3:15 AM, Andrew Purtell <
> > andrew.purtell@gmail.com>
> > > wrote:
> > >
> > >> I appreciate very much the earlier feedback about switching from RTC
> to
> > >> CTR. It helped me think about the essential thing I was after.
> > >>
> > >> I'm thinking of making a formal proposal to adopt this, with a VOTE:
> > >>
> > >>> After posting a patch to JIRA, after one week if there is no review
> or
> > >> veto, a committer can commit their own work.
> > >>
> > >> (On a bit of a lag...)
> > >
> > > What is the problem we are trying to solve? Blocked committers unable
> to
> > > get a review?
> > >
> > > Any commit can destabilize. Commits without peer review will
> destabilize
> > > more than those that have been reviewed. As a community, I'd think that
> > > anything we can do to act against entropy in the system would be
> primary
> > > above all other considerations especially given our project a mature
> > > datastore.
> > >
> > > In my own case, it seems to take me tens of patch postings to arrive at
> > > something a fellow reviewer thinks worthy of commit when working with a
> > > peer on an issue of substance. Often the final patch is radically
> > different
> > > from first posting. If auto-commit, the first version would just go in.
> > > Unless review, I'd be 'done'. If review, each iteration would be in the
> > > code base? Churn would go up. Commits would be more and smaller. Our
> > > project would be made of less coherent 'change sets'.
> > >
> > > I'd be with LarsH suggestion that committers should be allowed go ahead
> > > with one-liners or more substantial, non-controversial changes in site
> or
> > > doc but beyond this, I'd be against auto-commit.
> > >
> > >>
> > >>
> > >> Also, looking at https://hbase.apache.org/book.html#_decisions, I
> don't
> > >> think the "patch +1 policy" should remain because the trial OWNERS
> > concept
> > >> hasn't worked out, IMHO. The OWNERS concept requires a set of
> constantly
> > >> present and engaged owners, a resource demand that's hard to square
> with
> > >> the volunteer nature of our community. The amount of time any
> committer
> > or
> > >> PMC member has on this project is highly variable day to day and week
> to
> > >> week.  I'm also thinking of calling a VOTE to significantly revise or
> > >> strike this section.
> > > Yeah. OWNERS project failed. Its ghost is in effect in that we each go
> to
> > > the domain expert when its clear (e.g. Matteo or Stephen on pv2).
> > >
> > >
> > >> Both of these things have a common root: Volunteer time is a very
> > precious
> > >> commodity. Our community's supply of volunteer time fluctuates. I
> would
> > >> like to see committers be able to make progress with their own work
> > even in
> > >> periods when volunteer time is in very short supply, or when they are
> > >> working on niche concerns that simply do not draw sufficient interest
> > from
> > >> other committers. (This is different from work that people think isn't
> > >> appropriate - in that case ignoring it so it will go away would no
> > longer
> > >> be an option, a veto would be required if you want to stop something.)
> > > Committers should be spending more time reviewing their peers work (and
> > > just as importantly, the work of contributors)?
> > > St.Ack
> > >
> > >
> > >
> > >> On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <
> > andrew.purtell@gmail.com>
> > >> wrote:
> > >>
> > >>> Had this thought after getting back on the road. As an alternative to
> > any
> > >>> sweeping change we could do one incremental but very significant
> thing
> > >> that
> > >>> acknowledges our status as trusted and busy peers: After posting a
> > patch
> > >> to
> > >>> JIRA, after one week if there is no review or veto, a committer can
> > >> commit
> > >>> their own work.
> > >>>
> > >>>
> > >>>>> On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <olorinbant@gmail.com
> >
> > >>>> wrote:
> > >>>>
> > >>>> Just curious, I assume if this change is made, would it only apply
> to
> > >>>> master branch?
> > >>>>
> > >>>> -Mikhail
> > >>>>
> > >>>> On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
> > >>>> <an...@gmail.com> wrote:
> > >>>>> @dev is now CCed
> > >>>>>
> > >>>>> I didn't want to over structure the discussion with too much detail
> > up
> > >>> front. I do think CTR without supporting process or boundaries can be
> > >> more
> > >>> problematic than helpful. That could take the form of customarily
> > waiting
> > >>> for reviews before commit even under a CTR regime. I think this
> > >> discussion
> > >>> has been great so far. I would also add that CTR moves 'R' from a
> > gating
> > >>> requirement per commit (which can be hard to overcome for niche areas
> > or
> > >>> when volunteer resources are less available) more to RMs. will be
> back
> > >>> later with more.
> > >>>>>
> > >>>>>
> > >>>>>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com>
> > >>> wrote:
> > >>>>>>
> > >>>>>> I'd also favor having this discussion on dev@.
> > >>>>>>
> > >>>>>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <
> > ghelmling@gmail.com
> > >>>
> > >>> wrote:
> > >>>>>>>
> > >>>>>>> This is already a really interesting and meaningful discussion,
> and
> > >> is
> > >>>>>>> obviously important to the community.
> > >>>>>>>
> > >>>>>>> Is there any reason not to move this straight to the dev@ list?
> > >>>>>>>
> > >>>>>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <todd@cloudera.com
> >
> > >>> wrote:
> > >>>>>>>>
> > >>>>>>>> I'm not very active in HBase these days, so I won't cast a
> > non-zero
> > >>> vote,
> > >>>>>>>> but I'm -0 on this idea, for basically two reasons:
> > >>>>>>>>
> > >>>>>>>> 1) In my experience at a past job which used CTR, the reality
> was
> > >>> that it
> > >>>>>>>> was more like "Commit and maybe review" rather than "Commit then
> > >>> review".
> > >>>>>>>> It's always more fun (and often easier) to write new code than
> to
> > >>> review
> > >>>>>>>> other people's code, so if there isn't a requirement that all
> code
> > >>> gets
> > >>>>>>>> reviewed before commit, it's easy for the ratio of code written
> to
> > >>> code
> > >>>>>>>> reviewed to tend towards values significantly greater than 1:1
> > over
> > >>> time.
> > >>>>>>>> At my past job, this meant that a lot of code made it into
> > >>> production (it
> > >>>>>>>> was a website) that hadn't ever been reviewed, and in many cases
> > we
> > >>> found
> > >>>>>>>> bugs (or other issues) that would have definitely been caught
> by a
> > >>> good
> > >>>>>>>> code reviewer.
> > >>>>>>>>
> > >>>>>>>> 2) CTR has both the advantage and disadvantage of allowing areas
> > of
> > >>> code
> > >>>>>>> to
> > >>>>>>>> be evolved quickly by a single person. That could be seen as a
> > >> plus,
> > >>> in
> > >>>>>>>> that there are some areas which tend to get ignored because we
> > >> don't
> > >>>>>>> have a
> > >>>>>>>> critical mass of people reviewing patches in the area -- patches
> > >>> languish
> > >>>>>>>> in these areas currently. However, that could also be seen as a
> > >> good
> > >>>>>>> thing
> > >>>>>>>> from a "community over code" perspective -- it's better to not
> > have
> > >>> any
> > >>>>>>>> areas of code with bus-factor 1. Feeling the pain of not getting
> > >>> reviews
> > >>>>>>> in
> > >>>>>>>> these areas with only a single active committer encourages us to
> > >> find
> > >>>>>>>> solutions - either by deprecating "niche" features (as we once
> did
> > >>> with
> > >>>>>>>> various 'contrib' projects) or by recruiting new committers who
> > >> have
> > >>>>>>>> interest in maintaining that code area. Lifting review
> > restrictions
> > >>> would
> > >>>>>>>> allow us to sweep bus-factor issues under the rug.
> > >>>>>>>>
> > >>>>>>>> That said, I think CTR can be a valuable tool for "move fast on
> > new
> > >>>>>>>> experimentation" type projects -- I've participated in several
> > >>> feature
> > >>>>>>>> branches in HDFS where we operated on CTR on the branch. The
> > >>> assumption
> > >>>>>>> was
> > >>>>>>>> that, prior to merging the branch into trunk, all of the patches
> > >> (or
> > >>> a
> > >>>>>>>> consolidated mega-patch) would be thoroughly reviewed by several
> > >>> other
> > >>>>>>>> committers. I found this to work very well during early
> > >> development,
> > >>>>>>> since
> > >>>>>>>> I could hack on things and even commit pieces of code that had
> > >> known
> > >>>>>>>> issues/TODOs. For trickier patches on the CTR branch, I still
> > >> tended
> > >>> to
> > >>>>>>>> ping experienced contributors for their opinions and feedback
> > >> before
> > >>>>>>>> committing.
> > >>>>>>>>
> > >>>>>>>> Perhaps such a hybrid policy would work well in the context of
> > >> HBase
> > >>>>>>>> feature development as well?
> > >>>>>>>>
> > >>>>>>>> -Todd
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
> > >>> apurtell@apache.org>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Just thought of branch merge votes. Sorry for that omission. I
> > >>> think it
> > >>>>>>>>> makes sense to keep those, but let's discuss that too in this
> > >>> context.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
> > >>> apurtell@apache.org>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> As Greg Stein said on a thread over at general@incubator
> > >>>>>>>> ("[DISCUSSION]
> > >>>>>>>>>> Graduate Ignite from the Apache Incubator"):
> > >>>>>>>>>>
> > >>>>>>>>>> I always translate RTC as "we don't trust you, so somebody
> else
> > >>> must
> > >>>>>>>>>> approve anything you do." To me, that is a lousy basis for
> > >>> creating a
> > >>>>>>>>>> community. Trust and peer respect should be the basis, which
> > >>> implies
> > >>>>>>>> CTR.
> > >>>>>>>>>>
> > >>>>>>>>>> I happen to agree with this sentiment and would like to
> propose
> > >>>>>>>> switching
> > >>>>>>>>>> our consensus on commit procedure from RTC to CTR.
> Concurrently,
> > >> I
> > >>>>>>>> would
> > >>>>>>>>>> like to propose this consensus also include the following:
> > >>>>>>>>>> - Checkins should pass precommit or the committer should
> explain
> > >> on
> > >>>>>>> the
> > >>>>>>>>>> JIRA why precommit failures are in their best judgement not
> > >> related
> > >>>>>>>>>> - The PMC should be willing to, ultimately, revoke
> committership
> > >>>>>>> should
> > >>>>>>>>>> trust in any individual committer be discovered to be
> misplaced.
> > >> I
> > >>>>>>>> would
> > >>>>>>>>>> expect such a last resort to be exceedingly rare and likely
> > never
> > >>> to
> > >>>>>>>>> happen
> > >>>>>>>>>> because of course long before that we would be setting correct
> > >>> public
> > >>>>>>>>>> examples in the first place and respectfully counseling well
> > >>>>>>>> intentioned
> > >>>>>>>>>> committers who might stray.
> > >>>>>>>>>>
> > >>>>>>>>>> Depending on how the conversation proceeds here I would like
> to
> > >>>>>>> include
> > >>>>>>>>>> dev@ in this conversation at the earliest opportunity as
> well.
> > >>>>>>>>>>
> > >>>>>>>>>> Thoughts?
> > >>>>>>>>>>
> > >>>>>>>>>> --
> > >>>>>>>>>> Best regards,
> > >>>>>>>>>>
> > >>>>>>>>>> - Andy
> > >>>>>>>>>>
> > >>>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
> > >> Piet
> > >>>>>>>> Hein
> > >>>>>>>>>> (via Tom White)
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> Best regards,
> > >>>>>>>>>
> > >>>>>>>>> - Andy
> > >>>>>>>>>
> > >>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
> > >> Piet
> > >>>>>>> Hein
> > >>>>>>>>> (via Tom White)
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> Todd Lipcon
> > >>>>>>>> Software Engineer, Cloudera
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Sean
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Thanks,
> > >>>> Michael Antonov
> > >>
> >
>
>
>
> --
> Sean
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Switching from RTC to CTR

[Sean Busbey <bu...@cloudera.com>]

My apologies if this thread has run its course and I'm showing up late to
rehash things.

Here's a short list of the reasons I like RTC:

1) Number One With A Bullet: It puts committers and
non-committer-contributors on closer to equal footing. If I'm participating
in the project and I haven't been blessed with a commit bit, what am I
supposed to do after my week of having my patch sit around?

2) Community interaction. As Stack mentioned, having the communal norm of
reviews means that folks are more likely to see more of the code.

3) Everyone has a bad day. I totally identify with committership being a
sign of "I trust you" to project participants. But everyone has one of
those days where you're in a rush either because of work or life. Having
even a cursory additional set of eyes on things markedly increases the
quality of the overall code base over a long enough time line (at least in
my experience contributing to open source projects). So for me, the trust
is largely "to follow the rules" and "to provide feedback in reviews".

If we end up with some part of the code that isn't getting reviews, I'd
rather the PMC fix that problem instead of backslide on the three points
above.

That we don't have this problem right now is wonderful. I have been in / am
currently in some projects where the community is very near end-of-life by
the ASF's definition of 3 folks to approve a release. My observation of
those communities has the CTR ones much more a loose collection of people
scratching their own itch. When an ASF project gets to that point, what the
advantage over just going to the attic and keeping independent forks for
those few remaining folks?

I kind of view this like the ASF policy on only distributing PMC approved
releases. The advice from the foundation for folks who don't like the
limitation of waiting for a release is "make more releases." Similarly, I
think the problem of reviewer bandwidth is solved with "make more
committers."

I don't want to hijack this thread, but maybe we could have another one
about expectations for committership and ways the PMC could help get more
folks on the path?


On Sun, Aug 2, 2015 at 1:59 PM, Andrew Purtell <an...@gmail.com>
wrote:

> Agreed, committers should be spending more time reviewing others' work. We
> can ask. It may happen. It may work for a short while. It may not. Shrug.
> People will do what they want.
>
> I'm looking to make one substantial change that will allow committers to
> make progress even if there's nobody else around or interested for one
> week. It happens sometimes. I've already talked about my concerns on
> assuming a certain level of available volunteer bandwidth. Let me just say
> that things are great now, it's fantastic.
>
> We are pretty much on the honor system already. I don't buy the argument
> we can't trust that CTR or CTR after one week can work because committers,
> even if asked to customarily get a review before commit, may decide to
> check in unreviewed untested destabilizing changes. At least, In that case
> I'd argue we have a different problem. If you go back to my original mail,
> I do say we shouldn't undertake any change if we as PMC are unwilling to
> revoke the commit bit from committers who sadly demonstrate themselves
> unworthy of trust through checkins of toxic waste without review. Merit
> _can_ be un-earned. Commits can be reverted. Just because something is
> checked in does not mean it will go out in a release. We could put a
> nightly suite of regression tests in place. Both proposals I have made,
> especially the latter, are not a binary release of any attempt at quality
> control. We would have all of the consensus expectations of good committer
> behavior still in place.
>
> Let's assume someone checks in something without getting a review today.
> What would happen? Someone else would revert it and we'd have a discussion.
> If we were operating under CTR-after-one-week (or plain CTR) but with
> documented expectations that someone get a review first, what changes, in
> terms of quality control and project discipline? Maybe the difference is we
> could more easily justify revoking commit privileges? Maybe. I think every
> discussion like that would be strictly case by case though, a conversation
> between that person and the PMC, justification for revoking committer
> status wouldn't be rule based.
>
> What definitely would change is my well tested good faith change waiting
> for review after one week could go in no matter who's on vacation or
> whatever. Not saying this is a problem for me today. Like I said, things
> are great around here today. Awesome.
>
>
> > On Aug 2, 2015, at 10:52 AM, Stack <st...@duboce.net> wrote:
> >
> > On Fri, Jul 31, 2015 at 3:15 AM, Andrew Purtell <
> andrew.purtell@gmail.com>
> > wrote:
> >
> >> I appreciate very much the earlier feedback about switching from RTC to
> >> CTR. It helped me think about the essential thing I was after.
> >>
> >> I'm thinking of making a formal proposal to adopt this, with a VOTE:
> >>
> >>> After posting a patch to JIRA, after one week if there is no review or
> >> veto, a committer can commit their own work.
> >>
> >> (On a bit of a lag...)
> >
> > What is the problem we are trying to solve? Blocked committers unable to
> > get a review?
> >
> > Any commit can destabilize. Commits without peer review will destabilize
> > more than those that have been reviewed. As a community, I'd think that
> > anything we can do to act against entropy in the system would be primary
> > above all other considerations especially given our project a mature
> > datastore.
> >
> > In my own case, it seems to take me tens of patch postings to arrive at
> > something a fellow reviewer thinks worthy of commit when working with a
> > peer on an issue of substance. Often the final patch is radically
> different
> > from first posting. If auto-commit, the first version would just go in.
> > Unless review, I'd be 'done'. If review, each iteration would be in the
> > code base? Churn would go up. Commits would be more and smaller. Our
> > project would be made of less coherent 'change sets'.
> >
> > I'd be with LarsH suggestion that committers should be allowed go ahead
> > with one-liners or more substantial, non-controversial changes in site or
> > doc but beyond this, I'd be against auto-commit.
> >
> >>
> >>
> >> Also, looking at https://hbase.apache.org/book.html#_decisions, I don't
> >> think the "patch +1 policy" should remain because the trial OWNERS
> concept
> >> hasn't worked out, IMHO. The OWNERS concept requires a set of constantly
> >> present and engaged owners, a resource demand that's hard to square with
> >> the volunteer nature of our community. The amount of time any committer
> or
> >> PMC member has on this project is highly variable day to day and week to
> >> week.  I'm also thinking of calling a VOTE to significantly revise or
> >> strike this section.
> > Yeah. OWNERS project failed. Its ghost is in effect in that we each go to
> > the domain expert when its clear (e.g. Matteo or Stephen on pv2).
> >
> >
> >> Both of these things have a common root: Volunteer time is a very
> precious
> >> commodity. Our community's supply of volunteer time fluctuates. I would
> >> like to see committers be able to make progress with their own work
> even in
> >> periods when volunteer time is in very short supply, or when they are
> >> working on niche concerns that simply do not draw sufficient interest
> from
> >> other committers. (This is different from work that people think isn't
> >> appropriate - in that case ignoring it so it will go away would no
> longer
> >> be an option, a veto would be required if you want to stop something.)
> > Committers should be spending more time reviewing their peers work (and
> > just as importantly, the work of contributors)?
> > St.Ack
> >
> >
> >
> >> On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <
> andrew.purtell@gmail.com>
> >> wrote:
> >>
> >>> Had this thought after getting back on the road. As an alternative to
> any
> >>> sweeping change we could do one incremental but very significant thing
> >> that
> >>> acknowledges our status as trusted and busy peers: After posting a
> patch
> >> to
> >>> JIRA, after one week if there is no review or veto, a committer can
> >> commit
> >>> their own work.
> >>>
> >>>
> >>>>> On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <ol...@gmail.com>
> >>>> wrote:
> >>>>
> >>>> Just curious, I assume if this change is made, would it only apply to
> >>>> master branch?
> >>>>
> >>>> -Mikhail
> >>>>
> >>>> On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
> >>>> <an...@gmail.com> wrote:
> >>>>> @dev is now CCed
> >>>>>
> >>>>> I didn't want to over structure the discussion with too much detail
> up
> >>> front. I do think CTR without supporting process or boundaries can be
> >> more
> >>> problematic than helpful. That could take the form of customarily
> waiting
> >>> for reviews before commit even under a CTR regime. I think this
> >> discussion
> >>> has been great so far. I would also add that CTR moves 'R' from a
> gating
> >>> requirement per commit (which can be hard to overcome for niche areas
> or
> >>> when volunteer resources are less available) more to RMs. will be back
> >>> later with more.
> >>>>>
> >>>>>
> >>>>>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com>
> >>> wrote:
> >>>>>>
> >>>>>> I'd also favor having this discussion on dev@.
> >>>>>>
> >>>>>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <
> ghelmling@gmail.com
> >>>
> >>> wrote:
> >>>>>>>
> >>>>>>> This is already a really interesting and meaningful discussion, and
> >> is
> >>>>>>> obviously important to the community.
> >>>>>>>
> >>>>>>> Is there any reason not to move this straight to the dev@ list?
> >>>>>>>
> >>>>>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <to...@cloudera.com>
> >>> wrote:
> >>>>>>>>
> >>>>>>>> I'm not very active in HBase these days, so I won't cast a
> non-zero
> >>> vote,
> >>>>>>>> but I'm -0 on this idea, for basically two reasons:
> >>>>>>>>
> >>>>>>>> 1) In my experience at a past job which used CTR, the reality was
> >>> that it
> >>>>>>>> was more like "Commit and maybe review" rather than "Commit then
> >>> review".
> >>>>>>>> It's always more fun (and often easier) to write new code than to
> >>> review
> >>>>>>>> other people's code, so if there isn't a requirement that all code
> >>> gets
> >>>>>>>> reviewed before commit, it's easy for the ratio of code written to
> >>> code
> >>>>>>>> reviewed to tend towards values significantly greater than 1:1
> over
> >>> time.
> >>>>>>>> At my past job, this meant that a lot of code made it into
> >>> production (it
> >>>>>>>> was a website) that hadn't ever been reviewed, and in many cases
> we
> >>> found
> >>>>>>>> bugs (or other issues) that would have definitely been caught by a
> >>> good
> >>>>>>>> code reviewer.
> >>>>>>>>
> >>>>>>>> 2) CTR has both the advantage and disadvantage of allowing areas
> of
> >>> code
> >>>>>>> to
> >>>>>>>> be evolved quickly by a single person. That could be seen as a
> >> plus,
> >>> in
> >>>>>>>> that there are some areas which tend to get ignored because we
> >> don't
> >>>>>>> have a
> >>>>>>>> critical mass of people reviewing patches in the area -- patches
> >>> languish
> >>>>>>>> in these areas currently. However, that could also be seen as a
> >> good
> >>>>>>> thing
> >>>>>>>> from a "community over code" perspective -- it's better to not
> have
> >>> any
> >>>>>>>> areas of code with bus-factor 1. Feeling the pain of not getting
> >>> reviews
> >>>>>>> in
> >>>>>>>> these areas with only a single active committer encourages us to
> >> find
> >>>>>>>> solutions - either by deprecating "niche" features (as we once did
> >>> with
> >>>>>>>> various 'contrib' projects) or by recruiting new committers who
> >> have
> >>>>>>>> interest in maintaining that code area. Lifting review
> restrictions
> >>> would
> >>>>>>>> allow us to sweep bus-factor issues under the rug.
> >>>>>>>>
> >>>>>>>> That said, I think CTR can be a valuable tool for "move fast on
> new
> >>>>>>>> experimentation" type projects -- I've participated in several
> >>> feature
> >>>>>>>> branches in HDFS where we operated on CTR on the branch. The
> >>> assumption
> >>>>>>> was
> >>>>>>>> that, prior to merging the branch into trunk, all of the patches
> >> (or
> >>> a
> >>>>>>>> consolidated mega-patch) would be thoroughly reviewed by several
> >>> other
> >>>>>>>> committers. I found this to work very well during early
> >> development,
> >>>>>>> since
> >>>>>>>> I could hack on things and even commit pieces of code that had
> >> known
> >>>>>>>> issues/TODOs. For trickier patches on the CTR branch, I still
> >> tended
> >>> to
> >>>>>>>> ping experienced contributors for their opinions and feedback
> >> before
> >>>>>>>> committing.
> >>>>>>>>
> >>>>>>>> Perhaps such a hybrid policy would work well in the context of
> >> HBase
> >>>>>>>> feature development as well?
> >>>>>>>>
> >>>>>>>> -Todd
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
> >>> apurtell@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Just thought of branch merge votes. Sorry for that omission. I
> >>> think it
> >>>>>>>>> makes sense to keep those, but let's discuss that too in this
> >>> context.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
> >>> apurtell@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> As Greg Stein said on a thread over at general@incubator
> >>>>>>>> ("[DISCUSSION]
> >>>>>>>>>> Graduate Ignite from the Apache Incubator"):
> >>>>>>>>>>
> >>>>>>>>>> I always translate RTC as "we don't trust you, so somebody else
> >>> must
> >>>>>>>>>> approve anything you do." To me, that is a lousy basis for
> >>> creating a
> >>>>>>>>>> community. Trust and peer respect should be the basis, which
> >>> implies
> >>>>>>>> CTR.
> >>>>>>>>>>
> >>>>>>>>>> I happen to agree with this sentiment and would like to propose
> >>>>>>>> switching
> >>>>>>>>>> our consensus on commit procedure from RTC to CTR. Concurrently,
> >> I
> >>>>>>>> would
> >>>>>>>>>> like to propose this consensus also include the following:
> >>>>>>>>>> - Checkins should pass precommit or the committer should explain
> >> on
> >>>>>>> the
> >>>>>>>>>> JIRA why precommit failures are in their best judgement not
> >> related
> >>>>>>>>>> - The PMC should be willing to, ultimately, revoke committership
> >>>>>>> should
> >>>>>>>>>> trust in any individual committer be discovered to be misplaced.
> >> I
> >>>>>>>> would
> >>>>>>>>>> expect such a last resort to be exceedingly rare and likely
> never
> >>> to
> >>>>>>>>> happen
> >>>>>>>>>> because of course long before that we would be setting correct
> >>> public
> >>>>>>>>>> examples in the first place and respectfully counseling well
> >>>>>>>> intentioned
> >>>>>>>>>> committers who might stray.
> >>>>>>>>>>
> >>>>>>>>>> Depending on how the conversation proceeds here I would like to
> >>>>>>> include
> >>>>>>>>>> dev@ in this conversation at the earliest opportunity as well.
> >>>>>>>>>>
> >>>>>>>>>> Thoughts?
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Best regards,
> >>>>>>>>>>
> >>>>>>>>>> - Andy
> >>>>>>>>>>
> >>>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
> >> Piet
> >>>>>>>> Hein
> >>>>>>>>>> (via Tom White)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Best regards,
> >>>>>>>>>
> >>>>>>>>> - Andy
> >>>>>>>>>
> >>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
> >> Piet
> >>>>>>> Hein
> >>>>>>>>> (via Tom White)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Todd Lipcon
> >>>>>>>> Software Engineer, Cloudera
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Sean
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Thanks,
> >>>> Michael Antonov
> >>
>



-- 
Sean

Re: [DISCUSSION] Switching from RTC to CTR

[Enis Söztutar <en...@apache.org>]

I would like to echo the sentiment that from my experience CTR works for
projects early in the lifetime and have smaller community. Also, CTR has
the potential to pollute the commit history which would be pretty bad. For
HBase, what we have in affect seems to work IMO. Is there any specific
examples that adopting CTR will solve?

For trivial stuff, RM and release related checkins and backporting changes
for patches, etc are already CTR, no. I have seen myself and others push
checkins without reviews because a strict RTC is unrealistic (for example I
freely make changes for backporting a patch from master and directly commit
it, or do checkins for the release, etc). I have also done some of "I'll
commit this tomorrow unless objection" type of commits if I think that the
patch is trivial and there is no interest.

>From the book, this is what we have in
https://hbase.apache.org/book.html#_review

For non-trivial changes, it is required to get another committer to review
your own patches before commit. Use the *Submit Patch* button in JIRA, just
like other contributors, and then wait for a +1 response from another
committer before committing.
<https://hbase.apache.org/book.html#_reject>
It clearly says that +1 is only required for non-trivial changes. Depending
on the committers judgement the stuff that can go in without review, I
would argue that "I'll commit this unless objection" policy is good enough.
We can amend it to add a statement that if a committer believes a review is
optional, she can add a jira comment, and commit the patch after sufficient
time has passed. No need to quantify what "sufficient time" means. The
"trust" comes from trusting the committers judgement.

Enis


On Tue, Aug 11, 2015 at 1:49 AM, Andrew Purtell <ap...@apache.org> wrote:

> I don't think that's a digression from the discussion. I also think
> frequent integration tests will be useful whether RTC or CTR, just as a
> culture of code reviews is useful whether RTC or CTR.
>
> We could also look at resurrecting the static checking work. (Well, I guess
> that would be on me, HBASE-13365, etc.) When I was working on HBASE-13825
> it occurred to me it should be easy to make a static check for invocation
> of a builder's mergeFrom or mergeDelimitedFrom, and make that a warning.
> This is one of those numerous details code reviewers should have in mind
> but won't manage it consistently because they change over time and vary by
> code line.
>
>
>
> On Mon, Aug 10, 2015 at 9:53 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
>
> > Sorry to digress from the title main topic but is the concern is about
> > quality of the system as code enters the system or about trust?
> >
> > I lean towards "trust but verify".  I'd actually argue that we should
> have
> > more infrastructure in place to catch the bugs beyond just our unit test
> > infrastructure.  Ideally our integration tests would be run frequently
> > upstream on our mainline branches.   With those in place we could find
> > quality issues sooner rather than later.  With integration test run infra
> > in place we'd have a chance at catching errors even if unit tests are
> > lacking.
> >
> > Having more of this in place would help improve quality regardless of
> > which CTR or RTC schema we use because we'll have evidence of how much we
> > should trust and the quality of code based on execution.
> > Without that I think the current RTC on release branches and trunk, and
> CTR
> > for feature branches scheme the enforces "trust but verify".
> >
> > Jon.
> >
> > On Sun, Aug 2, 2015 at 11:59 AM, Andrew Purtell <
> andrew.purtell@gmail.com>
> > wrote:
> >
> > > Agreed, committers should be spending more time reviewing others' work.
> > We
> > > can ask. It may happen. It may work for a short while. It may not.
> Shrug.
> > > People will do what they want.
> > >
> > > I'm looking to make one substantial change that will allow committers
> to
> > > make progress even if there's nobody else around or interested for one
> > > week. It happens sometimes. I've already talked about my concerns on
> > > assuming a certain level of available volunteer bandwidth. Let me just
> > say
> > > that things are great now, it's fantastic.
> > >
> > > We are pretty much on the honor system already. I don't buy the
> argument
> > > we can't trust that CTR or CTR after one week can work because
> > committers,
> > > even if asked to customarily get a review before commit, may decide to
> > > check in unreviewed untested destabilizing changes. At least, In that
> > case
> > > I'd argue we have a different problem. If you go back to my original
> > mail,
> > > I do say we shouldn't undertake any change if we as PMC are unwilling
> to
> > > revoke the commit bit from committers who sadly demonstrate themselves
> > > unworthy of trust through checkins of toxic waste without review. Merit
> > > _can_ be un-earned. Commits can be reverted. Just because something is
> > > checked in does not mean it will go out in a release. We could put a
> > > nightly suite of regression tests in place. Both proposals I have made,
> > > especially the latter, are not a binary release of any attempt at
> quality
> > > control. We would have all of the consensus expectations of good
> > committer
> > > behavior still in place.
> > >
> > > Let's assume someone checks in something without getting a review
> today.
> > > What would happen? Someone else would revert it and we'd have a
> > discussion.
> > > If we were operating under CTR-after-one-week (or plain CTR) but with
> > > documented expectations that someone get a review first, what changes,
> in
> > > terms of quality control and project discipline? Maybe the difference
> is
> > we
> > > could more easily justify revoking commit privileges? Maybe. I think
> > every
> > > discussion like that would be strictly case by case though, a
> > conversation
> > > between that person and the PMC, justification for revoking committer
> > > status wouldn't be rule based.
> > >
> > > What definitely would change is my well tested good faith change
> waiting
> > > for review after one week could go in no matter who's on vacation or
> > > whatever. Not saying this is a problem for me today. Like I said,
> things
> > > are great around here today. Awesome.
> > >
> > >
> > > > On Aug 2, 2015, at 10:52 AM, Stack <st...@duboce.net> wrote:
> > > >
> > > > On Fri, Jul 31, 2015 at 3:15 AM, Andrew Purtell <
> > > andrew.purtell@gmail.com>
> > > > wrote:
> > > >
> > > >> I appreciate very much the earlier feedback about switching from RTC
> > to
> > > >> CTR. It helped me think about the essential thing I was after.
> > > >>
> > > >> I'm thinking of making a formal proposal to adopt this, with a VOTE:
> > > >>
> > > >>> After posting a patch to JIRA, after one week if there is no review
> > or
> > > >> veto, a committer can commit their own work.
> > > >>
> > > >> (On a bit of a lag...)
> > > >
> > > > What is the problem we are trying to solve? Blocked committers unable
> > to
> > > > get a review?
> > > >
> > > > Any commit can destabilize. Commits without peer review will
> > destabilize
> > > > more than those that have been reviewed. As a community, I'd think
> that
> > > > anything we can do to act against entropy in the system would be
> > primary
> > > > above all other considerations especially given our project a mature
> > > > datastore.
> > > >
> > > > In my own case, it seems to take me tens of patch postings to arrive
> at
> > > > something a fellow reviewer thinks worthy of commit when working
> with a
> > > > peer on an issue of substance. Often the final patch is radically
> > > different
> > > > from first posting. If auto-commit, the first version would just go
> in.
> > > > Unless review, I'd be 'done'. If review, each iteration would be in
> the
> > > > code base? Churn would go up. Commits would be more and smaller. Our
> > > > project would be made of less coherent 'change sets'.
> > > >
> > > > I'd be with LarsH suggestion that committers should be allowed go
> ahead
> > > > with one-liners or more substantial, non-controversial changes in
> site
> > or
> > > > doc but beyond this, I'd be against auto-commit.
> > > >
> > > >>
> > > >>
> > > >> Also, looking at https://hbase.apache.org/book.html#_decisions, I
> > don't
> > > >> think the "patch +1 policy" should remain because the trial OWNERS
> > > concept
> > > >> hasn't worked out, IMHO. The OWNERS concept requires a set of
> > constantly
> > > >> present and engaged owners, a resource demand that's hard to square
> > with
> > > >> the volunteer nature of our community. The amount of time any
> > committer
> > > or
> > > >> PMC member has on this project is highly variable day to day and
> week
> > to
> > > >> week.  I'm also thinking of calling a VOTE to significantly revise
> or
> > > >> strike this section.
> > > > Yeah. OWNERS project failed. Its ghost is in effect in that we each
> go
> > to
> > > > the domain expert when its clear (e.g. Matteo or Stephen on pv2).
> > > >
> > > >
> > > >> Both of these things have a common root: Volunteer time is a very
> > > precious
> > > >> commodity. Our community's supply of volunteer time fluctuates. I
> > would
> > > >> like to see committers be able to make progress with their own work
> > > even in
> > > >> periods when volunteer time is in very short supply, or when they
> are
> > > >> working on niche concerns that simply do not draw sufficient
> interest
> > > from
> > > >> other committers. (This is different from work that people think
> isn't
> > > >> appropriate - in that case ignoring it so it will go away would no
> > > longer
> > > >> be an option, a veto would be required if you want to stop
> something.)
> > > > Committers should be spending more time reviewing their peers work
> (and
> > > > just as importantly, the work of contributors)?
> > > > St.Ack
> > > >
> > > >
> > > >
> > > >> On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <
> > > andrew.purtell@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> Had this thought after getting back on the road. As an alternative
> to
> > > any
> > > >>> sweeping change we could do one incremental but very significant
> > thing
> > > >> that
> > > >>> acknowledges our status as trusted and busy peers: After posting a
> > > patch
> > > >> to
> > > >>> JIRA, after one week if there is no review or veto, a committer can
> > > >> commit
> > > >>> their own work.
> > > >>>
> > > >>>
> > > >>>>> On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <
> olorinbant@gmail.com
> > >
> > > >>>> wrote:
> > > >>>>
> > > >>>> Just curious, I assume if this change is made, would it only apply
> > to
> > > >>>> master branch?
> > > >>>>
> > > >>>> -Mikhail
> > > >>>>
> > > >>>> On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
> > > >>>> <an...@gmail.com> wrote:
> > > >>>>> @dev is now CCed
> > > >>>>>
> > > >>>>> I didn't want to over structure the discussion with too much
> detail
> > > up
> > > >>> front. I do think CTR without supporting process or boundaries can
> be
> > > >> more
> > > >>> problematic than helpful. That could take the form of customarily
> > > waiting
> > > >>> for reviews before commit even under a CTR regime. I think this
> > > >> discussion
> > > >>> has been great so far. I would also add that CTR moves 'R' from a
> > > gating
> > > >>> requirement per commit (which can be hard to overcome for niche
> areas
> > > or
> > > >>> when volunteer resources are less available) more to RMs. will be
> > back
> > > >>> later with more.
> > > >>>>>
> > > >>>>>
> > > >>>>>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <sean.busbey@gmail.com
> >
> > > >>> wrote:
> > > >>>>>>
> > > >>>>>> I'd also favor having this discussion on dev@.
> > > >>>>>>
> > > >>>>>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <
> > > ghelmling@gmail.com
> > > >>>
> > > >>> wrote:
> > > >>>>>>>
> > > >>>>>>> This is already a really interesting and meaningful discussion,
> > and
> > > >> is
> > > >>>>>>> obviously important to the community.
> > > >>>>>>>
> > > >>>>>>> Is there any reason not to move this straight to the dev@
> list?
> > > >>>>>>>
> > > >>>>>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <
> todd@cloudera.com
> > >
> > > >>> wrote:
> > > >>>>>>>>
> > > >>>>>>>> I'm not very active in HBase these days, so I won't cast a
> > > non-zero
> > > >>> vote,
> > > >>>>>>>> but I'm -0 on this idea, for basically two reasons:
> > > >>>>>>>>
> > > >>>>>>>> 1) In my experience at a past job which used CTR, the reality
> > was
> > > >>> that it
> > > >>>>>>>> was more like "Commit and maybe review" rather than "Commit
> then
> > > >>> review".
> > > >>>>>>>> It's always more fun (and often easier) to write new code than
> > to
> > > >>> review
> > > >>>>>>>> other people's code, so if there isn't a requirement that all
> > code
> > > >>> gets
> > > >>>>>>>> reviewed before commit, it's easy for the ratio of code
> written
> > to
> > > >>> code
> > > >>>>>>>> reviewed to tend towards values significantly greater than 1:1
> > > over
> > > >>> time.
> > > >>>>>>>> At my past job, this meant that a lot of code made it into
> > > >>> production (it
> > > >>>>>>>> was a website) that hadn't ever been reviewed, and in many
> cases
> > > we
> > > >>> found
> > > >>>>>>>> bugs (or other issues) that would have definitely been caught
> > by a
> > > >>> good
> > > >>>>>>>> code reviewer.
> > > >>>>>>>>
> > > >>>>>>>> 2) CTR has both the advantage and disadvantage of allowing
> areas
> > > of
> > > >>> code
> > > >>>>>>> to
> > > >>>>>>>> be evolved quickly by a single person. That could be seen as a
> > > >> plus,
> > > >>> in
> > > >>>>>>>> that there are some areas which tend to get ignored because we
> > > >> don't
> > > >>>>>>> have a
> > > >>>>>>>> critical mass of people reviewing patches in the area --
> patches
> > > >>> languish
> > > >>>>>>>> in these areas currently. However, that could also be seen as
> a
> > > >> good
> > > >>>>>>> thing
> > > >>>>>>>> from a "community over code" perspective -- it's better to not
> > > have
> > > >>> any
> > > >>>>>>>> areas of code with bus-factor 1. Feeling the pain of not
> getting
> > > >>> reviews
> > > >>>>>>> in
> > > >>>>>>>> these areas with only a single active committer encourages us
> to
> > > >> find
> > > >>>>>>>> solutions - either by deprecating "niche" features (as we once
> > did
> > > >>> with
> > > >>>>>>>> various 'contrib' projects) or by recruiting new committers
> who
> > > >> have
> > > >>>>>>>> interest in maintaining that code area. Lifting review
> > > restrictions
> > > >>> would
> > > >>>>>>>> allow us to sweep bus-factor issues under the rug.
> > > >>>>>>>>
> > > >>>>>>>> That said, I think CTR can be a valuable tool for "move fast
> on
> > > new
> > > >>>>>>>> experimentation" type projects -- I've participated in several
> > > >>> feature
> > > >>>>>>>> branches in HDFS where we operated on CTR on the branch. The
> > > >>> assumption
> > > >>>>>>> was
> > > >>>>>>>> that, prior to merging the branch into trunk, all of the
> patches
> > > >> (or
> > > >>> a
> > > >>>>>>>> consolidated mega-patch) would be thoroughly reviewed by
> several
> > > >>> other
> > > >>>>>>>> committers. I found this to work very well during early
> > > >> development,
> > > >>>>>>> since
> > > >>>>>>>> I could hack on things and even commit pieces of code that had
> > > >> known
> > > >>>>>>>> issues/TODOs. For trickier patches on the CTR branch, I still
> > > >> tended
> > > >>> to
> > > >>>>>>>> ping experienced contributors for their opinions and feedback
> > > >> before
> > > >>>>>>>> committing.
> > > >>>>>>>>
> > > >>>>>>>> Perhaps such a hybrid policy would work well in the context of
> > > >> HBase
> > > >>>>>>>> feature development as well?
> > > >>>>>>>>
> > > >>>>>>>> -Todd
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
> > > >>> apurtell@apache.org>
> > > >>>>>>>> wrote:
> > > >>>>>>>>
> > > >>>>>>>>> Just thought of branch merge votes. Sorry for that omission.
> I
> > > >>> think it
> > > >>>>>>>>> makes sense to keep those, but let's discuss that too in this
> > > >>> context.
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
> > > >>> apurtell@apache.org>
> > > >>>>>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> As Greg Stein said on a thread over at general@incubator
> > > >>>>>>>> ("[DISCUSSION]
> > > >>>>>>>>>> Graduate Ignite from the Apache Incubator"):
> > > >>>>>>>>>>
> > > >>>>>>>>>> I always translate RTC as "we don't trust you, so somebody
> > else
> > > >>> must
> > > >>>>>>>>>> approve anything you do." To me, that is a lousy basis for
> > > >>> creating a
> > > >>>>>>>>>> community. Trust and peer respect should be the basis, which
> > > >>> implies
> > > >>>>>>>> CTR.
> > > >>>>>>>>>>
> > > >>>>>>>>>> I happen to agree with this sentiment and would like to
> > propose
> > > >>>>>>>> switching
> > > >>>>>>>>>> our consensus on commit procedure from RTC to CTR.
> > Concurrently,
> > > >> I
> > > >>>>>>>> would
> > > >>>>>>>>>> like to propose this consensus also include the following:
> > > >>>>>>>>>> - Checkins should pass precommit or the committer should
> > explain
> > > >> on
> > > >>>>>>> the
> > > >>>>>>>>>> JIRA why precommit failures are in their best judgement not
> > > >> related
> > > >>>>>>>>>> - The PMC should be willing to, ultimately, revoke
> > committership
> > > >>>>>>> should
> > > >>>>>>>>>> trust in any individual committer be discovered to be
> > misplaced.
> > > >> I
> > > >>>>>>>> would
> > > >>>>>>>>>> expect such a last resort to be exceedingly rare and likely
> > > never
> > > >>> to
> > > >>>>>>>>> happen
> > > >>>>>>>>>> because of course long before that we would be setting
> correct
> > > >>> public
> > > >>>>>>>>>> examples in the first place and respectfully counseling well
> > > >>>>>>>> intentioned
> > > >>>>>>>>>> committers who might stray.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Depending on how the conversation proceeds here I would like
> > to
> > > >>>>>>> include
> > > >>>>>>>>>> dev@ in this conversation at the earliest opportunity as
> > well.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Thoughts?
> > > >>>>>>>>>>
> > > >>>>>>>>>> --
> > > >>>>>>>>>> Best regards,
> > > >>>>>>>>>>
> > > >>>>>>>>>> - Andy
> > > >>>>>>>>>>
> > > >>>>>>>>>> Problems worthy of attack prove their worth by hitting
> back. -
> > > >> Piet
> > > >>>>>>>> Hein
> > > >>>>>>>>>> (via Tom White)
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>>>>>> --
> > > >>>>>>>>> Best regards,
> > > >>>>>>>>>
> > > >>>>>>>>> - Andy
> > > >>>>>>>>>
> > > >>>>>>>>> Problems worthy of attack prove their worth by hitting back.
> -
> > > >> Piet
> > > >>>>>>> Hein
> > > >>>>>>>>> (via Tom White)
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> --
> > > >>>>>>>> Todd Lipcon
> > > >>>>>>>> Software Engineer, Cloudera
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> --
> > > >>>>>> Sean
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> --
> > > >>>> Thanks,
> > > >>>> Michael Antonov
> > > >>
> > >
> >
> >
> >
> > --
> > // Jonathan Hsieh (shay)
> > // HBase Tech Lead, Software Engineer, Cloudera
> > // jon@cloudera.com // @jmhsieh
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Re: [DISCUSSION] Switching from RTC to CTR

[Andrew Purtell <ap...@apache.org>]

I don't think that's a digression from the discussion. I also think
frequent integration tests will be useful whether RTC or CTR, just as a
culture of code reviews is useful whether RTC or CTR.

We could also look at resurrecting the static checking work. (Well, I guess
that would be on me, HBASE-13365, etc.) When I was working on HBASE-13825
it occurred to me it should be easy to make a static check for invocation
of a builder's mergeFrom or mergeDelimitedFrom, and make that a warning.
This is one of those numerous details code reviewers should have in mind
but won't manage it consistently because they change over time and vary by
code line.



On Mon, Aug 10, 2015 at 9:53 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:

> Sorry to digress from the title main topic but is the concern is about
> quality of the system as code enters the system or about trust?
>
> I lean towards "trust but verify".  I'd actually argue that we should have
> more infrastructure in place to catch the bugs beyond just our unit test
> infrastructure.  Ideally our integration tests would be run frequently
> upstream on our mainline branches.   With those in place we could find
> quality issues sooner rather than later.  With integration test run infra
> in place we'd have a chance at catching errors even if unit tests are
> lacking.
>
> Having more of this in place would help improve quality regardless of
> which CTR or RTC schema we use because we'll have evidence of how much we
> should trust and the quality of code based on execution.
> Without that I think the current RTC on release branches and trunk, and CTR
> for feature branches scheme the enforces "trust but verify".
>
> Jon.
>
> On Sun, Aug 2, 2015 at 11:59 AM, Andrew Purtell <an...@gmail.com>
> wrote:
>
> > Agreed, committers should be spending more time reviewing others' work.
> We
> > can ask. It may happen. It may work for a short while. It may not. Shrug.
> > People will do what they want.
> >
> > I'm looking to make one substantial change that will allow committers to
> > make progress even if there's nobody else around or interested for one
> > week. It happens sometimes. I've already talked about my concerns on
> > assuming a certain level of available volunteer bandwidth. Let me just
> say
> > that things are great now, it's fantastic.
> >
> > We are pretty much on the honor system already. I don't buy the argument
> > we can't trust that CTR or CTR after one week can work because
> committers,
> > even if asked to customarily get a review before commit, may decide to
> > check in unreviewed untested destabilizing changes. At least, In that
> case
> > I'd argue we have a different problem. If you go back to my original
> mail,
> > I do say we shouldn't undertake any change if we as PMC are unwilling to
> > revoke the commit bit from committers who sadly demonstrate themselves
> > unworthy of trust through checkins of toxic waste without review. Merit
> > _can_ be un-earned. Commits can be reverted. Just because something is
> > checked in does not mean it will go out in a release. We could put a
> > nightly suite of regression tests in place. Both proposals I have made,
> > especially the latter, are not a binary release of any attempt at quality
> > control. We would have all of the consensus expectations of good
> committer
> > behavior still in place.
> >
> > Let's assume someone checks in something without getting a review today.
> > What would happen? Someone else would revert it and we'd have a
> discussion.
> > If we were operating under CTR-after-one-week (or plain CTR) but with
> > documented expectations that someone get a review first, what changes, in
> > terms of quality control and project discipline? Maybe the difference is
> we
> > could more easily justify revoking commit privileges? Maybe. I think
> every
> > discussion like that would be strictly case by case though, a
> conversation
> > between that person and the PMC, justification for revoking committer
> > status wouldn't be rule based.
> >
> > What definitely would change is my well tested good faith change waiting
> > for review after one week could go in no matter who's on vacation or
> > whatever. Not saying this is a problem for me today. Like I said, things
> > are great around here today. Awesome.
> >
> >
> > > On Aug 2, 2015, at 10:52 AM, Stack <st...@duboce.net> wrote:
> > >
> > > On Fri, Jul 31, 2015 at 3:15 AM, Andrew Purtell <
> > andrew.purtell@gmail.com>
> > > wrote:
> > >
> > >> I appreciate very much the earlier feedback about switching from RTC
> to
> > >> CTR. It helped me think about the essential thing I was after.
> > >>
> > >> I'm thinking of making a formal proposal to adopt this, with a VOTE:
> > >>
> > >>> After posting a patch to JIRA, after one week if there is no review
> or
> > >> veto, a committer can commit their own work.
> > >>
> > >> (On a bit of a lag...)
> > >
> > > What is the problem we are trying to solve? Blocked committers unable
> to
> > > get a review?
> > >
> > > Any commit can destabilize. Commits without peer review will
> destabilize
> > > more than those that have been reviewed. As a community, I'd think that
> > > anything we can do to act against entropy in the system would be
> primary
> > > above all other considerations especially given our project a mature
> > > datastore.
> > >
> > > In my own case, it seems to take me tens of patch postings to arrive at
> > > something a fellow reviewer thinks worthy of commit when working with a
> > > peer on an issue of substance. Often the final patch is radically
> > different
> > > from first posting. If auto-commit, the first version would just go in.
> > > Unless review, I'd be 'done'. If review, each iteration would be in the
> > > code base? Churn would go up. Commits would be more and smaller. Our
> > > project would be made of less coherent 'change sets'.
> > >
> > > I'd be with LarsH suggestion that committers should be allowed go ahead
> > > with one-liners or more substantial, non-controversial changes in site
> or
> > > doc but beyond this, I'd be against auto-commit.
> > >
> > >>
> > >>
> > >> Also, looking at https://hbase.apache.org/book.html#_decisions, I
> don't
> > >> think the "patch +1 policy" should remain because the trial OWNERS
> > concept
> > >> hasn't worked out, IMHO. The OWNERS concept requires a set of
> constantly
> > >> present and engaged owners, a resource demand that's hard to square
> with
> > >> the volunteer nature of our community. The amount of time any
> committer
> > or
> > >> PMC member has on this project is highly variable day to day and week
> to
> > >> week.  I'm also thinking of calling a VOTE to significantly revise or
> > >> strike this section.
> > > Yeah. OWNERS project failed. Its ghost is in effect in that we each go
> to
> > > the domain expert when its clear (e.g. Matteo or Stephen on pv2).
> > >
> > >
> > >> Both of these things have a common root: Volunteer time is a very
> > precious
> > >> commodity. Our community's supply of volunteer time fluctuates. I
> would
> > >> like to see committers be able to make progress with their own work
> > even in
> > >> periods when volunteer time is in very short supply, or when they are
> > >> working on niche concerns that simply do not draw sufficient interest
> > from
> > >> other committers. (This is different from work that people think isn't
> > >> appropriate - in that case ignoring it so it will go away would no
> > longer
> > >> be an option, a veto would be required if you want to stop something.)
> > > Committers should be spending more time reviewing their peers work (and
> > > just as importantly, the work of contributors)?
> > > St.Ack
> > >
> > >
> > >
> > >> On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <
> > andrew.purtell@gmail.com>
> > >> wrote:
> > >>
> > >>> Had this thought after getting back on the road. As an alternative to
> > any
> > >>> sweeping change we could do one incremental but very significant
> thing
> > >> that
> > >>> acknowledges our status as trusted and busy peers: After posting a
> > patch
> > >> to
> > >>> JIRA, after one week if there is no review or veto, a committer can
> > >> commit
> > >>> their own work.
> > >>>
> > >>>
> > >>>>> On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <olorinbant@gmail.com
> >
> > >>>> wrote:
> > >>>>
> > >>>> Just curious, I assume if this change is made, would it only apply
> to
> > >>>> master branch?
> > >>>>
> > >>>> -Mikhail
> > >>>>
> > >>>> On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
> > >>>> <an...@gmail.com> wrote:
> > >>>>> @dev is now CCed
> > >>>>>
> > >>>>> I didn't want to over structure the discussion with too much detail
> > up
> > >>> front. I do think CTR without supporting process or boundaries can be
> > >> more
> > >>> problematic than helpful. That could take the form of customarily
> > waiting
> > >>> for reviews before commit even under a CTR regime. I think this
> > >> discussion
> > >>> has been great so far. I would also add that CTR moves 'R' from a
> > gating
> > >>> requirement per commit (which can be hard to overcome for niche areas
> > or
> > >>> when volunteer resources are less available) more to RMs. will be
> back
> > >>> later with more.
> > >>>>>
> > >>>>>
> > >>>>>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com>
> > >>> wrote:
> > >>>>>>
> > >>>>>> I'd also favor having this discussion on dev@.
> > >>>>>>
> > >>>>>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <
> > ghelmling@gmail.com
> > >>>
> > >>> wrote:
> > >>>>>>>
> > >>>>>>> This is already a really interesting and meaningful discussion,
> and
> > >> is
> > >>>>>>> obviously important to the community.
> > >>>>>>>
> > >>>>>>> Is there any reason not to move this straight to the dev@ list?
> > >>>>>>>
> > >>>>>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <todd@cloudera.com
> >
> > >>> wrote:
> > >>>>>>>>
> > >>>>>>>> I'm not very active in HBase these days, so I won't cast a
> > non-zero
> > >>> vote,
> > >>>>>>>> but I'm -0 on this idea, for basically two reasons:
> > >>>>>>>>
> > >>>>>>>> 1) In my experience at a past job which used CTR, the reality
> was
> > >>> that it
> > >>>>>>>> was more like "Commit and maybe review" rather than "Commit then
> > >>> review".
> > >>>>>>>> It's always more fun (and often easier) to write new code than
> to
> > >>> review
> > >>>>>>>> other people's code, so if there isn't a requirement that all
> code
> > >>> gets
> > >>>>>>>> reviewed before commit, it's easy for the ratio of code written
> to
> > >>> code
> > >>>>>>>> reviewed to tend towards values significantly greater than 1:1
> > over
> > >>> time.
> > >>>>>>>> At my past job, this meant that a lot of code made it into
> > >>> production (it
> > >>>>>>>> was a website) that hadn't ever been reviewed, and in many cases
> > we
> > >>> found
> > >>>>>>>> bugs (or other issues) that would have definitely been caught
> by a
> > >>> good
> > >>>>>>>> code reviewer.
> > >>>>>>>>
> > >>>>>>>> 2) CTR has both the advantage and disadvantage of allowing areas
> > of
> > >>> code
> > >>>>>>> to
> > >>>>>>>> be evolved quickly by a single person. That could be seen as a
> > >> plus,
> > >>> in
> > >>>>>>>> that there are some areas which tend to get ignored because we
> > >> don't
> > >>>>>>> have a
> > >>>>>>>> critical mass of people reviewing patches in the area -- patches
> > >>> languish
> > >>>>>>>> in these areas currently. However, that could also be seen as a
> > >> good
> > >>>>>>> thing
> > >>>>>>>> from a "community over code" perspective -- it's better to not
> > have
> > >>> any
> > >>>>>>>> areas of code with bus-factor 1. Feeling the pain of not getting
> > >>> reviews
> > >>>>>>> in
> > >>>>>>>> these areas with only a single active committer encourages us to
> > >> find
> > >>>>>>>> solutions - either by deprecating "niche" features (as we once
> did
> > >>> with
> > >>>>>>>> various 'contrib' projects) or by recruiting new committers who
> > >> have
> > >>>>>>>> interest in maintaining that code area. Lifting review
> > restrictions
> > >>> would
> > >>>>>>>> allow us to sweep bus-factor issues under the rug.
> > >>>>>>>>
> > >>>>>>>> That said, I think CTR can be a valuable tool for "move fast on
> > new
> > >>>>>>>> experimentation" type projects -- I've participated in several
> > >>> feature
> > >>>>>>>> branches in HDFS where we operated on CTR on the branch. The
> > >>> assumption
> > >>>>>>> was
> > >>>>>>>> that, prior to merging the branch into trunk, all of the patches
> > >> (or
> > >>> a
> > >>>>>>>> consolidated mega-patch) would be thoroughly reviewed by several
> > >>> other
> > >>>>>>>> committers. I found this to work very well during early
> > >> development,
> > >>>>>>> since
> > >>>>>>>> I could hack on things and even commit pieces of code that had
> > >> known
> > >>>>>>>> issues/TODOs. For trickier patches on the CTR branch, I still
> > >> tended
> > >>> to
> > >>>>>>>> ping experienced contributors for their opinions and feedback
> > >> before
> > >>>>>>>> committing.
> > >>>>>>>>
> > >>>>>>>> Perhaps such a hybrid policy would work well in the context of
> > >> HBase
> > >>>>>>>> feature development as well?
> > >>>>>>>>
> > >>>>>>>> -Todd
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
> > >>> apurtell@apache.org>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Just thought of branch merge votes. Sorry for that omission. I
> > >>> think it
> > >>>>>>>>> makes sense to keep those, but let's discuss that too in this
> > >>> context.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
> > >>> apurtell@apache.org>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> As Greg Stein said on a thread over at general@incubator
> > >>>>>>>> ("[DISCUSSION]
> > >>>>>>>>>> Graduate Ignite from the Apache Incubator"):
> > >>>>>>>>>>
> > >>>>>>>>>> I always translate RTC as "we don't trust you, so somebody
> else
> > >>> must
> > >>>>>>>>>> approve anything you do." To me, that is a lousy basis for
> > >>> creating a
> > >>>>>>>>>> community. Trust and peer respect should be the basis, which
> > >>> implies
> > >>>>>>>> CTR.
> > >>>>>>>>>>
> > >>>>>>>>>> I happen to agree with this sentiment and would like to
> propose
> > >>>>>>>> switching
> > >>>>>>>>>> our consensus on commit procedure from RTC to CTR.
> Concurrently,
> > >> I
> > >>>>>>>> would
> > >>>>>>>>>> like to propose this consensus also include the following:
> > >>>>>>>>>> - Checkins should pass precommit or the committer should
> explain
> > >> on
> > >>>>>>> the
> > >>>>>>>>>> JIRA why precommit failures are in their best judgement not
> > >> related
> > >>>>>>>>>> - The PMC should be willing to, ultimately, revoke
> committership
> > >>>>>>> should
> > >>>>>>>>>> trust in any individual committer be discovered to be
> misplaced.
> > >> I
> > >>>>>>>> would
> > >>>>>>>>>> expect such a last resort to be exceedingly rare and likely
> > never
> > >>> to
> > >>>>>>>>> happen
> > >>>>>>>>>> because of course long before that we would be setting correct
> > >>> public
> > >>>>>>>>>> examples in the first place and respectfully counseling well
> > >>>>>>>> intentioned
> > >>>>>>>>>> committers who might stray.
> > >>>>>>>>>>
> > >>>>>>>>>> Depending on how the conversation proceeds here I would like
> to
> > >>>>>>> include
> > >>>>>>>>>> dev@ in this conversation at the earliest opportunity as
> well.
> > >>>>>>>>>>
> > >>>>>>>>>> Thoughts?
> > >>>>>>>>>>
> > >>>>>>>>>> --
> > >>>>>>>>>> Best regards,
> > >>>>>>>>>>
> > >>>>>>>>>> - Andy
> > >>>>>>>>>>
> > >>>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
> > >> Piet
> > >>>>>>>> Hein
> > >>>>>>>>>> (via Tom White)
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> --
> > >>>>>>>>> Best regards,
> > >>>>>>>>>
> > >>>>>>>>> - Andy
> > >>>>>>>>>
> > >>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
> > >> Piet
> > >>>>>>> Hein
> > >>>>>>>>> (via Tom White)
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> Todd Lipcon
> > >>>>>>>> Software Engineer, Cloudera
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Sean
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> Thanks,
> > >>>> Michael Antonov
> > >>
> >
>
>
>
> --
> // Jonathan Hsieh (shay)
> // HBase Tech Lead, Software Engineer, Cloudera
> // jon@cloudera.com // @jmhsieh
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Re: [DISCUSSION] Switching from RTC to CTR

[Jonathan Hsieh <jo...@cloudera.com>]

Sorry to digress from the title main topic but is the concern is about
quality of the system as code enters the system or about trust?

I lean towards "trust but verify".  I'd actually argue that we should have
more infrastructure in place to catch the bugs beyond just our unit test
infrastructure.  Ideally our integration tests would be run frequently
upstream on our mainline branches.   With those in place we could find
quality issues sooner rather than later.  With integration test run infra
in place we'd have a chance at catching errors even if unit tests are
lacking.

Having more of this in place would help improve quality regardless of
which CTR or RTC schema we use because we'll have evidence of how much we
should trust and the quality of code based on execution.
Without that I think the current RTC on release branches and trunk, and CTR
for feature branches scheme the enforces "trust but verify".

Jon.

On Sun, Aug 2, 2015 at 11:59 AM, Andrew Purtell <an...@gmail.com>
wrote:

> Agreed, committers should be spending more time reviewing others' work. We
> can ask. It may happen. It may work for a short while. It may not. Shrug.
> People will do what they want.
>
> I'm looking to make one substantial change that will allow committers to
> make progress even if there's nobody else around or interested for one
> week. It happens sometimes. I've already talked about my concerns on
> assuming a certain level of available volunteer bandwidth. Let me just say
> that things are great now, it's fantastic.
>
> We are pretty much on the honor system already. I don't buy the argument
> we can't trust that CTR or CTR after one week can work because committers,
> even if asked to customarily get a review before commit, may decide to
> check in unreviewed untested destabilizing changes. At least, In that case
> I'd argue we have a different problem. If you go back to my original mail,
> I do say we shouldn't undertake any change if we as PMC are unwilling to
> revoke the commit bit from committers who sadly demonstrate themselves
> unworthy of trust through checkins of toxic waste without review. Merit
> _can_ be un-earned. Commits can be reverted. Just because something is
> checked in does not mean it will go out in a release. We could put a
> nightly suite of regression tests in place. Both proposals I have made,
> especially the latter, are not a binary release of any attempt at quality
> control. We would have all of the consensus expectations of good committer
> behavior still in place.
>
> Let's assume someone checks in something without getting a review today.
> What would happen? Someone else would revert it and we'd have a discussion.
> If we were operating under CTR-after-one-week (or plain CTR) but with
> documented expectations that someone get a review first, what changes, in
> terms of quality control and project discipline? Maybe the difference is we
> could more easily justify revoking commit privileges? Maybe. I think every
> discussion like that would be strictly case by case though, a conversation
> between that person and the PMC, justification for revoking committer
> status wouldn't be rule based.
>
> What definitely would change is my well tested good faith change waiting
> for review after one week could go in no matter who's on vacation or
> whatever. Not saying this is a problem for me today. Like I said, things
> are great around here today. Awesome.
>
>
> > On Aug 2, 2015, at 10:52 AM, Stack <st...@duboce.net> wrote:
> >
> > On Fri, Jul 31, 2015 at 3:15 AM, Andrew Purtell <
> andrew.purtell@gmail.com>
> > wrote:
> >
> >> I appreciate very much the earlier feedback about switching from RTC to
> >> CTR. It helped me think about the essential thing I was after.
> >>
> >> I'm thinking of making a formal proposal to adopt this, with a VOTE:
> >>
> >>> After posting a patch to JIRA, after one week if there is no review or
> >> veto, a committer can commit their own work.
> >>
> >> (On a bit of a lag...)
> >
> > What is the problem we are trying to solve? Blocked committers unable to
> > get a review?
> >
> > Any commit can destabilize. Commits without peer review will destabilize
> > more than those that have been reviewed. As a community, I'd think that
> > anything we can do to act against entropy in the system would be primary
> > above all other considerations especially given our project a mature
> > datastore.
> >
> > In my own case, it seems to take me tens of patch postings to arrive at
> > something a fellow reviewer thinks worthy of commit when working with a
> > peer on an issue of substance. Often the final patch is radically
> different
> > from first posting. If auto-commit, the first version would just go in.
> > Unless review, I'd be 'done'. If review, each iteration would be in the
> > code base? Churn would go up. Commits would be more and smaller. Our
> > project would be made of less coherent 'change sets'.
> >
> > I'd be with LarsH suggestion that committers should be allowed go ahead
> > with one-liners or more substantial, non-controversial changes in site or
> > doc but beyond this, I'd be against auto-commit.
> >
> >>
> >>
> >> Also, looking at https://hbase.apache.org/book.html#_decisions, I don't
> >> think the "patch +1 policy" should remain because the trial OWNERS
> concept
> >> hasn't worked out, IMHO. The OWNERS concept requires a set of constantly
> >> present and engaged owners, a resource demand that's hard to square with
> >> the volunteer nature of our community. The amount of time any committer
> or
> >> PMC member has on this project is highly variable day to day and week to
> >> week.  I'm also thinking of calling a VOTE to significantly revise or
> >> strike this section.
> > Yeah. OWNERS project failed. Its ghost is in effect in that we each go to
> > the domain expert when its clear (e.g. Matteo or Stephen on pv2).
> >
> >
> >> Both of these things have a common root: Volunteer time is a very
> precious
> >> commodity. Our community's supply of volunteer time fluctuates. I would
> >> like to see committers be able to make progress with their own work
> even in
> >> periods when volunteer time is in very short supply, or when they are
> >> working on niche concerns that simply do not draw sufficient interest
> from
> >> other committers. (This is different from work that people think isn't
> >> appropriate - in that case ignoring it so it will go away would no
> longer
> >> be an option, a veto would be required if you want to stop something.)
> > Committers should be spending more time reviewing their peers work (and
> > just as importantly, the work of contributors)?
> > St.Ack
> >
> >
> >
> >> On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <
> andrew.purtell@gmail.com>
> >> wrote:
> >>
> >>> Had this thought after getting back on the road. As an alternative to
> any
> >>> sweeping change we could do one incremental but very significant thing
> >> that
> >>> acknowledges our status as trusted and busy peers: After posting a
> patch
> >> to
> >>> JIRA, after one week if there is no review or veto, a committer can
> >> commit
> >>> their own work.
> >>>
> >>>
> >>>>> On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <ol...@gmail.com>
> >>>> wrote:
> >>>>
> >>>> Just curious, I assume if this change is made, would it only apply to
> >>>> master branch?
> >>>>
> >>>> -Mikhail
> >>>>
> >>>> On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
> >>>> <an...@gmail.com> wrote:
> >>>>> @dev is now CCed
> >>>>>
> >>>>> I didn't want to over structure the discussion with too much detail
> up
> >>> front. I do think CTR without supporting process or boundaries can be
> >> more
> >>> problematic than helpful. That could take the form of customarily
> waiting
> >>> for reviews before commit even under a CTR regime. I think this
> >> discussion
> >>> has been great so far. I would also add that CTR moves 'R' from a
> gating
> >>> requirement per commit (which can be hard to overcome for niche areas
> or
> >>> when volunteer resources are less available) more to RMs. will be back
> >>> later with more.
> >>>>>
> >>>>>
> >>>>>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com>
> >>> wrote:
> >>>>>>
> >>>>>> I'd also favor having this discussion on dev@.
> >>>>>>
> >>>>>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <
> ghelmling@gmail.com
> >>>
> >>> wrote:
> >>>>>>>
> >>>>>>> This is already a really interesting and meaningful discussion, and
> >> is
> >>>>>>> obviously important to the community.
> >>>>>>>
> >>>>>>> Is there any reason not to move this straight to the dev@ list?
> >>>>>>>
> >>>>>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <to...@cloudera.com>
> >>> wrote:
> >>>>>>>>
> >>>>>>>> I'm not very active in HBase these days, so I won't cast a
> non-zero
> >>> vote,
> >>>>>>>> but I'm -0 on this idea, for basically two reasons:
> >>>>>>>>
> >>>>>>>> 1) In my experience at a past job which used CTR, the reality was
> >>> that it
> >>>>>>>> was more like "Commit and maybe review" rather than "Commit then
> >>> review".
> >>>>>>>> It's always more fun (and often easier) to write new code than to
> >>> review
> >>>>>>>> other people's code, so if there isn't a requirement that all code
> >>> gets
> >>>>>>>> reviewed before commit, it's easy for the ratio of code written to
> >>> code
> >>>>>>>> reviewed to tend towards values significantly greater than 1:1
> over
> >>> time.
> >>>>>>>> At my past job, this meant that a lot of code made it into
> >>> production (it
> >>>>>>>> was a website) that hadn't ever been reviewed, and in many cases
> we
> >>> found
> >>>>>>>> bugs (or other issues) that would have definitely been caught by a
> >>> good
> >>>>>>>> code reviewer.
> >>>>>>>>
> >>>>>>>> 2) CTR has both the advantage and disadvantage of allowing areas
> of
> >>> code
> >>>>>>> to
> >>>>>>>> be evolved quickly by a single person. That could be seen as a
> >> plus,
> >>> in
> >>>>>>>> that there are some areas which tend to get ignored because we
> >> don't
> >>>>>>> have a
> >>>>>>>> critical mass of people reviewing patches in the area -- patches
> >>> languish
> >>>>>>>> in these areas currently. However, that could also be seen as a
> >> good
> >>>>>>> thing
> >>>>>>>> from a "community over code" perspective -- it's better to not
> have
> >>> any
> >>>>>>>> areas of code with bus-factor 1. Feeling the pain of not getting
> >>> reviews
> >>>>>>> in
> >>>>>>>> these areas with only a single active committer encourages us to
> >> find
> >>>>>>>> solutions - either by deprecating "niche" features (as we once did
> >>> with
> >>>>>>>> various 'contrib' projects) or by recruiting new committers who
> >> have
> >>>>>>>> interest in maintaining that code area. Lifting review
> restrictions
> >>> would
> >>>>>>>> allow us to sweep bus-factor issues under the rug.
> >>>>>>>>
> >>>>>>>> That said, I think CTR can be a valuable tool for "move fast on
> new
> >>>>>>>> experimentation" type projects -- I've participated in several
> >>> feature
> >>>>>>>> branches in HDFS where we operated on CTR on the branch. The
> >>> assumption
> >>>>>>> was
> >>>>>>>> that, prior to merging the branch into trunk, all of the patches
> >> (or
> >>> a
> >>>>>>>> consolidated mega-patch) would be thoroughly reviewed by several
> >>> other
> >>>>>>>> committers. I found this to work very well during early
> >> development,
> >>>>>>> since
> >>>>>>>> I could hack on things and even commit pieces of code that had
> >> known
> >>>>>>>> issues/TODOs. For trickier patches on the CTR branch, I still
> >> tended
> >>> to
> >>>>>>>> ping experienced contributors for their opinions and feedback
> >> before
> >>>>>>>> committing.
> >>>>>>>>
> >>>>>>>> Perhaps such a hybrid policy would work well in the context of
> >> HBase
> >>>>>>>> feature development as well?
> >>>>>>>>
> >>>>>>>> -Todd
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
> >>> apurtell@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Just thought of branch merge votes. Sorry for that omission. I
> >>> think it
> >>>>>>>>> makes sense to keep those, but let's discuss that too in this
> >>> context.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
> >>> apurtell@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> As Greg Stein said on a thread over at general@incubator
> >>>>>>>> ("[DISCUSSION]
> >>>>>>>>>> Graduate Ignite from the Apache Incubator"):
> >>>>>>>>>>
> >>>>>>>>>> I always translate RTC as "we don't trust you, so somebody else
> >>> must
> >>>>>>>>>> approve anything you do." To me, that is a lousy basis for
> >>> creating a
> >>>>>>>>>> community. Trust and peer respect should be the basis, which
> >>> implies
> >>>>>>>> CTR.
> >>>>>>>>>>
> >>>>>>>>>> I happen to agree with this sentiment and would like to propose
> >>>>>>>> switching
> >>>>>>>>>> our consensus on commit procedure from RTC to CTR. Concurrently,
> >> I
> >>>>>>>> would
> >>>>>>>>>> like to propose this consensus also include the following:
> >>>>>>>>>> - Checkins should pass precommit or the committer should explain
> >> on
> >>>>>>> the
> >>>>>>>>>> JIRA why precommit failures are in their best judgement not
> >> related
> >>>>>>>>>> - The PMC should be willing to, ultimately, revoke committership
> >>>>>>> should
> >>>>>>>>>> trust in any individual committer be discovered to be misplaced.
> >> I
> >>>>>>>> would
> >>>>>>>>>> expect such a last resort to be exceedingly rare and likely
> never
> >>> to
> >>>>>>>>> happen
> >>>>>>>>>> because of course long before that we would be setting correct
> >>> public
> >>>>>>>>>> examples in the first place and respectfully counseling well
> >>>>>>>> intentioned
> >>>>>>>>>> committers who might stray.
> >>>>>>>>>>
> >>>>>>>>>> Depending on how the conversation proceeds here I would like to
> >>>>>>> include
> >>>>>>>>>> dev@ in this conversation at the earliest opportunity as well.
> >>>>>>>>>>
> >>>>>>>>>> Thoughts?
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Best regards,
> >>>>>>>>>>
> >>>>>>>>>> - Andy
> >>>>>>>>>>
> >>>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
> >> Piet
> >>>>>>>> Hein
> >>>>>>>>>> (via Tom White)
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Best regards,
> >>>>>>>>>
> >>>>>>>>> - Andy
> >>>>>>>>>
> >>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
> >> Piet
> >>>>>>> Hein
> >>>>>>>>> (via Tom White)
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Todd Lipcon
> >>>>>>>> Software Engineer, Cloudera
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Sean
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Thanks,
> >>>> Michael Antonov
> >>
>



-- 
// Jonathan Hsieh (shay)
// HBase Tech Lead, Software Engineer, Cloudera
// jon@cloudera.com // @jmhsieh

Re: [DISCUSSION] Switching from RTC to CTR

[Andrew Purtell <an...@gmail.com>]

Let me add a side note. No matter what comes of this discussion, if you don't mind I'd like to thank the set of committers who go out of their way to review the work of others irrespective of any direct involvement in the work. You know who you are. You're what allows the project to make forward progress in my opinion. That's so important. An extra thank you for our RM crew, taking in additional work. If all of our 40-50 odd committers on the list were this engaged that would be amazing. (Of course, that's not how it works, people move in and out and onward, nor is that required of the role.)


> On Aug 2, 2015, at 11:59 AM, Andrew Purtell <an...@gmail.com> wrote:
> 
> Agreed, committers should be spending more time reviewing others' work. We can ask. It may happen. It may work for a short while. It may not. Shrug. People will do what they want. 
> 
> I'm looking to make one substantial change that will allow committers to make progress even if there's nobody else around or interested for one week. It happens sometimes. I've already talked about my concerns on assuming a certain level of available volunteer bandwidth. Let me just say that things are great now, it's fantastic. 
> 
> We are pretty much on the honor system already. I don't buy the argument we can't trust that CTR or CTR after one week can work because committers, even if asked to customarily get a review before commit, may decide to check in unreviewed untested destabilizing changes. At least, In that case I'd argue we have a different problem. If you go back to my original mail, I do say we shouldn't undertake any change if we as PMC are unwilling to revoke the commit bit from committers who sadly demonstrate themselves unworthy of trust through checkins of toxic waste without review. Merit _can_ be un-earned. Commits can be reverted. Just because something is checked in does not mean it will go out in a release. We could put a nightly suite of regression tests in place. Both proposals I have made, especially the latter, are not a binary release of any attempt at quality control. We would have all of the consensus expectations of good committer behavior still in place. 
> 
> Let's assume someone checks in something without getting a review today. What would happen? Someone else would revert it and we'd have a discussion. If we were operating under CTR-after-one-week (or plain CTR) but with documented expectations that someone get a review first, what changes, in terms of quality control and project discipline? Maybe the difference is we could more easily justify revoking commit privileges? Maybe. I think every discussion like that would be strictly case by case though, a conversation between that person and the PMC, justification for revoking committer status wouldn't be rule based. 
> 
> What definitely would change is my well tested good faith change waiting for review after one week could go in no matter who's on vacation or whatever. Not saying this is a problem for me today. Like I said, things are great around here today. Awesome. 
> 
> 
>> On Aug 2, 2015, at 10:52 AM, Stack <st...@duboce.net> wrote:
>> 
>> On Fri, Jul 31, 2015 at 3:15 AM, Andrew Purtell <an...@gmail.com>
>> wrote:
>> 
>>> I appreciate very much the earlier feedback about switching from RTC to
>>> CTR. It helped me think about the essential thing I was after.
>>> 
>>> I'm thinking of making a formal proposal to adopt this, with a VOTE:
>>> 
>>>> After posting a patch to JIRA, after one week if there is no review or
>>> veto, a committer can commit their own work.
>>> 
>>> (On a bit of a lag...)
>> 
>> What is the problem we are trying to solve? Blocked committers unable to
>> get a review?
>> 
>> Any commit can destabilize. Commits without peer review will destabilize
>> more than those that have been reviewed. As a community, I'd think that
>> anything we can do to act against entropy in the system would be primary
>> above all other considerations especially given our project a mature
>> datastore.
>> 
>> In my own case, it seems to take me tens of patch postings to arrive at
>> something a fellow reviewer thinks worthy of commit when working with a
>> peer on an issue of substance. Often the final patch is radically different
>> from first posting. If auto-commit, the first version would just go in.
>> Unless review, I'd be 'done'. If review, each iteration would be in the
>> code base? Churn would go up. Commits would be more and smaller. Our
>> project would be made of less coherent 'change sets'.
>> 
>> I'd be with LarsH suggestion that committers should be allowed go ahead
>> with one-liners or more substantial, non-controversial changes in site or
>> doc but beyond this, I'd be against auto-commit.
>> 
>>> 
>>> 
>>> Also, looking at https://hbase.apache.org/book.html#_decisions, I don't
>>> think the "patch +1 policy" should remain because the trial OWNERS concept
>>> hasn't worked out, IMHO. The OWNERS concept requires a set of constantly
>>> present and engaged owners, a resource demand that's hard to square with
>>> the volunteer nature of our community. The amount of time any committer or
>>> PMC member has on this project is highly variable day to day and week to
>>> week.  I'm also thinking of calling a VOTE to significantly revise or
>>> strike this section.
>> Yeah. OWNERS project failed. Its ghost is in effect in that we each go to
>> the domain expert when its clear (e.g. Matteo or Stephen on pv2).
>> 
>> 
>>> Both of these things have a common root: Volunteer time is a very precious
>>> commodity. Our community's supply of volunteer time fluctuates. I would
>>> like to see committers be able to make progress with their own work even in
>>> periods when volunteer time is in very short supply, or when they are
>>> working on niche concerns that simply do not draw sufficient interest from
>>> other committers. (This is different from work that people think isn't
>>> appropriate - in that case ignoring it so it will go away would no longer
>>> be an option, a veto would be required if you want to stop something.)
>> Committers should be spending more time reviewing their peers work (and
>> just as importantly, the work of contributors)?
>> St.Ack
>> 
>> 
>> 
>>> On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <an...@gmail.com>
>>> wrote:
>>> 
>>>> Had this thought after getting back on the road. As an alternative to any
>>>> sweeping change we could do one incremental but very significant thing
>>> that
>>>> acknowledges our status as trusted and busy peers: After posting a patch
>>> to
>>>> JIRA, after one week if there is no review or veto, a committer can
>>> commit
>>>> their own work.
>>>> 
>>>> 
>>>>>> On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <ol...@gmail.com>
>>>>> wrote:
>>>>> 
>>>>> Just curious, I assume if this change is made, would it only apply to
>>>>> master branch?
>>>>> 
>>>>> -Mikhail
>>>>> 
>>>>> On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
>>>>> <an...@gmail.com> wrote:
>>>>>> @dev is now CCed
>>>>>> 
>>>>>> I didn't want to over structure the discussion with too much detail up
>>>> front. I do think CTR without supporting process or boundaries can be
>>> more
>>>> problematic than helpful. That could take the form of customarily waiting
>>>> for reviews before commit even under a CTR regime. I think this
>>> discussion
>>>> has been great so far. I would also add that CTR moves 'R' from a gating
>>>> requirement per commit (which can be hard to overcome for niche areas or
>>>> when volunteer resources are less available) more to RMs. will be back
>>>> later with more.
>>>>>> 
>>>>>> 
>>>>>>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com>
>>>> wrote:
>>>>>>> 
>>>>>>> I'd also favor having this discussion on dev@.
>>>>>>> 
>>>>>>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <ghelmling@gmail.com
>>>> 
>>>> wrote:
>>>>>>>> 
>>>>>>>> This is already a really interesting and meaningful discussion, and
>>> is
>>>>>>>> obviously important to the community.
>>>>>>>> 
>>>>>>>> Is there any reason not to move this straight to the dev@ list?
>>>>>>>> 
>>>>>>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <to...@cloudera.com>
>>>> wrote:
>>>>>>>>> 
>>>>>>>>> I'm not very active in HBase these days, so I won't cast a non-zero
>>>> vote,
>>>>>>>>> but I'm -0 on this idea, for basically two reasons:
>>>>>>>>> 
>>>>>>>>> 1) In my experience at a past job which used CTR, the reality was
>>>> that it
>>>>>>>>> was more like "Commit and maybe review" rather than "Commit then
>>>> review".
>>>>>>>>> It's always more fun (and often easier) to write new code than to
>>>> review
>>>>>>>>> other people's code, so if there isn't a requirement that all code
>>>> gets
>>>>>>>>> reviewed before commit, it's easy for the ratio of code written to
>>>> code
>>>>>>>>> reviewed to tend towards values significantly greater than 1:1 over
>>>> time.
>>>>>>>>> At my past job, this meant that a lot of code made it into
>>>> production (it
>>>>>>>>> was a website) that hadn't ever been reviewed, and in many cases we
>>>> found
>>>>>>>>> bugs (or other issues) that would have definitely been caught by a
>>>> good
>>>>>>>>> code reviewer.
>>>>>>>>> 
>>>>>>>>> 2) CTR has both the advantage and disadvantage of allowing areas of
>>>> code
>>>>>>>> to
>>>>>>>>> be evolved quickly by a single person. That could be seen as a
>>> plus,
>>>> in
>>>>>>>>> that there are some areas which tend to get ignored because we
>>> don't
>>>>>>>> have a
>>>>>>>>> critical mass of people reviewing patches in the area -- patches
>>>> languish
>>>>>>>>> in these areas currently. However, that could also be seen as a
>>> good
>>>>>>>> thing
>>>>>>>>> from a "community over code" perspective -- it's better to not have
>>>> any
>>>>>>>>> areas of code with bus-factor 1. Feeling the pain of not getting
>>>> reviews
>>>>>>>> in
>>>>>>>>> these areas with only a single active committer encourages us to
>>> find
>>>>>>>>> solutions - either by deprecating "niche" features (as we once did
>>>> with
>>>>>>>>> various 'contrib' projects) or by recruiting new committers who
>>> have
>>>>>>>>> interest in maintaining that code area. Lifting review restrictions
>>>> would
>>>>>>>>> allow us to sweep bus-factor issues under the rug.
>>>>>>>>> 
>>>>>>>>> That said, I think CTR can be a valuable tool for "move fast on new
>>>>>>>>> experimentation" type projects -- I've participated in several
>>>> feature
>>>>>>>>> branches in HDFS where we operated on CTR on the branch. The
>>>> assumption
>>>>>>>> was
>>>>>>>>> that, prior to merging the branch into trunk, all of the patches
>>> (or
>>>> a
>>>>>>>>> consolidated mega-patch) would be thoroughly reviewed by several
>>>> other
>>>>>>>>> committers. I found this to work very well during early
>>> development,
>>>>>>>> since
>>>>>>>>> I could hack on things and even commit pieces of code that had
>>> known
>>>>>>>>> issues/TODOs. For trickier patches on the CTR branch, I still
>>> tended
>>>> to
>>>>>>>>> ping experienced contributors for their opinions and feedback
>>> before
>>>>>>>>> committing.
>>>>>>>>> 
>>>>>>>>> Perhaps such a hybrid policy would work well in the context of
>>> HBase
>>>>>>>>> feature development as well?
>>>>>>>>> 
>>>>>>>>> -Todd
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
>>>> apurtell@apache.org>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Just thought of branch merge votes. Sorry for that omission. I
>>>> think it
>>>>>>>>>> makes sense to keep those, but let's discuss that too in this
>>>> context.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
>>>> apurtell@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> As Greg Stein said on a thread over at general@incubator
>>>>>>>>> ("[DISCUSSION]
>>>>>>>>>>> Graduate Ignite from the Apache Incubator"):
>>>>>>>>>>> 
>>>>>>>>>>> I always translate RTC as "we don't trust you, so somebody else
>>>> must
>>>>>>>>>>> approve anything you do." To me, that is a lousy basis for
>>>> creating a
>>>>>>>>>>> community. Trust and peer respect should be the basis, which
>>>> implies
>>>>>>>>> CTR.
>>>>>>>>>>> 
>>>>>>>>>>> I happen to agree with this sentiment and would like to propose
>>>>>>>>> switching
>>>>>>>>>>> our consensus on commit procedure from RTC to CTR. Concurrently,
>>> I
>>>>>>>>> would
>>>>>>>>>>> like to propose this consensus also include the following:
>>>>>>>>>>> - Checkins should pass precommit or the committer should explain
>>> on
>>>>>>>> the
>>>>>>>>>>> JIRA why precommit failures are in their best judgement not
>>> related
>>>>>>>>>>> - The PMC should be willing to, ultimately, revoke committership
>>>>>>>> should
>>>>>>>>>>> trust in any individual committer be discovered to be misplaced.
>>> I
>>>>>>>>> would
>>>>>>>>>>> expect such a last resort to be exceedingly rare and likely never
>>>> to
>>>>>>>>>> happen
>>>>>>>>>>> because of course long before that we would be setting correct
>>>> public
>>>>>>>>>>> examples in the first place and respectfully counseling well
>>>>>>>>> intentioned
>>>>>>>>>>> committers who might stray.
>>>>>>>>>>> 
>>>>>>>>>>> Depending on how the conversation proceeds here I would like to
>>>>>>>> include
>>>>>>>>>>> dev@ in this conversation at the earliest opportunity as well.
>>>>>>>>>>> 
>>>>>>>>>>> Thoughts?
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> Best regards,
>>>>>>>>>>> 
>>>>>>>>>>> - Andy
>>>>>>>>>>> 
>>>>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
>>> Piet
>>>>>>>>> Hein
>>>>>>>>>>> (via Tom White)
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Best regards,
>>>>>>>>>> 
>>>>>>>>>> - Andy
>>>>>>>>>> 
>>>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
>>> Piet
>>>>>>>> Hein
>>>>>>>>>> (via Tom White)
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Todd Lipcon
>>>>>>>>> Software Engineer, Cloudera
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Sean
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Thanks,
>>>>> Michael Antonov
>>> 

Re: [DISCUSSION] Switching from RTC to CTR

[Andrew Purtell <an...@gmail.com>]

Agreed, committers should be spending more time reviewing others' work. We can ask. It may happen. It may work for a short while. It may not. Shrug. People will do what they want. 

I'm looking to make one substantial change that will allow committers to make progress even if there's nobody else around or interested for one week. It happens sometimes. I've already talked about my concerns on assuming a certain level of available volunteer bandwidth. Let me just say that things are great now, it's fantastic. 

We are pretty much on the honor system already. I don't buy the argument we can't trust that CTR or CTR after one week can work because committers, even if asked to customarily get a review before commit, may decide to check in unreviewed untested destabilizing changes. At least, In that case I'd argue we have a different problem. If you go back to my original mail, I do say we shouldn't undertake any change if we as PMC are unwilling to revoke the commit bit from committers who sadly demonstrate themselves unworthy of trust through checkins of toxic waste without review. Merit _can_ be un-earned. Commits can be reverted. Just because something is checked in does not mean it will go out in a release. We could put a nightly suite of regression tests in place. Both proposals I have made, especially the latter, are not a binary release of any attempt at quality control. We would have all of the consensus expectations of good committer behavior still in place. 

Let's assume someone checks in something without getting a review today. What would happen? Someone else would revert it and we'd have a discussion. If we were operating under CTR-after-one-week (or plain CTR) but with documented expectations that someone get a review first, what changes, in terms of quality control and project discipline? Maybe the difference is we could more easily justify revoking commit privileges? Maybe. I think every discussion like that would be strictly case by case though, a conversation between that person and the PMC, justification for revoking committer status wouldn't be rule based. 

What definitely would change is my well tested good faith change waiting for review after one week could go in no matter who's on vacation or whatever. Not saying this is a problem for me today. Like I said, things are great around here today. Awesome. 


> On Aug 2, 2015, at 10:52 AM, Stack <st...@duboce.net> wrote:
> 
> On Fri, Jul 31, 2015 at 3:15 AM, Andrew Purtell <an...@gmail.com>
> wrote:
> 
>> I appreciate very much the earlier feedback about switching from RTC to
>> CTR. It helped me think about the essential thing I was after.
>> 
>> I'm thinking of making a formal proposal to adopt this, with a VOTE:
>> 
>>> After posting a patch to JIRA, after one week if there is no review or
>> veto, a committer can commit their own work.
>> 
>> (On a bit of a lag...)
> 
> What is the problem we are trying to solve? Blocked committers unable to
> get a review?
> 
> Any commit can destabilize. Commits without peer review will destabilize
> more than those that have been reviewed. As a community, I'd think that
> anything we can do to act against entropy in the system would be primary
> above all other considerations especially given our project a mature
> datastore.
> 
> In my own case, it seems to take me tens of patch postings to arrive at
> something a fellow reviewer thinks worthy of commit when working with a
> peer on an issue of substance. Often the final patch is radically different
> from first posting. If auto-commit, the first version would just go in.
> Unless review, I'd be 'done'. If review, each iteration would be in the
> code base? Churn would go up. Commits would be more and smaller. Our
> project would be made of less coherent 'change sets'.
> 
> I'd be with LarsH suggestion that committers should be allowed go ahead
> with one-liners or more substantial, non-controversial changes in site or
> doc but beyond this, I'd be against auto-commit.
> 
>> 
>> 
>> Also, looking at https://hbase.apache.org/book.html#_decisions, I don't
>> think the "patch +1 policy" should remain because the trial OWNERS concept
>> hasn't worked out, IMHO. The OWNERS concept requires a set of constantly
>> present and engaged owners, a resource demand that's hard to square with
>> the volunteer nature of our community. The amount of time any committer or
>> PMC member has on this project is highly variable day to day and week to
>> week.  I'm also thinking of calling a VOTE to significantly revise or
>> strike this section.
> Yeah. OWNERS project failed. Its ghost is in effect in that we each go to
> the domain expert when its clear (e.g. Matteo or Stephen on pv2).
> 
> 
>> Both of these things have a common root: Volunteer time is a very precious
>> commodity. Our community's supply of volunteer time fluctuates. I would
>> like to see committers be able to make progress with their own work even in
>> periods when volunteer time is in very short supply, or when they are
>> working on niche concerns that simply do not draw sufficient interest from
>> other committers. (This is different from work that people think isn't
>> appropriate - in that case ignoring it so it will go away would no longer
>> be an option, a veto would be required if you want to stop something.)
> Committers should be spending more time reviewing their peers work (and
> just as importantly, the work of contributors)?
> St.Ack
> 
> 
> 
>> On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <an...@gmail.com>
>> wrote:
>> 
>>> Had this thought after getting back on the road. As an alternative to any
>>> sweeping change we could do one incremental but very significant thing
>> that
>>> acknowledges our status as trusted and busy peers: After posting a patch
>> to
>>> JIRA, after one week if there is no review or veto, a committer can
>> commit
>>> their own work.
>>> 
>>> 
>>>>> On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <ol...@gmail.com>
>>>> wrote:
>>>> 
>>>> Just curious, I assume if this change is made, would it only apply to
>>>> master branch?
>>>> 
>>>> -Mikhail
>>>> 
>>>> On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
>>>> <an...@gmail.com> wrote:
>>>>> @dev is now CCed
>>>>> 
>>>>> I didn't want to over structure the discussion with too much detail up
>>> front. I do think CTR without supporting process or boundaries can be
>> more
>>> problematic than helpful. That could take the form of customarily waiting
>>> for reviews before commit even under a CTR regime. I think this
>> discussion
>>> has been great so far. I would also add that CTR moves 'R' from a gating
>>> requirement per commit (which can be hard to overcome for niche areas or
>>> when volunteer resources are less available) more to RMs. will be back
>>> later with more.
>>>>> 
>>>>> 
>>>>>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com>
>>> wrote:
>>>>>> 
>>>>>> I'd also favor having this discussion on dev@.
>>>>>> 
>>>>>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <ghelmling@gmail.com
>>> 
>>> wrote:
>>>>>>> 
>>>>>>> This is already a really interesting and meaningful discussion, and
>> is
>>>>>>> obviously important to the community.
>>>>>>> 
>>>>>>> Is there any reason not to move this straight to the dev@ list?
>>>>>>> 
>>>>>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <to...@cloudera.com>
>>> wrote:
>>>>>>>> 
>>>>>>>> I'm not very active in HBase these days, so I won't cast a non-zero
>>> vote,
>>>>>>>> but I'm -0 on this idea, for basically two reasons:
>>>>>>>> 
>>>>>>>> 1) In my experience at a past job which used CTR, the reality was
>>> that it
>>>>>>>> was more like "Commit and maybe review" rather than "Commit then
>>> review".
>>>>>>>> It's always more fun (and often easier) to write new code than to
>>> review
>>>>>>>> other people's code, so if there isn't a requirement that all code
>>> gets
>>>>>>>> reviewed before commit, it's easy for the ratio of code written to
>>> code
>>>>>>>> reviewed to tend towards values significantly greater than 1:1 over
>>> time.
>>>>>>>> At my past job, this meant that a lot of code made it into
>>> production (it
>>>>>>>> was a website) that hadn't ever been reviewed, and in many cases we
>>> found
>>>>>>>> bugs (or other issues) that would have definitely been caught by a
>>> good
>>>>>>>> code reviewer.
>>>>>>>> 
>>>>>>>> 2) CTR has both the advantage and disadvantage of allowing areas of
>>> code
>>>>>>> to
>>>>>>>> be evolved quickly by a single person. That could be seen as a
>> plus,
>>> in
>>>>>>>> that there are some areas which tend to get ignored because we
>> don't
>>>>>>> have a
>>>>>>>> critical mass of people reviewing patches in the area -- patches
>>> languish
>>>>>>>> in these areas currently. However, that could also be seen as a
>> good
>>>>>>> thing
>>>>>>>> from a "community over code" perspective -- it's better to not have
>>> any
>>>>>>>> areas of code with bus-factor 1. Feeling the pain of not getting
>>> reviews
>>>>>>> in
>>>>>>>> these areas with only a single active committer encourages us to
>> find
>>>>>>>> solutions - either by deprecating "niche" features (as we once did
>>> with
>>>>>>>> various 'contrib' projects) or by recruiting new committers who
>> have
>>>>>>>> interest in maintaining that code area. Lifting review restrictions
>>> would
>>>>>>>> allow us to sweep bus-factor issues under the rug.
>>>>>>>> 
>>>>>>>> That said, I think CTR can be a valuable tool for "move fast on new
>>>>>>>> experimentation" type projects -- I've participated in several
>>> feature
>>>>>>>> branches in HDFS where we operated on CTR on the branch. The
>>> assumption
>>>>>>> was
>>>>>>>> that, prior to merging the branch into trunk, all of the patches
>> (or
>>> a
>>>>>>>> consolidated mega-patch) would be thoroughly reviewed by several
>>> other
>>>>>>>> committers. I found this to work very well during early
>> development,
>>>>>>> since
>>>>>>>> I could hack on things and even commit pieces of code that had
>> known
>>>>>>>> issues/TODOs. For trickier patches on the CTR branch, I still
>> tended
>>> to
>>>>>>>> ping experienced contributors for their opinions and feedback
>> before
>>>>>>>> committing.
>>>>>>>> 
>>>>>>>> Perhaps such a hybrid policy would work well in the context of
>> HBase
>>>>>>>> feature development as well?
>>>>>>>> 
>>>>>>>> -Todd
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
>>> apurtell@apache.org>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Just thought of branch merge votes. Sorry for that omission. I
>>> think it
>>>>>>>>> makes sense to keep those, but let's discuss that too in this
>>> context.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
>>> apurtell@apache.org>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> As Greg Stein said on a thread over at general@incubator
>>>>>>>> ("[DISCUSSION]
>>>>>>>>>> Graduate Ignite from the Apache Incubator"):
>>>>>>>>>> 
>>>>>>>>>> I always translate RTC as "we don't trust you, so somebody else
>>> must
>>>>>>>>>> approve anything you do." To me, that is a lousy basis for
>>> creating a
>>>>>>>>>> community. Trust and peer respect should be the basis, which
>>> implies
>>>>>>>> CTR.
>>>>>>>>>> 
>>>>>>>>>> I happen to agree with this sentiment and would like to propose
>>>>>>>> switching
>>>>>>>>>> our consensus on commit procedure from RTC to CTR. Concurrently,
>> I
>>>>>>>> would
>>>>>>>>>> like to propose this consensus also include the following:
>>>>>>>>>> - Checkins should pass precommit or the committer should explain
>> on
>>>>>>> the
>>>>>>>>>> JIRA why precommit failures are in their best judgement not
>> related
>>>>>>>>>> - The PMC should be willing to, ultimately, revoke committership
>>>>>>> should
>>>>>>>>>> trust in any individual committer be discovered to be misplaced.
>> I
>>>>>>>> would
>>>>>>>>>> expect such a last resort to be exceedingly rare and likely never
>>> to
>>>>>>>>> happen
>>>>>>>>>> because of course long before that we would be setting correct
>>> public
>>>>>>>>>> examples in the first place and respectfully counseling well
>>>>>>>> intentioned
>>>>>>>>>> committers who might stray.
>>>>>>>>>> 
>>>>>>>>>> Depending on how the conversation proceeds here I would like to
>>>>>>> include
>>>>>>>>>> dev@ in this conversation at the earliest opportunity as well.
>>>>>>>>>> 
>>>>>>>>>> Thoughts?
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Best regards,
>>>>>>>>>> 
>>>>>>>>>> - Andy
>>>>>>>>>> 
>>>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
>> Piet
>>>>>>>> Hein
>>>>>>>>>> (via Tom White)
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Best regards,
>>>>>>>>> 
>>>>>>>>> - Andy
>>>>>>>>> 
>>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
>> Piet
>>>>>>> Hein
>>>>>>>>> (via Tom White)
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Todd Lipcon
>>>>>>>> Software Engineer, Cloudera
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Sean
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Thanks,
>>>> Michael Antonov
>> 

Re: [DISCUSSION] Switching from RTC to CTR

[Stack <st...@duboce.net>]

On Fri, Jul 31, 2015 at 3:15 AM, Andrew Purtell <an...@gmail.com>
wrote:

> I appreciate very much the earlier feedback about switching from RTC to
> CTR. It helped me think about the essential thing I was after.
>
> I'm thinking of making a formal proposal to adopt this, with a VOTE:
>
> > After posting a patch to JIRA, after one week if there is no review or
> veto, a committer can commit their own work.
>
> (On a bit of a lag...)

What is the problem we are trying to solve? Blocked committers unable to
get a review?

Any commit can destabilize. Commits without peer review will destabilize
more than those that have been reviewed. As a community, I'd think that
anything we can do to act against entropy in the system would be primary
above all other considerations especially given our project a mature
datastore.

In my own case, it seems to take me tens of patch postings to arrive at
something a fellow reviewer thinks worthy of commit when working with a
peer on an issue of substance. Often the final patch is radically different
from first posting. If auto-commit, the first version would just go in.
Unless review, I'd be 'done'. If review, each iteration would be in the
code base? Churn would go up. Commits would be more and smaller. Our
project would be made of less coherent 'change sets'.

I'd be with LarsH suggestion that committers should be allowed go ahead
with one-liners or more substantial, non-controversial changes in site or
doc but beyond this, I'd be against auto-commit.

>
>
> Also, looking at https://hbase.apache.org/book.html#_decisions, I don't
> think the "patch +1 policy" should remain because the trial OWNERS concept
> hasn't worked out, IMHO. The OWNERS concept requires a set of constantly
> present and engaged owners, a resource demand that's hard to square with
> the volunteer nature of our community. The amount of time any committer or
> PMC member has on this project is highly variable day to day and week to
> week.  I'm also thinking of calling a VOTE to significantly revise or
> strike this section.
>
>
Yeah. OWNERS project failed. Its ghost is in effect in that we each go to
the domain expert when its clear (e.g. Matteo or Stephen on pv2).


> Both of these things have a common root: Volunteer time is a very precious
> commodity. Our community's supply of volunteer time fluctuates. I would
> like to see committers be able to make progress with their own work even in
> periods when volunteer time is in very short supply, or when they are
> working on niche concerns that simply do not draw sufficient interest from
> other committers. (This is different from work that people think isn't
> appropriate - in that case ignoring it so it will go away would no longer
> be an option, a veto would be required if you want to stop something.)
>
>
>
Committers should be spending more time reviewing their peers work (and
just as importantly, the work of contributors)?
St.Ack



> On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <an...@gmail.com>
> wrote:
>
> > Had this thought after getting back on the road. As an alternative to any
> > sweeping change we could do one incremental but very significant thing
> that
> > acknowledges our status as trusted and busy peers: After posting a patch
> to
> > JIRA, after one week if there is no review or veto, a committer can
> commit
> > their own work.
> >
> >
> > > On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <ol...@gmail.com>
> > wrote:
> > >
> > > Just curious, I assume if this change is made, would it only apply to
> > > master branch?
> > >
> > > -Mikhail
> > >
> > > On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
> > > <an...@gmail.com> wrote:
> > >> @dev is now CCed
> > >>
> > >> I didn't want to over structure the discussion with too much detail up
> > front. I do think CTR without supporting process or boundaries can be
> more
> > problematic than helpful. That could take the form of customarily waiting
> > for reviews before commit even under a CTR regime. I think this
> discussion
> > has been great so far. I would also add that CTR moves 'R' from a gating
> > requirement per commit (which can be hard to overcome for niche areas or
> > when volunteer resources are less available) more to RMs. will be back
> > later with more.
> > >>
> > >>
> > >>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com>
> > wrote:
> > >>>
> > >>> I'd also favor having this discussion on dev@.
> > >>>
> > >>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <ghelmling@gmail.com
> >
> > wrote:
> > >>>>
> > >>>> This is already a really interesting and meaningful discussion, and
> is
> > >>>> obviously important to the community.
> > >>>>
> > >>>> Is there any reason not to move this straight to the dev@ list?
> > >>>>
> > >>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <to...@cloudera.com>
> > wrote:
> > >>>>>
> > >>>>> I'm not very active in HBase these days, so I won't cast a non-zero
> > vote,
> > >>>>> but I'm -0 on this idea, for basically two reasons:
> > >>>>>
> > >>>>> 1) In my experience at a past job which used CTR, the reality was
> > that it
> > >>>>> was more like "Commit and maybe review" rather than "Commit then
> > review".
> > >>>>> It's always more fun (and often easier) to write new code than to
> > review
> > >>>>> other people's code, so if there isn't a requirement that all code
> > gets
> > >>>>> reviewed before commit, it's easy for the ratio of code written to
> > code
> > >>>>> reviewed to tend towards values significantly greater than 1:1 over
> > time.
> > >>>>> At my past job, this meant that a lot of code made it into
> > production (it
> > >>>>> was a website) that hadn't ever been reviewed, and in many cases we
> > found
> > >>>>> bugs (or other issues) that would have definitely been caught by a
> > good
> > >>>>> code reviewer.
> > >>>>>
> > >>>>> 2) CTR has both the advantage and disadvantage of allowing areas of
> > code
> > >>>> to
> > >>>>> be evolved quickly by a single person. That could be seen as a
> plus,
> > in
> > >>>>> that there are some areas which tend to get ignored because we
> don't
> > >>>> have a
> > >>>>> critical mass of people reviewing patches in the area -- patches
> > languish
> > >>>>> in these areas currently. However, that could also be seen as a
> good
> > >>>> thing
> > >>>>> from a "community over code" perspective -- it's better to not have
> > any
> > >>>>> areas of code with bus-factor 1. Feeling the pain of not getting
> > reviews
> > >>>> in
> > >>>>> these areas with only a single active committer encourages us to
> find
> > >>>>> solutions - either by deprecating "niche" features (as we once did
> > with
> > >>>>> various 'contrib' projects) or by recruiting new committers who
> have
> > >>>>> interest in maintaining that code area. Lifting review restrictions
> > would
> > >>>>> allow us to sweep bus-factor issues under the rug.
> > >>>>>
> > >>>>> That said, I think CTR can be a valuable tool for "move fast on new
> > >>>>> experimentation" type projects -- I've participated in several
> > feature
> > >>>>> branches in HDFS where we operated on CTR on the branch. The
> > assumption
> > >>>> was
> > >>>>> that, prior to merging the branch into trunk, all of the patches
> (or
> > a
> > >>>>> consolidated mega-patch) would be thoroughly reviewed by several
> > other
> > >>>>> committers. I found this to work very well during early
> development,
> > >>>> since
> > >>>>> I could hack on things and even commit pieces of code that had
> known
> > >>>>> issues/TODOs. For trickier patches on the CTR branch, I still
> tended
> > to
> > >>>>> ping experienced contributors for their opinions and feedback
> before
> > >>>>> committing.
> > >>>>>
> > >>>>> Perhaps such a hybrid policy would work well in the context of
> HBase
> > >>>>> feature development as well?
> > >>>>>
> > >>>>> -Todd
> > >>>>>
> > >>>>>
> > >>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
> > apurtell@apache.org>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Just thought of branch merge votes. Sorry for that omission. I
> > think it
> > >>>>>> makes sense to keep those, but let's discuss that too in this
> > context.
> > >>>>>>
> > >>>>>>
> > >>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
> > apurtell@apache.org>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> As Greg Stein said on a thread over at general@incubator
> > >>>>> ("[DISCUSSION]
> > >>>>>>> Graduate Ignite from the Apache Incubator"):
> > >>>>>>>
> > >>>>>>> I always translate RTC as "we don't trust you, so somebody else
> > must
> > >>>>>>> approve anything you do." To me, that is a lousy basis for
> > creating a
> > >>>>>>> community. Trust and peer respect should be the basis, which
> > implies
> > >>>>> CTR.
> > >>>>>>>
> > >>>>>>> I happen to agree with this sentiment and would like to propose
> > >>>>> switching
> > >>>>>>> our consensus on commit procedure from RTC to CTR. Concurrently,
> I
> > >>>>> would
> > >>>>>>> like to propose this consensus also include the following:
> > >>>>>>> - Checkins should pass precommit or the committer should explain
> on
> > >>>> the
> > >>>>>>> JIRA why precommit failures are in their best judgement not
> related
> > >>>>>>> - The PMC should be willing to, ultimately, revoke committership
> > >>>> should
> > >>>>>>> trust in any individual committer be discovered to be misplaced.
> I
> > >>>>> would
> > >>>>>>> expect such a last resort to be exceedingly rare and likely never
> > to
> > >>>>>> happen
> > >>>>>>> because of course long before that we would be setting correct
> > public
> > >>>>>>> examples in the first place and respectfully counseling well
> > >>>>> intentioned
> > >>>>>>> committers who might stray.
> > >>>>>>>
> > >>>>>>> Depending on how the conversation proceeds here I would like to
> > >>>> include
> > >>>>>>> dev@ in this conversation at the earliest opportunity as well.
> > >>>>>>>
> > >>>>>>> Thoughts?
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> Best regards,
> > >>>>>>>
> > >>>>>>>  - Andy
> > >>>>>>>
> > >>>>>>> Problems worthy of attack prove their worth by hitting back. -
> Piet
> > >>>>> Hein
> > >>>>>>> (via Tom White)
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Best regards,
> > >>>>>>
> > >>>>>>  - Andy
> > >>>>>>
> > >>>>>> Problems worthy of attack prove their worth by hitting back. -
> Piet
> > >>>> Hein
> > >>>>>> (via Tom White)
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Todd Lipcon
> > >>>>> Software Engineer, Cloudera
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Sean
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Michael Antonov
> >
>

Re: [DISCUSSION] Switching from RTC to CTR

[Andrew Purtell <an...@gmail.com>]

Sure, I'm not planning to do anything right away. Pretty busy here too. 


> On Jul 30, 2015, at 6:30 PM, Sean Busbey <bu...@cloudera.com> wrote:
> 
> I have some feedback on RTC v CTR I've been chewing on, but have been too
> buried in the licensing bog to get together. Can the vote wait till next
> week? That should still be plenty of time in the unlikely event board@
> wants a resolution.
> 
> -- 
> Sean
>> On Jul 30, 2015 8:16 PM, "Andrew Purtell" <an...@gmail.com> wrote:
>> 
>> I appreciate very much the earlier feedback about switching from RTC to
>> CTR. It helped me think about the essential thing I was after.
>> 
>> I'm thinking of making a formal proposal to adopt this, with a VOTE:
>> 
>>> After posting a patch to JIRA, after one week if there is no review or
>> veto, a committer can commit their own work.
>> 
>> It's important we discuss this and have a vote because the default
>> Foundation decision making process (
>> http://www.apache.org/foundation/voting.html) does not allow what would
>> amount to lazy consensus when RTC is in effect. Should my proposal pass, we
>> would arrive at a hybrid policy that is identical to the default Foundation
>> one *until* one week has elapsed after a code change is proposed. Then, for
>> a committer, for that one code change, they would be able to operate using
>> CTR. I think the HBase PMC is empowered to set this kind of policy for our
>> own project at our option. If you feel I am mistaken about that, please
>> speak up. Should the vote pass I will run it by board@ for review to be
>> sure.
>> 
>> We'd document this in the book:
>> https://hbase.apache.org/book.html#_decisions
>> 
>> Also, looking at https://hbase.apache.org/book.html#_decisions, I don't
>> think the "patch +1 policy" should remain because the trial OWNERS concept
>> hasn't worked out, IMHO. The OWNERS concept requires a set of constantly
>> present and engaged owners, a resource demand that's hard to square with
>> the volunteer nature of our community. The amount of time any committer or
>> PMC member has on this project is highly variable day to day and week to
>> week.  I'm also thinking of calling a VOTE to significantly revise or
>> strike this section.
>> 
>> Both of these things have a common root: Volunteer time is a very precious
>> commodity. Our community's supply of volunteer time fluctuates. I would
>> like to see committers be able to make progress with their own work even in
>> periods when volunteer time is in very short supply, or when they are
>> working on niche concerns that simply do not draw sufficient interest from
>> other committers. (This is different from work that people think isn't
>> appropriate - in that case ignoring it so it will go away would no longer
>> be an option, a veto would be required if you want to stop something.)
>> 
>> 
>> On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <an...@gmail.com>
>> wrote:
>> 
>>> Had this thought after getting back on the road. As an alternative to any
>>> sweeping change we could do one incremental but very significant thing
>> that
>>> acknowledges our status as trusted and busy peers: After posting a patch
>> to
>>> JIRA, after one week if there is no review or veto, a committer can
>> commit
>>> their own work.
>>> 
>>> 
>>>>> On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <ol...@gmail.com>
>>>> wrote:
>>>> 
>>>> Just curious, I assume if this change is made, would it only apply to
>>>> master branch?
>>>> 
>>>> -Mikhail
>>>> 
>>>> On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
>>>> <an...@gmail.com> wrote:
>>>>> @dev is now CCed
>>>>> 
>>>>> I didn't want to over structure the discussion with too much detail up
>>> front. I do think CTR without supporting process or boundaries can be
>> more
>>> problematic than helpful. That could take the form of customarily waiting
>>> for reviews before commit even under a CTR regime. I think this
>> discussion
>>> has been great so far. I would also add that CTR moves 'R' from a gating
>>> requirement per commit (which can be hard to overcome for niche areas or
>>> when volunteer resources are less available) more to RMs. will be back
>>> later with more.
>>>>> 
>>>>> 
>>>>>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com>
>>> wrote:
>>>>>> 
>>>>>> I'd also favor having this discussion on dev@.
>>>>>> 
>>>>>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <ghelmling@gmail.com
>>> 
>>> wrote:
>>>>>>> 
>>>>>>> This is already a really interesting and meaningful discussion, and
>> is
>>>>>>> obviously important to the community.
>>>>>>> 
>>>>>>> Is there any reason not to move this straight to the dev@ list?
>>>>>>> 
>>>>>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <to...@cloudera.com>
>>> wrote:
>>>>>>>> 
>>>>>>>> I'm not very active in HBase these days, so I won't cast a non-zero
>>> vote,
>>>>>>>> but I'm -0 on this idea, for basically two reasons:
>>>>>>>> 
>>>>>>>> 1) In my experience at a past job which used CTR, the reality was
>>> that it
>>>>>>>> was more like "Commit and maybe review" rather than "Commit then
>>> review".
>>>>>>>> It's always more fun (and often easier) to write new code than to
>>> review
>>>>>>>> other people's code, so if there isn't a requirement that all code
>>> gets
>>>>>>>> reviewed before commit, it's easy for the ratio of code written to
>>> code
>>>>>>>> reviewed to tend towards values significantly greater than 1:1 over
>>> time.
>>>>>>>> At my past job, this meant that a lot of code made it into
>>> production (it
>>>>>>>> was a website) that hadn't ever been reviewed, and in many cases we
>>> found
>>>>>>>> bugs (or other issues) that would have definitely been caught by a
>>> good
>>>>>>>> code reviewer.
>>>>>>>> 
>>>>>>>> 2) CTR has both the advantage and disadvantage of allowing areas of
>>> code
>>>>>>> to
>>>>>>>> be evolved quickly by a single person. That could be seen as a
>> plus,
>>> in
>>>>>>>> that there are some areas which tend to get ignored because we
>> don't
>>>>>>> have a
>>>>>>>> critical mass of people reviewing patches in the area -- patches
>>> languish
>>>>>>>> in these areas currently. However, that could also be seen as a
>> good
>>>>>>> thing
>>>>>>>> from a "community over code" perspective -- it's better to not have
>>> any
>>>>>>>> areas of code with bus-factor 1. Feeling the pain of not getting
>>> reviews
>>>>>>> in
>>>>>>>> these areas with only a single active committer encourages us to
>> find
>>>>>>>> solutions - either by deprecating "niche" features (as we once did
>>> with
>>>>>>>> various 'contrib' projects) or by recruiting new committers who
>> have
>>>>>>>> interest in maintaining that code area. Lifting review restrictions
>>> would
>>>>>>>> allow us to sweep bus-factor issues under the rug.
>>>>>>>> 
>>>>>>>> That said, I think CTR can be a valuable tool for "move fast on new
>>>>>>>> experimentation" type projects -- I've participated in several
>>> feature
>>>>>>>> branches in HDFS where we operated on CTR on the branch. The
>>> assumption
>>>>>>> was
>>>>>>>> that, prior to merging the branch into trunk, all of the patches
>> (or
>>> a
>>>>>>>> consolidated mega-patch) would be thoroughly reviewed by several
>>> other
>>>>>>>> committers. I found this to work very well during early
>> development,
>>>>>>> since
>>>>>>>> I could hack on things and even commit pieces of code that had
>> known
>>>>>>>> issues/TODOs. For trickier patches on the CTR branch, I still
>> tended
>>> to
>>>>>>>> ping experienced contributors for their opinions and feedback
>> before
>>>>>>>> committing.
>>>>>>>> 
>>>>>>>> Perhaps such a hybrid policy would work well in the context of
>> HBase
>>>>>>>> feature development as well?
>>>>>>>> 
>>>>>>>> -Todd
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
>>> apurtell@apache.org>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>>> Just thought of branch merge votes. Sorry for that omission. I
>>> think it
>>>>>>>>> makes sense to keep those, but let's discuss that too in this
>>> context.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
>>> apurtell@apache.org>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> As Greg Stein said on a thread over at general@incubator
>>>>>>>> ("[DISCUSSION]
>>>>>>>>>> Graduate Ignite from the Apache Incubator"):
>>>>>>>>>> 
>>>>>>>>>> I always translate RTC as "we don't trust you, so somebody else
>>> must
>>>>>>>>>> approve anything you do." To me, that is a lousy basis for
>>> creating a
>>>>>>>>>> community. Trust and peer respect should be the basis, which
>>> implies
>>>>>>>> CTR.
>>>>>>>>>> 
>>>>>>>>>> I happen to agree with this sentiment and would like to propose
>>>>>>>> switching
>>>>>>>>>> our consensus on commit procedure from RTC to CTR. Concurrently,
>> I
>>>>>>>> would
>>>>>>>>>> like to propose this consensus also include the following:
>>>>>>>>>> - Checkins should pass precommit or the committer should explain
>> on
>>>>>>> the
>>>>>>>>>> JIRA why precommit failures are in their best judgement not
>> related
>>>>>>>>>> - The PMC should be willing to, ultimately, revoke committership
>>>>>>> should
>>>>>>>>>> trust in any individual committer be discovered to be misplaced.
>> I
>>>>>>>> would
>>>>>>>>>> expect such a last resort to be exceedingly rare and likely never
>>> to
>>>>>>>>> happen
>>>>>>>>>> because of course long before that we would be setting correct
>>> public
>>>>>>>>>> examples in the first place and respectfully counseling well
>>>>>>>> intentioned
>>>>>>>>>> committers who might stray.
>>>>>>>>>> 
>>>>>>>>>> Depending on how the conversation proceeds here I would like to
>>>>>>> include
>>>>>>>>>> dev@ in this conversation at the earliest opportunity as well.
>>>>>>>>>> 
>>>>>>>>>> Thoughts?
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Best regards,
>>>>>>>>>> 
>>>>>>>>>> - Andy
>>>>>>>>>> 
>>>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
>> Piet
>>>>>>>> Hein
>>>>>>>>>> (via Tom White)
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> --
>>>>>>>>> Best regards,
>>>>>>>>> 
>>>>>>>>> - Andy
>>>>>>>>> 
>>>>>>>>> Problems worthy of attack prove their worth by hitting back. -
>> Piet
>>>>>>> Hein
>>>>>>>>> (via Tom White)
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> --
>>>>>>>> Todd Lipcon
>>>>>>>> Software Engineer, Cloudera
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Sean
>>>> 
>>>> 
>>>> 
>>>> --
>>>> Thanks,
>>>> Michael Antonov
>> 

Re: [DISCUSSION] Switching from RTC to CTR

[Sean Busbey <bu...@cloudera.com>]

I have some feedback on RTC v CTR I've been chewing on, but have been too
buried in the licensing bog to get together. Can the vote wait till next
week? That should still be plenty of time in the unlikely event board@
wants a resolution.

-- 
Sean
On Jul 30, 2015 8:16 PM, "Andrew Purtell" <an...@gmail.com> wrote:

> I appreciate very much the earlier feedback about switching from RTC to
> CTR. It helped me think about the essential thing I was after.
>
> I'm thinking of making a formal proposal to adopt this, with a VOTE:
>
> > After posting a patch to JIRA, after one week if there is no review or
> veto, a committer can commit their own work.
>
> It's important we discuss this and have a vote because the default
> Foundation decision making process (
> http://www.apache.org/foundation/voting.html) does not allow what would
> amount to lazy consensus when RTC is in effect. Should my proposal pass, we
> would arrive at a hybrid policy that is identical to the default Foundation
> one *until* one week has elapsed after a code change is proposed. Then, for
> a committer, for that one code change, they would be able to operate using
> CTR. I think the HBase PMC is empowered to set this kind of policy for our
> own project at our option. If you feel I am mistaken about that, please
> speak up. Should the vote pass I will run it by board@ for review to be
> sure.
>
> We'd document this in the book:
> https://hbase.apache.org/book.html#_decisions
>
> Also, looking at https://hbase.apache.org/book.html#_decisions, I don't
> think the "patch +1 policy" should remain because the trial OWNERS concept
> hasn't worked out, IMHO. The OWNERS concept requires a set of constantly
> present and engaged owners, a resource demand that's hard to square with
> the volunteer nature of our community. The amount of time any committer or
> PMC member has on this project is highly variable day to day and week to
> week.  I'm also thinking of calling a VOTE to significantly revise or
> strike this section.
>
> Both of these things have a common root: Volunteer time is a very precious
> commodity. Our community's supply of volunteer time fluctuates. I would
> like to see committers be able to make progress with their own work even in
> periods when volunteer time is in very short supply, or when they are
> working on niche concerns that simply do not draw sufficient interest from
> other committers. (This is different from work that people think isn't
> appropriate - in that case ignoring it so it will go away would no longer
> be an option, a veto would be required if you want to stop something.)
>
>
> On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <an...@gmail.com>
> wrote:
>
> > Had this thought after getting back on the road. As an alternative to any
> > sweeping change we could do one incremental but very significant thing
> that
> > acknowledges our status as trusted and busy peers: After posting a patch
> to
> > JIRA, after one week if there is no review or veto, a committer can
> commit
> > their own work.
> >
> >
> > > On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <ol...@gmail.com>
> > wrote:
> > >
> > > Just curious, I assume if this change is made, would it only apply to
> > > master branch?
> > >
> > > -Mikhail
> > >
> > > On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
> > > <an...@gmail.com> wrote:
> > >> @dev is now CCed
> > >>
> > >> I didn't want to over structure the discussion with too much detail up
> > front. I do think CTR without supporting process or boundaries can be
> more
> > problematic than helpful. That could take the form of customarily waiting
> > for reviews before commit even under a CTR regime. I think this
> discussion
> > has been great so far. I would also add that CTR moves 'R' from a gating
> > requirement per commit (which can be hard to overcome for niche areas or
> > when volunteer resources are less available) more to RMs. will be back
> > later with more.
> > >>
> > >>
> > >>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com>
> > wrote:
> > >>>
> > >>> I'd also favor having this discussion on dev@.
> > >>>
> > >>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <ghelmling@gmail.com
> >
> > wrote:
> > >>>>
> > >>>> This is already a really interesting and meaningful discussion, and
> is
> > >>>> obviously important to the community.
> > >>>>
> > >>>> Is there any reason not to move this straight to the dev@ list?
> > >>>>
> > >>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <to...@cloudera.com>
> > wrote:
> > >>>>>
> > >>>>> I'm not very active in HBase these days, so I won't cast a non-zero
> > vote,
> > >>>>> but I'm -0 on this idea, for basically two reasons:
> > >>>>>
> > >>>>> 1) In my experience at a past job which used CTR, the reality was
> > that it
> > >>>>> was more like "Commit and maybe review" rather than "Commit then
> > review".
> > >>>>> It's always more fun (and often easier) to write new code than to
> > review
> > >>>>> other people's code, so if there isn't a requirement that all code
> > gets
> > >>>>> reviewed before commit, it's easy for the ratio of code written to
> > code
> > >>>>> reviewed to tend towards values significantly greater than 1:1 over
> > time.
> > >>>>> At my past job, this meant that a lot of code made it into
> > production (it
> > >>>>> was a website) that hadn't ever been reviewed, and in many cases we
> > found
> > >>>>> bugs (or other issues) that would have definitely been caught by a
> > good
> > >>>>> code reviewer.
> > >>>>>
> > >>>>> 2) CTR has both the advantage and disadvantage of allowing areas of
> > code
> > >>>> to
> > >>>>> be evolved quickly by a single person. That could be seen as a
> plus,
> > in
> > >>>>> that there are some areas which tend to get ignored because we
> don't
> > >>>> have a
> > >>>>> critical mass of people reviewing patches in the area -- patches
> > languish
> > >>>>> in these areas currently. However, that could also be seen as a
> good
> > >>>> thing
> > >>>>> from a "community over code" perspective -- it's better to not have
> > any
> > >>>>> areas of code with bus-factor 1. Feeling the pain of not getting
> > reviews
> > >>>> in
> > >>>>> these areas with only a single active committer encourages us to
> find
> > >>>>> solutions - either by deprecating "niche" features (as we once did
> > with
> > >>>>> various 'contrib' projects) or by recruiting new committers who
> have
> > >>>>> interest in maintaining that code area. Lifting review restrictions
> > would
> > >>>>> allow us to sweep bus-factor issues under the rug.
> > >>>>>
> > >>>>> That said, I think CTR can be a valuable tool for "move fast on new
> > >>>>> experimentation" type projects -- I've participated in several
> > feature
> > >>>>> branches in HDFS where we operated on CTR on the branch. The
> > assumption
> > >>>> was
> > >>>>> that, prior to merging the branch into trunk, all of the patches
> (or
> > a
> > >>>>> consolidated mega-patch) would be thoroughly reviewed by several
> > other
> > >>>>> committers. I found this to work very well during early
> development,
> > >>>> since
> > >>>>> I could hack on things and even commit pieces of code that had
> known
> > >>>>> issues/TODOs. For trickier patches on the CTR branch, I still
> tended
> > to
> > >>>>> ping experienced contributors for their opinions and feedback
> before
> > >>>>> committing.
> > >>>>>
> > >>>>> Perhaps such a hybrid policy would work well in the context of
> HBase
> > >>>>> feature development as well?
> > >>>>>
> > >>>>> -Todd
> > >>>>>
> > >>>>>
> > >>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
> > apurtell@apache.org>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Just thought of branch merge votes. Sorry for that omission. I
> > think it
> > >>>>>> makes sense to keep those, but let's discuss that too in this
> > context.
> > >>>>>>
> > >>>>>>
> > >>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
> > apurtell@apache.org>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> As Greg Stein said on a thread over at general@incubator
> > >>>>> ("[DISCUSSION]
> > >>>>>>> Graduate Ignite from the Apache Incubator"):
> > >>>>>>>
> > >>>>>>> I always translate RTC as "we don't trust you, so somebody else
> > must
> > >>>>>>> approve anything you do." To me, that is a lousy basis for
> > creating a
> > >>>>>>> community. Trust and peer respect should be the basis, which
> > implies
> > >>>>> CTR.
> > >>>>>>>
> > >>>>>>> I happen to agree with this sentiment and would like to propose
> > >>>>> switching
> > >>>>>>> our consensus on commit procedure from RTC to CTR. Concurrently,
> I
> > >>>>> would
> > >>>>>>> like to propose this consensus also include the following:
> > >>>>>>> - Checkins should pass precommit or the committer should explain
> on
> > >>>> the
> > >>>>>>> JIRA why precommit failures are in their best judgement not
> related
> > >>>>>>> - The PMC should be willing to, ultimately, revoke committership
> > >>>> should
> > >>>>>>> trust in any individual committer be discovered to be misplaced.
> I
> > >>>>> would
> > >>>>>>> expect such a last resort to be exceedingly rare and likely never
> > to
> > >>>>>> happen
> > >>>>>>> because of course long before that we would be setting correct
> > public
> > >>>>>>> examples in the first place and respectfully counseling well
> > >>>>> intentioned
> > >>>>>>> committers who might stray.
> > >>>>>>>
> > >>>>>>> Depending on how the conversation proceeds here I would like to
> > >>>> include
> > >>>>>>> dev@ in this conversation at the earliest opportunity as well.
> > >>>>>>>
> > >>>>>>> Thoughts?
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> Best regards,
> > >>>>>>>
> > >>>>>>>  - Andy
> > >>>>>>>
> > >>>>>>> Problems worthy of attack prove their worth by hitting back. -
> Piet
> > >>>>> Hein
> > >>>>>>> (via Tom White)
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>> --
> > >>>>>> Best regards,
> > >>>>>>
> > >>>>>>  - Andy
> > >>>>>>
> > >>>>>> Problems worthy of attack prove their worth by hitting back. -
> Piet
> > >>>> Hein
> > >>>>>> (via Tom White)
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Todd Lipcon
> > >>>>> Software Engineer, Cloudera
> > >>>
> > >>>
> > >>>
> > >>> --
> > >>> Sean
> > >
> > >
> > >
> > > --
> > > Thanks,
> > > Michael Antonov
> >
>

Re: [DISCUSSION] Switching from RTC to CTR

[Andrew Purtell <an...@gmail.com>]

I appreciate very much the earlier feedback about switching from RTC to
CTR. It helped me think about the essential thing I was after.

I'm thinking of making a formal proposal to adopt this, with a VOTE:

> After posting a patch to JIRA, after one week if there is no review or
veto, a committer can commit their own work.

It's important we discuss this and have a vote because the default
Foundation decision making process (
http://www.apache.org/foundation/voting.html) does not allow what would
amount to lazy consensus when RTC is in effect. Should my proposal pass, we
would arrive at a hybrid policy that is identical to the default Foundation
one *until* one week has elapsed after a code change is proposed. Then, for
a committer, for that one code change, they would be able to operate using
CTR. I think the HBase PMC is empowered to set this kind of policy for our
own project at our option. If you feel I am mistaken about that, please
speak up. Should the vote pass I will run it by board@ for review to be
sure.

We'd document this in the book:
https://hbase.apache.org/book.html#_decisions

Also, looking at https://hbase.apache.org/book.html#_decisions, I don't
think the "patch +1 policy" should remain because the trial OWNERS concept
hasn't worked out, IMHO. The OWNERS concept requires a set of constantly
present and engaged owners, a resource demand that's hard to square with
the volunteer nature of our community. The amount of time any committer or
PMC member has on this project is highly variable day to day and week to
week.  I'm also thinking of calling a VOTE to significantly revise or
strike this section.

Both of these things have a common root: Volunteer time is a very precious
commodity. Our community's supply of volunteer time fluctuates. I would
like to see committers be able to make progress with their own work even in
periods when volunteer time is in very short supply, or when they are
working on niche concerns that simply do not draw sufficient interest from
other committers. (This is different from work that people think isn't
appropriate - in that case ignoring it so it will go away would no longer
be an option, a veto would be required if you want to stop something.)


On Wed, Jul 29, 2015 at 3:56 PM, Andrew Purtell <an...@gmail.com>
wrote:

> Had this thought after getting back on the road. As an alternative to any
> sweeping change we could do one incremental but very significant thing that
> acknowledges our status as trusted and busy peers: After posting a patch to
> JIRA, after one week if there is no review or veto, a committer can commit
> their own work.
>
>
> > On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <ol...@gmail.com>
> wrote:
> >
> > Just curious, I assume if this change is made, would it only apply to
> > master branch?
> >
> > -Mikhail
> >
> > On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
> > <an...@gmail.com> wrote:
> >> @dev is now CCed
> >>
> >> I didn't want to over structure the discussion with too much detail up
> front. I do think CTR without supporting process or boundaries can be more
> problematic than helpful. That could take the form of customarily waiting
> for reviews before commit even under a CTR regime. I think this discussion
> has been great so far. I would also add that CTR moves 'R' from a gating
> requirement per commit (which can be hard to overcome for niche areas or
> when volunteer resources are less available) more to RMs. will be back
> later with more.
> >>
> >>
> >>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com>
> wrote:
> >>>
> >>> I'd also favor having this discussion on dev@.
> >>>
> >>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <gh...@gmail.com>
> wrote:
> >>>>
> >>>> This is already a really interesting and meaningful discussion, and is
> >>>> obviously important to the community.
> >>>>
> >>>> Is there any reason not to move this straight to the dev@ list?
> >>>>
> >>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <to...@cloudera.com>
> wrote:
> >>>>>
> >>>>> I'm not very active in HBase these days, so I won't cast a non-zero
> vote,
> >>>>> but I'm -0 on this idea, for basically two reasons:
> >>>>>
> >>>>> 1) In my experience at a past job which used CTR, the reality was
> that it
> >>>>> was more like "Commit and maybe review" rather than "Commit then
> review".
> >>>>> It's always more fun (and often easier) to write new code than to
> review
> >>>>> other people's code, so if there isn't a requirement that all code
> gets
> >>>>> reviewed before commit, it's easy for the ratio of code written to
> code
> >>>>> reviewed to tend towards values significantly greater than 1:1 over
> time.
> >>>>> At my past job, this meant that a lot of code made it into
> production (it
> >>>>> was a website) that hadn't ever been reviewed, and in many cases we
> found
> >>>>> bugs (or other issues) that would have definitely been caught by a
> good
> >>>>> code reviewer.
> >>>>>
> >>>>> 2) CTR has both the advantage and disadvantage of allowing areas of
> code
> >>>> to
> >>>>> be evolved quickly by a single person. That could be seen as a plus,
> in
> >>>>> that there are some areas which tend to get ignored because we don't
> >>>> have a
> >>>>> critical mass of people reviewing patches in the area -- patches
> languish
> >>>>> in these areas currently. However, that could also be seen as a good
> >>>> thing
> >>>>> from a "community over code" perspective -- it's better to not have
> any
> >>>>> areas of code with bus-factor 1. Feeling the pain of not getting
> reviews
> >>>> in
> >>>>> these areas with only a single active committer encourages us to find
> >>>>> solutions - either by deprecating "niche" features (as we once did
> with
> >>>>> various 'contrib' projects) or by recruiting new committers who have
> >>>>> interest in maintaining that code area. Lifting review restrictions
> would
> >>>>> allow us to sweep bus-factor issues under the rug.
> >>>>>
> >>>>> That said, I think CTR can be a valuable tool for "move fast on new
> >>>>> experimentation" type projects -- I've participated in several
> feature
> >>>>> branches in HDFS where we operated on CTR on the branch. The
> assumption
> >>>> was
> >>>>> that, prior to merging the branch into trunk, all of the patches (or
> a
> >>>>> consolidated mega-patch) would be thoroughly reviewed by several
> other
> >>>>> committers. I found this to work very well during early development,
> >>>> since
> >>>>> I could hack on things and even commit pieces of code that had known
> >>>>> issues/TODOs. For trickier patches on the CTR branch, I still tended
> to
> >>>>> ping experienced contributors for their opinions and feedback before
> >>>>> committing.
> >>>>>
> >>>>> Perhaps such a hybrid policy would work well in the context of HBase
> >>>>> feature development as well?
> >>>>>
> >>>>> -Todd
> >>>>>
> >>>>>
> >>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <
> apurtell@apache.org>
> >>>>> wrote:
> >>>>>
> >>>>>> Just thought of branch merge votes. Sorry for that omission. I
> think it
> >>>>>> makes sense to keep those, but let's discuss that too in this
> context.
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <
> apurtell@apache.org>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> As Greg Stein said on a thread over at general@incubator
> >>>>> ("[DISCUSSION]
> >>>>>>> Graduate Ignite from the Apache Incubator"):
> >>>>>>>
> >>>>>>> I always translate RTC as "we don't trust you, so somebody else
> must
> >>>>>>> approve anything you do." To me, that is a lousy basis for
> creating a
> >>>>>>> community. Trust and peer respect should be the basis, which
> implies
> >>>>> CTR.
> >>>>>>>
> >>>>>>> I happen to agree with this sentiment and would like to propose
> >>>>> switching
> >>>>>>> our consensus on commit procedure from RTC to CTR. Concurrently, I
> >>>>> would
> >>>>>>> like to propose this consensus also include the following:
> >>>>>>> - Checkins should pass precommit or the committer should explain on
> >>>> the
> >>>>>>> JIRA why precommit failures are in their best judgement not related
> >>>>>>> - The PMC should be willing to, ultimately, revoke committership
> >>>> should
> >>>>>>> trust in any individual committer be discovered to be misplaced. I
> >>>>> would
> >>>>>>> expect such a last resort to be exceedingly rare and likely never
> to
> >>>>>> happen
> >>>>>>> because of course long before that we would be setting correct
> public
> >>>>>>> examples in the first place and respectfully counseling well
> >>>>> intentioned
> >>>>>>> committers who might stray.
> >>>>>>>
> >>>>>>> Depending on how the conversation proceeds here I would like to
> >>>> include
> >>>>>>> dev@ in this conversation at the earliest opportunity as well.
> >>>>>>>
> >>>>>>> Thoughts?
> >>>>>>>
> >>>>>>> --
> >>>>>>> Best regards,
> >>>>>>>
> >>>>>>>  - Andy
> >>>>>>>
> >>>>>>> Problems worthy of attack prove their worth by hitting back. - Piet
> >>>>> Hein
> >>>>>>> (via Tom White)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Best regards,
> >>>>>>
> >>>>>>  - Andy
> >>>>>>
> >>>>>> Problems worthy of attack prove their worth by hitting back. - Piet
> >>>> Hein
> >>>>>> (via Tom White)
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Todd Lipcon
> >>>>> Software Engineer, Cloudera
> >>>
> >>>
> >>>
> >>> --
> >>> Sean
> >
> >
> >
> > --
> > Thanks,
> > Michael Antonov
>

Re: [DISCUSSION] Switching from RTC to CTR

[Andrew Purtell <an...@gmail.com>]

Had this thought after getting back on the road. As an alternative to any sweeping change we could do one incremental but very significant thing that acknowledges our status as trusted and busy peers: After posting a patch to JIRA, after one week if there is no review or veto, a committer can commit their own work.


> On Jul 29, 2015, at 2:20 PM, Mikhail Antonov <ol...@gmail.com> wrote:
> 
> Just curious, I assume if this change is made, would it only apply to
> master branch?
> 
> -Mikhail
> 
> On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
> <an...@gmail.com> wrote:
>> @dev is now CCed
>> 
>> I didn't want to over structure the discussion with too much detail up front. I do think CTR without supporting process or boundaries can be more problematic than helpful. That could take the form of customarily waiting for reviews before commit even under a CTR regime. I think this discussion has been great so far. I would also add that CTR moves 'R' from a gating requirement per commit (which can be hard to overcome for niche areas or when volunteer resources are less available) more to RMs. will be back later with more.
>> 
>> 
>>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com> wrote:
>>> 
>>> I'd also favor having this discussion on dev@.
>>> 
>>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <gh...@gmail.com> wrote:
>>>> 
>>>> This is already a really interesting and meaningful discussion, and is
>>>> obviously important to the community.
>>>> 
>>>> Is there any reason not to move this straight to the dev@ list?
>>>> 
>>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <to...@cloudera.com> wrote:
>>>>> 
>>>>> I'm not very active in HBase these days, so I won't cast a non-zero vote,
>>>>> but I'm -0 on this idea, for basically two reasons:
>>>>> 
>>>>> 1) In my experience at a past job which used CTR, the reality was that it
>>>>> was more like "Commit and maybe review" rather than "Commit then review".
>>>>> It's always more fun (and often easier) to write new code than to review
>>>>> other people's code, so if there isn't a requirement that all code gets
>>>>> reviewed before commit, it's easy for the ratio of code written to code
>>>>> reviewed to tend towards values significantly greater than 1:1 over time.
>>>>> At my past job, this meant that a lot of code made it into production (it
>>>>> was a website) that hadn't ever been reviewed, and in many cases we found
>>>>> bugs (or other issues) that would have definitely been caught by a good
>>>>> code reviewer.
>>>>> 
>>>>> 2) CTR has both the advantage and disadvantage of allowing areas of code
>>>> to
>>>>> be evolved quickly by a single person. That could be seen as a plus, in
>>>>> that there are some areas which tend to get ignored because we don't
>>>> have a
>>>>> critical mass of people reviewing patches in the area -- patches languish
>>>>> in these areas currently. However, that could also be seen as a good
>>>> thing
>>>>> from a "community over code" perspective -- it's better to not have any
>>>>> areas of code with bus-factor 1. Feeling the pain of not getting reviews
>>>> in
>>>>> these areas with only a single active committer encourages us to find
>>>>> solutions - either by deprecating "niche" features (as we once did with
>>>>> various 'contrib' projects) or by recruiting new committers who have
>>>>> interest in maintaining that code area. Lifting review restrictions would
>>>>> allow us to sweep bus-factor issues under the rug.
>>>>> 
>>>>> That said, I think CTR can be a valuable tool for "move fast on new
>>>>> experimentation" type projects -- I've participated in several feature
>>>>> branches in HDFS where we operated on CTR on the branch. The assumption
>>>> was
>>>>> that, prior to merging the branch into trunk, all of the patches (or a
>>>>> consolidated mega-patch) would be thoroughly reviewed by several other
>>>>> committers. I found this to work very well during early development,
>>>> since
>>>>> I could hack on things and even commit pieces of code that had known
>>>>> issues/TODOs. For trickier patches on the CTR branch, I still tended to
>>>>> ping experienced contributors for their opinions and feedback before
>>>>> committing.
>>>>> 
>>>>> Perhaps such a hybrid policy would work well in the context of HBase
>>>>> feature development as well?
>>>>> 
>>>>> -Todd
>>>>> 
>>>>> 
>>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <ap...@apache.org>
>>>>> wrote:
>>>>> 
>>>>>> Just thought of branch merge votes. Sorry for that omission. I think it
>>>>>> makes sense to keep those, but let's discuss that too in this context.
>>>>>> 
>>>>>> 
>>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <ap...@apache.org>
>>>>>> wrote:
>>>>>> 
>>>>>>> As Greg Stein said on a thread over at general@incubator
>>>>> ("[DISCUSSION]
>>>>>>> Graduate Ignite from the Apache Incubator"):
>>>>>>> 
>>>>>>> I always translate RTC as "we don't trust you, so somebody else must
>>>>>>> approve anything you do." To me, that is a lousy basis for creating a
>>>>>>> community. Trust and peer respect should be the basis, which implies
>>>>> CTR.
>>>>>>> 
>>>>>>> I happen to agree with this sentiment and would like to propose
>>>>> switching
>>>>>>> our consensus on commit procedure from RTC to CTR. Concurrently, I
>>>>> would
>>>>>>> like to propose this consensus also include the following:
>>>>>>> - Checkins should pass precommit or the committer should explain on
>>>> the
>>>>>>> JIRA why precommit failures are in their best judgement not related
>>>>>>> - The PMC should be willing to, ultimately, revoke committership
>>>> should
>>>>>>> trust in any individual committer be discovered to be misplaced. I
>>>>> would
>>>>>>> expect such a last resort to be exceedingly rare and likely never to
>>>>>> happen
>>>>>>> because of course long before that we would be setting correct public
>>>>>>> examples in the first place and respectfully counseling well
>>>>> intentioned
>>>>>>> committers who might stray.
>>>>>>> 
>>>>>>> Depending on how the conversation proceeds here I would like to
>>>> include
>>>>>>> dev@ in this conversation at the earliest opportunity as well.
>>>>>>> 
>>>>>>> Thoughts?
>>>>>>> 
>>>>>>> --
>>>>>>> Best regards,
>>>>>>> 
>>>>>>>  - Andy
>>>>>>> 
>>>>>>> Problems worthy of attack prove their worth by hitting back. - Piet
>>>>> Hein
>>>>>>> (via Tom White)
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Best regards,
>>>>>> 
>>>>>>  - Andy
>>>>>> 
>>>>>> Problems worthy of attack prove their worth by hitting back. - Piet
>>>> Hein
>>>>>> (via Tom White)
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Todd Lipcon
>>>>> Software Engineer, Cloudera
>>> 
>>> 
>>> 
>>> --
>>> Sean
> 
> 
> 
> -- 
> Thanks,
> Michael Antonov

Re: [DISCUSSION] Switching from RTC to CTR

[Mikhail Antonov <ol...@gmail.com>]

Just curious, I assume if this change is made, would it only apply to
master branch?

-Mikhail

On Wed, Jul 29, 2015 at 2:09 PM, Andrew Purtell
<an...@gmail.com> wrote:
> @dev is now CCed
>
> I didn't want to over structure the discussion with too much detail up front. I do think CTR without supporting process or boundaries can be more problematic than helpful. That could take the form of customarily waiting for reviews before commit even under a CTR regime. I think this discussion has been great so far. I would also add that CTR moves 'R' from a gating requirement per commit (which can be hard to overcome for niche areas or when volunteer resources are less available) more to RMs. will be back later with more.
>
>
>> On Jul 29, 2015, at 1:36 PM, Sean Busbey <se...@gmail.com> wrote:
>>
>> I'd also favor having this discussion on dev@.
>>
>>> On Wed, Jul 29, 2015 at 2:29 PM, Gary Helmling <gh...@gmail.com> wrote:
>>>
>>> This is already a really interesting and meaningful discussion, and is
>>> obviously important to the community.
>>>
>>> Is there any reason not to move this straight to the dev@ list?
>>>
>>>> On Wed, Jul 29, 2015 at 11:40 AM Todd Lipcon <to...@cloudera.com> wrote:
>>>>
>>>> I'm not very active in HBase these days, so I won't cast a non-zero vote,
>>>> but I'm -0 on this idea, for basically two reasons:
>>>>
>>>> 1) In my experience at a past job which used CTR, the reality was that it
>>>> was more like "Commit and maybe review" rather than "Commit then review".
>>>> It's always more fun (and often easier) to write new code than to review
>>>> other people's code, so if there isn't a requirement that all code gets
>>>> reviewed before commit, it's easy for the ratio of code written to code
>>>> reviewed to tend towards values significantly greater than 1:1 over time.
>>>> At my past job, this meant that a lot of code made it into production (it
>>>> was a website) that hadn't ever been reviewed, and in many cases we found
>>>> bugs (or other issues) that would have definitely been caught by a good
>>>> code reviewer.
>>>>
>>>> 2) CTR has both the advantage and disadvantage of allowing areas of code
>>> to
>>>> be evolved quickly by a single person. That could be seen as a plus, in
>>>> that there are some areas which tend to get ignored because we don't
>>> have a
>>>> critical mass of people reviewing patches in the area -- patches languish
>>>> in these areas currently. However, that could also be seen as a good
>>> thing
>>>> from a "community over code" perspective -- it's better to not have any
>>>> areas of code with bus-factor 1. Feeling the pain of not getting reviews
>>> in
>>>> these areas with only a single active committer encourages us to find
>>>> solutions - either by deprecating "niche" features (as we once did with
>>>> various 'contrib' projects) or by recruiting new committers who have
>>>> interest in maintaining that code area. Lifting review restrictions would
>>>> allow us to sweep bus-factor issues under the rug.
>>>>
>>>> That said, I think CTR can be a valuable tool for "move fast on new
>>>> experimentation" type projects -- I've participated in several feature
>>>> branches in HDFS where we operated on CTR on the branch. The assumption
>>> was
>>>> that, prior to merging the branch into trunk, all of the patches (or a
>>>> consolidated mega-patch) would be thoroughly reviewed by several other
>>>> committers. I found this to work very well during early development,
>>> since
>>>> I could hack on things and even commit pieces of code that had known
>>>> issues/TODOs. For trickier patches on the CTR branch, I still tended to
>>>> ping experienced contributors for their opinions and feedback before
>>>> committing.
>>>>
>>>> Perhaps such a hybrid policy would work well in the context of HBase
>>>> feature development as well?
>>>>
>>>> -Todd
>>>>
>>>>
>>>> On Wed, Jul 29, 2015 at 11:27 AM, Andrew Purtell <ap...@apache.org>
>>>> wrote:
>>>>
>>>>> Just thought of branch merge votes. Sorry for that omission. I think it
>>>>> makes sense to keep those, but let's discuss that too in this context.
>>>>>
>>>>>
>>>>> On Wed, Jul 29, 2015 at 11:26 AM, Andrew Purtell <ap...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> As Greg Stein said on a thread over at general@incubator
>>>> ("[DISCUSSION]
>>>>>> Graduate Ignite from the Apache Incubator"):
>>>>>>
>>>>>> I always translate RTC as "we don't trust you, so somebody else must
>>>>>> approve anything you do." To me, that is a lousy basis for creating a
>>>>>> community. Trust and peer respect should be the basis, which implies
>>>> CTR.
>>>>>>
>>>>>> I happen to agree with this sentiment and would like to propose
>>>> switching
>>>>>> our consensus on commit procedure from RTC to CTR. Concurrently, I
>>>> would
>>>>>> like to propose this consensus also include the following:
>>>>>> - Checkins should pass precommit or the committer should explain on
>>> the
>>>>>> JIRA why precommit failures are in their best judgement not related
>>>>>> - The PMC should be willing to, ultimately, revoke committership
>>> should
>>>>>> trust in any individual committer be discovered to be misplaced. I
>>>> would
>>>>>> expect such a last resort to be exceedingly rare and likely never to
>>>>> happen
>>>>>> because of course long before that we would be setting correct public
>>>>>> examples in the first place and respectfully counseling well
>>>> intentioned
>>>>>> committers who might stray.
>>>>>>
>>>>>> Depending on how the conversation proceeds here I would like to
>>> include
>>>>>> dev@ in this conversation at the earliest opportunity as well.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> --
>>>>>> Best regards,
>>>>>>
>>>>>>   - Andy
>>>>>>
>>>>>> Problems worthy of attack prove their worth by hitting back. - Piet
>>>> Hein
>>>>>> (via Tom White)
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best regards,
>>>>>
>>>>>   - Andy
>>>>>
>>>>> Problems worthy of attack prove their worth by hitting back. - Piet
>>> Hein
>>>>> (via Tom White)
>>>>
>>>>
>>>>
>>>> --
>>>> Todd Lipcon
>>>> Software Engineer, Cloudera
>>
>>
>>
>> --
>> Sean



-- 
Thanks,
Michael Antonov