You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@hbase.apache.org by yu...@gmail.com on 2011/12/12 15:43:46 UTC

Code review request for hbase-4120 table priority

Hi,
4120 has gone through more than 20 revisions. 

Please provide your comments. 

I plan to integrate it this week. 

Thanks

Re: Code review request for hbase-4120 table priority

Posted by Todd Lipcon <to...@cloudera.com>.

On Mon, Dec 12, 2011 at 4:30 PM, Andrew Purtell <ap...@apache.org> wrote:
> I've become aware of several private forks of HDFS and HBase. Too bad. A pooling of dev resources would have almost surely have been better.

BTW, to this point -- some of the private forks of HDFS and HBase are
due to the opposite problem. For example, FB branched off at 0.89 and
still uses that branch in production because they've seen the trunk
moving too *fast* and accepting too much new stuff. (at least
according to emails a couple months ago - apologies if I misunderstood
that and putting words in people's mouths)

So in summary I think we have to find the balance. IMO our balance has
been too far towards moving fast and not towards stability in the last
year. As we grow up we need to shift back towards stability of the
core.

-Todd
-- 
Todd Lipcon
Software Engineer, Cloudera

Re: Code review request for hbase-4120 table priority

Posted by Stack <st...@duboce.net>.

On Mon, Dec 12, 2011 at 11:23 PM, lars hofhansl <lh...@yahoo.com> wrote:
> While I haven't looked (in depth) at the patch, yet, this is definitely a feature that will be extremely helpful
> for Salesforce's multitenant architecture to isolate tenants and services from each other.
>
> While we don't have HBase in our production data centers, yet (working on it), I am certain that we will use this feature
> eventually.
>

I think it fair to say we all need better support for multi-tenancy
not only in hbase itself but also down through the hadoop layers.

St.Ack

Re: a feature branch sounds like a great idea, but... (was Re: Code review request for hbase-4120 table priority)

Posted by lars hofhansl <lh...@yahoo.com>.

I would also agree with Andy.

If multiple contributors/committers wanted to collaborate on a patch they can always use git and github (or whatever method *they* prefer)
for their collaborationand produce a patch when they're done.


In any event, the end result would just be a patch that (hopefully) eventually is accepted and applied to main HBase repository.

I don't think we need (and want) to mandate any policy here, nor do we want major features languish in a separate repository
that likely nobody is going to use.


-- Lars


----- Original Message -----
From: Ted Yu <yu...@gmail.com>
To: dev@hbase.apache.org
Cc: Andrew Purtell <ap...@apache.org>
Sent: Friday, December 16, 2011 11:43 AM
Subject: Re: a feature branch sounds like a great idea, but... (was Re: Code review request for hbase-4120 table priority)

Jonathan:
If the feature is very big and cannot be partitioned, I agree with what you
said.

I can think of few examples of such features. Take a look at HBASE-2856,
HBASE-451, etc. They got completed without resorting to feature branch.

Cheers

On Fri, Dec 16, 2011 at 10:43 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:

> I'll make a few points and then leave it to you all to decide on how
> to proceed.  (Ideally, Lui Jia would chime in as well!)
>
> I agree about the perceived importance of being in an asf repo, which
> is why being a branch in the asf repo seems like an appealing approach
> -- it addresses the desire to be inclusive, and to address the
> concerns of more conservative members of the community.
>
> I'd argue that difficult isn't the same as impossible -- other
> projects have managed branches just fine.  Hadoop is an example that
> has had many branches (HA, append, security, etc) that
> for-better-or-for-worse eventually were or will be merged or
> abandoned.
>
> A feature branch allows the patch to get in which allows other folks
> to work on the patch.  Instead of a single mega-patch by a single
> person, the basics could be committed in the branch and follow on work
> could be broken up into manageably sized, quicker to review patches
> done by multiple people.  When it is ready, the merge happens.
> Admittedly I've only done merges in github-land, but since most of us
> I assume use git, the merge could happen there and could potentially
> just be applied to svn as a series of patches on mainline trunk when
> it is time.
>
> The feature branches that have the most merge pains ones that have
> cross-cutting changes.  In the particular case of HBASE-4120, the code
> only touches 3-4 existing files (RPC related) so I'd think the merge
> pain is relatively low (unless we change the rpc servers anytime
> soon).
>
> Jon.
>
> On Dec 14, 2011, at 12:13, Ted Yu <yu...@gmail.com> wrote:
>
> > I agree with Andy.
> >
> > Maintaining a patch that cleanly applies to TRUNK is a lot easier.
> >
> > Cheers
> >
> > On Wed, Dec 14, 2011 at 11:41 AM, Andrew Purtell <apurtell@apache.org
> >wrote:
> >
> >> We have discussed feature branches in the past, and even attempted it
> once
> >> or twice (if I recall correctly, both with coprocessors and security),
> but
> >> Subversion merge support is lacking. I assert ASF tooling is suboptimal
> for
> >> managing a long lived feature branch. I'm happy to be properly educated
> if
> >> you disagree.
> >>
> >> Otherwise, this leads the discussion to external hosting of feature
> >> branches under development, where there is more appropriate tooling,
> such
> >> as up on GitHub.
> >>
> >> Trend put up our security work on a public GitHub repository. I
> routinely
> >> advertised it in any security related discussion. However, it was
> ignored
> >> by the community: As far as I know, nobody ever checked it out and tried
> >> it. Only code in the ASF repo seemed to matter.
> >>
> >> Best regards,
> >>
> >>
> >>   - Andy
> >>
> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >> (via Tom White)
> >>
> >>
> >> ----- Original Message -----
> >>> From: Jonathan Hsieh <jo...@cloudera.com>
> >>> To: dev@hbase.apache.org
> >>> Cc: Andrew Purtell <ap...@apache.org>
> >>> Sent: Tuesday, December 13, 2011 11:57 AM
> >>> Subject: Re: Code review request for hbase-4120 table priority
> >>>
> >>> Note: I've only done a quick look at the jira and the code.  The high
> >> level
> >>> design document/approach seems reasonable and I think most agree that
> >> this
> >>> is a useful feature and that a lot of effort has gone into it.
> >>>
> >>> The feature is off by default -- I can see one main difference in this
> >>> situation compared to other major newish generally-considered
> >> experimental
> >>> or incomplete features (replication, off-heap slab cache, online schema
> >>> changes).  This feature doesn't have one of the current HBase
> committers
> >>> using/testing it in their production  environments or in their test
> >>> environment.
> >>>
> >>> This seems perfect for *a feature branch* as we talked briefly about at
> >> the
> >>> Pow-wow.  There seem to be some problems identified that will result in
> >>> follow on issues (races mentioned).  Using a branch would:
> >>> * make it available at apache allows devs to test it
> >>> * allows a committer who is championing this to test it by using it
> more
> >>> and to iron out glaring problems in environment,
> >>> * encourages and shepards the contributor allowing them to justify
> >>> continued effort,
> >>> * allows all of us to defer the decision to fold the feature into 0.94
> >> (or
> >>> 0.96, or later) when more folks are familiar or comfortable with it.
> >>>
> >>> Who knows, maybe some of the TaoBao folks will eventually become
> >> committers.
> >>>
> >>> Jon.
> >>>
> >>> On Tue, Dec 13, 2011 at 12:42 AM, <yu...@gmail.com> wrote:
> >>>
> >>>> Thanks for the suggestion, Lars.
> >>>> The original scope for 4120 is bigger than the latest patch which only
> >>>> covers table priorities.
> >>>>
> >>>> Let's perform more reviews for the current patch. We can create more
> >>>> subtasks for the umbrella feature.
> >>>>
> >>>> Cheers
> >>>>
> >>>>
> >>>>
> >>>> On Dec 12, 2011, at 11:23 PM, lars hofhansl <lh...@yahoo.com>
> >>> wrote:
> >>>>
> >>>>> While I haven't looked (in depth) at the patch, yet, this is
> >>> definitely
> >>>> a feature that will be extremely helpful
> >>>>> for Salesforce's multitenant architecture to isolate tenants and
> >>>> services from each other.
> >>>>>
> >>>>> While we don't have HBase in our production data centers, yet
> >>> (working
> >>>> on it), I am certain that we will use this feature
> >>>>> eventually.
> >>>>>
> >>>>> Would it help to break the patch into multiple smaller patches?
> >>>>>
> >>>>> Off the bat I think of:
> >>>>> 1. the grouping logic
> >>>>> 2. regionserver configuration (caching, etc) per group
> >>>>> 3. table priorities
> >>>>> 4. etc... (folks who have actually looked at the patch can probably
> >>>> identify better demarcations between the aspects of this change.)
> >>>>>
> >>>>> That would certainly make it more manageable for me - personally - to
> >>>> review the code.
> >>>>>
> >>>>> -- Lars
> >>>>>
> >>>>>
> >>>>> ----- Original Message -----
> >>>>> From: Todd Lipcon <to...@cloudera.com>
> >>>>> To: dev@hbase.apache.org; Andrew Purtell <ap...@apache.org>
> >>>>> Cc:
> >>>>> Sent: Monday, December 12, 2011 4:55 PM
> >>>>> Subject: Re: Code review request for hbase-4120 table priority
> >>>>>
> >>>>> On Mon, Dec 12, 2011 at 4:36 PM, Andrew Purtell
> >>> <ap...@apache.org>
> >>>> wrote:
> >>>>>>
> >>>>>> HBase as a project should not have as a criteria for inclusion of
> >>> some
> >>>> feature that Cloudera and SU and Facebook run it. Core managed to
> >> escape
> >>>> Yahoo. Let's not run history in reverse here in HBase land. And,
> >>> actually,
> >>>> this makes it worse, because the the occurrence that a number of core
> >> HBase
> >>>> users (multiple) will all need something is substantially less likely
> >> than
> >>>> if one might find it useful; or, maybe, only users outside of those
> >> with
> >>>> such self-appointed attitude, yet perhaps a community multiples in
> >> size of
> >>>> "core users".
> >>>>>
> >>>>> It's not about Cloudera/SU/FB - it's about code that will be
> >>> supported
> >>>>> by people who are committed to the project. TrendMicro certainly fits
> >>>>> the bill. I of course mean no offense to Lu Jia, but neither he nor
> >>>>> Taobao has made continued contributions in the past - just one other
> >>>>> bug fix beyond the HBASE-4120 project.
> >>>>>
> >>>>> If we have a few of the core people committed to running this in
> >>>>> production and supporting it in the future, I'm all for it (just
> >>> like
> >>>>> I am +1 on security). I just want to avoid repeating mistakes like
> >> the
> >>>>> Avro server which isn't really supported despite being in our
> >>>>> codebase. (You'll note this was a Cloudera contribution but from a
> >>>>> contributor who was doing this in his spare time rather than part of
> >>>>> job responsibilities, and we have never run it in production
> >>>>> scenarios)
> >>>>>
> >>>>> I am consistently conservative on what goes into the project because
> >>>>> we have to stand behind what we release. I certainly don't think
> >>> _all_
> >>>>> core people should find every feature useful (eg REST and Thrift are
> >>>>> examples of some things which are useless to many but I think make
> >>>>> sense). But if _no_ core people see a feature as a requirement then
> >>>>> I'd rather let it bake until we have many people requesting it.
> >>>>> Otherwise people download HBase, try out these "fringe"
> >>> features, and
> >>>>> get a bad taste in their mouth when they've bit-rot across several
> >>>>> versions of little usage.
> >>>>>
> >>>>> -Todd
> >>>>> --
> >>>>> Todd Lipcon
> >>>>> Software Engineer, Cloudera
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> // Jonathan Hsieh (shay)
> >>> // Software Engineer, Cloudera
> >>> // jon@cloudera.com
> >>>
> >>
>

Re: a feature branch sounds like a great idea, but... (was Re: Code review request for hbase-4120 table priority)

Posted by Ted Yu <yu...@gmail.com>.

Jonathan:
If the feature is very big and cannot be partitioned, I agree with what you
said.

I can think of few examples of such features. Take a look at HBASE-2856,
HBASE-451, etc. They got completed without resorting to feature branch.

Cheers

On Fri, Dec 16, 2011 at 10:43 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:

> I'll make a few points and then leave it to you all to decide on how
> to proceed.  (Ideally, Lui Jia would chime in as well!)
>
> I agree about the perceived importance of being in an asf repo, which
> is why being a branch in the asf repo seems like an appealing approach
> -- it addresses the desire to be inclusive, and to address the
> concerns of more conservative members of the community.
>
> I'd argue that difficult isn't the same as impossible -- other
> projects have managed branches just fine.  Hadoop is an example that
> has had many branches (HA, append, security, etc) that
> for-better-or-for-worse eventually were or will be merged or
> abandoned.
>
> A feature branch allows the patch to get in which allows other folks
> to work on the patch.  Instead of a single mega-patch by a single
> person, the basics could be committed in the branch and follow on work
> could be broken up into manageably sized, quicker to review patches
> done by multiple people.  When it is ready, the merge happens.
> Admittedly I've only done merges in github-land, but since most of us
> I assume use git, the merge could happen there and could potentially
> just be applied to svn as a series of patches on mainline trunk when
> it is time.
>
> The feature branches that have the most merge pains ones that have
> cross-cutting changes.  In the particular case of HBASE-4120, the code
> only touches 3-4 existing files (RPC related) so I'd think the merge
> pain is relatively low (unless we change the rpc servers anytime
> soon).
>
> Jon.
>
> On Dec 14, 2011, at 12:13, Ted Yu <yu...@gmail.com> wrote:
>
> > I agree with Andy.
> >
> > Maintaining a patch that cleanly applies to TRUNK is a lot easier.
> >
> > Cheers
> >
> > On Wed, Dec 14, 2011 at 11:41 AM, Andrew Purtell <apurtell@apache.org
> >wrote:
> >
> >> We have discussed feature branches in the past, and even attempted it
> once
> >> or twice (if I recall correctly, both with coprocessors and security),
> but
> >> Subversion merge support is lacking. I assert ASF tooling is suboptimal
> for
> >> managing a long lived feature branch. I'm happy to be properly educated
> if
> >> you disagree.
> >>
> >> Otherwise, this leads the discussion to external hosting of feature
> >> branches under development, where there is more appropriate tooling,
> such
> >> as up on GitHub.
> >>
> >> Trend put up our security work on a public GitHub repository. I
> routinely
> >> advertised it in any security related discussion. However, it was
> ignored
> >> by the community: As far as I know, nobody ever checked it out and tried
> >> it. Only code in the ASF repo seemed to matter.
> >>
> >> Best regards,
> >>
> >>
> >>   - Andy
> >>
> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >> (via Tom White)
> >>
> >>
> >> ----- Original Message -----
> >>> From: Jonathan Hsieh <jo...@cloudera.com>
> >>> To: dev@hbase.apache.org
> >>> Cc: Andrew Purtell <ap...@apache.org>
> >>> Sent: Tuesday, December 13, 2011 11:57 AM
> >>> Subject: Re: Code review request for hbase-4120 table priority
> >>>
> >>> Note: I've only done a quick look at the jira and the code.  The high
> >> level
> >>> design document/approach seems reasonable and I think most agree that
> >> this
> >>> is a useful feature and that a lot of effort has gone into it.
> >>>
> >>> The feature is off by default -- I can see one main difference in this
> >>> situation compared to other major newish generally-considered
> >> experimental
> >>> or incomplete features (replication, off-heap slab cache, online schema
> >>> changes).  This feature doesn't have one of the current HBase
> committers
> >>> using/testing it in their production  environments or in their test
> >>> environment.
> >>>
> >>> This seems perfect for *a feature branch* as we talked briefly about at
> >> the
> >>> Pow-wow.  There seem to be some problems identified that will result in
> >>> follow on issues (races mentioned).  Using a branch would:
> >>> * make it available at apache allows devs to test it
> >>> * allows a committer who is championing this to test it by using it
> more
> >>> and to iron out glaring problems in environment,
> >>> * encourages and shepards the contributor allowing them to justify
> >>> continued effort,
> >>> * allows all of us to defer the decision to fold the feature into 0.94
> >> (or
> >>> 0.96, or later) when more folks are familiar or comfortable with it.
> >>>
> >>> Who knows, maybe some of the TaoBao folks will eventually become
> >> committers.
> >>>
> >>> Jon.
> >>>
> >>> On Tue, Dec 13, 2011 at 12:42 AM, <yu...@gmail.com> wrote:
> >>>
> >>>> Thanks for the suggestion, Lars.
> >>>> The original scope for 4120 is bigger than the latest patch which only
> >>>> covers table priorities.
> >>>>
> >>>> Let's perform more reviews for the current patch. We can create more
> >>>> subtasks for the umbrella feature.
> >>>>
> >>>> Cheers
> >>>>
> >>>>
> >>>>
> >>>> On Dec 12, 2011, at 11:23 PM, lars hofhansl <lh...@yahoo.com>
> >>> wrote:
> >>>>
> >>>>> While I haven't looked (in depth) at the patch, yet, this is
> >>> definitely
> >>>> a feature that will be extremely helpful
> >>>>> for Salesforce's multitenant architecture to isolate tenants and
> >>>> services from each other.
> >>>>>
> >>>>> While we don't have HBase in our production data centers, yet
> >>> (working
> >>>> on it), I am certain that we will use this feature
> >>>>> eventually.
> >>>>>
> >>>>> Would it help to break the patch into multiple smaller patches?
> >>>>>
> >>>>> Off the bat I think of:
> >>>>> 1. the grouping logic
> >>>>> 2. regionserver configuration (caching, etc) per group
> >>>>> 3. table priorities
> >>>>> 4. etc... (folks who have actually looked at the patch can probably
> >>>> identify better demarcations between the aspects of this change.)
> >>>>>
> >>>>> That would certainly make it more manageable for me - personally - to
> >>>> review the code.
> >>>>>
> >>>>> -- Lars
> >>>>>
> >>>>>
> >>>>> ----- Original Message -----
> >>>>> From: Todd Lipcon <to...@cloudera.com>
> >>>>> To: dev@hbase.apache.org; Andrew Purtell <ap...@apache.org>
> >>>>> Cc:
> >>>>> Sent: Monday, December 12, 2011 4:55 PM
> >>>>> Subject: Re: Code review request for hbase-4120 table priority
> >>>>>
> >>>>> On Mon, Dec 12, 2011 at 4:36 PM, Andrew Purtell
> >>> <ap...@apache.org>
> >>>> wrote:
> >>>>>>
> >>>>>> HBase as a project should not have as a criteria for inclusion of
> >>> some
> >>>> feature that Cloudera and SU and Facebook run it. Core managed to
> >> escape
> >>>> Yahoo. Let's not run history in reverse here in HBase land. And,
> >>> actually,
> >>>> this makes it worse, because the the occurrence that a number of core
> >> HBase
> >>>> users (multiple) will all need something is substantially less likely
> >> than
> >>>> if one might find it useful; or, maybe, only users outside of those
> >> with
> >>>> such self-appointed attitude, yet perhaps a community multiples in
> >> size of
> >>>> "core users".
> >>>>>
> >>>>> It's not about Cloudera/SU/FB - it's about code that will be
> >>> supported
> >>>>> by people who are committed to the project. TrendMicro certainly fits
> >>>>> the bill. I of course mean no offense to Lu Jia, but neither he nor
> >>>>> Taobao has made continued contributions in the past - just one other
> >>>>> bug fix beyond the HBASE-4120 project.
> >>>>>
> >>>>> If we have a few of the core people committed to running this in
> >>>>> production and supporting it in the future, I'm all for it (just
> >>> like
> >>>>> I am +1 on security). I just want to avoid repeating mistakes like
> >> the
> >>>>> Avro server which isn't really supported despite being in our
> >>>>> codebase. (You'll note this was a Cloudera contribution but from a
> >>>>> contributor who was doing this in his spare time rather than part of
> >>>>> job responsibilities, and we have never run it in production
> >>>>> scenarios)
> >>>>>
> >>>>> I am consistently conservative on what goes into the project because
> >>>>> we have to stand behind what we release. I certainly don't think
> >>> _all_
> >>>>> core people should find every feature useful (eg REST and Thrift are
> >>>>> examples of some things which are useless to many but I think make
> >>>>> sense). But if _no_ core people see a feature as a requirement then
> >>>>> I'd rather let it bake until we have many people requesting it.
> >>>>> Otherwise people download HBase, try out these "fringe"
> >>> features, and
> >>>>> get a bad taste in their mouth when they've bit-rot across several
> >>>>> versions of little usage.
> >>>>>
> >>>>> -Todd
> >>>>> --
> >>>>> Todd Lipcon
> >>>>> Software Engineer, Cloudera
> >>>>>
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> // Jonathan Hsieh (shay)
> >>> // Software Engineer, Cloudera
> >>> // jon@cloudera.com
> >>>
> >>
>

Re: a feature branch sounds like a great idea, but... (was Re: Code review request for hbase-4120 table priority)

Posted by Jonathan Hsieh <jo...@cloudera.com>.

I'll make a few points and then leave it to you all to decide on how
to proceed.  (Ideally, Lui Jia would chime in as well!)

I agree about the perceived importance of being in an asf repo, which
is why being a branch in the asf repo seems like an appealing approach
-- it addresses the desire to be inclusive, and to address the
concerns of more conservative members of the community.

I'd argue that difficult isn't the same as impossible -- other
projects have managed branches just fine.  Hadoop is an example that
has had many branches (HA, append, security, etc) that
for-better-or-for-worse eventually were or will be merged or
abandoned.

A feature branch allows the patch to get in which allows other folks
to work on the patch.  Instead of a single mega-patch by a single
person, the basics could be committed in the branch and follow on work
could be broken up into manageably sized, quicker to review patches
done by multiple people.  When it is ready, the merge happens.
Admittedly I've only done merges in github-land, but since most of us
I assume use git, the merge could happen there and could potentially
just be applied to svn as a series of patches on mainline trunk when
it is time.

The feature branches that have the most merge pains ones that have
cross-cutting changes.  In the particular case of HBASE-4120, the code
only touches 3-4 existing files (RPC related) so I'd think the merge
pain is relatively low (unless we change the rpc servers anytime
soon).

Jon.

On Dec 14, 2011, at 12:13, Ted Yu <yu...@gmail.com> wrote:

> I agree with Andy.
>
> Maintaining a patch that cleanly applies to TRUNK is a lot easier.
>
> Cheers
>
> On Wed, Dec 14, 2011 at 11:41 AM, Andrew Purtell <ap...@apache.org>wrote:
>
>> We have discussed feature branches in the past, and even attempted it once
>> or twice (if I recall correctly, both with coprocessors and security), but
>> Subversion merge support is lacking. I assert ASF tooling is suboptimal for
>> managing a long lived feature branch. I'm happy to be properly educated if
>> you disagree.
>>
>> Otherwise, this leads the discussion to external hosting of feature
>> branches under development, where there is more appropriate tooling, such
>> as up on GitHub.
>>
>> Trend put up our security work on a public GitHub repository. I routinely
>> advertised it in any security related discussion. However, it was ignored
>> by the community: As far as I know, nobody ever checked it out and tried
>> it. Only code in the ASF repo seemed to matter.
>>
>> Best regards,
>>
>>
>>   - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>
>>
>> ----- Original Message -----
>>> From: Jonathan Hsieh <jo...@cloudera.com>
>>> To: dev@hbase.apache.org
>>> Cc: Andrew Purtell <ap...@apache.org>
>>> Sent: Tuesday, December 13, 2011 11:57 AM
>>> Subject: Re: Code review request for hbase-4120 table priority
>>>
>>> Note: I've only done a quick look at the jira and the code.  The high
>> level
>>> design document/approach seems reasonable and I think most agree that
>> this
>>> is a useful feature and that a lot of effort has gone into it.
>>>
>>> The feature is off by default -- I can see one main difference in this
>>> situation compared to other major newish generally-considered
>> experimental
>>> or incomplete features (replication, off-heap slab cache, online schema
>>> changes).  This feature doesn't have one of the current HBase committers
>>> using/testing it in their production  environments or in their test
>>> environment.
>>>
>>> This seems perfect for *a feature branch* as we talked briefly about at
>> the
>>> Pow-wow.  There seem to be some problems identified that will result in
>>> follow on issues (races mentioned).  Using a branch would:
>>> * make it available at apache allows devs to test it
>>> * allows a committer who is championing this to test it by using it more
>>> and to iron out glaring problems in environment,
>>> * encourages and shepards the contributor allowing them to justify
>>> continued effort,
>>> * allows all of us to defer the decision to fold the feature into 0.94
>> (or
>>> 0.96, or later) when more folks are familiar or comfortable with it.
>>>
>>> Who knows, maybe some of the TaoBao folks will eventually become
>> committers.
>>>
>>> Jon.
>>>
>>> On Tue, Dec 13, 2011 at 12:42 AM, <yu...@gmail.com> wrote:
>>>
>>>> Thanks for the suggestion, Lars.
>>>> The original scope for 4120 is bigger than the latest patch which only
>>>> covers table priorities.
>>>>
>>>> Let's perform more reviews for the current patch. We can create more
>>>> subtasks for the umbrella feature.
>>>>
>>>> Cheers
>>>>
>>>>
>>>>
>>>> On Dec 12, 2011, at 11:23 PM, lars hofhansl <lh...@yahoo.com>
>>> wrote:
>>>>
>>>>> While I haven't looked (in depth) at the patch, yet, this is
>>> definitely
>>>> a feature that will be extremely helpful
>>>>> for Salesforce's multitenant architecture to isolate tenants and
>>>> services from each other.
>>>>>
>>>>> While we don't have HBase in our production data centers, yet
>>> (working
>>>> on it), I am certain that we will use this feature
>>>>> eventually.
>>>>>
>>>>> Would it help to break the patch into multiple smaller patches?
>>>>>
>>>>> Off the bat I think of:
>>>>> 1. the grouping logic
>>>>> 2. regionserver configuration (caching, etc) per group
>>>>> 3. table priorities
>>>>> 4. etc... (folks who have actually looked at the patch can probably
>>>> identify better demarcations between the aspects of this change.)
>>>>>
>>>>> That would certainly make it more manageable for me - personally - to
>>>> review the code.
>>>>>
>>>>> -- Lars
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>> From: Todd Lipcon <to...@cloudera.com>
>>>>> To: dev@hbase.apache.org; Andrew Purtell <ap...@apache.org>
>>>>> Cc:
>>>>> Sent: Monday, December 12, 2011 4:55 PM
>>>>> Subject: Re: Code review request for hbase-4120 table priority
>>>>>
>>>>> On Mon, Dec 12, 2011 at 4:36 PM, Andrew Purtell
>>> <ap...@apache.org>
>>>> wrote:
>>>>>>
>>>>>> HBase as a project should not have as a criteria for inclusion of
>>> some
>>>> feature that Cloudera and SU and Facebook run it. Core managed to
>> escape
>>>> Yahoo. Let's not run history in reverse here in HBase land. And,
>>> actually,
>>>> this makes it worse, because the the occurrence that a number of core
>> HBase
>>>> users (multiple) will all need something is substantially less likely
>> than
>>>> if one might find it useful; or, maybe, only users outside of those
>> with
>>>> such self-appointed attitude, yet perhaps a community multiples in
>> size of
>>>> "core users".
>>>>>
>>>>> It's not about Cloudera/SU/FB - it's about code that will be
>>> supported
>>>>> by people who are committed to the project. TrendMicro certainly fits
>>>>> the bill. I of course mean no offense to Lu Jia, but neither he nor
>>>>> Taobao has made continued contributions in the past - just one other
>>>>> bug fix beyond the HBASE-4120 project.
>>>>>
>>>>> If we have a few of the core people committed to running this in
>>>>> production and supporting it in the future, I'm all for it (just
>>> like
>>>>> I am +1 on security). I just want to avoid repeating mistakes like
>> the
>>>>> Avro server which isn't really supported despite being in our
>>>>> codebase. (You'll note this was a Cloudera contribution but from a
>>>>> contributor who was doing this in his spare time rather than part of
>>>>> job responsibilities, and we have never run it in production
>>>>> scenarios)
>>>>>
>>>>> I am consistently conservative on what goes into the project because
>>>>> we have to stand behind what we release. I certainly don't think
>>> _all_
>>>>> core people should find every feature useful (eg REST and Thrift are
>>>>> examples of some things which are useless to many but I think make
>>>>> sense). But if _no_ core people see a feature as a requirement then
>>>>> I'd rather let it bake until we have many people requesting it.
>>>>> Otherwise people download HBase, try out these "fringe"
>>> features, and
>>>>> get a bad taste in their mouth when they've bit-rot across several
>>>>> versions of little usage.
>>>>>
>>>>> -Todd
>>>>> --
>>>>> Todd Lipcon
>>>>> Software Engineer, Cloudera
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> // Jonathan Hsieh (shay)
>>> // Software Engineer, Cloudera
>>> // jon@cloudera.com
>>>
>>

Re: a feature branch sounds like a great idea, but... (was Re: Code review request for hbase-4120 table priority)

Posted by Ted Yu <yu...@gmail.com>.

I agree with Andy.

Maintaining a patch that cleanly applies to TRUNK is a lot easier.

Cheers

On Wed, Dec 14, 2011 at 11:41 AM, Andrew Purtell <ap...@apache.org>wrote:

> We have discussed feature branches in the past, and even attempted it once
> or twice (if I recall correctly, both with coprocessors and security), but
> Subversion merge support is lacking. I assert ASF tooling is suboptimal for
> managing a long lived feature branch. I'm happy to be properly educated if
> you disagree.
>
> Otherwise, this leads the discussion to external hosting of feature
> branches under development, where there is more appropriate tooling, such
> as up on GitHub.
>
> Trend put up our security work on a public GitHub repository. I routinely
> advertised it in any security related discussion. However, it was ignored
> by the community: As far as I know, nobody ever checked it out and tried
> it. Only code in the ASF repo seemed to matter.
>
> Best regards,
>
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
>
> ----- Original Message -----
> > From: Jonathan Hsieh <jo...@cloudera.com>
> > To: dev@hbase.apache.org
> > Cc: Andrew Purtell <ap...@apache.org>
> > Sent: Tuesday, December 13, 2011 11:57 AM
> > Subject: Re: Code review request for hbase-4120 table priority
> >
> > Note: I've only done a quick look at the jira and the code.  The high
> level
> > design document/approach seems reasonable and I think most agree that
> this
> > is a useful feature and that a lot of effort has gone into it.
> >
> > The feature is off by default -- I can see one main difference in this
> > situation compared to other major newish generally-considered
> experimental
> > or incomplete features (replication, off-heap slab cache, online schema
> > changes).  This feature doesn't have one of the current HBase committers
> > using/testing it in their production  environments or in their test
> > environment.
> >
> > This seems perfect for *a feature branch* as we talked briefly about at
> the
> > Pow-wow.  There seem to be some problems identified that will result in
> > follow on issues (races mentioned).  Using a branch would:
> > * make it available at apache allows devs to test it
> > * allows a committer who is championing this to test it by using it more
> > and to iron out glaring problems in environment,
> > * encourages and shepards the contributor allowing them to justify
> > continued effort,
> > * allows all of us to defer the decision to fold the feature into 0.94
> (or
> > 0.96, or later) when more folks are familiar or comfortable with it.
> >
> > Who knows, maybe some of the TaoBao folks will eventually become
> committers.
> >
> > Jon.
> >
> > On Tue, Dec 13, 2011 at 12:42 AM, <yu...@gmail.com> wrote:
> >
> >>  Thanks for the suggestion, Lars.
> >>  The original scope for 4120 is bigger than the latest patch which only
> >>  covers table priorities.
> >>
> >>  Let's perform more reviews for the current patch. We can create more
> >>  subtasks for the umbrella feature.
> >>
> >>  Cheers
> >>
> >>
> >>
> >>  On Dec 12, 2011, at 11:23 PM, lars hofhansl <lh...@yahoo.com>
> > wrote:
> >>
> >>  > While I haven't looked (in depth) at the patch, yet, this is
> > definitely
> >>  a feature that will be extremely helpful
> >>  > for Salesforce's multitenant architecture to isolate tenants and
> >>  services from each other.
> >>  >
> >>  > While we don't have HBase in our production data centers, yet
> > (working
> >>  on it), I am certain that we will use this feature
> >>  > eventually.
> >>  >
> >>  > Would it help to break the patch into multiple smaller patches?
> >>  >
> >>  > Off the bat I think of:
> >>  > 1. the grouping logic
> >>  > 2. regionserver configuration (caching, etc) per group
> >>  > 3. table priorities
> >>  > 4. etc... (folks who have actually looked at the patch can probably
> >>  identify better demarcations between the aspects of this change.)
> >>  >
> >>  > That would certainly make it more manageable for me - personally - to
> >>  review the code.
> >>  >
> >>  > -- Lars
> >>  >
> >>  >
> >>  > ----- Original Message -----
> >>  > From: Todd Lipcon <to...@cloudera.com>
> >>  > To: dev@hbase.apache.org; Andrew Purtell <ap...@apache.org>
> >>  > Cc:
> >>  > Sent: Monday, December 12, 2011 4:55 PM
> >>  > Subject: Re: Code review request for hbase-4120 table priority
> >>  >
> >>  > On Mon, Dec 12, 2011 at 4:36 PM, Andrew Purtell
> > <ap...@apache.org>
> >>  wrote:
> >>  >>
> >>  >> HBase as a project should not have as a criteria for inclusion of
> > some
> >>  feature that Cloudera and SU and Facebook run it. Core managed to
> escape
> >>  Yahoo. Let's not run history in reverse here in HBase land. And,
> > actually,
> >>  this makes it worse, because the the occurrence that a number of core
> HBase
> >>  users (multiple) will all need something is substantially less likely
> than
> >>  if one might find it useful; or, maybe, only users outside of those
> with
> >>  such self-appointed attitude, yet perhaps a community multiples in
> size of
> >>  "core users".
> >>  >
> >>  > It's not about Cloudera/SU/FB - it's about code that will be
> > supported
> >>  > by people who are committed to the project. TrendMicro certainly fits
> >>  > the bill. I of course mean no offense to Lu Jia, but neither he nor
> >>  > Taobao has made continued contributions in the past - just one other
> >>  > bug fix beyond the HBASE-4120 project.
> >>  >
> >>  > If we have a few of the core people committed to running this in
> >>  > production and supporting it in the future, I'm all for it (just
> > like
> >>  > I am +1 on security). I just want to avoid repeating mistakes like
> the
> >>  > Avro server which isn't really supported despite being in our
> >>  > codebase. (You'll note this was a Cloudera contribution but from a
> >>  > contributor who was doing this in his spare time rather than part of
> >>  > job responsibilities, and we have never run it in production
> >>  > scenarios)
> >>  >
> >>  > I am consistently conservative on what goes into the project because
> >>  > we have to stand behind what we release. I certainly don't think
> > _all_
> >>  > core people should find every feature useful (eg REST and Thrift are
> >>  > examples of some things which are useless to many but I think make
> >>  > sense). But if _no_ core people see a feature as a requirement then
> >>  > I'd rather let it bake until we have many people requesting it.
> >>  > Otherwise people download HBase, try out these "fringe"
> > features, and
> >>  > get a bad taste in their mouth when they've bit-rot across several
> >>  > versions of little usage.
> >>  >
> >>  > -Todd
> >>  > --
> >>  > Todd Lipcon
> >>  > Software Engineer, Cloudera
> >>  >
> >>
> >
> >
> >
> > --
> > // Jonathan Hsieh (shay)
> > // Software Engineer, Cloudera
> > // jon@cloudera.com
> >
>

a feature branch sounds like a great idea, but... (was Re: Code review request for hbase-4120 table priority)

Posted by Andrew Purtell <ap...@apache.org>.

We have discussed feature branches in the past, and even attempted it once or twice (if I recall correctly, both with coprocessors and security), but Subversion merge support is lacking. I assert ASF tooling is suboptimal for managing a long lived feature branch. I'm happy to be properly educated if you disagree.

Otherwise, this leads the discussion to external hosting of feature branches under development, where there is more appropriate tooling, such as up on GitHub.

Trend put up our security work on a public GitHub repository. I routinely advertised it in any security related discussion. However, it was ignored by the community: As far as I know, nobody ever checked it out and tried it. Only code in the ASF repo seemed to matter.

Best regards,


   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)


----- Original Message -----
> From: Jonathan Hsieh <jo...@cloudera.com>
> To: dev@hbase.apache.org
> Cc: Andrew Purtell <ap...@apache.org>
> Sent: Tuesday, December 13, 2011 11:57 AM
> Subject: Re: Code review request for hbase-4120 table priority
> 
> Note: I've only done a quick look at the jira and the code.  The high level
> design document/approach seems reasonable and I think most agree that this
> is a useful feature and that a lot of effort has gone into it.
> 
> The feature is off by default -- I can see one main difference in this
> situation compared to other major newish generally-considered experimental
> or incomplete features (replication, off-heap slab cache, online schema
> changes).  This feature doesn't have one of the current HBase committers
> using/testing it in their production  environments or in their test
> environment.
> 
> This seems perfect for *a feature branch* as we talked briefly about at the
> Pow-wow.  There seem to be some problems identified that will result in
> follow on issues (races mentioned).  Using a branch would:
> * make it available at apache allows devs to test it
> * allows a committer who is championing this to test it by using it more
> and to iron out glaring problems in environment,
> * encourages and shepards the contributor allowing them to justify
> continued effort,
> * allows all of us to defer the decision to fold the feature into 0.94 (or
> 0.96, or later) when more folks are familiar or comfortable with it.
> 
> Who knows, maybe some of the TaoBao folks will eventually become committers.
> 
> Jon.
> 
> On Tue, Dec 13, 2011 at 12:42 AM, <yu...@gmail.com> wrote:
> 
>>  Thanks for the suggestion, Lars.
>>  The original scope for 4120 is bigger than the latest patch which only
>>  covers table priorities.
>> 
>>  Let's perform more reviews for the current patch. We can create more
>>  subtasks for the umbrella feature.
>> 
>>  Cheers
>> 
>> 
>> 
>>  On Dec 12, 2011, at 11:23 PM, lars hofhansl <lh...@yahoo.com> 
> wrote:
>> 
>>  > While I haven't looked (in depth) at the patch, yet, this is 
> definitely
>>  a feature that will be extremely helpful
>>  > for Salesforce's multitenant architecture to isolate tenants and
>>  services from each other.
>>  >
>>  > While we don't have HBase in our production data centers, yet 
> (working
>>  on it), I am certain that we will use this feature
>>  > eventually.
>>  >
>>  > Would it help to break the patch into multiple smaller patches?
>>  >
>>  > Off the bat I think of:
>>  > 1. the grouping logic
>>  > 2. regionserver configuration (caching, etc) per group
>>  > 3. table priorities
>>  > 4. etc... (folks who have actually looked at the patch can probably
>>  identify better demarcations between the aspects of this change.)
>>  >
>>  > That would certainly make it more manageable for me - personally - to
>>  review the code.
>>  >
>>  > -- Lars
>>  >
>>  >
>>  > ----- Original Message -----
>>  > From: Todd Lipcon <to...@cloudera.com>
>>  > To: dev@hbase.apache.org; Andrew Purtell <ap...@apache.org>
>>  > Cc:
>>  > Sent: Monday, December 12, 2011 4:55 PM
>>  > Subject: Re: Code review request for hbase-4120 table priority
>>  >
>>  > On Mon, Dec 12, 2011 at 4:36 PM, Andrew Purtell 
> <ap...@apache.org>
>>  wrote:
>>  >>
>>  >> HBase as a project should not have as a criteria for inclusion of 
> some
>>  feature that Cloudera and SU and Facebook run it. Core managed to escape
>>  Yahoo. Let's not run history in reverse here in HBase land. And, 
> actually,
>>  this makes it worse, because the the occurrence that a number of core HBase
>>  users (multiple) will all need something is substantially less likely than
>>  if one might find it useful; or, maybe, only users outside of those with
>>  such self-appointed attitude, yet perhaps a community multiples in size of
>>  "core users".
>>  >
>>  > It's not about Cloudera/SU/FB - it's about code that will be 
> supported
>>  > by people who are committed to the project. TrendMicro certainly fits
>>  > the bill. I of course mean no offense to Lu Jia, but neither he nor
>>  > Taobao has made continued contributions in the past - just one other
>>  > bug fix beyond the HBASE-4120 project.
>>  >
>>  > If we have a few of the core people committed to running this in
>>  > production and supporting it in the future, I'm all for it (just 
> like
>>  > I am +1 on security). I just want to avoid repeating mistakes like the
>>  > Avro server which isn't really supported despite being in our
>>  > codebase. (You'll note this was a Cloudera contribution but from a
>>  > contributor who was doing this in his spare time rather than part of
>>  > job responsibilities, and we have never run it in production
>>  > scenarios)
>>  >
>>  > I am consistently conservative on what goes into the project because
>>  > we have to stand behind what we release. I certainly don't think 
> _all_
>>  > core people should find every feature useful (eg REST and Thrift are
>>  > examples of some things which are useless to many but I think make
>>  > sense). But if _no_ core people see a feature as a requirement then
>>  > I'd rather let it bake until we have many people requesting it.
>>  > Otherwise people download HBase, try out these "fringe" 
> features, and
>>  > get a bad taste in their mouth when they've bit-rot across several
>>  > versions of little usage.
>>  >
>>  > -Todd
>>  > --
>>  > Todd Lipcon
>>  > Software Engineer, Cloudera
>>  >
>> 
> 
> 
> 
> -- 
> // Jonathan Hsieh (shay)
> // Software Engineer, Cloudera
> // jon@cloudera.com
>

Re: Code review request for hbase-4120 table priority

Posted by Stack <st...@duboce.net>.

On Tue, Dec 13, 2011 at 11:57 AM, Jonathan Hsieh <jo...@cloudera.com> wrote:
> Note: I've only done a quick look at the jira and the code.  The high level
> design document/approach seems reasonable and I think most agree that this
> is a useful feature and that a lot of effort has gone into it.
>

Agreed.


> The feature is off by default -- I can see one main difference in this
> situation compared to other major newish generally-considered experimental
> or incomplete features (replication, off-heap slab cache, online schema
> changes).  This feature doesn't have one of the current HBase committers
> using/testing it in their production  environments or in their test
> environment.
>

A couple of the lads are shaping up to back this feature.


> This seems perfect for *a feature branch* as we talked briefly about at the
> Pow-wow.  There seem to be some problems identified that will result in
> follow on issues (races mentioned).  Using a branch would:
> * make it available at apache allows devs to test it
> * allows a committer who is championing this to test it by using it more
> and to iron out glaring problems in environment,
> * encourages and shepards the contributor allowing them to justify
> continued effort,
> * allows all of us to defer the decision to fold the feature into 0.94 (or
> 0.96, or later) when more folks are familiar or comfortable with it.
>

Agree.

> Who knows, maybe some of the TaoBao folks will eventually become committers.
>

Looking forward to the day....

St.Ack

Re: Code review request for hbase-4120 table priority

Posted by Jonathan Hsieh <jo...@cloudera.com>.

Note: I've only done a quick look at the jira and the code.  The high level
design document/approach seems reasonable and I think most agree that this
is a useful feature and that a lot of effort has gone into it.

The feature is off by default -- I can see one main difference in this
situation compared to other major newish generally-considered experimental
or incomplete features (replication, off-heap slab cache, online schema
changes).  This feature doesn't have one of the current HBase committers
using/testing it in their production  environments or in their test
environment.

This seems perfect for *a feature branch* as we talked briefly about at the
Pow-wow.  There seem to be some problems identified that will result in
follow on issues (races mentioned).  Using a branch would:
* make it available at apache allows devs to test it
* allows a committer who is championing this to test it by using it more
and to iron out glaring problems in environment,
* encourages and shepards the contributor allowing them to justify
continued effort,
* allows all of us to defer the decision to fold the feature into 0.94 (or
0.96, or later) when more folks are familiar or comfortable with it.

Who knows, maybe some of the TaoBao folks will eventually become committers.

Jon.

On Tue, Dec 13, 2011 at 12:42 AM, <yu...@gmail.com> wrote:

> Thanks for the suggestion, Lars.
> The original scope for 4120 is bigger than the latest patch which only
> covers table priorities.
>
> Let's perform more reviews for the current patch. We can create more
> subtasks for the umbrella feature.
>
> Cheers
>
>
>
> On Dec 12, 2011, at 11:23 PM, lars hofhansl <lh...@yahoo.com> wrote:
>
> > While I haven't looked (in depth) at the patch, yet, this is definitely
> a feature that will be extremely helpful
> > for Salesforce's multitenant architecture to isolate tenants and
> services from each other.
> >
> > While we don't have HBase in our production data centers, yet (working
> on it), I am certain that we will use this feature
> > eventually.
> >
> > Would it help to break the patch into multiple smaller patches?
> >
> > Off the bat I think of:
> > 1. the grouping logic
> > 2. regionserver configuration (caching, etc) per group
> > 3. table priorities
> > 4. etc... (folks who have actually looked at the patch can probably
> identify better demarcations between the aspects of this change.)
> >
> > That would certainly make it more manageable for me - personally - to
> review the code.
> >
> > -- Lars
> >
> >
> > ----- Original Message -----
> > From: Todd Lipcon <to...@cloudera.com>
> > To: dev@hbase.apache.org; Andrew Purtell <ap...@apache.org>
> > Cc:
> > Sent: Monday, December 12, 2011 4:55 PM
> > Subject: Re: Code review request for hbase-4120 table priority
> >
> > On Mon, Dec 12, 2011 at 4:36 PM, Andrew Purtell <ap...@apache.org>
> wrote:
> >>
> >> HBase as a project should not have as a criteria for inclusion of some
> feature that Cloudera and SU and Facebook run it. Core managed to escape
> Yahoo. Let's not run history in reverse here in HBase land. And, actually,
> this makes it worse, because the the occurrence that a number of core HBase
> users (multiple) will all need something is substantially less likely than
> if one might find it useful; or, maybe, only users outside of those with
> such self-appointed attitude, yet perhaps a community multiples in size of
> "core users".
> >
> > It's not about Cloudera/SU/FB - it's about code that will be supported
> > by people who are committed to the project. TrendMicro certainly fits
> > the bill. I of course mean no offense to Lu Jia, but neither he nor
> > Taobao has made continued contributions in the past - just one other
> > bug fix beyond the HBASE-4120 project.
> >
> > If we have a few of the core people committed to running this in
> > production and supporting it in the future, I'm all for it (just like
> > I am +1 on security). I just want to avoid repeating mistakes like the
> > Avro server which isn't really supported despite being in our
> > codebase. (You'll note this was a Cloudera contribution but from a
> > contributor who was doing this in his spare time rather than part of
> > job responsibilities, and we have never run it in production
> > scenarios)
> >
> > I am consistently conservative on what goes into the project because
> > we have to stand behind what we release. I certainly don't think _all_
> > core people should find every feature useful (eg REST and Thrift are
> > examples of some things which are useless to many but I think make
> > sense). But if _no_ core people see a feature as a requirement then
> > I'd rather let it bake until we have many people requesting it.
> > Otherwise people download HBase, try out these "fringe" features, and
> > get a bad taste in their mouth when they've bit-rot across several
> > versions of little usage.
> >
> > -Todd
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>



-- 
// Jonathan Hsieh (shay)
// Software Engineer, Cloudera
// jon@cloudera.com

Re: Code review request for hbase-4120 table priority

Posted by yu...@gmail.com.

Thanks for the suggestion, Lars. 
The original scope for 4120 is bigger than the latest patch which only covers table priorities. 

Let's perform more reviews for the current patch. We can create more subtasks for the umbrella feature. 

Cheers



On Dec 12, 2011, at 11:23 PM, lars hofhansl <lh...@yahoo.com> wrote:

> While I haven't looked (in depth) at the patch, yet, this is definitely a feature that will be extremely helpful
> for Salesforce's multitenant architecture to isolate tenants and services from each other.
> 
> While we don't have HBase in our production data centers, yet (working on it), I am certain that we will use this feature
> eventually.
> 
> Would it help to break the patch into multiple smaller patches?
> 
> Off the bat I think of:
> 1. the grouping logic
> 2. regionserver configuration (caching, etc) per group
> 3. table priorities
> 4. etc... (folks who have actually looked at the patch can probably identify better demarcations between the aspects of this change.)
> 
> That would certainly make it more manageable for me - personally - to review the code.
> 
> -- Lars
> 
> 
> ----- Original Message -----
> From: Todd Lipcon <to...@cloudera.com>
> To: dev@hbase.apache.org; Andrew Purtell <ap...@apache.org>
> Cc: 
> Sent: Monday, December 12, 2011 4:55 PM
> Subject: Re: Code review request for hbase-4120 table priority
> 
> On Mon, Dec 12, 2011 at 4:36 PM, Andrew Purtell <ap...@apache.org> wrote:
>> 
>> HBase as a project should not have as a criteria for inclusion of some feature that Cloudera and SU and Facebook run it. Core managed to escape Yahoo. Let's not run history in reverse here in HBase land. And, actually, this makes it worse, because the the occurrence that a number of core HBase users (multiple) will all need something is substantially less likely than if one might find it useful; or, maybe, only users outside of those with such self-appointed attitude, yet perhaps a community multiples in size of "core users".
> 
> It's not about Cloudera/SU/FB - it's about code that will be supported
> by people who are committed to the project. TrendMicro certainly fits
> the bill. I of course mean no offense to Lu Jia, but neither he nor
> Taobao has made continued contributions in the past - just one other
> bug fix beyond the HBASE-4120 project.
> 
> If we have a few of the core people committed to running this in
> production and supporting it in the future, I'm all for it (just like
> I am +1 on security). I just want to avoid repeating mistakes like the
> Avro server which isn't really supported despite being in our
> codebase. (You'll note this was a Cloudera contribution but from a
> contributor who was doing this in his spare time rather than part of
> job responsibilities, and we have never run it in production
> scenarios)
> 
> I am consistently conservative on what goes into the project because
> we have to stand behind what we release. I certainly don't think _all_
> core people should find every feature useful (eg REST and Thrift are
> examples of some things which are useless to many but I think make
> sense). But if _no_ core people see a feature as a requirement then
> I'd rather let it bake until we have many people requesting it.
> Otherwise people download HBase, try out these "fringe" features, and
> get a bad taste in their mouth when they've bit-rot across several
> versions of little usage.
> 
> -Todd
> -- 
> Todd Lipcon
> Software Engineer, Cloudera
>

Re: Code review request for hbase-4120 table priority

Posted by lars hofhansl <lh...@yahoo.com>.

While I haven't looked (in depth) at the patch, yet, this is definitely a feature that will be extremely helpful
for Salesforce's multitenant architecture to isolate tenants and services from each other.

While we don't have HBase in our production data centers, yet (working on it), I am certain that we will use this feature
eventually.

Would it help to break the patch into multiple smaller patches?

Off the bat I think of:
1. the grouping logic
2. regionserver configuration (caching, etc) per group
3. table priorities
4. etc... (folks who have actually looked at the patch can probably identify better demarcations between the aspects of this change.)

That would certainly make it more manageable for me - personally - to review the code.

-- Lars

----- Original Message -----
From: Todd Lipcon <to...@cloudera.com>
To: dev@hbase.apache.org; Andrew Purtell <ap...@apache.org>
Cc: 
Sent: Monday, December 12, 2011 4:55 PM
Subject: Re: Code review request for hbase-4120 table priority

On Mon, Dec 12, 2011 at 4:36 PM, Andrew Purtell <ap...@apache.org> wrote:
>
> HBase as a project should not have as a criteria for inclusion of some feature that Cloudera and SU and Facebook run it. Core managed to escape Yahoo. Let's not run history in reverse here in HBase land. And, actually, this makes it worse, because the the occurrence that a number of core HBase users (multiple) will all need something is substantially less likely than if one might find it useful; or, maybe, only users outside of those with such self-appointed attitude, yet perhaps a community multiples in size of "core users".

It's not about Cloudera/SU/FB - it's about code that will be supported
by people who are committed to the project. TrendMicro certainly fits
the bill. I of course mean no offense to Lu Jia, but neither he nor
Taobao has made continued contributions in the past - just one other
bug fix beyond the HBASE-4120 project.

If we have a few of the core people committed to running this in
production and supporting it in the future, I'm all for it (just like
I am +1 on security). I just want to avoid repeating mistakes like the
Avro server which isn't really supported despite being in our
codebase. (You'll note this was a Cloudera contribution but from a
contributor who was doing this in his spare time rather than part of
job responsibilities, and we have never run it in production
scenarios)

I am consistently conservative on what goes into the project because
we have to stand behind what we release. I certainly don't think _all_
core people should find every feature useful (eg REST and Thrift are
examples of some things which are useless to many but I think make
sense). But if _no_ core people see a feature as a requirement then
I'd rather let it bake until we have many people requesting it.
Otherwise people download HBase, try out these "fringe" features, and
get a bad taste in their mouth when they've bit-rot across several
versions of little usage.

-Todd
-- 
Todd Lipcon
Software Engineer, Cloudera

Re: Code review request for hbase-4120 table priority

Posted by yu...@gmail.com.

I will be supporting table priority and related changes in the foreseeable future. 

Cheers



On Dec 12, 2011, at 6:55 PM, Todd Lipcon <to...@cloudera.com> wrote:

> On Mon, Dec 12, 2011 at 4:36 PM, Andrew Purtell <ap...@apache.org> wrote:
>> 
>> HBase as a project should not have as a criteria for inclusion of some feature that Cloudera and SU and Facebook run it. Core managed to escape Yahoo. Let's not run history in reverse here in HBase land. And, actually, this makes it worse, because the the occurrence that a number of core HBase users (multiple) will all need something is substantially less likely than if one might find it useful; or, maybe, only users outside of those with such self-appointed attitude, yet perhaps a community multiples in size of "core users".
> 
> It's not about Cloudera/SU/FB - it's about code that will be supported
> by people who are committed to the project. TrendMicro certainly fits
> the bill. I of course mean no offense to Lu Jia, but neither he nor
> Taobao has made continued contributions in the past - just one other
> bug fix beyond the HBASE-4120 project.
> 
> If we have a few of the core people committed to running this in
> production and supporting it in the future, I'm all for it (just like
> I am +1 on security). I just want to avoid repeating mistakes like the
> Avro server which isn't really supported despite being in our
> codebase. (You'll note this was a Cloudera contribution but from a
> contributor who was doing this in his spare time rather than part of
> job responsibilities, and we have never run it in production
> scenarios)
> 
> I am consistently conservative on what goes into the project because
> we have to stand behind what we release. I certainly don't think _all_
> core people should find every feature useful (eg REST and Thrift are
> examples of some things which are useless to many but I think make
> sense). But if _no_ core people see a feature as a requirement then
> I'd rather let it bake until we have many people requesting it.
> Otherwise people download HBase, try out these "fringe" features, and
> get a bad taste in their mouth when they've bit-rot across several
> versions of little usage.
> 
> -Todd
> -- 
> Todd Lipcon
> Software Engineer, Cloudera

Re: Code review request for hbase-4120 table priority

Posted by Andrew Purtell <ap...@yahoo.com>.

No that's not correct. 

I believe I was told off by Nicolas and Taobao should look to contribute elsewhere, new features to some other open source project. 

On Dec 12, 2011, at 10:24 PM, Stack <st...@duboce.net> wrote:

> On Mon, Dec 12, 2011 at 10:14 PM, Andrew Purtell <ap...@yahoo.com> wrote:
>>> There are other open source projects out
>>> there that aren't used for as critical data and would love to have that
>>> next feature that might differentiate them.
>> 
>> Message received. A new level of "maturity". Go elsewhere.
>> 
> 
> Am I reading the above correctly Andrew when I interpret it as: "For a
> stable project, go elsewhere?"
> St.Ack

Re: Code review request for hbase-4120 table priority

Posted by Stack <st...@duboce.net>.

On Mon, Dec 12, 2011 at 10:14 PM, Andrew Purtell <ap...@yahoo.com> wrote:
>> There are other open source projects out
>> there that aren't used for as critical data and would love to have that
>> next feature that might differentiate them.
>
> Message received. A new level of "maturity". Go elsewhere.
>

Am I reading the above correctly Andrew when I interpret it as: "For a
stable project, go elsewhere?"
St.Ack

Re: Code review request for hbase-4120 table priority

Posted by Andrew Purtell <ap...@yahoo.com>.

>   I
> would expect that a committer wanting to use this feature in his company
> for production would do a much more thorough analysis than I did.

I haven't looked at the patch yet. This is early, would be for 0.94 and 0.92 isn't even out yet, etc. 

Why make this personal?

> There are other open source projects out
> there that aren't used for as critical data and would love to have that
> next feature that might differentiate them. 

Message received. A new level of "maturity". Go elsewhere. 

Best regards,

    - Andy


On Dec 12, 2011, at 7:12 PM, Nicolas Spiegelberg <ns...@fb.com> wrote:

>> I would really like to see the isolation feature in 0.94+, so I intend to
>> work with Jia and, if there is a successful result, support it going
>> forward in a manner like Stargate, even though I may not run it
>> personally (like I don't run Stargate). I take an expansive view of what
>> open source projects should accept...
>> 
>> On the other hand there are users and committers of HBase for whom
>> stability is paramount. I expect the tension :-); it will make us a
>> healthier project.
>> 
>> But we cannot have a new feature acceptance criteria that requires
>> several "core users" run it in production.
> 
> I think we should be really careful here because a laissez-fare attitude
> will keep the project from reaching a new level of maturity.  There are a
> number of committers from various companies who are using this for
> business-critical applications, want to provide users with a build that
> they can use for business-critical applications, and independently coming
> to the same conclusion that we need a more stable trunk.  In fact, it was
> important enough for us to discuss in depth at the HBase pow-pow a couple
> weeks ago, with 30+? people.  There are other open source projects out
> there that aren't used for as critical data and would love to have that
> next feature that might differentiate them.  As a maturing product, we
> don't need a ton of new features as much as we need our features to work
> properly and have some well-thought tweaks for completeness.
> 
> That said, this thread has been split between a discussion about
> HBASE-4120 and a discussion about open source inclusivity.
> 
> I appreciate the effort that Tao Bao is putting into HBASE-4120.  I'm glad
> that they have plans to use this in a production environment.  I'm glad to
> hear that there are active companies that also want to use this feature in
> a critical environment.  That said, I gave an initial scan for HBASE-4120
> and found race conditions & necessary design refactoring.  This was just
> an introductory scan, so I didn't look very hard and expected to find more
> issues in a later scan.  I have not looked at the diff recently because of
> prod release issues; but I find it disturbing that other reviewers didn't
> find those issues, since I waited over a month to review that patch.  I
> would expect that a committer wanting to use this feature in his company
> for production would do a much more thorough analysis than I did.
>

Re: Code review request for hbase-4120 table priority

Posted by Nicolas Spiegelberg <ns...@fb.com>.

>I would really like to see the isolation feature in 0.94+, so I intend to
>work with Jia and, if there is a successful result, support it going
>forward in a manner like Stargate, even though I may not run it
>personally (like I don't run Stargate). I take an expansive view of what
>open source projects should accept...
>
>On the other hand there are users and committers of HBase for whom
>stability is paramount. I expect the tension :-); it will make us a
>healthier project.
>
>But we cannot have a new feature acceptance criteria that requires
>several "core users" run it in production.

I think we should be really careful here because a laissez-fare attitude
will keep the project from reaching a new level of maturity.  There are a
number of committers from various companies who are using this for
business-critical applications, want to provide users with a build that
they can use for business-critical applications, and independently coming
to the same conclusion that we need a more stable trunk.  In fact, it was
important enough for us to discuss in depth at the HBase pow-pow a couple
weeks ago, with 30+? people.  There are other open source projects out
there that aren't used for as critical data and would love to have that
next feature that might differentiate them.  As a maturing product, we
don't need a ton of new features as much as we need our features to work
properly and have some well-thought tweaks for completeness.

That said, this thread has been split between a discussion about
HBASE-4120 and a discussion about open source inclusivity.

I appreciate the effort that Tao Bao is putting into HBASE-4120.  I'm glad
that they have plans to use this in a production environment.  I'm glad to
hear that there are active companies that also want to use this feature in
a critical environment.  That said, I gave an initial scan for HBASE-4120
and found race conditions & necessary design refactoring.  This was just
an introductory scan, so I didn't look very hard and expected to find more
issues in a later scan.  I have not looked at the diff recently because of
prod release issues; but I find it disturbing that other reviewers didn't
find those issues, since I waited over a month to review that patch.  I
would expect that a committer wanting to use this feature in his company
for production would do a much more thorough analysis than I did.

Re: Code review request for hbase-4120 table priority

Posted by Andrew Purtell <ap...@apache.org>.

Hi Todd,

> It's not about Cloudera/SU/FB - it's about code that will be supported
> by people who are committed to the project. 


Fine, but that is not what you stated exactly, so I felt it important to -1 the language in your previous email.

> If we have a few of the core people committed to running this in
> production and supporting it in the future, I'm all for it (just like
> I am +1 on security). I just want to avoid repeating mistakes like the
> Avro server which isn't really supported despite being in our
> codebase. 


My point of view here is that what you said is fine, but it would have been better if you stopped at "If we have a few of the core people committed".

I would really like to see the isolation feature in 0.94+, so I intend to work with Jia and, if there is a successful result, support it going forward in a manner like Stargate, even though I may not run it personally (like I don't run Stargate). I take an expansive view of what open source projects should accept...


> I just want to avoid repeating mistakes like the
> Avro server which isn't really supported despite being in our
> codebase.  


... even though it may lead to situations like that.

On the other hand there are users and committers of HBase for whom stability is paramount. I expect the tension :-); it will make us a healthier project.

But we cannot have a new feature acceptance criteria that requires several "core users" run it in production.

Best regards,


   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)


----- Original Message -----
> From: Todd Lipcon <to...@cloudera.com>
> To: dev@hbase.apache.org; Andrew Purtell <ap...@apache.org>
> Cc: 
> Sent: Monday, December 12, 2011 4:55 PM
> Subject: Re: Code review request for hbase-4120 table priority
> 
> On Mon, Dec 12, 2011 at 4:36 PM, Andrew Purtell <ap...@apache.org> 
> wrote:
>> 
>>  HBase as a project should not have as a criteria for inclusion of some 
> feature that Cloudera and SU and Facebook run it. Core managed to escape Yahoo. 
> Let's not run history in reverse here in HBase land. And, actually, this 
> makes it worse, because the the occurrence that a number of core HBase users 
> (multiple) will all need something is substantially less likely than if one 
> might find it useful; or, maybe, only users outside of those with such 
> self-appointed attitude, yet perhaps a community multiples in size of "core 
> users".
> 
> It's not about Cloudera/SU/FB - it's about code that will be supported
> by people who are committed to the project. TrendMicro certainly fits
> the bill. I of course mean no offense to Lu Jia, but neither he nor
> Taobao has made continued contributions in the past - just one other
> bug fix beyond the HBASE-4120 project.
> 
> If we have a few of the core people committed to running this in
> production and supporting it in the future, I'm all for it (just like
> I am +1 on security). I just want to avoid repeating mistakes like the
> Avro server which isn't really supported despite being in our
> codebase. (You'll note this was a Cloudera contribution but from a
> contributor who was doing this in his spare time rather than part of
> job responsibilities, and we have never run it in production
> scenarios)
> 
> I am consistently conservative on what goes into the project because
> we have to stand behind what we release. I certainly don't think _all_
> core people should find every feature useful (eg REST and Thrift are
> examples of some things which are useless to many but I think make
> sense). But if _no_ core people see a feature as a requirement then
> I'd rather let it bake until we have many people requesting it.
> Otherwise people download HBase, try out these "fringe" features, and
> get a bad taste in their mouth when they've bit-rot across several
> versions of little usage.
> 
> -Todd
> -- 
> Todd Lipcon
> Software Engineer, Cloudera
>

Re: Code review request for hbase-4120 table priority

Posted by Todd Lipcon <to...@cloudera.com>.

On Mon, Dec 12, 2011 at 4:36 PM, Andrew Purtell <ap...@apache.org> wrote:
>
> HBase as a project should not have as a criteria for inclusion of some feature that Cloudera and SU and Facebook run it. Core managed to escape Yahoo. Let's not run history in reverse here in HBase land. And, actually, this makes it worse, because the the occurrence that a number of core HBase users (multiple) will all need something is substantially less likely than if one might find it useful; or, maybe, only users outside of those with such self-appointed attitude, yet perhaps a community multiples in size of "core users".

It's not about Cloudera/SU/FB - it's about code that will be supported
by people who are committed to the project. TrendMicro certainly fits
the bill. I of course mean no offense to Lu Jia, but neither he nor
Taobao has made continued contributions in the past - just one other
bug fix beyond the HBASE-4120 project.

If we have a few of the core people committed to running this in
production and supporting it in the future, I'm all for it (just like
I am +1 on security). I just want to avoid repeating mistakes like the
Avro server which isn't really supported despite being in our
codebase. (You'll note this was a Cloudera contribution but from a
contributor who was doing this in his spare time rather than part of
job responsibilities, and we have never run it in production
scenarios)

I am consistently conservative on what goes into the project because
we have to stand behind what we release. I certainly don't think _all_
core people should find every feature useful (eg REST and Thrift are
examples of some things which are useless to many but I think make
sense). But if _no_ core people see a feature as a requirement then
I'd rather let it bake until we have many people requesting it.
Otherwise people download HBase, try out these "fringe" features, and
get a bad taste in their mouth when they've bit-rot across several
versions of little usage.

-Todd
--
Todd Lipcon
Software Engineer, Cloudera

Re: Code review request for hbase-4120 table priority

Posted by Andrew Purtell <ap...@apache.org>.

By the way, -1 to this as a criteria:

> If it's completely a coprocessor, then it seems we should let it bake
> on github and only incorporate in core if we find that a number of the
> core HBase users are using it in production. 

 
HBase as a project should not have as a criteria for inclusion of some feature that Cloudera and SU and Facebook run it. Core managed to escape Yahoo. Let's not run history in reverse here in HBase land. And, actually, this makes it worse, because the the occurrence that a number of core HBase users (multiple) will all need something is substantially less likely than if one might find it useful; or, maybe, only users outside of those with such self-appointed attitude, yet perhaps a community multiples in size of "core users". 

Best regards,


   - Andy


Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)


----- Original Message -----
> From: Andrew Purtell <ap...@apache.org>
> To: "dev@hbase.apache.org" <de...@hbase.apache.org>
> Cc: 
> Sent: Monday, December 12, 2011 4:30 PM
> Subject: Re: Code review request for hbase-4120 table priority
> 
> HBASE-4120 deals with only the RPC prioritization parts. This cannot be 
> implemented as a coprocessor.
> 
>>  If it's completely a coprocessor, then it seems we should let it bake
>>  on github and only incorporate in core if we find that a number of the
>>  core HBase users are using it in production.  
> 
> 
> The addition of the coprocessor framework is from my point of view a double 
> edged sword. It can be a tool to clarify core and reduce maintenance burden on 
> the community for various boutique functions. On the other hand it can be used 
> to freeze further feature development and leave users who need such features out 
> in the cold to go their own way or make a private fork. 
> 
> I've become aware of several private forks of HDFS and HBase. Too bad. A 
> pooling of dev resources would have almost surely have been better.
> 
> HBASE-4120 and successor issues are / will be an attempt to take what runs in 
> production at Taobao and upstream it. They could just do a code drop on GitHub 
> of what they have now. Would be much easier than reimplementing it as a 
> coprocessor. However they have incentive here to give back to upstream, 
> additional co-development with the community, for inclusion of their work in the 
> distribution. What is the incentive if we want them to make the reimplementation 
> effort only to then just drop that on GitHub?
> 
> There are some cases where it would benefit all users to include 
> coprocessor-based features in tree: Like security. Perhaps like constraints. 
> Like the isolation/allocation stuff, for perhaps 0.94. Perhaps secondary 
> indexing. In all of these cases, though the majority of the implementation is as 
> coprocessor, there must be some changes to core -- to the coprocessor framework, 
> generally -- to support it. A coprocessor can be included in the tree as a maven 
> module. Without such inclusion, the rationale for not dropping the enabling 
> logic in core may be lacking ... "oh, this just supports some GitHub 
> thing".
> 
> Best regards,
> 
> 
>    - Andy
> 
> Problems worthy of attack prove their worth by hitting back. - Piet Hein (via 
> Tom White)
> 
> 
> ----- Original Message -----
>>  From: Todd Lipcon <to...@cloudera.com>
>>  To: dev@hbase.apache.org
>>  Cc: 
>>  Sent: Monday, December 12, 2011 4:03 PM
>>  Subject: Re: Code review request for hbase-4120 table priority
>> 
>>  If it's completely a coprocessor, then it seems we should let it bake
>>  on github and only incorporate in core if we find that a number of the
>>  core HBase users are using it in production. Am I misunderstanding the
>>  implementation? (haven't looked at the most recent patch)
>> 
>>  -Todd
>> 
>>  On Mon, Dec 12, 2011 at 3:50 PM,  <yu...@gmail.com> wrote:
>>>   Waiting for review comments from other committers.
>>>   The implementation is pluggable by using coprocessors.
>>> 
>>>   Cheers
>>> 
>>> 
>>> 
>>>   On Dec 12, 2011, at 5:43 PM, Stack <st...@duboce.net> wrote:
>>> 
>>>>   On Mon, Dec 12, 2011 at 6:43 AM,  <yu...@gmail.com> 
> wrote:
>>>>>   Hi,
>>>>>   4120 has gone through more than 20 revisions.
>>>>> 
>>>>>   Please provide your comments.
>>>>> 
>>>>>   I plan to integrate it this week.
>>>>> 
>>>> 
>>>>   I'd suggest hold on commit until some other committers have 
> had a
>>>>   looksee.  This is an important feature that we need to get right 
> and
>>>>   there is no need to rush it in.
>>>> 
>>>>   Thanks Ted (and thanks for the reviews so far),
>>>>   St.Ack
>> 
>> 
>> 
>>  -- 
>>  Todd Lipcon
>>  Software Engineer, Cloudera
>> 
>

Re: Code review request for hbase-4120 table priority

Posted by Andrew Purtell <ap...@apache.org>.

HBASE-4120 deals with only the RPC prioritization parts. This cannot be implemented as a coprocessor.

> If it's completely a coprocessor, then it seems we should let it bake
> on github and only incorporate in core if we find that a number of the
> core HBase users are using it in production.  

The addition of the coprocessor framework is from my point of view a double edged sword. It can be a tool to clarify core and reduce maintenance burden on the community for various boutique functions. On the other hand it can be used to freeze further feature development and leave users who need such features out in the cold to go their own way or make a private fork. 

I've become aware of several private forks of HDFS and HBase. Too bad. A pooling of dev resources would have almost surely have been better.

HBASE-4120 and successor issues are / will be an attempt to take what runs in production at Taobao and upstream it. They could just do a code drop on GitHub of what they have now. Would be much easier than reimplementing it as a coprocessor. However they have incentive here to give back to upstream, additional co-development with the community, for inclusion of their work in the distribution. What is the incentive if we want them to make the reimplementation effort only to then just drop that on GitHub?

There are some cases where it would benefit all users to include coprocessor-based features in tree: Like security. Perhaps like constraints. Like the isolation/allocation stuff, for perhaps 0.94. Perhaps secondary indexing. In all of these cases, though the majority of the implementation is as coprocessor, there must be some changes to core -- to the coprocessor framework, generally -- to support it. A coprocessor can be included in the tree as a maven module. Without such inclusion, the rationale for not dropping the enabling logic in core may be lacking ... "oh, this just supports some GitHub thing".

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)

----- Original Message -----
> From: Todd Lipcon <to...@cloudera.com>
> To: dev@hbase.apache.org
> Cc: 
> Sent: Monday, December 12, 2011 4:03 PM
> Subject: Re: Code review request for hbase-4120 table priority
> 
> If it's completely a coprocessor, then it seems we should let it bake
> on github and only incorporate in core if we find that a number of the
> core HBase users are using it in production. Am I misunderstanding the
> implementation? (haven't looked at the most recent patch)
> 
> -Todd
> 
> On Mon, Dec 12, 2011 at 3:50 PM,  <yu...@gmail.com> wrote:
>>  Waiting for review comments from other committers.
>>  The implementation is pluggable by using coprocessors.
>> 
>>  Cheers
>> 
>> 
>> 
>>  On Dec 12, 2011, at 5:43 PM, Stack <st...@duboce.net> wrote:
>> 
>>>  On Mon, Dec 12, 2011 at 6:43 AM,  <yu...@gmail.com> wrote:
>>>>  Hi,
>>>>  4120 has gone through more than 20 revisions.
>>>> 
>>>>  Please provide your comments.
>>>> 
>>>>  I plan to integrate it this week.
>>>> 
>>> 
>>>  I'd suggest hold on commit until some other committers have had a
>>>  looksee.  This is an important feature that we need to get right and
>>>  there is no need to rush it in.
>>> 
>>>  Thanks Ted (and thanks for the reviews so far),
>>>  St.Ack
> 
> 
> 
> -- 
> Todd Lipcon
> Software Engineer, Cloudera
>

Re: Code review request for hbase-4120 table priority

Posted by yu...@gmail.com.

This feature is used in production at Taobao (China's EBay). You can find related description about cluster size, etc on the Jira. 

My understanding for hbase-4120 is that we are making the code conform to Apache hbase standard. 

There is related change for web UI, etc. 
Without clear feedback from Apache hbase, it is somehow difficult for the contributor to pursue further development. 

Regards

On Dec 12, 2011, at 6:03 PM, Todd Lipcon <to...@cloudera.com> wrote:

> If it's completely a coprocessor, then it seems we should let it bake
> on github and only incorporate in core if we find that a number of the
> core HBase users are using it in production. Am I misunderstanding the
> implementation? (haven't looked at the most recent patch)
> 
> -Todd
> 
> On Mon, Dec 12, 2011 at 3:50 PM,  <yu...@gmail.com> wrote:
>> Waiting for review comments from other committers.
>> The implementation is pluggable by using coprocessors.
>> 
>> Cheers
>> 
>> 
>> 
>> On Dec 12, 2011, at 5:43 PM, Stack <st...@duboce.net> wrote:
>> 
>>> On Mon, Dec 12, 2011 at 6:43 AM,  <yu...@gmail.com> wrote:
>>>> Hi,
>>>> 4120 has gone through more than 20 revisions.
>>>> 
>>>> Please provide your comments.
>>>> 
>>>> I plan to integrate it this week.
>>>> 
>>> 
>>> I'd suggest hold on commit until some other committers have had a
>>> looksee.  This is an important feature that we need to get right and
>>> there is no need to rush it in.
>>> 
>>> Thanks Ted (and thanks for the reviews so far),
>>> St.Ack
> 
> 
> 
> -- 
> Todd Lipcon
> Software Engineer, Cloudera

Re: Code review request for hbase-4120 table priority

Posted by Todd Lipcon <to...@cloudera.com>.

If it's completely a coprocessor, then it seems we should let it bake
on github and only incorporate in core if we find that a number of the
core HBase users are using it in production. Am I misunderstanding the
implementation? (haven't looked at the most recent patch)

-Todd

On Mon, Dec 12, 2011 at 3:50 PM,  <yu...@gmail.com> wrote:
> Waiting for review comments from other committers.
> The implementation is pluggable by using coprocessors.
>
> Cheers
>
>
>
> On Dec 12, 2011, at 5:43 PM, Stack <st...@duboce.net> wrote:
>
>> On Mon, Dec 12, 2011 at 6:43 AM,  <yu...@gmail.com> wrote:
>>> Hi,
>>> 4120 has gone through more than 20 revisions.
>>>
>>> Please provide your comments.
>>>
>>> I plan to integrate it this week.
>>>
>>
>> I'd suggest hold on commit until some other committers have had a
>> looksee.  This is an important feature that we need to get right and
>> there is no need to rush it in.
>>
>> Thanks Ted (and thanks for the reviews so far),
>> St.Ack



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: Code review request for hbase-4120 table priority

Posted by yu...@gmail.com.

Waiting for review comments from other committers. 
The implementation is pluggable by using coprocessors. 

Cheers



On Dec 12, 2011, at 5:43 PM, Stack <st...@duboce.net> wrote:

> On Mon, Dec 12, 2011 at 6:43 AM,  <yu...@gmail.com> wrote:
>> Hi,
>> 4120 has gone through more than 20 revisions.
>> 
>> Please provide your comments.
>> 
>> I plan to integrate it this week.
>> 
> 
> I'd suggest hold on commit until some other committers have had a
> looksee.  This is an important feature that we need to get right and
> there is no need to rush it in.
> 
> Thanks Ted (and thanks for the reviews so far),
> St.Ack

Re: Code review request for hbase-4120 table priority

Posted by Stack <st...@duboce.net>.

On Mon, Dec 12, 2011 at 6:43 AM,  <yu...@gmail.com> wrote:
> Hi,
> 4120 has gone through more than 20 revisions.
>
> Please provide your comments.
>
> I plan to integrate it this week.
>

I'd suggest hold on commit until some other committers have had a
looksee.  This is an important feature that we need to get right and
there is no need to rush it in.

Thanks Ted (and thanks for the reviews so far),
St.Ack