HBASE-4213

[Ted Yu <yu...@gmail.com>]

Todd/Andy:
Instant schema change is an important feature.

Can you let us know the criteria for HBASE-4213 to graduate ?
Does HBASE-4370 have to be implemented ?

Thanks

Re: HBASE-4213

[Subbu M Iyer <ms...@gmail.com>]

Andy/Todd,

Yes I agree as well to branching as it will give us the confidence of
not breaking anything in the trunk at the same time give us a good
platform to iteratively address all the major concerns and vet out all
the open holes.

Given all the different critical requirements plus number of edge
cases to address i think it wouldn't be advisable/nor feasible to
introduce this feature in one big-bang, but rather iteratively
address/review/commit to its branch until it matures.

+1 to branching. I also think its a good idea to list down or capture
all the different criteria, edge cases, suggestions we wanted to be
addressed so that we don't miss any.

To start I will iterate the list of interesting requirements we
received in the original Jira and please feel free to add more as you
see fit.

thanks
Subbu

On Thu, Sep 15, 2011 at 8:46 AM, Andrew Purtell <ap...@apache.org> wrote:
>> From: Todd Lipcon <to...@cloudera.com>
>
>> I'd also be happy to commit what we have to a new branch, then do
>> followup work on that branch until people feel comfortable merging it.
>
> Ted, Subbu, this sounds like a good idea. What do you think?
>
>
> Best regards,
>
>
>   - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
>
>
> ----- Original Message -----
>> From: Todd Lipcon <to...@cloudera.com>
>> To: Andrew Purtell <ap...@apache.org>
>> Cc: Ted Yu <yu...@gmail.com>; Subbu M Iyer <ms...@gmail.com>; "dev@hbase.apache.org" <de...@hbase.apache.org>
>> Sent: Thursday, September 15, 2011 8:38 AM
>> Subject: Re: HBASE-4213
>>
>> On Thu, Sep 15, 2011 at 8:33 AM, Andrew Purtell <ap...@apache.org>
>> wrote:
>>>  My recommendation is to begin adding test cases that test RS and ZK quorum
>> peer failures happening while the schema update is in progress, and insure that
>> failures are handled and the update process recovers and succeeds. Once there is
>> a set of such tests we can evaluate their coverage and introduce the feature. It
>> would still be risky, but there would be some basis for believing it to be
>> practical.
>>
>> +1. I'd also be happy to commit what we have to a new branch, then do
>> followup work on that branch until people feel comfortable merging it.
>> Branching gives us the plus of having smaller patches to review,
>> without the risk introducd by merging half-done things in trunk.
>>
>> BTW: I realize I'm on the more conservative side in the community -
>> please hold me to the same or higher standards :) I don't trust my own
>> code more than anyone else's!
>>
>> -Todd
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
>

Re: HBASE-4213

[Andrew Purtell <ap...@apache.org>]

> From: Todd Lipcon <to...@cloudera.com>

> I'd also be happy to commit what we have to a new branch, then do
> followup work on that branch until people feel comfortable merging it.

Ted, Subbu, this sounds like a good idea. What do you think?


Best regards,


  - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)


----- Original Message -----
> From: Todd Lipcon <to...@cloudera.com>
> To: Andrew Purtell <ap...@apache.org>
> Cc: Ted Yu <yu...@gmail.com>; Subbu M Iyer <ms...@gmail.com>; "dev@hbase.apache.org" <de...@hbase.apache.org>
> Sent: Thursday, September 15, 2011 8:38 AM
> Subject: Re: HBASE-4213
> 
> On Thu, Sep 15, 2011 at 8:33 AM, Andrew Purtell <ap...@apache.org> 
> wrote:
>>  My recommendation is to begin adding test cases that test RS and ZK quorum 
> peer failures happening while the schema update is in progress, and insure that 
> failures are handled and the update process recovers and succeeds. Once there is 
> a set of such tests we can evaluate their coverage and introduce the feature. It 
> would still be risky, but there would be some basis for believing it to be 
> practical.
> 
> +1. I'd also be happy to commit what we have to a new branch, then do
> followup work on that branch until people feel comfortable merging it.
> Branching gives us the plus of having smaller patches to review,
> without the risk introducd by merging half-done things in trunk.
> 
> BTW: I realize I'm on the more conservative side in the community -
> please hold me to the same or higher standards :) I don't trust my own
> code more than anyone else's!
> 
> -Todd
> -- 
> Todd Lipcon
> Software Engineer, Cloudera
>

Re: HBASE-4213

[Todd Lipcon <to...@cloudera.com>]

On Thu, Sep 15, 2011 at 8:33 AM, Andrew Purtell <ap...@apache.org> wrote:
> My recommendation is to begin adding test cases that test RS and ZK quorum peer failures happening while the schema update is in progress, and insure that failures are handled and the update process recovers and succeeds. Once there is a set of such tests we can evaluate their coverage and introduce the feature. It would still be risky, but there would be some basis for believing it to be practical.

+1. I'd also be happy to commit what we have to a new branch, then do
followup work on that branch until people feel comfortable merging it.
Branching gives us the plus of having smaller patches to review,
without the risk introducd by merging half-done things in trunk.

BTW: I realize I'm on the more conservative side in the community -
please hold me to the same or higher standards :) I don't trust my own
code more than anyone else's!

-Todd
-- 
Todd Lipcon
Software Engineer, Cloudera

Re: HBASE-4213

[Andrew Purtell <ap...@apache.org>]

> From: Todd Lipcon <to...@cloudera.com>
> If a feature doesn't properly handle error conditions, it shouldn't be
> committed, IMO.

This is my concern as well. I had initially given the feature a +1 to get it in so further development can be done, but upon further consideration (and Todd's code review) I'm concerned it won't be useful in practice as currently implemented.

Attempting a schema update would require a completely healthy cluster and no failure could occur during that time. In other words, the cluster would have to be quiesced first. That is not practical, so the feature is not useful.


My recommendation is to begin adding test cases that test RS and ZK quorum peer failures happening while the schema update is in progress, and insure that failures are handled and the update process recovers and succeeds. Once there is a set of such tests we can evaluate their coverage and introduce the feature. It would still be risky, but there would be some basis for believing it to be practical.


Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)


>________________________________
>From: Todd Lipcon <to...@cloudera.com>
>To: Ted Yu <yu...@gmail.com>
>Cc: Andrew Purtell <ap...@apache.org>; Subbu M Iyer <ms...@gmail.com>; dev@hbase.apache.org
>Sent: Wednesday, September 14, 2011 11:57 PM
>Subject: Re: HBASE-4213
>
>On Wed, Sep 14, 2011 at 2:51 PM, Ted Yu <yu...@gmail.com> wrote:
>> Todd/Andy:
>> Instant schema change is an important feature.
>>
>> Can you let us know the criteria for HBASE-4213 to graduate ?
>> Does HBASE-4370 have to be implemented ?
>
>In my opinion, the criteria is the same as any other. Reviewers should
>be convinced it doesn't introduce bugs or APIs we don't want to
>maintain, and the feature should be usable by ordinary folks. There
>should also be sufficient testing including fault scenarios. If a
>feature doesn't properly handle error conditions, it shouldn't be
>committed, IMO.
>
>-Todd
>-- 
>Todd Lipcon
>Software Engineer, Cloudera
>
>
> 

Re: HBASE-4213

[Todd Lipcon <to...@cloudera.com>]

On Wed, Sep 14, 2011 at 2:51 PM, Ted Yu <yu...@gmail.com> wrote:
> Todd/Andy:
> Instant schema change is an important feature.
>
> Can you let us know the criteria for HBASE-4213 to graduate ?
> Does HBASE-4370 have to be implemented ?

In my opinion, the criteria is the same as any other. Reviewers should
be convinced it doesn't introduce bugs or APIs we don't want to
maintain, and the feature should be usable by ordinary folks. There
should also be sufficient testing including fault scenarios. If a
feature doesn't properly handle error conditions, it shouldn't be
committed, IMO.

-Todd
-- 
Todd Lipcon
Software Engineer, Cloudera