Best way to help on beta2?

[Josh Elser <el...@apache.org>]

Hi folks,

Just curious where I can offer a hand on the remaining 2.0.0-beta2 work. 
I know the theme was upgrade, and I also saw the good test-fixing work 
that's also happening now.

Anything still pending on the radar I can help out with? Feel free to 
directly ping for any help on reviews.

- Josh

Re: Best way to help on beta2?

[Josh Elser <el...@apache.org>]

Thanks for the great update. Glad we have folks tightly holding on to 
the reins.

On 2/21/18 6:02 AM, 张铎(Duo Zhang) wrote:
> I think we could target to GA, not only beta-2. At least we need to pass a
> 10b ITBLL before releasing GA, and then, make sure we have a good
> performance compare to 1.x release lines.

Yeah, I think as long as we're propagating the beta moniker, we can punt 
down the road. Good that we all look at it now though.

> The problem here is that these tasks are not independent... ITBLL will not
> pass since we still have several broken UTs... And before passing ITBLL,
> focusing on performance is not a good idea since we may impact performance
> a lot when fixing some critical bugs...

I know Ted and Sergey S have been looking indirectly into performance. I 
think with the change in accounting in some places (counting ops in one 
multi() differently IIRC), perf was reasonable, but I'd have to defer to 
them for the hard details. I could be inventing that to stay positive :)

> So I think the first priority stuff is to fix the broken UTs. It will be
> good if you can help here Josh.

Got it!

<snip/>

>> In general, test fixing has been taking forever...  (Almost 200 issues
>> since beta-1). The flakey list was long and if you dug, there was a dirty
>> root cause that needed addressing. We are almost there now thanks to the
>> great work of a bunch of you all [1] (Duo has done a mountain of work in
>> here). The nightlies are starting to pass with hbase2 on hadoop2 [2] and we
>> are now into test failures running hbase2 over hadoop3 (Drob has some fun
>> fixes coming here). Correct me if I have it wrong but I'd think it'd be
>> hard to release an hbase2 if unit tests don't pass (most of the time).

I've seen the massive commits Duo has landed :). Agree with ya on 
stable(ish) tests.

>> Thereafter, its test and doc, mostly[3]. Help is needed here, in particular
>> enabling new stuff and trying it. Little has been done on scale and perf
>> that I know of. I was going to take a look at a rolling upgrade to see what
>> is involved and a last run through the compatibility report. Making this
>> work might have to be a post-2.0.0 thing; hbase-2.0.0 has been going on too
>> long.
>>
>> Was hoping to beta-2 this weekend or start of next week.
>>
>> Thanks,
>> St.Ack
>>
>> 1.
>> https://builds.apache.org/view/H-L/view/HBase/job/HBase-
>> Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html
>> 2. https://builds.apache.org/job/HBase%20Nightly/job/branch-2/
>> 3. https://issues.apache.org/jira/projects/HBASE/versions/12340862

Thanks for the great details. Will try to help this along.

Re: Best way to help on beta2?

["张铎 (Duo Zhang)" <pa...@gmail.com>]

I think we could target to GA, not only beta-2. At least we need to pass a
10b ITBLL before releasing GA, and then, make sure we have a good
performance compare to 1.x release lines.

The problem here is that these tasks are not independent... ITBLL will not
pass since we still have several broken UTs... And before passing ITBLL,
focusing on performance is not a good idea since we may impact performance
a lot when fixing some critical bugs...

So I think the first priority stuff is to fix the broken UTs. It will be
good if you can help here Josh.

Thanks.

2018-02-21 14:44 GMT+08:00 Stack <st...@duboce.net>:

> On Tue, Feb 20, 2018 at 11:36 AM, Josh Elser <el...@apache.org> wrote:
>
> > Hi folks,
> >
> > Just curious where I can offer a hand on the remaining 2.0.0-beta2 work.
> I
> > know the theme was upgrade, and I also saw the good test-fixing work
> that's
> > also happening now.
> >
> > Anything still pending on the radar I can help out with? Feel free to
> > directly ping for any help on reviews.
> >
> > - Josh
> >
>
> Thanks Josh for asking.
>
> You have one you are trying to land and I just asked you take a look at
> another that I think lands in your corner.
>
> In general, test fixing has been taking forever...  (Almost 200 issues
> since beta-1). The flakey list was long and if you dug, there was a dirty
> root cause that needed addressing. We are almost there now thanks to the
> great work of a bunch of you all [1] (Duo has done a mountain of work in
> here). The nightlies are starting to pass with hbase2 on hadoop2 [2] and we
> are now into test failures running hbase2 over hadoop3 (Drob has some fun
> fixes coming here). Correct me if I have it wrong but I'd think it'd be
> hard to release an hbase2 if unit tests don't pass (most of the time).
>
> Thereafter, its test and doc, mostly[3]. Help is needed here, in particular
> enabling new stuff and trying it. Little has been done on scale and perf
> that I know of. I was going to take a look at a rolling upgrade to see what
> is involved and a last run through the compatibility report. Making this
> work might have to be a post-2.0.0 thing; hbase-2.0.0 has been going on too
> long.
>
> Was hoping to beta-2 this weekend or start of next week.
>
> Thanks,
> St.Ack
>
> 1.
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-
> Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html
> 2. https://builds.apache.org/job/HBase%20Nightly/job/branch-2/
> 3. https://issues.apache.org/jira/projects/HBASE/versions/12340862
>

Re: Best way to help on beta2?

[Stack <st...@duboce.net>]

On Tue, Feb 20, 2018 at 11:36 AM, Josh Elser <el...@apache.org> wrote:

> Hi folks,
>
> Just curious where I can offer a hand on the remaining 2.0.0-beta2 work. I
> know the theme was upgrade, and I also saw the good test-fixing work that's
> also happening now.
>
> Anything still pending on the radar I can help out with? Feel free to
> directly ping for any help on reviews.
>
> - Josh
>

Thanks Josh for asking.

You have one you are trying to land and I just asked you take a look at
another that I think lands in your corner.

In general, test fixing has been taking forever...  (Almost 200 issues
since beta-1). The flakey list was long and if you dug, there was a dirty
root cause that needed addressing. We are almost there now thanks to the
great work of a bunch of you all [1] (Duo has done a mountain of work in
here). The nightlies are starting to pass with hbase2 on hadoop2 [2] and we
are now into test failures running hbase2 over hadoop3 (Drob has some fun
fixes coming here). Correct me if I have it wrong but I'd think it'd be
hard to release an hbase2 if unit tests don't pass (most of the time).

Thereafter, its test and doc, mostly[3]. Help is needed here, in particular
enabling new stuff and trying it. Little has been done on scale and perf
that I know of. I was going to take a look at a rolling upgrade to see what
is involved and a last run through the compatibility report. Making this
work might have to be a post-2.0.0 thing; hbase-2.0.0 has been going on too
long.

Was hoping to beta-2 this weekend or start of next week.

Thanks,
St.Ack

1.
https://builds.apache.org/view/H-L/view/HBase/job/HBase-Find-Flaky-Tests-branch2.0/lastSuccessfulBuild/artifact/dashboard.html
2. https://builds.apache.org/job/HBase%20Nightly/job/branch-2/
3. https://issues.apache.org/jira/projects/HBASE/versions/12340862