0.95 and 0.96 remaining issues

[Stack <st...@duboce.net>]

Let me freshen up this thread.  Let me also petition that we as a project
renew our committal to pushing out 0.96.  We are almost there.  Lets do one
last 'Development Release', a 0.95.2, and then a 0.96 release in August?

Here is how I see it.  Please feel free to add/pushback:

+ There are no blockers [1], just a few criticals, a few of which we should
at least fix: failed hadoop2 unit tests, replication fixup, etc. (The
migration polishing should probably be a blocker).  Is the list of issues
against 0.95.2 up-to-date?  Do all agree the list of criticals complete?
 Are there any blockers lurking?  Please speak up if there is anything we
need include.
+ From Enis's list below, to be done still is namespaces and finishing
distributed log replay. IMO, distributed log replay should not hold up
0.96.  We can turn it on whenever it is done.  Namespaces is the long pole
and progress seems slow.  Do we hold up the release for them?  How can we
hurry this effort along?  Swat team descends on Y!?
+ Is anyone testing?  Integration tests fail on ec2 build from time to time
[2].  Our Elliott dug in on one of the failures a few days back and found
legit issue w/ no retry on admin tasks (I heart hbase-it tests).  Our unit
test story is better [3] but there are still the odd failures.
+ The build is still not baked; publishing a different hbase for hadoop1
and hadoop2 is still not done (This is my charge).

Your 0.96 RM,
St.Ack


1.
https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
2. http://54.241.6.143/
3. https://builds.apache.org/view/H-L/view/HBase/job/hbase-0.95/


On Tue, Apr 2, 2013 at 5:59 PM, Enis Söztutar <en...@hortonworks.com> wrote:

> Hi devs,
>
> We(HWX) had some discussions with Stack this morning about the eventual
> 0.96 release, and 0.95 stuff. Having 0.96 out sooner rather than later is
> becoming crucial as time passes. We've talked about some of the significant
> work items, and what areas to focus on.
>
> Some of the notes are:
>
>  - HBASE-8015. Support for Namespaces (Francis owns it, and the patch looks
> mostly done. We should get this in since this is the singularity)
>  - HBASE-7999 Add 'system' tables (this may turn into just adding a system
> namespace, and be done with it. Depends on HBASE-8015. )
>  - HBASE-8093. Change the META name (can be done together as a part of
> HBASE-8015, HBASE-7999)
>  - HBASE-3787 Increment is non-idempotent but client retries RPC. Nonce
> issue (idempotent) -> critical, Enis or Sergey will work on this.
>  - Tarball, maven -> Enis, Giri, Nick, Stack
>  - Jeffrey's log replay patch (It allows 2sec recovery for accepting writes
> for regions. We can get this in disabled by default, and keep both
> distributed log splitting, and log replay. In 96, we can make it rock solid
> afterwards)
>  - Region locality fixes hbase-7932/7942 (Devaraj says that he was able to
> convince the hdfs guys to accept the patch.)
>  - IPC review (DD, Stack)
>  - Cell stuff review
>  - Cell support in Filters etc. -> These are important but no blockers,
> maybe we can defer them to 1.0.
>  - HBASE-7897 Add support for tags to Cell Interface
>  - MTTR -> non blocker. The more we have the better
>  - Formal hadoop2 support -> DD, Ted, Nick, hadoop-2.0.4-alpha issue.
>  - prefix-tree module, storage module, mapred? -> Enis push for decision
> here.
>  - HBASE-7704. migration tool that checks presence of HFile V1 files.
> (Stack)
>  - Snapshots, any work left? -> Ted will review
>  - HBASE-8045. Fix .META. migration after HBASE-3171 (Stack)
>  - IO fencing, write compactions to hlog (Stack spend some time on this,
> pretty important)
>  - Unit test fixes
>  - How are we going to test this
>
> Kudos to Stack for driving the first RC for 0.95. He says that he wants to
> do 2-week release cycles, and at some point, we can just label 0.95.x as
> the 0.96.0 release.
>
> We'll also go over the issues and re-kindle the discussions, re-assign,
> bump to 0.95.1 etc.
>
> Most of these have been discussed in the issues, or elsewhere, but we would
> love your input/help on these. Any more issues, that should be/should not
> be blockers for 0.96? Any issues that has to go with the singularity?
>
> Enis
>

Re: 0.95 and 0.96 remaining issues

[Nick Dimiduk <nd...@gmail.com>]

I want to see initial data type APIs ship out with 0.95.2. A patch for
ordered byte serialization is up (HBASE-8201) and is nearing steady-state.
However, sershe is the only person who's left feedback. I just posted an
early patch for the data type API itself (HBASE-8693). It should get some
eyes from all manor of interested parties, but I'll settle for folk from
Phoenix for now.

Should these tasks be escalated to criticals in order to grab attention?

Additional comments inline.

On Mon, Jun 24, 2013 at 9:53 AM, Stack <st...@duboce.net> wrote:

> Lets do one last 'Development Release', a 0.95.2, and then a 0.96 release
> in August?
>

Woo hoo!

+ There are no blockers [1], just a few criticals, a few of which we should
> at least fix: failed hadoop2 unit tests


+1

replication fixup


+1

(The migration polishing should probably be a blocker)


+1

Namespaces is the long pole and progress seems slow.  Do we hold up the
> release for them?  How can we hurry this effort along?  Swat team descends
> on Y!?
>

It would be a shame to not get a decision on this in for 0.96.

+ Is anyone testing?  Integration tests fail on ec2 build from time to time
> [2].  Our Elliott dug in on one of the failures a few days back and found
> legit issue w/ no retry on admin tasks (I heart hbase-it tests).  Our unit
> test story is better [3] but there are still the odd failures.
>

With the creation of the new list, noticing these issues is going to push
further back-burner. Nannying this stuff should retain focus. I'll
volunteer to track on these issues as I see them.

Notice though that some of the more recent failures are caused by lack of
disk space on the Jenkins build host.

+ The build is still not baked; publishing a different hbase for hadoop1
> and hadoop2 is still not done (This is my charge).
>
> Your 0.96 RM,
> St.Ack
>
>
> 1.
>
> https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
> 2. http://54.241.6.143/
> 3. https://builds.apache.org/view/H-L/view/HBase/job/hbase-0.95/
>
>
> On Tue, Apr 2, 2013 at 5:59 PM, Enis Söztutar <en...@hortonworks.com>
> wrote:
>
> > Hi devs,
> >
> > We(HWX) had some discussions with Stack this morning about the eventual
> > 0.96 release, and 0.95 stuff. Having 0.96 out sooner rather than later is
> > becoming crucial as time passes. We've talked about some of the
> significant
> > work items, and what areas to focus on.
> >
> > Some of the notes are:
> >
> >  - HBASE-8015. Support for Namespaces (Francis owns it, and the patch
> looks
> > mostly done. We should get this in since this is the singularity)
> >  - HBASE-7999 Add 'system' tables (this may turn into just adding a
> system
> > namespace, and be done with it. Depends on HBASE-8015. )
> >  - HBASE-8093. Change the META name (can be done together as a part of
> > HBASE-8015, HBASE-7999)
> >  - HBASE-3787 Increment is non-idempotent but client retries RPC. Nonce
> > issue (idempotent) -> critical, Enis or Sergey will work on this.
> >  - Tarball, maven -> Enis, Giri, Nick, Stack
> >  - Jeffrey's log replay patch (It allows 2sec recovery for accepting
> writes
> > for regions. We can get this in disabled by default, and keep both
> > distributed log splitting, and log replay. In 96, we can make it rock
> solid
> > afterwards)
> >  - Region locality fixes hbase-7932/7942 (Devaraj says that he was able
> to
> > convince the hdfs guys to accept the patch.)
> >  - IPC review (DD, Stack)
> >  - Cell stuff review
> >  - Cell support in Filters etc. -> These are important but no blockers,
> > maybe we can defer them to 1.0.
> >  - HBASE-7897 Add support for tags to Cell Interface
> >  - MTTR -> non blocker. The more we have the better
> >  - Formal hadoop2 support -> DD, Ted, Nick, hadoop-2.0.4-alpha issue.
> >  - prefix-tree module, storage module, mapred? -> Enis push for decision
> > here.
> >  - HBASE-7704. migration tool that checks presence of HFile V1 files.
> > (Stack)
> >  - Snapshots, any work left? -> Ted will review
> >  - HBASE-8045. Fix .META. migration after HBASE-3171 (Stack)
> >  - IO fencing, write compactions to hlog (Stack spend some time on this,
> > pretty important)
> >  - Unit test fixes
> >  - How are we going to test this
> >
> > Kudos to Stack for driving the first RC for 0.95. He says that he wants
> to
> > do 2-week release cycles, and at some point, we can just label 0.95.x as
> > the 0.96.0 release.
> >
> > We'll also go over the issues and re-kindle the discussions, re-assign,
> > bump to 0.95.1 etc.
> >
> > Most of these have been discussed in the issues, or elsewhere, but we
> would
> > love your input/help on these. Any more issues, that should be/should not
> > be blockers for 0.96? Any issues that has to go with the singularity?
> >
> > Enis
> >
>