Working towards a dev release

[Todd Lipcon <to...@cloudera.com>]

Hey everyone,

I'd like to start planning towards a development release this month. Here's
my thinking on timeline, working backwards:

*Tues, 6/29 - Hadoop Summit, official first release of dev series*

It would be great to have a release at this point that we think is
moderately usable (ie not significantly less stable than 0.20.n has been).
Perhaps some of the new features (eg replication, coprocessors, etc) would
still be in progress but I think it would be great to be able to talk up a
working installable-today release at the summit.

Thinking backwards, from here:

*Tues, 6/22 - Major patches committed, first "usable" builds w/ new features
*

If all the real development features were committed by 6/22, that would give
us a week to bang on it suss out the bugs. Those of you working on larger
things (eg master fixup), do you think you'll be ready for it to be
committed at this point?

*Tues, 6/15 - Major patches in review*
*
*
Given the scope of some of the changes in the works (particularly thinking
about master work being done at FB right now) I think it would be excellent
if the rest of us could start looking at patches by 6/15. Fine if there are
still known bugs, but this will give us some days to look at the patch and
work in parallel on fixing them.

Do these target dates seem reasonable to everyone?

-Todd

-- 
Todd Lipcon
Software Engineer, Cloudera

Re: Working towards a dev release

[Stack <st...@duboce.net>]

On Fri, Jun 4, 2010 at 12:58 AM, Todd Lipcon <to...@cloudera.com> wrote:
> On Fri, Jun 4, 2010 at 12:50 AM, Dhruba Borthakur <dh...@gmail.com> wrote:
>
>> This also means that we should target a Hadoop release from the
>> branch-0.20-append, isn't it?
>>
>>
> Probably

I'd say that sounds great Dhruba.  HBase bundles a hadoop.  I'd
imagine the branch-0.20-append is what we'd bundle in this case.  Its
what we'll tell users install, it or optionally cdh3b2.

St.Ack

Re: Working towards a dev release

[Todd Lipcon <to...@cloudera.com>]

On Fri, Jun 4, 2010 at 12:50 AM, Dhruba Borthakur <dh...@gmail.com> wrote:

> This also means that we should target a Hadoop release from the
> branch-0.20-append, isn't it?
>
>
Probably, though I'm still a little confused as to who will be testing that
branch. I plan on concentrating my testing on top of CDH3b2, and I imagine
you guys are testing based on your internal branch. So we can certainly do
an append branch release, but I think we'll have to count on community
testing a bit more.

>
> On Thu, Jun 3, 2010 at 10:12 PM, Stack <st...@duboce.net> wrote:
>
> > Dates look good to me.
> >
> > There are too many open issues currently filed against
> > http://su.pr/698kib  If fellas have a moment, take a look.  See what
> > can be purged (Are there any we should pull in that are not currently
> > in the list?)  Adopt those you think you'll be able to work on.  Looks
> > like a few have wrong priority; e.g. master rewrite is major true, but
> > probably should be a blocker.
>

I took a swing through some of the blockers to try to triage some down.
While we have a ton of open issues, we should try to close out the real show
stoppers first. So long as trunk works generally as well as 0.20.5 I think
we'll be able to get some users trying out a dev release by offering the
durability carrot (plus the few big new features we've already got done)

-Todd

>
> > Good stuff,
> > St.Ack
> >
> >
> >
> > On Thu, Jun 3, 2010 at 10:01 PM, Todd Lipcon <to...@cloudera.com> wrote:
> > > Hey everyone,
> > >
> > > I'd like to start planning towards a development release this month.
> > Here's
> > > my thinking on timeline, working backwards:
> > >
> > > *Tues, 6/29 - Hadoop Summit, official first release of dev series*
> > >
> > > It would be great to have a release at this point that we think is
> > > moderately usable (ie not significantly less stable than 0.20.n has
> > been).
> > > Perhaps some of the new features (eg replication, coprocessors, etc)
> > would
> > > still be in progress but I think it would be great to be able to talk
> up
> > a
> > > working installable-today release at the summit.
> > >
> > > Thinking backwards, from here:
> > >
> > > *Tues, 6/22 - Major patches committed, first "usable" builds w/ new
> > features
> > > *
> > >
> > > If all the real development features were committed by 6/22, that would
> > give
> > > us a week to bang on it suss out the bugs. Those of you working on
> larger
> > > things (eg master fixup), do you think you'll be ready for it to be
> > > committed at this point?
> > >
> > > *Tues, 6/15 - Major patches in review*
> > > *
> > > *
> > > Given the scope of some of the changes in the works (particularly
> > thinking
> > > about master work being done at FB right now) I think it would be
> > excellent
> > > if the rest of us could start looking at patches by 6/15. Fine if there
> > are
> > > still known bugs, but this will give us some days to look at the patch
> > and
> > > work in parallel on fixing them.
> > >
> > > Do these target dates seem reasonable to everyone?
> > >
> > > -Todd
> > >
> > > --
> > > Todd Lipcon
> > > Software Engineer, Cloudera
> > >
> >
>
>
>
> --
> Connect to me at http://www.facebook.com/dhruba
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Re: Working towards a dev release

[Dhruba Borthakur <dh...@gmail.com>]

This also means that we should target a Hadoop release from the
branch-0.20-append, isn't it?

thanks
dhruba

On Thu, Jun 3, 2010 at 10:12 PM, Stack <st...@duboce.net> wrote:

> Dates look good to me.
>
> There are too many open issues currently filed against
> http://su.pr/698kib  If fellas have a moment, take a look.  See what
> can be purged (Are there any we should pull in that are not currently
> in the list?)  Adopt those you think you'll be able to work on.  Looks
> like a few have wrong priority; e.g. master rewrite is major true, but
> probably should be a blocker.
>
> Good stuff,
> St.Ack
>
>
>
> On Thu, Jun 3, 2010 at 10:01 PM, Todd Lipcon <to...@cloudera.com> wrote:
> > Hey everyone,
> >
> > I'd like to start planning towards a development release this month.
> Here's
> > my thinking on timeline, working backwards:
> >
> > *Tues, 6/29 - Hadoop Summit, official first release of dev series*
> >
> > It would be great to have a release at this point that we think is
> > moderately usable (ie not significantly less stable than 0.20.n has
> been).
> > Perhaps some of the new features (eg replication, coprocessors, etc)
> would
> > still be in progress but I think it would be great to be able to talk up
> a
> > working installable-today release at the summit.
> >
> > Thinking backwards, from here:
> >
> > *Tues, 6/22 - Major patches committed, first "usable" builds w/ new
> features
> > *
> >
> > If all the real development features were committed by 6/22, that would
> give
> > us a week to bang on it suss out the bugs. Those of you working on larger
> > things (eg master fixup), do you think you'll be ready for it to be
> > committed at this point?
> >
> > *Tues, 6/15 - Major patches in review*
> > *
> > *
> > Given the scope of some of the changes in the works (particularly
> thinking
> > about master work being done at FB right now) I think it would be
> excellent
> > if the rest of us could start looking at patches by 6/15. Fine if there
> are
> > still known bugs, but this will give us some days to look at the patch
> and
> > work in parallel on fixing them.
> >
> > Do these target dates seem reasonable to everyone?
> >
> > -Todd
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>



-- 
Connect to me at http://www.facebook.com/dhruba

Re: Working towards a dev release

[Stack <st...@duboce.net>]

Dates look good to me.

There are too many open issues currently filed against
http://su.pr/698kib  If fellas have a moment, take a look.  See what
can be purged (Are there any we should pull in that are not currently
in the list?)  Adopt those you think you'll be able to work on.  Looks
like a few have wrong priority; e.g. master rewrite is major true, but
probably should be a blocker.

Good stuff,
St.Ack



On Thu, Jun 3, 2010 at 10:01 PM, Todd Lipcon <to...@cloudera.com> wrote:
> Hey everyone,
>
> I'd like to start planning towards a development release this month. Here's
> my thinking on timeline, working backwards:
>
> *Tues, 6/29 - Hadoop Summit, official first release of dev series*
>
> It would be great to have a release at this point that we think is
> moderately usable (ie not significantly less stable than 0.20.n has been).
> Perhaps some of the new features (eg replication, coprocessors, etc) would
> still be in progress but I think it would be great to be able to talk up a
> working installable-today release at the summit.
>
> Thinking backwards, from here:
>
> *Tues, 6/22 - Major patches committed, first "usable" builds w/ new features
> *
>
> If all the real development features were committed by 6/22, that would give
> us a week to bang on it suss out the bugs. Those of you working on larger
> things (eg master fixup), do you think you'll be ready for it to be
> committed at this point?
>
> *Tues, 6/15 - Major patches in review*
> *
> *
> Given the scope of some of the changes in the works (particularly thinking
> about master work being done at FB right now) I think it would be excellent
> if the rest of us could start looking at patches by 6/15. Fine if there are
> still known bugs, but this will give us some days to look at the patch and
> work in parallel on fixing them.
>
> Do these target dates seem reasonable to everyone?
>
> -Todd
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>

RE: Working towards a dev release

[Jonathan Gray <jg...@facebook.com>]

I'll be putting up patches to jira/reviewboard when they're ready for consumption.  There's no unified thing that would actually compile and make sense in a github or I would just push it out.  All of my compact/split stuff was based on branch and was before the master changes which impact it so have work to do before it's ready for review.

We are going to break stuff up into smaller chunks so I won't just post a single gigantic patch.  That may include some jiras that are just refactors but don't change anything.  This will allow actual logic changes to be more concentrated.

JG

> -----Original Message-----
> From: Todd Lipcon [mailto:todd@cloudera.com]
> Sent: Saturday, June 05, 2010 10:40 AM
> To: dev@hbase.apache.org
> Subject: Re: Working towards a dev release
> 
> On Sat, Jun 5, 2010 at 12:46 PM, Jonathan Gray <jg...@facebook.com>
> wrote:
> 
> > The dates sound good, at least on master side of things.
> >
> > I've got a bunch of half-done stuff on the compaction/flush/split
> side of
> > things that I've been holding off on until the master stuff pushes
> through.
> >  I suspect 6/22 is enough time to finish my stuff, but I may not have
> all
> > patches up by 6/15.  I'll keep that as the target and see how far I
> can get.
> >  I'd really like to at least get the refactor and HBASE-2375 in
> before the
> > dev release, some of the other changes can wait.
> >
> >
> Great. Any chance you could push those works-in-progress off to github?
> We
> could get some more eyes on it early that way, so review will go faster
> when
> it's ready to go?
> 
> -Todd
> 
> 
> > Also, +1 on doing a dev release of the hadoop branch to ship the dev
> > release with.  Part of the way we can get it tested is by putting it
> out
> > there to the community.
> >
> > JG
> >
> > > -----Original Message-----
> > > From: Todd Lipcon [mailto:todd@cloudera.com]
> > > Sent: Thursday, June 03, 2010 10:02 PM
> > > To: dev
> > > Subject: Working towards a dev release
> > >
> > > Hey everyone,
> > >
> > > I'd like to start planning towards a development release this
> month.
> > > Here's
> > > my thinking on timeline, working backwards:
> > >
> > > *Tues, 6/29 - Hadoop Summit, official first release of dev series*
> > >
> > > It would be great to have a release at this point that we think is
> > > moderately usable (ie not significantly less stable than 0.20.n has
> > > been).
> > > Perhaps some of the new features (eg replication, coprocessors,
> etc)
> > > would
> > > still be in progress but I think it would be great to be able to
> talk
> > > up a
> > > working installable-today release at the summit.
> > >
> > > Thinking backwards, from here:
> > >
> > > *Tues, 6/22 - Major patches committed, first "usable" builds w/ new
> > > features
> > > *
> > >
> > > If all the real development features were committed by 6/22, that
> would
> > > give
> > > us a week to bang on it suss out the bugs. Those of you working on
> > > larger
> > > things (eg master fixup), do you think you'll be ready for it to be
> > > committed at this point?
> > >
> > > *Tues, 6/15 - Major patches in review*
> > > *
> > > *
> > > Given the scope of some of the changes in the works (particularly
> > > thinking
> > > about master work being done at FB right now) I think it would be
> > > excellent
> > > if the rest of us could start looking at patches by 6/15. Fine if
> there
> > > are
> > > still known bugs, but this will give us some days to look at the
> patch
> > > and
> > > work in parallel on fixing them.
> > >
> > > Do these target dates seem reasonable to everyone?
> > >
> > > -Todd
> > >
> > > --
> > > Todd Lipcon
> > > Software Engineer, Cloudera
> >
> 
> 
> 
> --
> Todd Lipcon
> Software Engineer, Cloudera

Re: Working towards a dev release

[Todd Lipcon <to...@cloudera.com>]

On Sat, Jun 5, 2010 at 12:46 PM, Jonathan Gray <jg...@facebook.com> wrote:

> The dates sound good, at least on master side of things.
>
> I've got a bunch of half-done stuff on the compaction/flush/split side of
> things that I've been holding off on until the master stuff pushes through.
>  I suspect 6/22 is enough time to finish my stuff, but I may not have all
> patches up by 6/15.  I'll keep that as the target and see how far I can get.
>  I'd really like to at least get the refactor and HBASE-2375 in before the
> dev release, some of the other changes can wait.
>
>
Great. Any chance you could push those works-in-progress off to github? We
could get some more eyes on it early that way, so review will go faster when
it's ready to go?

-Todd


> Also, +1 on doing a dev release of the hadoop branch to ship the dev
> release with.  Part of the way we can get it tested is by putting it out
> there to the community.
>
> JG
>
> > -----Original Message-----
> > From: Todd Lipcon [mailto:todd@cloudera.com]
> > Sent: Thursday, June 03, 2010 10:02 PM
> > To: dev
> > Subject: Working towards a dev release
> >
> > Hey everyone,
> >
> > I'd like to start planning towards a development release this month.
> > Here's
> > my thinking on timeline, working backwards:
> >
> > *Tues, 6/29 - Hadoop Summit, official first release of dev series*
> >
> > It would be great to have a release at this point that we think is
> > moderately usable (ie not significantly less stable than 0.20.n has
> > been).
> > Perhaps some of the new features (eg replication, coprocessors, etc)
> > would
> > still be in progress but I think it would be great to be able to talk
> > up a
> > working installable-today release at the summit.
> >
> > Thinking backwards, from here:
> >
> > *Tues, 6/22 - Major patches committed, first "usable" builds w/ new
> > features
> > *
> >
> > If all the real development features were committed by 6/22, that would
> > give
> > us a week to bang on it suss out the bugs. Those of you working on
> > larger
> > things (eg master fixup), do you think you'll be ready for it to be
> > committed at this point?
> >
> > *Tues, 6/15 - Major patches in review*
> > *
> > *
> > Given the scope of some of the changes in the works (particularly
> > thinking
> > about master work being done at FB right now) I think it would be
> > excellent
> > if the rest of us could start looking at patches by 6/15. Fine if there
> > are
> > still known bugs, but this will give us some days to look at the patch
> > and
> > work in parallel on fixing them.
> >
> > Do these target dates seem reasonable to everyone?
> >
> > -Todd
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
>



-- 
Todd Lipcon
Software Engineer, Cloudera

RE: Working towards a dev release

[Jonathan Gray <jg...@facebook.com>]

The dates sound good, at least on master side of things.

I've got a bunch of half-done stuff on the compaction/flush/split side of things that I've been holding off on until the master stuff pushes through.  I suspect 6/22 is enough time to finish my stuff, but I may not have all patches up by 6/15.  I'll keep that as the target and see how far I can get.  I'd really like to at least get the refactor and HBASE-2375 in before the dev release, some of the other changes can wait.

Also, +1 on doing a dev release of the hadoop branch to ship the dev release with.  Part of the way we can get it tested is by putting it out there to the community.

JG

> -----Original Message-----
> From: Todd Lipcon [mailto:todd@cloudera.com]
> Sent: Thursday, June 03, 2010 10:02 PM
> To: dev
> Subject: Working towards a dev release
> 
> Hey everyone,
> 
> I'd like to start planning towards a development release this month.
> Here's
> my thinking on timeline, working backwards:
> 
> *Tues, 6/29 - Hadoop Summit, official first release of dev series*
> 
> It would be great to have a release at this point that we think is
> moderately usable (ie not significantly less stable than 0.20.n has
> been).
> Perhaps some of the new features (eg replication, coprocessors, etc)
> would
> still be in progress but I think it would be great to be able to talk
> up a
> working installable-today release at the summit.
> 
> Thinking backwards, from here:
> 
> *Tues, 6/22 - Major patches committed, first "usable" builds w/ new
> features
> *
> 
> If all the real development features were committed by 6/22, that would
> give
> us a week to bang on it suss out the bugs. Those of you working on
> larger
> things (eg master fixup), do you think you'll be ready for it to be
> committed at this point?
> 
> *Tues, 6/15 - Major patches in review*
> *
> *
> Given the scope of some of the changes in the works (particularly
> thinking
> about master work being done at FB right now) I think it would be
> excellent
> if the rest of us could start looking at patches by 6/15. Fine if there
> are
> still known bugs, but this will give us some days to look at the patch
> and
> work in parallel on fixing them.
> 
> Do these target dates seem reasonable to everyone?
> 
> -Todd
> 
> --
> Todd Lipcon
> Software Engineer, Cloudera