Re: old master to 0.90 and new master to 0.92? (was RE: Millions of photos into Hbase)

[Jean-Daniel Cryans <jd...@apache.org>]

So while you were in the plane a few things were discussed :)

In the thread "[VOTE] Release 'development release' HBase 0.89.2010830
rc2", it was decided that we sink RC2 in favor of a new RC with a
rollback of the zk-based assignment. This is what we're running here,
and Ryan published our repo yesterday
http://github.com/stumbleupon/hbase  (see the top of the CHANGES file
for what we added)

Also some of your arguments correlate those of Todd, expressed in the
thread "Next release" that was posted on 09/15. Stack and Andrew
expressed their opinions in favor of Todd's option #1 (although Stack
said that he would decide tomorrow if he changes his opinion after
some more work on the master).

My personal opinion leans towards Todd's option #2. 0.20.0 is now more
than a year old, we are able to release almost production-ready code
that is durable, so why not move forward with it (this is what we did
here). There are a few issues to fix, but it's small compared to
getting the new master in shape plus getting the rest working (like
the replication code's interaction with ZooKeeper that needs to be
redone).

J-D

On Tue, Sep 21, 2010 at 4:49 AM, Jonathan Gray <jg...@facebook.com> wrote:
> Deep within my soul I do not want to do this.
>
> But it might be practical.
>
> FB is going into prod w/ the old master and we've been doing work stabilizing it in ways that do not apply at all to the new one (especially around zk).  That'll make having two different active branches a bit of a nightmare but right now some of these patches are not even available as we've been operating under the assumption that the release would be on the new master (and the patches don't apply on trunk).
>
> I guess SU is also putting an 0.89 old master release into prod.
>
> What's a little unfortunate is that the 0.89 releases include the unnecessary move of some transition communication into ZK.  That was put in as a first step towards new master before we decided to branch it off.
>
> The question for me would be whether we think it's at all feasible to get the new master 0.90 released in time for HW.  If not, maybe we should consider taking an 0.89 branch, making it an 0.90 branch, and focus on stabilizing for release in time for HW, as Todd suggests.
>
> There are other ramifications of this.  It would be very difficult for me to get my flush/compact/split improvements in to the old master as the new implementation I've been working on relies completely on the new stuff.  But maybe better to punt that for 0.92 as well so can really nail it?
>
> The other factor is the enormous amount of ZK improvements that were done in the new master branch.  It's a real fuckin mess in 0.89 releases, though I've done a bit of cleanup already towards making it at least tenable for production.
>
> My main concern is that this move will push back any release of the new master significantly.  There are countless improvements in the codebase that came along with the rewrite, well beyond just zk transitions.  But doing a production release in time for HW is probably the most important thing.
>
> JG
>
>> -----Original Message-----
>> From: Todd Lipcon [mailto:todd@cloudera.com]
>> Sent: Tuesday, September 21, 2010 8:10 AM
>> To: dev
>> Subject: Re: Millions of photos into Hbase
>>
>> On Mon, Sep 20, 2010 at 5:47 PM, Ryan Rawson <ry...@gmail.com>
>> wrote:
>> > Hey,
>> >
>> > There is actually only 1 active branch of hbase, that being the 0.89
>> > release, which is based on 'trunk'.  We have snapshotted a series of
>> > 0.89 "developer releases" in hopes that people would try them our and
>> > start thinking about the next major version.  One of these is what SU
>> > is running prod on.
>> >
>> > At this point tracking 0.89 and which ones are the 'best' peach sets
>> > to run is a bit of a contact sport, but if you are serious about not
>> > losing data it is worthwhile.  SU is based on the most recent DR with
>> > a few minor patches of our own concoction brought in.  If current
>> > works, but some Master ops are slow, and there are a few patches on
>> > top of that.  I'll poke about and see if its possible to publish to a
>> > github branch or something.
>> >
>>
>> This is why I kind of want to release the latest 0.89 dev release as
>> 0.90, and push off the new master stuff as 0.92 :)
>>
>> -Todd
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>