You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Jean-Daniel Cryans <jd...@apache.org> on 2010/04/06 02:37:24 UTC
Vote on 0.20.3.1
Hey devs,
There are a good bunch of fixes in the branch that lots of people
could have used recently. It's hard to tell exactly when 0.20.4 will
be released but I think we need a bug fix release ASAP.
I propose that we tag rev# 919707 (just before the backport of group
commit) and apply a couple of the other biggest fixes that happened
after that:
HBASE-2174 Stop from resolving HRegionServer addresses to names using
DNS on every heartbeat
HBASE-2308 Fix the bin/rename_table.rb script, make it work again
HBASE-2023 Client sync block can cause 1 thread of a multi-threaded
client to block all others
HBASE-2305 Client port for ZK has no default
HBASE-2323 filter.RegexStringComparator does not work with certain bytes
HBASE-2147 run zookeeper in the same jvm as master during non-distributed mode
HBASE-2355 Unsynchronized logWriters map is mutated from several
threads in HLog splitting
HBASE-2358 Store doReconstructionLog will fail if oldlogfile.log is
empty and won't load region
HBASE-2365 Double-assignment around split
HBASE-2087 The wait on compaction because "Too many store files"
holds up all flushing
HBASE-2252 Mapping a very big table kills region servers
We could do without 2308,2305,2147 but the rest is pretty important.
The work would be done at StumbleUpon to generate the RC. What do you
guys think?
J-D
Re: Vote on 0.20.3.1
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Could be too.
I forgot to add to the list:
HBASE-2277 Update branch to hadoop 0.20.2
J-D
On Mon, Apr 5, 2010 at 5:41 PM, Todd Lipcon <to...@cloudera.com> wrote:
> +1, I think all of these are solid improvements to known bugs. Only
> question: why not just make this 0.20.4, and then make the next one be
> 0.20.5?
>
> -Todd
>
> On Mon, Apr 5, 2010 at 5:37 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
>
>> Hey devs,
>>
>> There are a good bunch of fixes in the branch that lots of people
>> could have used recently. It's hard to tell exactly when 0.20.4 will
>> be released but I think we need a bug fix release ASAP.
>>
>> I propose that we tag rev# 919707 (just before the backport of group
>> commit) and apply a couple of the other biggest fixes that happened
>> after that:
>>
>> HBASE-2174 Stop from resolving HRegionServer addresses to names using
>> DNS on every heartbeat
>> HBASE-2308 Fix the bin/rename_table.rb script, make it work again
>> HBASE-2023 Client sync block can cause 1 thread of a multi-threaded
>> client to block all others
>> HBASE-2305 Client port for ZK has no default
>> HBASE-2323 filter.RegexStringComparator does not work with certain bytes
>> HBASE-2147 run zookeeper in the same jvm as master during non-distributed
>> mode
>> HBASE-2355 Unsynchronized logWriters map is mutated from several
>> threads in HLog splitting
>> HBASE-2358 Store doReconstructionLog will fail if oldlogfile.log is
>> empty and won't load region
>> HBASE-2365 Double-assignment around split
>> HBASE-2087 The wait on compaction because "Too many store files"
>> holds up all flushing
>> HBASE-2252 Mapping a very big table kills region servers
>>
>> We could do without 2308,2305,2147 but the rest is pretty important.
>> The work would be done at StumbleUpon to generate the RC. What do you
>> guys think?
>>
>> J-D
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
Re: Vote on 0.20.3.1
Posted by Todd Lipcon <to...@cloudera.com>.
+1, I think all of these are solid improvements to known bugs. Only
question: why not just make this 0.20.4, and then make the next one be
0.20.5?
-Todd
On Mon, Apr 5, 2010 at 5:37 PM, Jean-Daniel Cryans <jd...@apache.org>wrote:
> Hey devs,
>
> There are a good bunch of fixes in the branch that lots of people
> could have used recently. It's hard to tell exactly when 0.20.4 will
> be released but I think we need a bug fix release ASAP.
>
> I propose that we tag rev# 919707 (just before the backport of group
> commit) and apply a couple of the other biggest fixes that happened
> after that:
>
> HBASE-2174 Stop from resolving HRegionServer addresses to names using
> DNS on every heartbeat
> HBASE-2308 Fix the bin/rename_table.rb script, make it work again
> HBASE-2023 Client sync block can cause 1 thread of a multi-threaded
> client to block all others
> HBASE-2305 Client port for ZK has no default
> HBASE-2323 filter.RegexStringComparator does not work with certain bytes
> HBASE-2147 run zookeeper in the same jvm as master during non-distributed
> mode
> HBASE-2355 Unsynchronized logWriters map is mutated from several
> threads in HLog splitting
> HBASE-2358 Store doReconstructionLog will fail if oldlogfile.log is
> empty and won't load region
> HBASE-2365 Double-assignment around split
> HBASE-2087 The wait on compaction because "Too many store files"
> holds up all flushing
> HBASE-2252 Mapping a very big table kills region servers
>
> We could do without 2308,2305,2147 but the rest is pretty important.
> The work would be done at StumbleUpon to generate the RC. What do you
> guys think?
>
> J-D
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: Vote on 0.20.3.1
Posted by Andrew Purtell <ap...@apache.org>.
I will probably merge in 118c6e5 soon though, but with flushlogentries set to 100 instead of 1 in hbase-default.xml.
- Andy
> From: Andrew Purtell <ap...@apache.org>
> Subject: Re: Vote on 0.20.3.1
> To: hbase-dev@hadoop.apache.org
> Date: Wednesday, April 7, 2010, 10:30 AM
> For example I have an internal build
> that is current to 0.20 branch but excludes the following
> commits:
>
> commit 118c6e5ba091ed96b23b74a6523eb14b4bbddc13
> Author: Michael Stack <st...@apache.org>
> Date: Sat Mar 6 22:38:30 2010 +0000
> HBASE-2298 Backport of "HLog Group Commit" to 0.20 branch
>
> commit cb11e28f7d15e1c8175aed0faf74834bbc387717
> Author: Michael Stack <st...@apache.org>
> Date: Wed Mar 10 15:20:36 2010 +0000
> HBASE-2234 Roll Hlog if any datanode in the write pipeline
> dies
>
> commit 61b0ac467cc48aed5455ad97044139f3aa33d2d1
> Author: Michael Stack <st...@apache.org>
> Date: Thu Mar 18 23:51:49 2010 +0000
> HBASE-2283 row level atomicity
>
> commit 2163795ff6eb868425e1c16d5cefbcf3859cb4c9
> Author: Michael Stack <st...@apache.org>
> Date: Fri Mar 26 17:38:40 2010 +0000
> HBASE-2340 Add end-to-end test of sync/flush
>
> commit 63276ee240a9c39e690990213657984d764cb15b
> Author: Michael Stack <st...@apache.org>
> Date: Sun Mar 28 05:26:41 2010 +0000
> HBASE-2359 WALEdit doesn't implement HeapSize
>
> commit dfe47eaa91e441a1a4a337f787ee1897aed5b0b2
> Author: Michael Stack <st...@apache.org>
> Date: Sun Mar 28 05:05:22 2010 +0000
> HBASE-2338 log recovery: deleted items may be
> resurrected
>
> commit 2a5618cfe034a2c666869bb284e8e15384b5e10a
> Author: Michael Stack <st...@apache.org>
> Date: Thu Apr 1 06:42:31 2010 +0000
> HBASE-2338 log recovery: deleted items may be
> resurrected; 2nd attempt...
>
[...]
> > I would personally prefer a 0.20.4 release that does
> > not depend on having the patched version of Hadoop
> > jars included, since Hadoop 0.20.2 does not have 200
> > or 826, and our next release will target it.
Re: Vote on 0.20.3.1
Posted by Andrew Purtell <ap...@apache.org>.
For example I have an internal build that is current to 0.20 branch but excludes the following commits:
commit 118c6e5ba091ed96b23b74a6523eb14b4bbddc13
Author: Michael Stack <st...@apache.org>
Date: Sat Mar 6 22:38:30 2010 +0000
HBASE-2298 Backport of "HLog Group Commit" to 0.20 branch
commit cb11e28f7d15e1c8175aed0faf74834bbc387717
Author: Michael Stack <st...@apache.org>
Date: Wed Mar 10 15:20:36 2010 +0000
HBASE-2234 Roll Hlog if any datanode in the write pipeline dies
commit 61b0ac467cc48aed5455ad97044139f3aa33d2d1
Author: Michael Stack <st...@apache.org>
Date: Thu Mar 18 23:51:49 2010 +0000
HBASE-2283 row level atomicity
commit 2163795ff6eb868425e1c16d5cefbcf3859cb4c9
Author: Michael Stack <st...@apache.org>
Date: Fri Mar 26 17:38:40 2010 +0000
HBASE-2340 Add end-to-end test of sync/flush
commit 63276ee240a9c39e690990213657984d764cb15b
Author: Michael Stack <st...@apache.org>
Date: Sun Mar 28 05:26:41 2010 +0000
HBASE-2359 WALEdit doesn't implement HeapSize
commit dfe47eaa91e441a1a4a337f787ee1897aed5b0b2
Author: Michael Stack <st...@apache.org>
Date: Sun Mar 28 05:05:22 2010 +0000
HBASE-2338 log recovery: deleted items may be resurrected
commit 2a5618cfe034a2c666869bb284e8e15384b5e10a
Author: Michael Stack <st...@apache.org>
Date: Thu Apr 1 06:42:31 2010 +0000
HBASE-2338 log recovery: deleted items may be resurrected;
2nd attempt...
> We all agreed to work on and commit some changes which
> produce more than an incremental release, something in
> between a fork and an incremental release if I can be a
> little cute about it. We are walking back a little from that
> now that a release vetted on (an also vetted) HDFS-200 is a
> month or two away, so we can get something out the door now
> that fixes the warts in 0.20.3.
>
> I would personally prefer a 0.20.4 release that does not
> depend on having the patched version of Hadoop jars
> included, since Hadoop 0.20.2 does not have 200 or 826, and
> our next release will target it.
Re: Vote on 0.20.3.1
Posted by Andrew Purtell <ap...@apache.org>.
I think J-D is doing the right thing here.
We all agreed to work on and commit some changes which produce more than an incremental release, something in between a fork and an incremental release if I can be a little cute about it. We are walking back a little from that now that a release vetted on (an also vetted) HDFS-200 is a month or two away, so we can get something out the door now that fixes the warts in 0.20.3.
I would personally prefer a 0.20.4 release that does not depend on having the patched version of Hadoop jars included, since Hadoop 0.20.2 does not have 200 or 826, and our next release will target it.
> I think you and I can agree that all of those changes that
> came during that last month are disruptive, and even tho
> they are reviewed and tested we still need to go through
> some QA. In the mean time, new comers as well as regulars
> are hitting the same (fixed in branch) bugs over and over
> again.
>
> So to release a new version we need to tag something, we
> could just tag 919707 and be done with it but I think there
> are a couple of fixes that need to make it to 0.20.4 that
> were committed after that. Committing patches to a tag
> doesn't make sense, it's not a tag anymore. So to make it
> clear, I started a branch that people can commit to
> whatever they feel like is needed for our next
> release.
Re: Vote on 0.20.3.1
Posted by Jean-Daniel Cryans <jd...@apache.org>.
I think you and I can agree that all of those changes that came during
that last month are disruptive, and even tho they are reviewed and
tested we still need to go through some QA. In the mean time, new
comers as well as regulars are hitting the same (fixed in branch) bugs
over and over again.
So to release a new version we need to tag something, we could just
tag 919707 and be done with it but I think there are a couple of fixes
that need to make it to 0.20.4 that were committed after that.
Committing patches to a tag doesn't make sense, it's not a tag
anymore. So to make it clear, I started a branch that people can
commit to whatever they feel like is needed for our next release.
> Branching your release branch is... not a good idea.
Sorry but it's _our_ release, and if you think it's not a good idea
please do cast a -1 vote and propose an alternative.
J-D
On Tue, Apr 6, 2010 at 10:24 PM, Ryan Rawson <ry...@gmail.com> wrote:
> I don't get it - why did we commit all those things to 0.20 branch if
> they are not suitable for the next release?
>
> If we did accidentally commit things that are not suitable for 0.20.4,
> then we should revert them asap and make a 0.20.4 release from branch.
> Branching your release branch is... not a good idea.
>
> On Tue, Apr 6, 2010 at 10:50 AM, Jean-Daniel Cryans <jd...@apache.org> wrote:
>> I created a new branch from rev 919707 from which we will be able to tag 0.20.4:
>>
>> https://svn.apache.org/repos/asf/hadoop/hbase/branches/0.20_pre_durability/
>>
>> And I committed the following on top of it:
>>
>> HBASE-2034 Client sync block can cause 1 thread of a multi-threaded client
>> to block all others (Karthik Ranganathan via Stack)
>> HBASE-2355 Unsynchronized logWriters map is mutated from several threads in
>> HLog splitting (Todd Lipcon via Andrew Purtell)
>> HBASE-2358 Store doReconstructionLog will fail if oldlogfile.log is
>> empty and won't load region (Cosmin Lehene via Stack)
>> HBASE-2365 Double-assignment around split
>> HBASE-2174 Stop from resolving HRegionServer addresses to names using DNS on
>> every heartbeat (Karthik Ranganathan via Stack)
>> HBASE-2087 The wait on compaction because "Too many store files"
>> holds up all flushing
>> HBASE-2252 Mapping a very big table kills region servers
>> HBASE-2277 Update branch to hadoop 0.20.2
>>
>> HBASE-2248 should be added when it's ready for commit, then if anyone
>> else thinks its missing others jiras feel free to commit them (or ask
>> a committer to do it).
>>
>> J-D
>>
>> On Tue, Apr 6, 2010 at 10:31 AM, Andrew Purtell <ap...@apache.org> wrote:
>>> Yes, I mean something ready to eval and vote by the end of the week.
>>>
>>> 2248 should go in for a 0.20.4 RC I think, so we can vet it sooner rather than later. Can always take it out if there are problems.
>>>
>>> We will have enough changes to sort out for 0.20.5 (HDFS-200 repercussions) already.
>>>
>>> - Andy
>>>
>>>> From: Jean-Daniel Cryans
>>>>
>>>> Well we have to put up a RC, this would be when we have
>>>> enough votes?
>>>>
>>>> There's also the question if 2248 should be included.
>>>>
>>>> J-D
>>>>
>>
>
Re: Vote on 0.20.3.1
Posted by Jean-Daniel Cryans <jd...@apache.org>.
>
> I agree with Ryan that branching the release branch can get pretty
> confusing. I've only seen a version number like 0.20.3.1 once before, and
> that was for a completely botched release, where the .1 was a very very
> minor fix on top of the first release. In this case, even though there
> aren't any giant changes slated, there are changes that are large enough
> that it seems dishonest to call it a "doubly minor" release on top of
> 0.20.3.
We already agreed to call it 0.20.4 instead of 0.20.3.1
>
> Regarding the bigger changes we have slated for 0.20 branch (eg durability
> fixes, 2248), why not just call it 0.21 at that point? We've already decided
> that future versions of HBase will be compatible with multiple versions of
> Hadoop, so we can add these major changes for 0.21 and then make 0.22 the
> next major release with replication, mvn, etc?
This is for another discussion, I'd like to end this one first.
>
> -Todd
>
Re: Vote on 0.20.3.1
Posted by Andrew Purtell <ap...@apache.org>.
> From: Ryan Rawson
> What happens if we need to patch 0.20.4?
All fixes for 0.20.4 would go in 0.20.5-dev as is typically done unless its an exceptional circumstance.
> And make a new release based on 0.20.4 but in 2 weeks but
> not before the 0.20 branch is ready?
>
> How can we guarantee that this won't happen?
We can't.
So here's the deal:
- We don't want to wait for 0.20.X + HDFS-200 because that will take too long.
- Current 0.20.4-dev has blockers due to recent changes that J-D has volunteered to cherry pick out so we can get a clean release now.
Good enough all things considered?
I think so.
- Andy
Re: Vote on 0.20.3.1
Posted by Stack <st...@duboce.net>.
On Wed, Apr 7, 2010 at 1:35 PM, Ryan Rawson <ry...@gmail.com> wrote:
> What happens if we need to patch 0.20.4? And make a new release based
> on 0.20.4 but in 2 weeks but not before the 0.20 branch is ready?
>
The though is that we will fix in 0.20.5. We won't put fixes on the
branch of branch.
> How can we guarantee that this won't happen?
>
We can't. We're gambling that 0.20.5 (made from head of the branch)
will be available in time to take over if 0.20.4 has really bad
issues.
St.Ack
Re: Vote on 0.20.3.1
Posted by Ryan Rawson <ry...@gmail.com>.
What happens if we need to patch 0.20.4? And make a new release based
on 0.20.4 but in 2 weeks but not before the 0.20 branch is ready?
How can we guarantee that this won't happen?
On Wed, Apr 7, 2010 at 1:29 PM, Jonathan Gray <jg...@facebook.com> wrote:
> The issue would be that we break out 0.20.4 to make an RC. Now we have to
> patch 3 separate trees for each bug fix (0.20.4, 0.20.5, and 0.21).
>
> Even given that, I'm still +1 on moving ahead with this plan. Once we
> release 0.20.4, dev goes back to how we are operating now against branch and
> trunk.
>
> JG
>
> On 4/7/10 12:57 PM, "Andrew Purtell" <ap...@apache.org> wrote:
>
>> I thought we agreed no third branch. What we are discussing now is 0.20.4.
>> Everything else is 0.20.5.
>>
>> If 0.20.4 release is 0.20.4-dev minus some changes, but 0.20.5 (and
>> 0.20.5-dev) is whatever went into 0.20.4 plus what was left out, then there is
>> some temporary discontinuity on the branch at the release (elided commits) but
>> that is immediately healed after the release when 0.20.5-dev is opened. So
>> from the users point of view there is no discontinuity. Trouble with 0.20.4?
>> Fix in 0.20.5-dev per usual. Or am I missing something?
>>
>>> From: Ryan Rawson <ry...@gmail.com>
>>> Subject: Re: Vote on 0.20.3.1
>>> To: hbase-dev@hadoop.apache.org
>>> Date: Wednesday, April 7, 2010, 12:16 PM
>>> If we made a 3rd branch we now have 3 hbase versions with 3
>>> distinct code lineages. Will we patch this new 0.20.4 version
>>> as bugs come up?
>>
>>
>>
>>
>>
>
>
Re: Vote on 0.20.3.1
Posted by Ryan Rawson <ry...@gmail.com>.
I like this. With no durability goal, we should be able to hit this in
short order. HDFS is not ready for the claim as of yet.
On Wed, Apr 7, 2010 at 5:25 PM, Stack <st...@duboce.net> wrote:
> I went through the list issues that we had filed against a 0.20.4. I
> moved out the issues that depend on hdfs-200, post rpc-version change,
> etc. Outstanding are ~15 issues: http://su.pr/1TLvb7 A good few are
> patch available. A couple of others are small doc issues. I'll take
> care of the easy ones this evening. Please go over the list
> yourselves to see what else we should include. As is, the list
> includes hbase-2248. I think we should try and get this one in if
> only to make the release a little interesting (and to fix the
> deadlock). We also need to fix the json jar issue so we have chance
> of being debian package.
>
> St.Ack
> P.S. I haven't voted yet. At the moment I'm +0. If hbase-2248 goes
> in, I'd go +1.
>
> On Wed, Apr 7, 2010 at 3:40 PM, Andrew Purtell <ap...@yahoo.com> wrote:
>> Implied is we move people to 0.20.5 rather than do a patch on 0.20.4, beyond the RC period.
>>
>> Hence it's even more important that J-D cherry pick bugfixes out of 0.20.4-dev for an 0.20.4 RC and release.
>>
>> Is that acceptable?
>>
>> - Andy
>>
>>
>>> From: Andrew Purtell <ap...@apache.org>
>>> Subject: Re: Vote on 0.20.3.1
>>> To: "hbase-dev@hadoop.apache.org" <hb...@hadoop.apache.org>
>>> Date: Wednesday, April 7, 2010, 3:35 PM
>>> > From: Jonathan Gray
>>> >
>>> > The issue would be that we break out 0.20.4 to make an
>>> RC.
>>> > Now we have to patch 3 separate trees for each bug fix
>>> (0.20.4,
>>> > 0.20.5, and 0.21).
>>>
>>> Ok, I grant you that. But only during the RC.
>>>
>>> - Andy
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>>
>
Re: Vote on 0.20.3.1
Posted by Stack <st...@duboce.net>.
I went through the list issues that we had filed against a 0.20.4. I
moved out the issues that depend on hdfs-200, post rpc-version change,
etc. Outstanding are ~15 issues: http://su.pr/1TLvb7 A good few are
patch available. A couple of others are small doc issues. I'll take
care of the easy ones this evening. Please go over the list
yourselves to see what else we should include. As is, the list
includes hbase-2248. I think we should try and get this one in if
only to make the release a little interesting (and to fix the
deadlock). We also need to fix the json jar issue so we have chance
of being debian package.
St.Ack
P.S. I haven't voted yet. At the moment I'm +0. If hbase-2248 goes
in, I'd go +1.
On Wed, Apr 7, 2010 at 3:40 PM, Andrew Purtell <ap...@yahoo.com> wrote:
> Implied is we move people to 0.20.5 rather than do a patch on 0.20.4, beyond the RC period.
>
> Hence it's even more important that J-D cherry pick bugfixes out of 0.20.4-dev for an 0.20.4 RC and release.
>
> Is that acceptable?
>
> - Andy
>
>
>> From: Andrew Purtell <ap...@apache.org>
>> Subject: Re: Vote on 0.20.3.1
>> To: "hbase-dev@hadoop.apache.org" <hb...@hadoop.apache.org>
>> Date: Wednesday, April 7, 2010, 3:35 PM
>> > From: Jonathan Gray
>> >
>> > The issue would be that we break out 0.20.4 to make an
>> RC.
>> > Now we have to patch 3 separate trees for each bug fix
>> (0.20.4,
>> > 0.20.5, and 0.21).
>>
>> Ok, I grant you that. But only during the RC.
>>
>> - Andy
>>
>>
>>
>>
>>
>
>
>
>
>
Re: Vote on 0.20.3.1
Posted by Andrew Purtell <ap...@yahoo.com>.
Implied is we move people to 0.20.5 rather than do a patch on 0.20.4, beyond the RC period.
Hence it's even more important that J-D cherry pick bugfixes out of 0.20.4-dev for an 0.20.4 RC and release.
Is that acceptable?
- Andy
> From: Andrew Purtell <ap...@apache.org>
> Subject: Re: Vote on 0.20.3.1
> To: "hbase-dev@hadoop.apache.org" <hb...@hadoop.apache.org>
> Date: Wednesday, April 7, 2010, 3:35 PM
> > From: Jonathan Gray
> >
> > The issue would be that we break out 0.20.4 to make an
> RC.
> > Now we have to patch 3 separate trees for each bug fix
> (0.20.4,
> > 0.20.5, and 0.21).
>
> Ok, I grant you that. But only during the RC.
>
> - Andy
>
>
>
>
>
Re: Vote on 0.20.3.1
Posted by Andrew Purtell <ap...@apache.org>.
> From: Jonathan Gray
>
> The issue would be that we break out 0.20.4 to make an RC.
> Now we have to patch 3 separate trees for each bug fix (0.20.4,
> 0.20.5, and 0.21).
Ok, I grant you that. But only during the RC.
- Andy
Re: Vote on 0.20.3.1
Posted by Jonathan Gray <jg...@facebook.com>.
The issue would be that we break out 0.20.4 to make an RC. Now we have to
patch 3 separate trees for each bug fix (0.20.4, 0.20.5, and 0.21).
Even given that, I'm still +1 on moving ahead with this plan. Once we
release 0.20.4, dev goes back to how we are operating now against branch and
trunk.
JG
On 4/7/10 12:57 PM, "Andrew Purtell" <ap...@apache.org> wrote:
> I thought we agreed no third branch. What we are discussing now is 0.20.4.
> Everything else is 0.20.5.
>
> If 0.20.4 release is 0.20.4-dev minus some changes, but 0.20.5 (and
> 0.20.5-dev) is whatever went into 0.20.4 plus what was left out, then there is
> some temporary discontinuity on the branch at the release (elided commits) but
> that is immediately healed after the release when 0.20.5-dev is opened. So
> from the users point of view there is no discontinuity. Trouble with 0.20.4?
> Fix in 0.20.5-dev per usual. Or am I missing something?
>
>> From: Ryan Rawson <ry...@gmail.com>
>> Subject: Re: Vote on 0.20.3.1
>> To: hbase-dev@hadoop.apache.org
>> Date: Wednesday, April 7, 2010, 12:16 PM
>> If we made a 3rd branch we now have 3 hbase versions with 3
>> distinct code lineages. Will we patch this new 0.20.4 version
>> as bugs come up?
>
>
>
>
>
Re: Vote on 0.20.3.1
Posted by Andrew Purtell <ap...@apache.org>.
I thought we agreed no third branch. What we are discussing now is 0.20.4. Everything else is 0.20.5.
If 0.20.4 release is 0.20.4-dev minus some changes, but 0.20.5 (and 0.20.5-dev) is whatever went into 0.20.4 plus what was left out, then there is some temporary discontinuity on the branch at the release (elided commits) but that is immediately healed after the release when 0.20.5-dev is opened. So from the users point of view there is no discontinuity. Trouble with 0.20.4? Fix in 0.20.5-dev per usual. Or am I missing something?
> From: Ryan Rawson <ry...@gmail.com>
> Subject: Re: Vote on 0.20.3.1
> To: hbase-dev@hadoop.apache.org
> Date: Wednesday, April 7, 2010, 12:16 PM
> If we made a 3rd branch we now have 3 hbase versions with 3
> distinct code lineages. Will we patch this new 0.20.4 version
> as bugs come up?
Re: Vote on 0.20.3.1
Posted by Ryan Rawson <ry...@gmail.com>.
If we made a 3rd branch we now have 3 hbase versions with 3 distinct
code lineages. Will we patch this new 0.20.4 version as bugs come up?
Or will there be a recommendation to move to a wholly new code line?
If branch isn't ready today, why will it be ready in 4 weeks?
On Wed, Apr 7, 2010 at 12:14 PM, Stack <st...@duboce.net> wrote:
> On Wed, Apr 7, 2010 at 11:34 AM, Todd Lipcon <to...@cloudera.com> wrote:
>> ...
>> I agree with Ryan that branching the release branch can get pretty
>> confusing. I've only seen a version number like 0.20.3.1 once before, and
>> that was for a completely botched release, where the .1 was a very very
>> minor fix on top of the first release. In this case, even though there
>> aren't any giant changes slated, there are changes that are large enough
>> that it seems dishonest to call it a "doubly minor" release on top of
>> 0.20.3.
>>
>
> I should have been clearer. I was referring to the physical branch in
> svn distinct from what we call the release.
>
> The consensus seemed to be going in the direction of calling the
> (branch of a branch) release 0.20.4 rather than 0.20.3.1 as a branch
> of a branch would usually imply. I'd be fine with that.
>
> Up to this, if truth be told, our patch releases have been more than
> just bug fixes. They've also included improvements and even new
> features. This comes of our being strait-jacketed by our legacy tie
> to hadoop versioning (major and minor only IIUC, no space for patch
> number as in 0.20.2 is the major version 20 and the minor version 2)
> compounded by the lack of a release 0.21 to free up space for changes.
> I liked 0.20.3.1. because it conveyed notion that this would be a
> patch release.... but I'm easy with what its called.
>
>
> St.Ack
>
>>
>>
>>>
>>> My current dilemma is that I think a release should include HBASE-2322
>>> "deadlock between put and cacheflusher in 0.20 branch" -- this seemed
>>> easy for me to trigger bulk uploading and IMO, a blocker -- but the
>>> fix for the deadlock is HBASE-2248 "Provide new non-copy mechanism to
>>> assure atomic reads in get and scan", a critical fix but a big change
>>> in how Gets work. I'd like to ship with HBASE-2248 -- for one of the
>>> reasons why, see comments in HBASE-2322 -- but I'd be uncomfortable
>>> doing so w/o a bunch of testing first.
>>>
>>> What do you all think?
>>>
>>> St.Ack
>>>
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
>
Re: Vote on 0.20.3.1
Posted by Stack <st...@duboce.net>.
On Wed, Apr 7, 2010 at 11:34 AM, Todd Lipcon <to...@cloudera.com> wrote:
> ...
> I agree with Ryan that branching the release branch can get pretty
> confusing. I've only seen a version number like 0.20.3.1 once before, and
> that was for a completely botched release, where the .1 was a very very
> minor fix on top of the first release. In this case, even though there
> aren't any giant changes slated, there are changes that are large enough
> that it seems dishonest to call it a "doubly minor" release on top of
> 0.20.3.
>
I should have been clearer. I was referring to the physical branch in
svn distinct from what we call the release.
The consensus seemed to be going in the direction of calling the
(branch of a branch) release 0.20.4 rather than 0.20.3.1 as a branch
of a branch would usually imply. I'd be fine with that.
Up to this, if truth be told, our patch releases have been more than
just bug fixes. They've also included improvements and even new
features. This comes of our being strait-jacketed by our legacy tie
to hadoop versioning (major and minor only IIUC, no space for patch
number as in 0.20.2 is the major version 20 and the minor version 2)
compounded by the lack of a release 0.21 to free up space for changes.
I liked 0.20.3.1. because it conveyed notion that this would be a
patch release.... but I'm easy with what its called.
St.Ack
>
>
>>
>> My current dilemma is that I think a release should include HBASE-2322
>> "deadlock between put and cacheflusher in 0.20 branch" -- this seemed
>> easy for me to trigger bulk uploading and IMO, a blocker -- but the
>> fix for the deadlock is HBASE-2248 "Provide new non-copy mechanism to
>> assure atomic reads in get and scan", a critical fix but a big change
>> in how Gets work. I'd like to ship with HBASE-2248 -- for one of the
>> reasons why, see comments in HBASE-2322 -- but I'd be uncomfortable
>> doing so w/o a bunch of testing first.
>>
>> What do you all think?
>>
>> St.Ack
>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
Re: Vote on 0.20.3.1
Posted by Todd Lipcon <to...@cloudera.com>.
On Wed, Apr 7, 2010 at 11:21 AM, Stack <st...@duboce.net> wrote:
> On Tue, Apr 6, 2010 at 10:24 PM, Ryan Rawson <ry...@gmail.com> wrote:
> > I don't get it - why did we commit all those things to 0.20 branch if
> > they are not suitable for the next release?
> >
> > If we did accidentally commit things that are not suitable for 0.20.4,
> > then we should revert them asap and make a 0.20.4 release from branch.
> > Branching your release branch is... not a good idea.
> >
>
> I don't see any issue with branching a release branch.
>
> Regards 0.20.4, all in branch needs to make it into a release but the
> proposed 0.20.4 is currently incomplete. The suggestion is that
> meantime, make a release that leaves out the half-done stuff --tested
> data durability, etc. -- and ship critical bug fixes only.
>
I agree with Ryan that branching the release branch can get pretty
confusing. I've only seen a version number like 0.20.3.1 once before, and
that was for a completely botched release, where the .1 was a very very
minor fix on top of the first release. In this case, even though there
aren't any giant changes slated, there are changes that are large enough
that it seems dishonest to call it a "doubly minor" release on top of
0.20.3.
Regarding the bigger changes we have slated for 0.20 branch (eg durability
fixes, 2248), why not just call it 0.21 at that point? We've already decided
that future versions of HBase will be compatible with multiple versions of
Hadoop, so we can add these major changes for 0.21 and then make 0.22 the
next major release with replication, mvn, etc?
-Todd
>
> My current dilemma is that I think a release should include HBASE-2322
> "deadlock between put and cacheflusher in 0.20 branch" -- this seemed
> easy for me to trigger bulk uploading and IMO, a blocker -- but the
> fix for the deadlock is HBASE-2248 "Provide new non-copy mechanism to
> assure atomic reads in get and scan", a critical fix but a big change
> in how Gets work. I'd like to ship with HBASE-2248 -- for one of the
> reasons why, see comments in HBASE-2322 -- but I'd be uncomfortable
> doing so w/o a bunch of testing first.
>
> What do you all think?
>
> St.Ack
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: Vote on 0.20.3.1
Posted by Stack <st...@duboce.net>.
On Tue, Apr 6, 2010 at 10:24 PM, Ryan Rawson <ry...@gmail.com> wrote:
> I don't get it - why did we commit all those things to 0.20 branch if
> they are not suitable for the next release?
>
> If we did accidentally commit things that are not suitable for 0.20.4,
> then we should revert them asap and make a 0.20.4 release from branch.
> Branching your release branch is... not a good idea.
>
I don't see any issue with branching a release branch.
Regards 0.20.4, all in branch needs to make it into a release but the
proposed 0.20.4 is currently incomplete. The suggestion is that
meantime, make a release that leaves out the half-done stuff --tested
data durability, etc. -- and ship critical bug fixes only.
My current dilemma is that I think a release should include HBASE-2322
"deadlock between put and cacheflusher in 0.20 branch" -- this seemed
easy for me to trigger bulk uploading and IMO, a blocker -- but the
fix for the deadlock is HBASE-2248 "Provide new non-copy mechanism to
assure atomic reads in get and scan", a critical fix but a big change
in how Gets work. I'd like to ship with HBASE-2248 -- for one of the
reasons why, see comments in HBASE-2322 -- but I'd be uncomfortable
doing so w/o a bunch of testing first.
What do you all think?
St.Ack
Re: Vote on 0.20.3.1
Posted by Ryan Rawson <ry...@gmail.com>.
I don't get it - why did we commit all those things to 0.20 branch if
they are not suitable for the next release?
If we did accidentally commit things that are not suitable for 0.20.4,
then we should revert them asap and make a 0.20.4 release from branch.
Branching your release branch is... not a good idea.
On Tue, Apr 6, 2010 at 10:50 AM, Jean-Daniel Cryans <jd...@apache.org> wrote:
> I created a new branch from rev 919707 from which we will be able to tag 0.20.4:
>
> https://svn.apache.org/repos/asf/hadoop/hbase/branches/0.20_pre_durability/
>
> And I committed the following on top of it:
>
> HBASE-2034 Client sync block can cause 1 thread of a multi-threaded client
> to block all others (Karthik Ranganathan via Stack)
> HBASE-2355 Unsynchronized logWriters map is mutated from several threads in
> HLog splitting (Todd Lipcon via Andrew Purtell)
> HBASE-2358 Store doReconstructionLog will fail if oldlogfile.log is
> empty and won't load region (Cosmin Lehene via Stack)
> HBASE-2365 Double-assignment around split
> HBASE-2174 Stop from resolving HRegionServer addresses to names using DNS on
> every heartbeat (Karthik Ranganathan via Stack)
> HBASE-2087 The wait on compaction because "Too many store files"
> holds up all flushing
> HBASE-2252 Mapping a very big table kills region servers
> HBASE-2277 Update branch to hadoop 0.20.2
>
> HBASE-2248 should be added when it's ready for commit, then if anyone
> else thinks its missing others jiras feel free to commit them (or ask
> a committer to do it).
>
> J-D
>
> On Tue, Apr 6, 2010 at 10:31 AM, Andrew Purtell <ap...@apache.org> wrote:
>> Yes, I mean something ready to eval and vote by the end of the week.
>>
>> 2248 should go in for a 0.20.4 RC I think, so we can vet it sooner rather than later. Can always take it out if there are problems.
>>
>> We will have enough changes to sort out for 0.20.5 (HDFS-200 repercussions) already.
>>
>> - Andy
>>
>>> From: Jean-Daniel Cryans
>>>
>>> Well we have to put up a RC, this would be when we have
>>> enough votes?
>>>
>>> There's also the question if 2248 should be included.
>>>
>>> J-D
>>>
>
Re: Vote on 0.20.3.1
Posted by Jean-Daniel Cryans <jd...@apache.org>.
I created a new branch from rev 919707 from which we will be able to tag 0.20.4:
https://svn.apache.org/repos/asf/hadoop/hbase/branches/0.20_pre_durability/
And I committed the following on top of it:
HBASE-2034 Client sync block can cause 1 thread of a multi-threaded client
to block all others (Karthik Ranganathan via Stack)
HBASE-2355 Unsynchronized logWriters map is mutated from several threads in
HLog splitting (Todd Lipcon via Andrew Purtell)
HBASE-2358 Store doReconstructionLog will fail if oldlogfile.log is
empty and won't load region (Cosmin Lehene via Stack)
HBASE-2365 Double-assignment around split
HBASE-2174 Stop from resolving HRegionServer addresses to names using DNS on
every heartbeat (Karthik Ranganathan via Stack)
HBASE-2087 The wait on compaction because "Too many store files"
holds up all flushing
HBASE-2252 Mapping a very big table kills region servers
HBASE-2277 Update branch to hadoop 0.20.2
HBASE-2248 should be added when it's ready for commit, then if anyone
else thinks its missing others jiras feel free to commit them (or ask
a committer to do it).
J-D
On Tue, Apr 6, 2010 at 10:31 AM, Andrew Purtell <ap...@apache.org> wrote:
> Yes, I mean something ready to eval and vote by the end of the week.
>
> 2248 should go in for a 0.20.4 RC I think, so we can vet it sooner rather than later. Can always take it out if there are problems.
>
> We will have enough changes to sort out for 0.20.5 (HDFS-200 repercussions) already.
>
> - Andy
>
>> From: Jean-Daniel Cryans
>>
>> Well we have to put up a RC, this would be when we have
>> enough votes?
>>
>> There's also the question if 2248 should be included.
>>
>> J-D
>>
RE: Vote on 0.20.3.1
Posted by Jonathan Gray <jg...@facebook.com>.
I'm +1 on rolling an 0.20.4 RC (not 0.20.3.1) and moving our current 0.20.4 to 0.20.5
I would actually like to see 2248 put into a release ASAP so we can work out any new potential issues sooner than later.
JG
> -----Original Message-----
> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-
> Daniel Cryans
> Sent: Monday, April 05, 2010 6:52 PM
> To: hbase-dev@hadoop.apache.org; apurtell@apache.org
> Subject: Re: Vote on 0.20.3.1
>
> Well we have to put up a RC, this would be when we have enough votes?
>
> There's also the question if 2248 should be included.
>
> J-D
>
> On Mon, Apr 5, 2010 at 6:49 PM, Andrew Purtell <ap...@apache.org>
> wrote:
> > +1
> >
> > What is the release schedule? By end of this week?
> >
> > Why not make it 0.20.4 as Todd asks?
> >
> > - Andy
> >
> >> From: Jean-Daniel Cryans
> >> I propose that we tag rev# 919707 (just before the backport
> >> of group commit) and apply a couple of the other biggest
> >> fixes that happened after that:
> >>
> >> HBASE-2174 Stop from resolving HRegionServer addresses to
> >> names using DNS on every heartbeat
> >> HBASE-2308 Fix the bin/rename_table.rb script, make it work
> >> again
> >> HBASE-2023 Client sync block can cause 1 thread of a
> >> multi-threaded client to block all others
> >> HBASE-2305 Client port for ZK has no default
> >> HBASE-2323 filter.RegexStringComparator does not work with
> >> certain bytes
> >> HBASE-2147 run zookeeper in the same jvm as master
> >> during non-distributed mode
> >> HBASE-2355 Unsynchronized logWriters map is mutated from
> >> several threads in HLog splitting
> >> HBASE-2358 Store doReconstructionLog will fail if
> >> oldlogfile.log is empty and won't load region
> >> HBASE-2365 Double-assignment around split
> >> HBASE-2087 The wait on compaction because "Too many
> >> store files" holds up all flushing
> >> HBASE-2252 Mapping a very big table kills region
> >> servers
> >>
> >> We could do without 2308,2305,2147 but the rest is pretty
> >> important.
> >
> >
> >
> >
> >
> >
Re: Vote on 0.20.3.1
Posted by Andrew Purtell <ap...@apache.org>.
Yes, I mean something ready to eval and vote by the end of the week.
2248 should go in for a 0.20.4 RC I think, so we can vet it sooner rather than later. Can always take it out if there are problems.
We will have enough changes to sort out for 0.20.5 (HDFS-200 repercussions) already.
- Andy
> From: Jean-Daniel Cryans
>
> Well we have to put up a RC, this would be when we have
> enough votes?
>
> There's also the question if 2248 should be included.
>
> J-D
>
> On Mon, Apr 5, 2010 at 6:49 PM, Andrew Purtell <ap...@apache.org>
> wrote:
> > +1
> >
> > What is the release schedule? By end of this week?
> >
> > Why not make it 0.20.4 as Todd asks?
Re: Vote on 0.20.3.1
Posted by Jean-Daniel Cryans <jd...@apache.org>.
Well we have to put up a RC, this would be when we have enough votes?
There's also the question if 2248 should be included.
J-D
On Mon, Apr 5, 2010 at 6:49 PM, Andrew Purtell <ap...@apache.org> wrote:
> +1
>
> What is the release schedule? By end of this week?
>
> Why not make it 0.20.4 as Todd asks?
>
> - Andy
>
>> From: Jean-Daniel Cryans
>> I propose that we tag rev# 919707 (just before the backport
>> of group commit) and apply a couple of the other biggest
>> fixes that happened after that:
>>
>> HBASE-2174 Stop from resolving HRegionServer addresses to
>> names using DNS on every heartbeat
>> HBASE-2308 Fix the bin/rename_table.rb script, make it work
>> again
>> HBASE-2023 Client sync block can cause 1 thread of a
>> multi-threaded client to block all others
>> HBASE-2305 Client port for ZK has no default
>> HBASE-2323 filter.RegexStringComparator does not work with
>> certain bytes
>> HBASE-2147 run zookeeper in the same jvm as master
>> during non-distributed mode
>> HBASE-2355 Unsynchronized logWriters map is mutated from
>> several threads in HLog splitting
>> HBASE-2358 Store doReconstructionLog will fail if
>> oldlogfile.log is empty and won't load region
>> HBASE-2365 Double-assignment around split
>> HBASE-2087 The wait on compaction because "Too many
>> store files" holds up all flushing
>> HBASE-2252 Mapping a very big table kills region
>> servers
>>
>> We could do without 2308,2305,2147 but the rest is pretty
>> important.
>
>
>
>
>
>
Re: Vote on 0.20.3.1
Posted by Andrew Purtell <ap...@apache.org>.
+1
What is the release schedule? By end of this week?
Why not make it 0.20.4 as Todd asks?
- Andy
> From: Jean-Daniel Cryans
> I propose that we tag rev# 919707 (just before the backport
> of group commit) and apply a couple of the other biggest
> fixes that happened after that:
>
> HBASE-2174 Stop from resolving HRegionServer addresses to
> names using DNS on every heartbeat
> HBASE-2308 Fix the bin/rename_table.rb script, make it work
> again
> HBASE-2023 Client sync block can cause 1 thread of a
> multi-threaded client to block all others
> HBASE-2305 Client port for ZK has no default
> HBASE-2323 filter.RegexStringComparator does not work with
> certain bytes
> HBASE-2147 run zookeeper in the same jvm as master
> during non-distributed mode
> HBASE-2355 Unsynchronized logWriters map is mutated from
> several threads in HLog splitting
> HBASE-2358 Store doReconstructionLog will fail if
> oldlogfile.log is empty and won't load region
> HBASE-2365 Double-assignment around split
> HBASE-2087 The wait on compaction because "Too many
> store files" holds up all flushing
> HBASE-2252 Mapping a very big table kills region
> servers
>
> We could do without 2308,2305,2147 but the rest is pretty
> important.