You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by Jean-Daniel Cryans <jd...@apache.org> on 2010/09/25 01:03:27 UTC
[VOTE] Release 'development release' HBase 0.89.2010924 rc1?
The 0.89.20100830 DR branch was cancelled, here's the new RC off a new branch.
As discussed, this release candidate contains a revert of HBASE-2694
which means that it is back on the "very" old master. It is also very
similar to what we run here in production.
Sources and binaries can be found here:
http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
Documentation:
http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/hbase-0.89.20100924/docs/
Here's the list of everything I added since moving from 0830:
HBASE-3008 Memstore.updateColumnValue passes wrong flag to heapSizeChange
HBASE-3035 Bandaid for HBASE-2990
HBASE-2643 Figure how to deal with eof splitting logs
HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads - to HBase
HBASE-3006 Reading compressed HFile blocks causes way too many DFS RPC calls
severly impacting performance
HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
HBASE-2992 [replication] MalformedObjectNameException in ReplicationMetrics
HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
pals) for 0.89
HBASE-3033 [replication] ReplicationSink.replicateEntries improvements
HBASE-2997 Performance fixes - profiler driven
HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2 only)
Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
just figured it while reading the old voting thread).
Should we release this as the next "Development Release"? Please cast
your vote by Wednesday, September 29th.
Thanks,
The HBase Team
Re: [VOTE] Release 'development release' HBase 0.89.2010924 rc1?
Posted by Ryan Rawson <ry...@gmail.com>.
yes it is ICV only, and most prevalent on tables that are heavily/only icv.
you can always kill -9 the RS to force log recovery and all will be
well. assuming you can take the outage :-)
On Mon, Oct 4, 2010 at 8:51 PM, Stack <st...@duboce.net> wrote:
> Sure. That caveat about no warranty, do not use in "production", is
> on there already. And the bug is in ICVs only, right? We can release
> w/ warning that ICVers need to apply the patch, np.
>
> Good stuff,
> St.Ack
>
>
> On Mon, Oct 4, 2010 at 8:40 PM, Ryan Rawson <ry...@gmail.com> wrote:
>> we could yes. with the caveat that no production use/data loss ahoy.
>>
>> -ryan
>>
>> On Mon, Oct 4, 2010 at 8:38 PM, Stack <st...@duboce.net> wrote:
>>> On Mon, Oct 4, 2010 at 4:56 PM, Ryan Rawson <ry...@gmail.com> wrote:
>>>> I ran ycsb on it for a while and it looked ok... but we really cant
>>>> ship without the fix to that bug, it has the possibility of causing
>>>> serious data loss for heavy users of ICV.
>>>>
>>>
>>> We can ship the DR though, right? 0.90.0RC1 is just around the corner!
>>> St.Ack
>>>
>>>
>>>> -ryan
>>>>
>>>> On Mon, Oct 4, 2010 at 3:41 PM, Jonathan Gray <jg...@facebook.com> wrote:
>>>>> +1
>>>>>
>>>>> I took it for a test drive today and tested all the basic stuff. No performance stuff but I think enough for my vote.
>>>>>
>>>>> JG
>>>>>
>>>>>> -----Original Message-----
>>>>>> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-
>>>>>> Daniel Cryans
>>>>>> Sent: Monday, October 04, 2010 10:56 AM
>>>>>> To: dev@hbase.apache.org
>>>>>> Subject: Re: [VOTE] Release 'development release' HBase 0.89.2010924
>>>>>> rc1?
>>>>>>
>>>>>> My vote is obviously +1, although we hit a bug this weekend regarding
>>>>>> HBASE-3008 (for which we'll post a patch soon). Over time, the
>>>>>> memstore size of regions with ICVs grows negative, which means that
>>>>>> those regions can't flush and when you close them you basically lose
>>>>>> all the data since the last flush (since on close it won't flush
>>>>>> either). We solved this by disabling ICVs to those tables (basically
>>>>>> setting the async ICV queues in the thrift servers to -1), copied the
>>>>>> data to another cluster, restarted the cluster with the fix,
>>>>>> re-imported the data, then re-enabled the ICVs.
>>>>>>
>>>>>> I don't think this is a blocker for a DR, as it only affects users
>>>>>> doing only tons of ICVs on particular tables, but it should be
>>>>>> disclosed somewhere.
>>>>>>
>>>>>> J-D
>>>>>>
>>>>>> On Fri, Sep 24, 2010 at 4:03 PM, Jean-Daniel Cryans
>>>>>> <jd...@apache.org> wrote:
>>>>>> > The 0.89.20100830 DR branch was cancelled, here's the new RC off a
>>>>>> new branch.
>>>>>> >
>>>>>> > As discussed, this release candidate contains a revert of HBASE-2694
>>>>>> > which means that it is back on the "very" old master. It is also very
>>>>>> > similar to what we run here in production.
>>>>>> >
>>>>>> > Sources and binaries can be found here:
>>>>>> >
>>>>>> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
>>>>>> >
>>>>>> > Documentation:
>>>>>> >
>>>>>> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-
>>>>>> 1/hbase-0.89.20100924/docs/
>>>>>> >
>>>>>> > Here's the list of everything I added since moving from 0830:
>>>>>> >
>>>>>> > HBASE-3008 Memstore.updateColumnValue passes wrong flag to
>>>>>> heapSizeChange
>>>>>> > HBASE-3035 Bandaid for HBASE-2990
>>>>>> > HBASE-2643 Figure how to deal with eof splitting logs
>>>>>> > HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads -
>>>>>> to HBase
>>>>>> > HBASE-3006 Reading compressed HFile blocks causes way too many DFS
>>>>>> RPC calls
>>>>>> > severly impacting performance
>>>>>> > HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
>>>>>> > HBASE-2992 [replication] MalformedObjectNameException in
>>>>>> ReplicationMetrics
>>>>>> > HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
>>>>>> > pals) for 0.89
>>>>>> > HBASE-3033 [replication] ReplicationSink.replicateEntries
>>>>>> improvements
>>>>>> > HBASE-2997 Performance fixes - profiler driven
>>>>>> > HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2
>>>>>> only)
>>>>>> >
>>>>>> > Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
>>>>>> > just figured it while reading the old voting thread).
>>>>>> >
>>>>>> > Should we release this as the next "Development Release"? Please cast
>>>>>> > your vote by Wednesday, September 29th.
>>>>>> >
>>>>>> > Thanks,
>>>>>> >
>>>>>> > The HBase Team
>>>>>> >
>>>>>
>>>>
>>>
>>
>
Re: [VOTE] Release 'development release' HBase 0.89.2010924 rc1?
Posted by Stack <st...@duboce.net>.
Sure. That caveat about no warranty, do not use in "production", is
on there already. And the bug is in ICVs only, right? We can release
w/ warning that ICVers need to apply the patch, np.
Good stuff,
St.Ack
On Mon, Oct 4, 2010 at 8:40 PM, Ryan Rawson <ry...@gmail.com> wrote:
> we could yes. with the caveat that no production use/data loss ahoy.
>
> -ryan
>
> On Mon, Oct 4, 2010 at 8:38 PM, Stack <st...@duboce.net> wrote:
>> On Mon, Oct 4, 2010 at 4:56 PM, Ryan Rawson <ry...@gmail.com> wrote:
>>> I ran ycsb on it for a while and it looked ok... but we really cant
>>> ship without the fix to that bug, it has the possibility of causing
>>> serious data loss for heavy users of ICV.
>>>
>>
>> We can ship the DR though, right? 0.90.0RC1 is just around the corner!
>> St.Ack
>>
>>
>>> -ryan
>>>
>>> On Mon, Oct 4, 2010 at 3:41 PM, Jonathan Gray <jg...@facebook.com> wrote:
>>>> +1
>>>>
>>>> I took it for a test drive today and tested all the basic stuff. No performance stuff but I think enough for my vote.
>>>>
>>>> JG
>>>>
>>>>> -----Original Message-----
>>>>> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-
>>>>> Daniel Cryans
>>>>> Sent: Monday, October 04, 2010 10:56 AM
>>>>> To: dev@hbase.apache.org
>>>>> Subject: Re: [VOTE] Release 'development release' HBase 0.89.2010924
>>>>> rc1?
>>>>>
>>>>> My vote is obviously +1, although we hit a bug this weekend regarding
>>>>> HBASE-3008 (for which we'll post a patch soon). Over time, the
>>>>> memstore size of regions with ICVs grows negative, which means that
>>>>> those regions can't flush and when you close them you basically lose
>>>>> all the data since the last flush (since on close it won't flush
>>>>> either). We solved this by disabling ICVs to those tables (basically
>>>>> setting the async ICV queues in the thrift servers to -1), copied the
>>>>> data to another cluster, restarted the cluster with the fix,
>>>>> re-imported the data, then re-enabled the ICVs.
>>>>>
>>>>> I don't think this is a blocker for a DR, as it only affects users
>>>>> doing only tons of ICVs on particular tables, but it should be
>>>>> disclosed somewhere.
>>>>>
>>>>> J-D
>>>>>
>>>>> On Fri, Sep 24, 2010 at 4:03 PM, Jean-Daniel Cryans
>>>>> <jd...@apache.org> wrote:
>>>>> > The 0.89.20100830 DR branch was cancelled, here's the new RC off a
>>>>> new branch.
>>>>> >
>>>>> > As discussed, this release candidate contains a revert of HBASE-2694
>>>>> > which means that it is back on the "very" old master. It is also very
>>>>> > similar to what we run here in production.
>>>>> >
>>>>> > Sources and binaries can be found here:
>>>>> >
>>>>> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
>>>>> >
>>>>> > Documentation:
>>>>> >
>>>>> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-
>>>>> 1/hbase-0.89.20100924/docs/
>>>>> >
>>>>> > Here's the list of everything I added since moving from 0830:
>>>>> >
>>>>> > HBASE-3008 Memstore.updateColumnValue passes wrong flag to
>>>>> heapSizeChange
>>>>> > HBASE-3035 Bandaid for HBASE-2990
>>>>> > HBASE-2643 Figure how to deal with eof splitting logs
>>>>> > HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads -
>>>>> to HBase
>>>>> > HBASE-3006 Reading compressed HFile blocks causes way too many DFS
>>>>> RPC calls
>>>>> > severly impacting performance
>>>>> > HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
>>>>> > HBASE-2992 [replication] MalformedObjectNameException in
>>>>> ReplicationMetrics
>>>>> > HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
>>>>> > pals) for 0.89
>>>>> > HBASE-3033 [replication] ReplicationSink.replicateEntries
>>>>> improvements
>>>>> > HBASE-2997 Performance fixes - profiler driven
>>>>> > HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2
>>>>> only)
>>>>> >
>>>>> > Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
>>>>> > just figured it while reading the old voting thread).
>>>>> >
>>>>> > Should we release this as the next "Development Release"? Please cast
>>>>> > your vote by Wednesday, September 29th.
>>>>> >
>>>>> > Thanks,
>>>>> >
>>>>> > The HBase Team
>>>>> >
>>>>
>>>
>>
>
Re: [VOTE] Release 'development release' HBase 0.89.2010924 rc1?
Posted by Ryan Rawson <ry...@gmail.com>.
we could yes. with the caveat that no production use/data loss ahoy.
-ryan
On Mon, Oct 4, 2010 at 8:38 PM, Stack <st...@duboce.net> wrote:
> On Mon, Oct 4, 2010 at 4:56 PM, Ryan Rawson <ry...@gmail.com> wrote:
>> I ran ycsb on it for a while and it looked ok... but we really cant
>> ship without the fix to that bug, it has the possibility of causing
>> serious data loss for heavy users of ICV.
>>
>
> We can ship the DR though, right? 0.90.0RC1 is just around the corner!
> St.Ack
>
>
>> -ryan
>>
>> On Mon, Oct 4, 2010 at 3:41 PM, Jonathan Gray <jg...@facebook.com> wrote:
>>> +1
>>>
>>> I took it for a test drive today and tested all the basic stuff. No performance stuff but I think enough for my vote.
>>>
>>> JG
>>>
>>>> -----Original Message-----
>>>> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-
>>>> Daniel Cryans
>>>> Sent: Monday, October 04, 2010 10:56 AM
>>>> To: dev@hbase.apache.org
>>>> Subject: Re: [VOTE] Release 'development release' HBase 0.89.2010924
>>>> rc1?
>>>>
>>>> My vote is obviously +1, although we hit a bug this weekend regarding
>>>> HBASE-3008 (for which we'll post a patch soon). Over time, the
>>>> memstore size of regions with ICVs grows negative, which means that
>>>> those regions can't flush and when you close them you basically lose
>>>> all the data since the last flush (since on close it won't flush
>>>> either). We solved this by disabling ICVs to those tables (basically
>>>> setting the async ICV queues in the thrift servers to -1), copied the
>>>> data to another cluster, restarted the cluster with the fix,
>>>> re-imported the data, then re-enabled the ICVs.
>>>>
>>>> I don't think this is a blocker for a DR, as it only affects users
>>>> doing only tons of ICVs on particular tables, but it should be
>>>> disclosed somewhere.
>>>>
>>>> J-D
>>>>
>>>> On Fri, Sep 24, 2010 at 4:03 PM, Jean-Daniel Cryans
>>>> <jd...@apache.org> wrote:
>>>> > The 0.89.20100830 DR branch was cancelled, here's the new RC off a
>>>> new branch.
>>>> >
>>>> > As discussed, this release candidate contains a revert of HBASE-2694
>>>> > which means that it is back on the "very" old master. It is also very
>>>> > similar to what we run here in production.
>>>> >
>>>> > Sources and binaries can be found here:
>>>> >
>>>> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
>>>> >
>>>> > Documentation:
>>>> >
>>>> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-
>>>> 1/hbase-0.89.20100924/docs/
>>>> >
>>>> > Here's the list of everything I added since moving from 0830:
>>>> >
>>>> > HBASE-3008 Memstore.updateColumnValue passes wrong flag to
>>>> heapSizeChange
>>>> > HBASE-3035 Bandaid for HBASE-2990
>>>> > HBASE-2643 Figure how to deal with eof splitting logs
>>>> > HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads -
>>>> to HBase
>>>> > HBASE-3006 Reading compressed HFile blocks causes way too many DFS
>>>> RPC calls
>>>> > severly impacting performance
>>>> > HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
>>>> > HBASE-2992 [replication] MalformedObjectNameException in
>>>> ReplicationMetrics
>>>> > HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
>>>> > pals) for 0.89
>>>> > HBASE-3033 [replication] ReplicationSink.replicateEntries
>>>> improvements
>>>> > HBASE-2997 Performance fixes - profiler driven
>>>> > HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2
>>>> only)
>>>> >
>>>> > Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
>>>> > just figured it while reading the old voting thread).
>>>> >
>>>> > Should we release this as the next "Development Release"? Please cast
>>>> > your vote by Wednesday, September 29th.
>>>> >
>>>> > Thanks,
>>>> >
>>>> > The HBase Team
>>>> >
>>>
>>
>
Re: [VOTE] Release 'development release' HBase 0.89.2010924 rc1?
Posted by Jean-Daniel Cryans <jd...@apache.org>.
We already have 3 binding +1s, so the vote has passed.
J-D
On Mon, Oct 4, 2010 at 8:38 PM, Stack <st...@duboce.net> wrote:
> On Mon, Oct 4, 2010 at 4:56 PM, Ryan Rawson <ry...@gmail.com> wrote:
>> I ran ycsb on it for a while and it looked ok... but we really cant
>> ship without the fix to that bug, it has the possibility of causing
>> serious data loss for heavy users of ICV.
>>
>
> We can ship the DR though, right? 0.90.0RC1 is just around the corner!
> St.Ack
>
>
>> -ryan
>>
>> On Mon, Oct 4, 2010 at 3:41 PM, Jonathan Gray <jg...@facebook.com> wrote:
>>> +1
>>>
>>> I took it for a test drive today and tested all the basic stuff. No performance stuff but I think enough for my vote.
>>>
>>> JG
>>>
>>>> -----Original Message-----
>>>> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-
>>>> Daniel Cryans
>>>> Sent: Monday, October 04, 2010 10:56 AM
>>>> To: dev@hbase.apache.org
>>>> Subject: Re: [VOTE] Release 'development release' HBase 0.89.2010924
>>>> rc1?
>>>>
>>>> My vote is obviously +1, although we hit a bug this weekend regarding
>>>> HBASE-3008 (for which we'll post a patch soon). Over time, the
>>>> memstore size of regions with ICVs grows negative, which means that
>>>> those regions can't flush and when you close them you basically lose
>>>> all the data since the last flush (since on close it won't flush
>>>> either). We solved this by disabling ICVs to those tables (basically
>>>> setting the async ICV queues in the thrift servers to -1), copied the
>>>> data to another cluster, restarted the cluster with the fix,
>>>> re-imported the data, then re-enabled the ICVs.
>>>>
>>>> I don't think this is a blocker for a DR, as it only affects users
>>>> doing only tons of ICVs on particular tables, but it should be
>>>> disclosed somewhere.
>>>>
>>>> J-D
>>>>
>>>> On Fri, Sep 24, 2010 at 4:03 PM, Jean-Daniel Cryans
>>>> <jd...@apache.org> wrote:
>>>> > The 0.89.20100830 DR branch was cancelled, here's the new RC off a
>>>> new branch.
>>>> >
>>>> > As discussed, this release candidate contains a revert of HBASE-2694
>>>> > which means that it is back on the "very" old master. It is also very
>>>> > similar to what we run here in production.
>>>> >
>>>> > Sources and binaries can be found here:
>>>> >
>>>> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
>>>> >
>>>> > Documentation:
>>>> >
>>>> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-
>>>> 1/hbase-0.89.20100924/docs/
>>>> >
>>>> > Here's the list of everything I added since moving from 0830:
>>>> >
>>>> > HBASE-3008 Memstore.updateColumnValue passes wrong flag to
>>>> heapSizeChange
>>>> > HBASE-3035 Bandaid for HBASE-2990
>>>> > HBASE-2643 Figure how to deal with eof splitting logs
>>>> > HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads -
>>>> to HBase
>>>> > HBASE-3006 Reading compressed HFile blocks causes way too many DFS
>>>> RPC calls
>>>> > severly impacting performance
>>>> > HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
>>>> > HBASE-2992 [replication] MalformedObjectNameException in
>>>> ReplicationMetrics
>>>> > HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
>>>> > pals) for 0.89
>>>> > HBASE-3033 [replication] ReplicationSink.replicateEntries
>>>> improvements
>>>> > HBASE-2997 Performance fixes - profiler driven
>>>> > HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2
>>>> only)
>>>> >
>>>> > Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
>>>> > just figured it while reading the old voting thread).
>>>> >
>>>> > Should we release this as the next "Development Release"? Please cast
>>>> > your vote by Wednesday, September 29th.
>>>> >
>>>> > Thanks,
>>>> >
>>>> > The HBase Team
>>>> >
>>>
>>
>
Re: [VOTE] Release 'development release' HBase 0.89.2010924 rc1?
Posted by Stack <st...@duboce.net>.
On Mon, Oct 4, 2010 at 4:56 PM, Ryan Rawson <ry...@gmail.com> wrote:
> I ran ycsb on it for a while and it looked ok... but we really cant
> ship without the fix to that bug, it has the possibility of causing
> serious data loss for heavy users of ICV.
>
We can ship the DR though, right? 0.90.0RC1 is just around the corner!
St.Ack
> -ryan
>
> On Mon, Oct 4, 2010 at 3:41 PM, Jonathan Gray <jg...@facebook.com> wrote:
>> +1
>>
>> I took it for a test drive today and tested all the basic stuff. No performance stuff but I think enough for my vote.
>>
>> JG
>>
>>> -----Original Message-----
>>> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-
>>> Daniel Cryans
>>> Sent: Monday, October 04, 2010 10:56 AM
>>> To: dev@hbase.apache.org
>>> Subject: Re: [VOTE] Release 'development release' HBase 0.89.2010924
>>> rc1?
>>>
>>> My vote is obviously +1, although we hit a bug this weekend regarding
>>> HBASE-3008 (for which we'll post a patch soon). Over time, the
>>> memstore size of regions with ICVs grows negative, which means that
>>> those regions can't flush and when you close them you basically lose
>>> all the data since the last flush (since on close it won't flush
>>> either). We solved this by disabling ICVs to those tables (basically
>>> setting the async ICV queues in the thrift servers to -1), copied the
>>> data to another cluster, restarted the cluster with the fix,
>>> re-imported the data, then re-enabled the ICVs.
>>>
>>> I don't think this is a blocker for a DR, as it only affects users
>>> doing only tons of ICVs on particular tables, but it should be
>>> disclosed somewhere.
>>>
>>> J-D
>>>
>>> On Fri, Sep 24, 2010 at 4:03 PM, Jean-Daniel Cryans
>>> <jd...@apache.org> wrote:
>>> > The 0.89.20100830 DR branch was cancelled, here's the new RC off a
>>> new branch.
>>> >
>>> > As discussed, this release candidate contains a revert of HBASE-2694
>>> > which means that it is back on the "very" old master. It is also very
>>> > similar to what we run here in production.
>>> >
>>> > Sources and binaries can be found here:
>>> >
>>> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
>>> >
>>> > Documentation:
>>> >
>>> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-
>>> 1/hbase-0.89.20100924/docs/
>>> >
>>> > Here's the list of everything I added since moving from 0830:
>>> >
>>> > HBASE-3008 Memstore.updateColumnValue passes wrong flag to
>>> heapSizeChange
>>> > HBASE-3035 Bandaid for HBASE-2990
>>> > HBASE-2643 Figure how to deal with eof splitting logs
>>> > HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads -
>>> to HBase
>>> > HBASE-3006 Reading compressed HFile blocks causes way too many DFS
>>> RPC calls
>>> > severly impacting performance
>>> > HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
>>> > HBASE-2992 [replication] MalformedObjectNameException in
>>> ReplicationMetrics
>>> > HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
>>> > pals) for 0.89
>>> > HBASE-3033 [replication] ReplicationSink.replicateEntries
>>> improvements
>>> > HBASE-2997 Performance fixes - profiler driven
>>> > HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2
>>> only)
>>> >
>>> > Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
>>> > just figured it while reading the old voting thread).
>>> >
>>> > Should we release this as the next "Development Release"? Please cast
>>> > your vote by Wednesday, September 29th.
>>> >
>>> > Thanks,
>>> >
>>> > The HBase Team
>>> >
>>
>
Re: [VOTE] Release 'development release' HBase 0.89.2010924 rc1?
Posted by Ryan Rawson <ry...@gmail.com>.
I ran ycsb on it for a while and it looked ok... but we really cant
ship without the fix to that bug, it has the possibility of causing
serious data loss for heavy users of ICV.
-ryan
On Mon, Oct 4, 2010 at 3:41 PM, Jonathan Gray <jg...@facebook.com> wrote:
> +1
>
> I took it for a test drive today and tested all the basic stuff. No performance stuff but I think enough for my vote.
>
> JG
>
>> -----Original Message-----
>> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-
>> Daniel Cryans
>> Sent: Monday, October 04, 2010 10:56 AM
>> To: dev@hbase.apache.org
>> Subject: Re: [VOTE] Release 'development release' HBase 0.89.2010924
>> rc1?
>>
>> My vote is obviously +1, although we hit a bug this weekend regarding
>> HBASE-3008 (for which we'll post a patch soon). Over time, the
>> memstore size of regions with ICVs grows negative, which means that
>> those regions can't flush and when you close them you basically lose
>> all the data since the last flush (since on close it won't flush
>> either). We solved this by disabling ICVs to those tables (basically
>> setting the async ICV queues in the thrift servers to -1), copied the
>> data to another cluster, restarted the cluster with the fix,
>> re-imported the data, then re-enabled the ICVs.
>>
>> I don't think this is a blocker for a DR, as it only affects users
>> doing only tons of ICVs on particular tables, but it should be
>> disclosed somewhere.
>>
>> J-D
>>
>> On Fri, Sep 24, 2010 at 4:03 PM, Jean-Daniel Cryans
>> <jd...@apache.org> wrote:
>> > The 0.89.20100830 DR branch was cancelled, here's the new RC off a
>> new branch.
>> >
>> > As discussed, this release candidate contains a revert of HBASE-2694
>> > which means that it is back on the "very" old master. It is also very
>> > similar to what we run here in production.
>> >
>> > Sources and binaries can be found here:
>> >
>> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
>> >
>> > Documentation:
>> >
>> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-
>> 1/hbase-0.89.20100924/docs/
>> >
>> > Here's the list of everything I added since moving from 0830:
>> >
>> > HBASE-3008 Memstore.updateColumnValue passes wrong flag to
>> heapSizeChange
>> > HBASE-3035 Bandaid for HBASE-2990
>> > HBASE-2643 Figure how to deal with eof splitting logs
>> > HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads -
>> to HBase
>> > HBASE-3006 Reading compressed HFile blocks causes way too many DFS
>> RPC calls
>> > severly impacting performance
>> > HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
>> > HBASE-2992 [replication] MalformedObjectNameException in
>> ReplicationMetrics
>> > HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
>> > pals) for 0.89
>> > HBASE-3033 [replication] ReplicationSink.replicateEntries
>> improvements
>> > HBASE-2997 Performance fixes - profiler driven
>> > HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2
>> only)
>> >
>> > Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
>> > just figured it while reading the old voting thread).
>> >
>> > Should we release this as the next "Development Release"? Please cast
>> > your vote by Wednesday, September 29th.
>> >
>> > Thanks,
>> >
>> > The HBase Team
>> >
>
RE: [VOTE] Release 'development release' HBase 0.89.2010924 rc1?
Posted by Jonathan Gray <jg...@facebook.com>.
+1
I took it for a test drive today and tested all the basic stuff. No performance stuff but I think enough for my vote.
JG
> -----Original Message-----
> From: jdcryans@gmail.com [mailto:jdcryans@gmail.com] On Behalf Of Jean-
> Daniel Cryans
> Sent: Monday, October 04, 2010 10:56 AM
> To: dev@hbase.apache.org
> Subject: Re: [VOTE] Release 'development release' HBase 0.89.2010924
> rc1?
>
> My vote is obviously +1, although we hit a bug this weekend regarding
> HBASE-3008 (for which we'll post a patch soon). Over time, the
> memstore size of regions with ICVs grows negative, which means that
> those regions can't flush and when you close them you basically lose
> all the data since the last flush (since on close it won't flush
> either). We solved this by disabling ICVs to those tables (basically
> setting the async ICV queues in the thrift servers to -1), copied the
> data to another cluster, restarted the cluster with the fix,
> re-imported the data, then re-enabled the ICVs.
>
> I don't think this is a blocker for a DR, as it only affects users
> doing only tons of ICVs on particular tables, but it should be
> disclosed somewhere.
>
> J-D
>
> On Fri, Sep 24, 2010 at 4:03 PM, Jean-Daniel Cryans
> <jd...@apache.org> wrote:
> > The 0.89.20100830 DR branch was cancelled, here's the new RC off a
> new branch.
> >
> > As discussed, this release candidate contains a revert of HBASE-2694
> > which means that it is back on the "very" old master. It is also very
> > similar to what we run here in production.
> >
> > Sources and binaries can be found here:
> >
> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
> >
> > Documentation:
> >
> > http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-
> 1/hbase-0.89.20100924/docs/
> >
> > Here's the list of everything I added since moving from 0830:
> >
> > HBASE-3008 Memstore.updateColumnValue passes wrong flag to
> heapSizeChange
> > HBASE-3035 Bandaid for HBASE-2990
> > HBASE-2643 Figure how to deal with eof splitting logs
> > HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads -
> to HBase
> > HBASE-3006 Reading compressed HFile blocks causes way too many DFS
> RPC calls
> > severly impacting performance
> > HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
> > HBASE-2992 [replication] MalformedObjectNameException in
> ReplicationMetrics
> > HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
> > pals) for 0.89
> > HBASE-3033 [replication] ReplicationSink.replicateEntries
> improvements
> > HBASE-2997 Performance fixes - profiler driven
> > HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2
> only)
> >
> > Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
> > just figured it while reading the old voting thread).
> >
> > Should we release this as the next "Development Release"? Please cast
> > your vote by Wednesday, September 29th.
> >
> > Thanks,
> >
> > The HBase Team
> >
Re: [VOTE] Release 'development release' HBase 0.89.2010924 rc1?
Posted by Jean-Daniel Cryans <jd...@apache.org>.
My vote is obviously +1, although we hit a bug this weekend regarding
HBASE-3008 (for which we'll post a patch soon). Over time, the
memstore size of regions with ICVs grows negative, which means that
those regions can't flush and when you close them you basically lose
all the data since the last flush (since on close it won't flush
either). We solved this by disabling ICVs to those tables (basically
setting the async ICV queues in the thrift servers to -1), copied the
data to another cluster, restarted the cluster with the fix,
re-imported the data, then re-enabled the ICVs.
I don't think this is a blocker for a DR, as it only affects users
doing only tons of ICVs on particular tables, but it should be
disclosed somewhere.
J-D
On Fri, Sep 24, 2010 at 4:03 PM, Jean-Daniel Cryans <jd...@apache.org> wrote:
> The 0.89.20100830 DR branch was cancelled, here's the new RC off a new branch.
>
> As discussed, this release candidate contains a revert of HBASE-2694
> which means that it is back on the "very" old master. It is also very
> similar to what we run here in production.
>
> Sources and binaries can be found here:
>
> http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
>
> Documentation:
>
> http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/hbase-0.89.20100924/docs/
>
> Here's the list of everything I added since moving from 0830:
>
> HBASE-3008 Memstore.updateColumnValue passes wrong flag to heapSizeChange
> HBASE-3035 Bandaid for HBASE-2990
> HBASE-2643 Figure how to deal with eof splitting logs
> HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads - to HBase
> HBASE-3006 Reading compressed HFile blocks causes way too many DFS RPC calls
> severly impacting performance
> HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
> HBASE-2992 [replication] MalformedObjectNameException in ReplicationMetrics
> HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
> pals) for 0.89
> HBASE-3033 [replication] ReplicationSink.replicateEntries improvements
> HBASE-2997 Performance fixes - profiler driven
> HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2 only)
>
> Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
> just figured it while reading the old voting thread).
>
> Should we release this as the next "Development Release"? Please cast
> your vote by Wednesday, September 29th.
>
> Thanks,
>
> The HBase Team
>
Re: [VOTE] Release 'development release' HBase 0.89.2010924 rc1?
Posted by Todd Lipcon <to...@cloudera.com>.
+0 from me - I did some brief testing but not quite enough for a full +1.
Was testing on a new cluster and I had forgotten to set up ulimit, so it
exploded after a few hours. Will upgrade to a +1 later this week if I have
time to run a longer test.
-Todd
On Mon, Oct 4, 2010 at 8:33 AM, Cosmin Lehene <cl...@adobe.com> wrote:
> +1
> I tested the build over the weekend running MR jobs and some write
> performance testing. It looks good.
>
> Cosmin
>
> On Sep 25, 2010, at 2:38 AM, Stack wrote:
>
> > Changing my vote after J-D and I did a bit of digging. HBASE-2986 is
> > a fix for HBASE-1845, the issue that added multi*; i.e. multiget,
> > multidelete, etc. This 0.89 was cut from the branch before hbase-1845
> > was applied.
> >
> > +1
> >
> > St.Ack
> >
> >
> > On Fri, Sep 24, 2010 at 4:21 PM, Stack <st...@duboce.net> wrote:
> >> -1 (Sorry). We need HBASE-2986. Without it client hangs if a split
> >> in the midst of a put.
> >>
> >> Otherwise, I'd already checked it out -- doc looks good, ran under
> >> load on a cluster -- and would have +1'd it only for your calling out
> >> the absence of HBASE-2986 J-D.
> >>
> >> St.Ack
> >>
> >>
> >> On Fri, Sep 24, 2010 at 4:03 PM, Jean-Daniel Cryans <
> jdcryans@apache.org> wrote:
> >>> The 0.89.20100830 DR branch was cancelled, here's the new RC off a new
> branch.
> >>>
> >>> As discussed, this release candidate contains a revert of HBASE-2694
> >>> which means that it is back on the "very" old master. It is also very
> >>> similar to what we run here in production.
> >>>
> >>> Sources and binaries can be found here:
> >>>
> >>> http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
> >>>
> >>> Documentation:
> >>>
> >>>
> http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/hbase-0.89.20100924/docs/
> >>>
> >>> Here's the list of everything I added since moving from 0830:
> >>>
> >>> HBASE-3008 Memstore.updateColumnValue passes wrong flag to
> heapSizeChange
> >>> HBASE-3035 Bandaid for HBASE-2990
> >>> HBASE-2643 Figure how to deal with eof splitting logs
> >>> HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads -
> to HBase
> >>> HBASE-3006 Reading compressed HFile blocks causes way too many DFS
> RPC calls
> >>> severly impacting performance
> >>> HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
> >>> HBASE-2992 [replication] MalformedObjectNameException in
> ReplicationMetrics
> >>> HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
> >>> pals) for 0.89
> >>> HBASE-3033 [replication] ReplicationSink.replicateEntries
> improvements
> >>> HBASE-2997 Performance fixes - profiler driven
> >>> HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2 only)
> >>>
> >>> Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
> >>> just figured it while reading the old voting thread).
> >>>
> >>> Should we release this as the next "Development Release"? Please cast
> >>> your vote by Wednesday, September 29th.
> >>>
> >>> Thanks,
> >>>
> >>> The HBase Team
> >>>
> >>
>
>
--
Todd Lipcon
Software Engineer, Cloudera
Re: [VOTE] Release 'development release' HBase 0.89.2010924 rc1?
Posted by Cosmin Lehene <cl...@adobe.com>.
+1
I tested the build over the weekend running MR jobs and some write performance testing. It looks good.
Cosmin
On Sep 25, 2010, at 2:38 AM, Stack wrote:
> Changing my vote after J-D and I did a bit of digging. HBASE-2986 is
> a fix for HBASE-1845, the issue that added multi*; i.e. multiget,
> multidelete, etc. This 0.89 was cut from the branch before hbase-1845
> was applied.
>
> +1
>
> St.Ack
>
>
> On Fri, Sep 24, 2010 at 4:21 PM, Stack <st...@duboce.net> wrote:
>> -1 (Sorry). We need HBASE-2986. Without it client hangs if a split
>> in the midst of a put.
>>
>> Otherwise, I'd already checked it out -- doc looks good, ran under
>> load on a cluster -- and would have +1'd it only for your calling out
>> the absence of HBASE-2986 J-D.
>>
>> St.Ack
>>
>>
>> On Fri, Sep 24, 2010 at 4:03 PM, Jean-Daniel Cryans <jd...@apache.org> wrote:
>>> The 0.89.20100830 DR branch was cancelled, here's the new RC off a new branch.
>>>
>>> As discussed, this release candidate contains a revert of HBASE-2694
>>> which means that it is back on the "very" old master. It is also very
>>> similar to what we run here in production.
>>>
>>> Sources and binaries can be found here:
>>>
>>> http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
>>>
>>> Documentation:
>>>
>>> http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/hbase-0.89.20100924/docs/
>>>
>>> Here's the list of everything I added since moving from 0830:
>>>
>>> HBASE-3008 Memstore.updateColumnValue passes wrong flag to heapSizeChange
>>> HBASE-3035 Bandaid for HBASE-2990
>>> HBASE-2643 Figure how to deal with eof splitting logs
>>> HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads - to HBase
>>> HBASE-3006 Reading compressed HFile blocks causes way too many DFS RPC calls
>>> severly impacting performance
>>> HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
>>> HBASE-2992 [replication] MalformedObjectNameException in ReplicationMetrics
>>> HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
>>> pals) for 0.89
>>> HBASE-3033 [replication] ReplicationSink.replicateEntries improvements
>>> HBASE-2997 Performance fixes - profiler driven
>>> HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2 only)
>>>
>>> Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
>>> just figured it while reading the old voting thread).
>>>
>>> Should we release this as the next "Development Release"? Please cast
>>> your vote by Wednesday, September 29th.
>>>
>>> Thanks,
>>>
>>> The HBase Team
>>>
>>
Re: [VOTE] Release 'development release' HBase 0.89.2010924 rc1?
Posted by Stack <st...@duboce.net>.
Changing my vote after J-D and I did a bit of digging. HBASE-2986 is
a fix for HBASE-1845, the issue that added multi*; i.e. multiget,
multidelete, etc. This 0.89 was cut from the branch before hbase-1845
was applied.
+1
St.Ack
On Fri, Sep 24, 2010 at 4:21 PM, Stack <st...@duboce.net> wrote:
> -1 (Sorry). We need HBASE-2986. Without it client hangs if a split
> in the midst of a put.
>
> Otherwise, I'd already checked it out -- doc looks good, ran under
> load on a cluster -- and would have +1'd it only for your calling out
> the absence of HBASE-2986 J-D.
>
> St.Ack
>
>
> On Fri, Sep 24, 2010 at 4:03 PM, Jean-Daniel Cryans <jd...@apache.org> wrote:
>> The 0.89.20100830 DR branch was cancelled, here's the new RC off a new branch.
>>
>> As discussed, this release candidate contains a revert of HBASE-2694
>> which means that it is back on the "very" old master. It is also very
>> similar to what we run here in production.
>>
>> Sources and binaries can be found here:
>>
>> http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
>>
>> Documentation:
>>
>> http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/hbase-0.89.20100924/docs/
>>
>> Here's the list of everything I added since moving from 0830:
>>
>> HBASE-3008 Memstore.updateColumnValue passes wrong flag to heapSizeChange
>> HBASE-3035 Bandaid for HBASE-2990
>> HBASE-2643 Figure how to deal with eof splitting logs
>> HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads - to HBase
>> HBASE-3006 Reading compressed HFile blocks causes way too many DFS RPC calls
>> severly impacting performance
>> HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
>> HBASE-2992 [replication] MalformedObjectNameException in ReplicationMetrics
>> HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
>> pals) for 0.89
>> HBASE-3033 [replication] ReplicationSink.replicateEntries improvements
>> HBASE-2997 Performance fixes - profiler driven
>> HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2 only)
>>
>> Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
>> just figured it while reading the old voting thread).
>>
>> Should we release this as the next "Development Release"? Please cast
>> your vote by Wednesday, September 29th.
>>
>> Thanks,
>>
>> The HBase Team
>>
>
Re: [VOTE] Release 'development release' HBase 0.89.2010924 rc1?
Posted by Stack <st...@duboce.net>.
-1 (Sorry). We need HBASE-2986. Without it client hangs if a split
in the midst of a put.
Otherwise, I'd already checked it out -- doc looks good, ran under
load on a cluster -- and would have +1'd it only for your calling out
the absence of HBASE-2986 J-D.
St.Ack
On Fri, Sep 24, 2010 at 4:03 PM, Jean-Daniel Cryans <jd...@apache.org> wrote:
> The 0.89.20100830 DR branch was cancelled, here's the new RC off a new branch.
>
> As discussed, this release candidate contains a revert of HBASE-2694
> which means that it is back on the "very" old master. It is also very
> similar to what we run here in production.
>
> Sources and binaries can be found here:
>
> http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/
>
> Documentation:
>
> http://people.apache.org/~jdcryans/hbase-0.89.20100924-candidate-1/hbase-0.89.20100924/docs/
>
> Here's the list of everything I added since moving from 0830:
>
> HBASE-3008 Memstore.updateColumnValue passes wrong flag to heapSizeChange
> HBASE-3035 Bandaid for HBASE-2990
> HBASE-2643 Figure how to deal with eof splitting logs
> HBASE-2941 port HADOOP-6713 - threading scalability for RPC reads - to HBase
> HBASE-3006 Reading compressed HFile blocks causes way too many DFS RPC calls
> severly impacting performance
> HBASE-2989 [replication] RSM won't cleanup after locking if 0 peers
> HBASE-2992 [replication] MalformedObjectNameException in ReplicationMetrics
> HBASE-3034 Revert the regions assignment part of HBASE-2694 (and
> pals) for 0.89
> HBASE-3033 [replication] ReplicationSink.replicateEntries improvements
> HBASE-2997 Performance fixes - profiler driven
> HBASE-2889 Tool to look at HLogs -- parse and tail -f (patch #2 only)
>
> Unfortunately I forgot to add HBASE-2986 like Stack asked (sorry, I
> just figured it while reading the old voting thread).
>
> Should we release this as the next "Development Release"? Please cast
> your vote by Wednesday, September 29th.
>
> Thanks,
>
> The HBase Team
>