You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by lars hofhansl <lh...@yahoo.com> on 2012/10/17 05:25:45 UTC
Set scanner caching to a better default?
We just ran into this again today, where we forgot to set scanner caching and observed bad performance.
The default of 1 does not seem to make any sense (except for very specific case of large/wide rows).
Any value between 10 and 1000 should be OK, really. Maybe the default should be 100.
This would also go some way to avoid the perception that HBase is slow for folks who are just playing around with it.
Thoughts?
-- Lars
Re: [DISCUSS] Set scanner caching to a better default? HBASE-7008
Posted by lars hofhansl <lh...@yahoo.com>.
I agree. Let me make it so.
________________________________
From: Stack <st...@duboce.net>
To: dev@hbase.apache.org; lars hofhansl <lh...@yahoo.com>
Sent: Saturday, October 27, 2012 6:56 PM
Subject: Re: [DISCUSS] Set scanner caching to a better default? HBASE-7008
On Sat, Oct 27, 2012 at 5:46 PM, lars hofhansl <lh...@yahoo.com> wrote:
> We have patch now (which also disables Nagle's algorithm): HBASE-7008
> The question is now: Should we only do this for 0.96 or also for the upcoming 0.94 point release (0.94.3).
> (See also discussion on the issue)
>
Only for 0.96 IMO. Changing defaults under folks in low-level ways
could surprise; a point release shouldn't.
St.Ack
Re: [DISCUSS] Set scanner caching to a better default? HBASE-7008
Posted by Stack <st...@duboce.net>.
On Sat, Oct 27, 2012 at 5:46 PM, lars hofhansl <lh...@yahoo.com> wrote:
> We have patch now (which also disables Nagle's algorithm): HBASE-7008
> The question is now: Should we only do this for 0.96 or also for the upcoming 0.94 point release (0.94.3).
> (See also discussion on the issue)
>
Only for 0.96 IMO. Changing defaults under folks in low-level ways
could surprise; a point release shouldn't.
St.Ack
Re: [DISCUSS] Set scanner caching to a better default? HBASE-7008
Posted by tsuna <ts...@gmail.com>.
On Sat, Oct 27, 2012 at 5:46 PM, lars hofhansl <lh...@yahoo.com> wrote:
> Any value between 10 and 1000 should be OK, really. Maybe the default should be 100.
FWIW, asynchbase uses 128 rows as the default for its scanners, and
4096 maximum KVs by default too.
--
Benoit "tsuna" Sigoure
[DISCUSS] Set scanner caching to a better default? HBASE-7008
Posted by lars hofhansl <lh...@yahoo.com>.
We have patch now (which also disables Nagle's algorithm): HBASE-7008
The question is now: Should we only do this for 0.96 or also for the upcoming 0.94 point release (0.94.3).
(See also discussion on the issue)
-- Lars
________________________________
From: lars hofhansl <lh...@yahoo.com>
To: hbase-dev <de...@hbase.apache.org>
Sent: Tuesday, October 16, 2012 8:25 PM
Subject: Set scanner caching to a better default?
We just ran into this again today, where we forgot to set scanner caching and observed bad performance.
The default of 1 does not seem to make any sense (except for very specific case of large/wide rows).
Any value between 10 and 1000 should be OK, really. Maybe the default should be 100.
This would also go some way to avoid the perception that HBase is slow for folks who are just playing around with it.
Thoughts?
-- Lars
Re: Set scanner caching to a better default?
Posted by Andrew Purtell <ap...@apache.org>.
I set that to 100 typically.
On Tuesday, October 16, 2012, lars hofhansl wrote:
> We just ran into this again today, where we forgot to set scanner caching
> and observed bad performance.
> The default of 1 does not seem to make any sense (except for very specific
> case of large/wide rows).
>
> Any value between 10 and 1000 should be OK, really. Maybe the default
> should be 100.
>
> This would also go some way to avoid the perception that HBase is slow for
> folks who are just playing around with it.
>
>
> Thoughts?
>
>
> -- Lars
>
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: Set scanner caching to a better default?
Posted by Doug Meil <do...@explorysmedical.com>.
100 is a good idea. It's one of the most common questions on the
dist-list (e.g., hey my MR job is slow? answer: set caching to something
more than 1).
On 10/17/12 1:12 AM, "Stack" <st...@duboce.net> wrote:
>On Tue, Oct 16, 2012 at 8:25 PM, lars hofhansl <lh...@yahoo.com>
>wrote:
>> We just ran into this again today, where we forgot to set scanner
>>caching and observed bad performance.
>> The default of 1 does not seem to make any sense (except for very
>>specific case of large/wide rows).
>>
>> Any value between 10 and 1000 should be OK, really. Maybe the default
>>should be 100.
>>
>> This would also go some way to avoid the perception that HBase is slow
>>for folks who are just playing around with it.
>>
>
>I'd say all our defaults could do w/ an edit but am fine starting w/
>this one alone (Or we have the UI come w/ flashing neon saying the
>configs are super conservative and must be tuned).
>
>St.Ack
>
Re: Set scanner caching to a better default?
Posted by Stack <st...@duboce.net>.
On Tue, Oct 16, 2012 at 8:25 PM, lars hofhansl <lh...@yahoo.com> wrote:
> We just ran into this again today, where we forgot to set scanner caching and observed bad performance.
> The default of 1 does not seem to make any sense (except for very specific case of large/wide rows).
>
> Any value between 10 and 1000 should be OK, really. Maybe the default should be 100.
>
> This would also go some way to avoid the perception that HBase is slow for folks who are just playing around with it.
>
I'd say all our defaults could do w/ an edit but am fine starting w/
this one alone (Or we have the UI come w/ flashing neon saying the
configs are super conservative and must be tuned).
St.Ack