You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Rod MacDonald <r....@gmail.com> on 2010/11/25 19:06:59 UTC
Region server memory configuration question
What is the relationship between the following configuration parameters?
hfile.block.cache.size
hbase.regionserver.global.memstore.upperLimit
Is having the sum of the two values >= 1 a problem?
That is, will the region server have problems with OutOfMemory if
hfile.block.cache.size=.6 and
hbase.regionserver.global.memstore.upperLimit=.4?
Thanks,
Rod
Re: Region server memory configuration question
Posted by Lars George <la...@gmail.com>.
Hi Rod,
That setting controls the memstores, not the block cache. Could rephrase your question because I am not sure what you are asking for.
Lars
On Nov 25, 2010, at 21:41, Rod MacDonald <r....@gmail.com> wrote:
> Thanks Lars.
> Is there any caches controlled by the
> hbase.regionserver.global.memstore.upperLimit
> other than the block cache?
> Rod
>
> On Thu, Nov 25, 2010 at 2:00 PM, Lars George <la...@gmail.com> wrote:
>
>> Hi Rod,
>>
>> I guess you will have to be careful setting them right. The
>> hbase.regionserver.global.memstore.upperLimit does not allocate
>> anything but acts as a barrier or threshold to flush data out. If you
>> were to set this to 1.0 then it would never trigger, while only
>> relying on the other triggers to flush data. And with
>> hfile.block.cache.size you could overdo it and have the server trying
>> to store to much data in memory triggering mayhem along the way.
>>
>> No hard and fast rule but keep them close to what they are set to and
>> test any change in either in any direction very thoroughly.
>>
>> Lars
>>
>> On Thu, Nov 25, 2010 at 7:06 PM, Rod MacDonald <r....@gmail.com>
>> wrote:
>>> What is the relationship between the following configuration parameters?
>>> hfile.block.cache.size
>>> hbase.regionserver.global.memstore.upperLimit
>>>
>>> Is having the sum of the two values >= 1 a problem?
>>> That is, will the region server have problems with OutOfMemory if
>>> hfile.block.cache.size=.6 and
>>> hbase.regionserver.global.memstore.upperLimit=.4?
>>>
>>> Thanks,
>>> Rod
>>>
>>
Re: Region server memory configuration question
Posted by Rod MacDonald <r....@gmail.com>.
Thanks Lars.
Is there any caches controlled by the
hbase.regionserver.global.memstore.upperLimit
other than the block cache?
Rod
On Thu, Nov 25, 2010 at 2:00 PM, Lars George <la...@gmail.com> wrote:
> Hi Rod,
>
> I guess you will have to be careful setting them right. The
> hbase.regionserver.global.memstore.upperLimit does not allocate
> anything but acts as a barrier or threshold to flush data out. If you
> were to set this to 1.0 then it would never trigger, while only
> relying on the other triggers to flush data. And with
> hfile.block.cache.size you could overdo it and have the server trying
> to store to much data in memory triggering mayhem along the way.
>
> No hard and fast rule but keep them close to what they are set to and
> test any change in either in any direction very thoroughly.
>
> Lars
>
> On Thu, Nov 25, 2010 at 7:06 PM, Rod MacDonald <r....@gmail.com>
> wrote:
> > What is the relationship between the following configuration parameters?
> > hfile.block.cache.size
> > hbase.regionserver.global.memstore.upperLimit
> >
> > Is having the sum of the two values >= 1 a problem?
> > That is, will the region server have problems with OutOfMemory if
> > hfile.block.cache.size=.6 and
> > hbase.regionserver.global.memstore.upperLimit=.4?
> >
> > Thanks,
> > Rod
> >
>
Re: Region server memory configuration question
Posted by Lars George <la...@gmail.com>.
Hi Rod,
I guess you will have to be careful setting them right. The
hbase.regionserver.global.memstore.upperLimit does not allocate
anything but acts as a barrier or threshold to flush data out. If you
were to set this to 1.0 then it would never trigger, while only
relying on the other triggers to flush data. And with
hfile.block.cache.size you could overdo it and have the server trying
to store to much data in memory triggering mayhem along the way.
No hard and fast rule but keep them close to what they are set to and
test any change in either in any direction very thoroughly.
Lars
On Thu, Nov 25, 2010 at 7:06 PM, Rod MacDonald <r....@gmail.com> wrote:
> What is the relationship between the following configuration parameters?
> hfile.block.cache.size
> hbase.regionserver.global.memstore.upperLimit
>
> Is having the sum of the two values >= 1 a problem?
> That is, will the region server have problems with OutOfMemory if
> hfile.block.cache.size=.6 and
> hbase.regionserver.global.memstore.upperLimit=.4?
>
> Thanks,
> Rod
>