Question about memory usage

[Giorgos Margaritis <gm...@gmail.com>]

Hello,
I'm studying HBase code, and especially memstore flushing code. From what I
understand, 40% of memory is used for buffering writes
(hbase.regionserver.global.memstore.upperLimit) and 20% for block cache
(hfile.block.cache.size). Where is the rest of memory used?

Re: Question about memory usage

[Stack <st...@duboce.net>]

General operations.  Edits have to get from rpc to the memstore and
then from memstore out to the filesystem.  They also have to come back
off the filesystem and up into block cache -- in most cases but not
all -- then get put out on the outgoing rpc.  Then there are the
background maintenance tasks.

If you'd like to learn more, dump the heap and bring it up in a
profiler.  Navigation is a little complicated by the fact that HBase
is all byte arrays for the most part, but if you trace a few, you'll
see roots other than memstore and cache.

Hope this helps,
St.Ack

On Sun, Apr 10, 2011 at 12:19 PM, Giorgos Margaritis <gm...@gmail.com> wrote:
> Hello,
> I'm studying HBase code, and especially memstore flushing code. From what I
> understand, 40% of memory is used for buffering writes
> (hbase.regionserver.global.memstore.upperLimit) and 20% for block cache
> (hfile.block.cache.size). Where is the rest of memory used?
>