You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Zhoushuaifeng <zh...@huawei.com> on 2011/04/22 04:00:04 UTC
Is there any influence to put performance when
"Block cache LRU eviction" occur?
I'm running random scan and put performance test at the same time, found that put performance descend a lot. Don't know why.
Looking into the logs, found that "Block cache LRU eviction" occured almost once per second. But it only last about 10 ms, so, will it bring so much influence to put performance? And how to improve it?
hbase-root-regionserver-c3s5.log.2011-04-19:2011-04-19 21:00:00,067 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 163.35 MB of total=1.36 GB
hbase-root-regionserver-c3s5.log.2011-04-19:2011-04-19 21:00:00,081 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=163.39 MB, total=1.2 GB, single=973.98 MB, multi=400.97 MB, memory=0 KB
Zhou Shuaifeng(Frank)
Re: Is there any influence to put performance when
"Block cache LRU eviction" occur?
Posted by Zhoushuaifeng <zh...@huawei.com>.
Hbase 0.90.1, hadoop0.20.2+append
Put op use putlist(2000). About 5000 regions, 512MB max size, 50 ipc handler each RS. Totally 8 RS.
Both put and scan have 500 threads.
Because we use random start key to scan, cache hitratio is very low.
2011-04-19 20:16:52,038 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=1.23 GB, free=378.14 MB, max=1.59 GB, blocks=19741, accesses=10958888, hits=2870, hitRatio=0.02%%, cachingAccesses=227733, cachingHits=2870, cachingHitsRatio=1.26%%, evictions=79, evicted=205121, evictedPerRun=2596.46826171875
2011-04-19 20:21:52,038 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: LRU Stats: total=1.27 GB, free=329.64 MB, max=1.59 GB, blocks=20511, accesses=11097514, hits=4399, hitRatio=0.03%%, cachingAccesses=326115, cachingHits=4399, cachingHitsRatio=1.34%%, evictions=116, evicted=301193, evictedPerRun=2596.491455078125
Zhou Shuaifeng(Frank)
-----邮件原件-----
发件人: saint.ack@gmail.com [mailto:saint.ack@gmail.com] 代表 Stack
发送时间: 2011年4月22日 11:11
收件人: user@hbase.apache.org
抄送: Yanlijun
主题: Re: Is there any influence to put performance when "Block cache LRU eviction" occur?
On Thu, Apr 21, 2011 at 7:00 PM, Zhoushuaifeng <zh...@huawei.com> wrote:
> I'm running random scan and put performance test at the same time, found that put performance descend a lot. Don't know why.
> Looking into the logs, found that "Block cache LRU eviction" occured almost once per second. But it only last about 10 ms, so, will it bring so much influence to put performance? And how to improve it?
>
I would think that there would be little to no relation between put
and block cache evictions.
> hbase-root-regionserver-c3s5.log.2011-04-19:2011-04-19 21:00:00,067 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 163.35 MB of total=1.36 GB
> hbase-root-regionserver-c3s5.log.2011-04-19:2011-04-19 21:00:00,081 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=163.39 MB, total=1.2 GB, single=973.98 MB, multi=400.97 MB, memory=0 KB
>
It looks like each time it runs its evictly ~10 percent. You are
doing lots of scans that go via the block cache?
Regards the puts, tell us more. How many clients, what type of
clients, regions, regionservers, versions of hbase and hadoop, etc.
You've seen this page? http://hbase.apache.org/book.html#performance
St.Ack
Re: Is there any influence to put performance when "Block cache LRU
eviction" occur?
Posted by Stack <st...@duboce.net>.
On Thu, Apr 21, 2011 at 7:00 PM, Zhoushuaifeng <zh...@huawei.com> wrote:
> I'm running random scan and put performance test at the same time, found that put performance descend a lot. Don't know why.
> Looking into the logs, found that "Block cache LRU eviction" occured almost once per second. But it only last about 10 ms, so, will it bring so much influence to put performance? And how to improve it?
>
I would think that there would be little to no relation between put
and block cache evictions.
> hbase-root-regionserver-c3s5.log.2011-04-19:2011-04-19 21:00:00,067 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction started; Attempting to free 163.35 MB of total=1.36 GB
> hbase-root-regionserver-c3s5.log.2011-04-19:2011-04-19 21:00:00,081 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Block cache LRU eviction completed; freed=163.39 MB, total=1.2 GB, single=973.98 MB, multi=400.97 MB, memory=0 KB
>
It looks like each time it runs its evictly ~10 percent. You are
doing lots of scans that go via the block cache?
Regards the puts, tell us more. How many clients, what type of
clients, regions, regionservers, versions of hbase and hadoop, etc.
You've seen this page? http://hbase.apache.org/book.html#performance
St.Ack