You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Danil Lipovoy (Jira)" <ji...@apache.org> on 2020/02/24 08:18:00 UTC
[jira] [Created] (HBASE-23887) BlockCache performance improve
Danil Lipovoy created HBASE-23887:
-------------------------------------
Summary: BlockCache performance improve
Key: HBASE-23887
URL: https://issues.apache.org/jira/browse/HBASE-23887
Project: HBase
Issue Type: New Feature
Reporter: Danil Lipovoy
Attachments: 2jira.png
Hi!
I first time here, sorry if something wrong.
I want propose how to improve performance when data in HFiles much more than BlockChache (usual story in BigData). The idea - caching only part of DATA blocks. It is good becouse LruBlockCache starts to work and save huge amount of GC. See the picture in attachment with test below. Requests per second is higher, GC is lower.
The key point of the code:
Added the parameter: *hbase.lru.cache.data.block.percent* which by default = 100
But if we set it 0-99, then will work the next logic:
*public void* cacheBlock(BlockCacheKey cacheKey, Cacheable buf, *boolean* inMemory) {
*if* (_cacheDataBlockPercent_ != 100 && buf.getBlockType().isData()) {
*if* (cacheKey.getOffset() % 100 >= _cacheDataBlockPercent_)
*return*;
}
...
}
Descriptions of the test:
3 nodes E5-2698 v4 @ 2.20GHz, 700 Gb Mem.
3 RegionServers
6 tables by 64 regions by 1.88 Gb data in each = 720 Gb total (only FAST_DIFF)
Total BlockCache Size = 38 Gb
Random read in 24 threads
I am going to make Pull Request, hope it is right way to make some contribution in this cool product. Correct me please if something wrong.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [jira] [Created] (HBASE-23887) BlockCache performance improve
Posted by pustota <bo...@gmail.com>.
Hi *! I've tested Cassandra and HBase in the same conditions.
HBase loosing without the improvement and shows the same results when it
enabled. By the way with another improvement
(https://issues.apache.org/jira/browse/HDFS-15202) HB beat CS.
Someone interested to make our product faster? When we operate with really
BigData time is really important, isn't?
Test results (4 nodes):
CS -> 250 000 ops
HD vanila -> 100 000 ops
HB improvement 23887 -> 250 000
HB 23887 + HDFS 15202 -> 300 000
--
Sent from: http://apache-hbase.679495.n3.nabble.com/HBase-Developer-f679493.html
Re: [jira] [Created] (HBASE-23887) BlockCache performance improve
Posted by pustota <bo...@gmail.com>.
Hi, all! Could you take a look at the feature?
BlockCache performance improvement
<https://issues.apache.org/jira/browse/HBASE-23887>
--
Sent from: http://apache-hbase.679495.n3.nabble.com/HBase-Developer-f679493.html