You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "mingjian (JIRA)" <ji...@apache.org> on 2011/07/18 03:48:00 UTC
[jira] [Updated] (HBASE-4111) readBlock in HFile.Reader not 64 KB,
but 5~6MB
[ https://issues.apache.org/jira/browse/HBASE-4111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
mingjian updated HBASE-4111:
----------------------------
Description:
We found a strange problem in our read test.
It is a 5 nodes cluster.Four of our 5 regionservers set "hfile.block.cache.size"=0.4, one of them is 0.1(we call it node A). When we random read from a 2TB data table we found node A's network reached 100MB, and others are less than 10MB. So the read speed is low.
We set node A's "hfile.block.cache.size"=0.2, then all the nodes's network are 10MB, that's right. To find why is this we debug with btrace and find "readBlock" in HFile.Reader become abnormal.We know hbase read a block which is 64 KB from disks and put it into blockcache. But when we set "hfile.block.cache.size"=0.1, it is not 64KB, it is 5~6MB one time after about 1 minute we restart hbase.
Why not 64 KB? The btrace code and results are in the attachments.
was:
We found a strange problem in our read test.
It is a 5 nodes cluster.Four of our 5 regionservers set "hfile.block.cache.size"=0.4, one of them is 0.1(we call it node A). When we random read from a 2TB data table we found node A's network reached 100MB, and others are less than 10MB. So the read speed is low.
We set node A's "hfile.block.cache.size"=0.2, then all the nodes's network are 10MB, that's right. To find why is this we debug with btrace and find "readBlock" in HFile.Reader become abnormal.We know hbase read a block which is 64 KB from disks and put it into blockcache. But when we set "hfile.block.cache.size"=0.1, it is not 64KB, it is 5~6MB one time after about 1 minute we restart hbase.
Why not 64 KB? The btrace code and results are in the attachments.
> readBlock in HFile.Reader not 64 KB, but 5~6MB
> ----------------------------------------------
>
> Key: HBASE-4111
> URL: https://issues.apache.org/jira/browse/HBASE-4111
> Project: HBase
> Issue Type: Bug
> Components: io, regionserver
> Reporter: mingjian
> Assignee: mingjian
> Attachments: code and result.txt
>
>
> We found a strange problem in our read test.
> It is a 5 nodes cluster.Four of our 5 regionservers set "hfile.block.cache.size"=0.4, one of them is 0.1(we call it node A). When we random read from a 2TB data table we found node A's network reached 100MB, and others are less than 10MB. So the read speed is low.
> We set node A's "hfile.block.cache.size"=0.2, then all the nodes's network are 10MB, that's right. To find why is this we debug with btrace and find "readBlock" in HFile.Reader become abnormal.We know hbase read a block which is 64 KB from disks and put it into blockcache. But when we set "hfile.block.cache.size"=0.1, it is not 64KB, it is 5~6MB one time after about 1 minute we restart hbase.
> Why not 64 KB? The btrace code and results are in the attachments.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira