You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2011/02/20 07:58:38 UTC
[jira] Commented: (HBASE-3551) Loaded hfile indexes occupy a good
chunk of heap; look into shrinking the amount used and/or evicting unused
indices
[ https://issues.apache.org/jira/browse/HBASE-3551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997097#comment-12997097 ]
Andrew Purtell commented on HBASE-3551:
---------------------------------------
We talked before about using a fixed resource pool for file handles / storefile indices and loading or unloading on a LRU basis (HBASE-24, HBASE-2751, etc.)
> Loaded hfile indexes occupy a good chunk of heap; look into shrinking the amount used and/or evicting unused indices
> --------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-3551
> URL: https://issues.apache.org/jira/browse/HBASE-3551
> Project: HBase
> Issue Type: Improvement
> Reporter: stack
>
> I hung with a user Marc and we were looking over configs and his cluster profile up on ec2. One thing we noticed was that his 100+ 1G regions of two families had ~2.5G of heap resident. We did a bit of math and couldn't get to 2.5G so that needs looking into. Even still, 2.5G is a bunch of heap to give over to indices (He actually OOME'd when he had his RS heap set to just 3G; we shouldn't OOME, we should just run slower). It sounds like he needs the indices loaded but still, for some cases we should drop indices for unaccessed files.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira