You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "Lars George (JIRA)" <ji...@apache.org> on 2012/08/02 08:09:02 UTC
[jira] [Commented] (HBASE-6497) Revisit HLog sizing and roll
parameters
[ https://issues.apache.org/jira/browse/HBASE-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427130#comment-13427130 ]
Lars George commented on HBASE-6497:
------------------------------------
The goal in designing a proper HBase schema is to maximize heap usage across all regions, which can lead to the situation where the WALs (aka HLog's) are required to be kept for a considerable amount of time.
The last iteration on WAL properties added a configurable block size, as well as threshold percentage to roll the log before it completely fills the single HDFS block (see HBASE-1394).
I am questioning if this is still in issue, maybe even in the light of recent improvements on log performance, for example HBASE-5699 and HBASE-4608.
At the least, I would like to figure out, if we should increase the WAL size to 512MB, to avoid getting into early flushing situations, impacting the overall I/O. Isn't HBASE-1364 helping to split larger logs (though not the logs themselves but distributed across the region servers obviously). I am not sure if the log splitting prefers block local nodes first, so that there is no remote reading though.
Questions:
# Is there a need to keep the logs small (typically 64-128 depending on the HDFS config)?
# Should we go multiple blocks?
# Do we still need the logroll multiplier?
# Should we increase the maxlogs number (default is 32)?
> Revisit HLog sizing and roll parameters
> ---------------------------------------
>
> Key: HBASE-6497
> URL: https://issues.apache.org/jira/browse/HBASE-6497
> Project: HBase
> Issue Type: Improvement
> Components: regionserver
> Reporter: Lars George
>
> The last major update to the HLog sizing and roll features were done in HBASE-1394. I am proposing to revisit these settings to overcome recent issues where the HLog becomes a major bottleneck.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira