You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by "Andrew Purtell (JIRA)" <ji...@apache.org> on 2009/12/19 22:47:18 UTC
[jira] Commented: (HBASE-1416) Pool of commit loggers in each
HRegionServer
[ https://issues.apache.org/jira/browse/HBASE-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792910#action_12792910 ]
Andrew Purtell commented on HBASE-1416:
---------------------------------------
Do we still want to do this? Seems some of Ryan's ideas have been implemented as HBASE-1939 already.
I was playing around with this the other day after committing HBASE-2059. On that note, implementing an HLog.Writer that stripes edits over a pool of logs under a directory instead of a single file works ok on read and write paths but split and roll code assumes the log is a single file so barf when they access a dir instead; the HLog.Writer interface would need to be widened with split and roll methods to make that work cleanly. Anyway, sync is still a log pole and with striped writes over a pool of HLogs there are more dirty blocks to sync. Unless the current edit load is high we end up dirtying small bits of a bunch of blocks where instead we should just write them all into one as they'll all fit in there.
> Pool of commit loggers in each HRegionServer
> --------------------------------------------
>
> Key: HBASE-1416
> URL: https://issues.apache.org/jira/browse/HBASE-1416
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Fix For: 0.21.0
>
>
> HBASE-1394 discusses pools of loggers as means of our being able to dump out the logs faster; commit log is log pole in a write transaction. This issue is about implementing the pool of writers.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Commented: (HBASE-1416) Pool of commit loggers in each
HRegionServer
Posted by Ryan Rawson <ry...@gmail.com>.
With deferred log flush we can choose between high durability and speed on a
per table basis. I think we should shelf this for now, since with write
buffering in the client and the adjusted sync calls, we are very fast at
bulk data import.
On Dec 19, 2009 1:47 PM, "Andrew Purtell (JIRA)" <ji...@apache.org> wrote:
[
https://issues.apache.org/jira/browse/HBASE-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792910#action_12792910]
Andrew Purtell commented on HBASE-1416:
---------------------------------------
Do we still want to do this? Seems some of Ryan's ideas have been
implemented as HBASE-1939 already.
I was playing around with this the other day after committing HBASE-2059. On
that note, implementing an HLog.Writer that stripes edits over a pool of
logs under a directory instead of a single file works ok on read and write
paths but split and roll code assumes the log is a single file so barf when
they access a dir instead; the HLog.Writer interface would need to be
widened with split and roll methods to make that work cleanly. Anyway, sync
is still a log pole and with striped writes over a pool of HLogs there are
more dirty blocks to sync. Unless the current edit load is high we end up
dirtying small bits of a bunch of blocks where instead we should just write
them all into one as they'll all fit in there.
> Pool of commit loggers in each HRegionServer >
-------------------------------------------- > > ...