Re: [jira] Commented: (HBASE-1416) Pool of commit loggers in each HRegionServer

[Ryan Rawson <ry...@gmail.com>]

With deferred log flush we can choose between high durability and speed on a
per table basis. I think we should shelf this for now, since with write
buffering in the client and the adjusted sync calls, we are very fast at
bulk data import.

On Dec 19, 2009 1:47 PM, "Andrew Purtell (JIRA)" <ji...@apache.org> wrote:


   [
https://issues.apache.org/jira/browse/HBASE-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12792910#action_12792910]

Andrew Purtell commented on HBASE-1416:
---------------------------------------
Do we still want to do this? Seems some of Ryan's ideas have been
implemented as HBASE-1939 already.

I was playing around with this the other day after committing HBASE-2059. On
that note, implementing an HLog.Writer that stripes edits over a pool of
logs under a directory instead of a single file works ok on read and write
paths but split and roll code assumes the log is a single file so barf when
they access a dir instead; the HLog.Writer interface would need to be
widened with split and roll methods to make that work cleanly. Anyway, sync
is still a log pole and with striped writes over a pool of HLogs there are
more dirty blocks to sync. Unless the current edit load is high we end up
dirtying small bits of a bunch of blocks where instead we should just write
them all into one as they'll all fit in there.

> Pool of commit loggers in each HRegionServer >
-------------------------------------------- > > ...