[jira] [Commented] (HBASE-14540) Write Ahead Log Batching Optimization

["John Leach (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941155#comment-14941155 ] 

John Leach commented on HBASE-14540:
------------------------------------

I did not run this on HBase based benchmarks but I did run this while we (SpliceMachine) were running TPCC benchmarks and it showed a significant improvement (2x).

Also we were able to get rid of these types of error messages.
{NOFORMAT}
wal.FSHLog: Slow sync cost
{NOFORMAT}


> Write Ahead Log Batching Optimization
> -------------------------------------
>
>                 Key: HBASE-14540
>                 URL: https://issues.apache.org/jira/browse/HBASE-14540
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: John Leach
>         Attachments: HBaseWALBlockingWaitStrategy.java
>
>
> The new write ahead log mechanism seems to batch too few mutations when running inside the disruptor.  As we scaled our load up (many threads with small writes), we saw the number of hdfs sync operations grow in concert with the number of writes.  Generally, one would expect the size of the batches to grow but the number of actual sync operations to settle. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)