[jira] [Commented] (ACCUMULO-1083) add concurrency to HDFS write-ahead log

["John Vines (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/ACCUMULO-1083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13596103#comment-13596103 ] 

John Vines commented on ACCUMULO-1083:
--------------------------------------

In the general scope of walog performance, I have a few questions that may or may not have been explored in development and testing.
# Hadoop sync/appends have gone through several iterations since they were introduced into the core release. Does anyone have any info on the performance of appends in the various hadoop releases? I'm thinking [~billie.rinaldi] or [~tlipcon] may have some insight.
# [~ecn]Can you provide some general insight on how the new waloggers are implemented? I have vague recollection of the incoming Mutations pooling with another thread constantly re-syncing. Does this achieve the same result as the group commits we had added recently?
# General community question- do we have any standard benchmarks or performance charts for append, or even general HDFS performance, in regard to system configuration? Specifically filesystem use, system configuration for disks, mounting options, and the like?

Thanks
                
> add concurrency to HDFS write-ahead log
> ---------------------------------------
>
>                 Key: ACCUMULO-1083
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1083
>             Project: Accumulo
>          Issue Type: Improvement
>          Components: tserver
>            Reporter: Adam Fuchs
>             Fix For: 1.6.0
>
>         Attachments: walog-performance.jpg
>
>
> When running tablet servers on beefy nodes (lots of disks), the write-ahead log can be a serious bottleneck. Today we ran a test of 1.5-SNAPSHOT on an 8-node (plus a master node) cluster in which the nodes had 32 cores and 15 drives each. Running with write-ahead log off resulted in a >4x performance improvement sustained over a long period.
> I believe the culprit is that the WAL is only using one file at a time per tablet server, which means HDFS is only appending to one drive (plus replicas). If we increase the number of concurrent WAL files supported on a tablet server we could probably drastically improve the performance on systems with many disks. As it stands, I believe Accumulo is significantly more optimized for a larger number of smaller nodes (3-4 drives).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira