You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-issues@hadoop.apache.org by "Matt Foley (JIRA)" <ji...@apache.org> on 2011/07/05 22:18:16 UTC

[jira] [Commented] (HDFS-2126) Improve Namenode startup time [umbrella task]

    [ https://issues.apache.org/jira/browse/HDFS-2126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13060108#comment-13060108 ] 

Matt Foley commented on HDFS-2126:
----------------------------------

HDFS-1295 decreased IBR processing time by more than 1/3.  On the server-class machines in our test cluster, it enabled processing speeds of about 450 Block Reports per minute.  Testing also established that even in large clusters it is still best to start all the Datanodes together, and not try to stagger them manually; the Namenode does not have difficulty enqueuing the flow.  

HDFS-1541 and HDFS-1767 help prevent stochastic errors in IBR processing.  It is hard to measure the exact improvement, but they eliminate the two most frequently seen causes of re-processing of nodes.

> Improve Namenode startup time [umbrella task]
> ---------------------------------------------
>
>                 Key: HDFS-2126
>                 URL: https://issues.apache.org/jira/browse/HDFS-2126
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: name-node
>    Affects Versions: 0.20.2
>            Reporter: Matt Foley
>
> This is an umbrella task to group the improvements in Namenode startup latency made over the last few months, and track remaining ideas.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira