You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-issues@hadoop.apache.org by "Kihwal Lee (JIRA)" <ji...@apache.org> on 2012/10/18 23:46:03 UTC

[jira] [Updated] (HDFS-4075) Reduce recommissioning overhead

     [ https://issues.apache.org/jira/browse/HDFS-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kihwal Lee updated HDFS-4075:
-----------------------------

    Attachment: hdfs-4075.patch

The patch demotes the logging level of over-replicated block invalidation to DEBUG. On recommissioning, it prints out the summary instead.

This patch does not cleanly apply to branch-0.23, but it is due to a simple context difference.
                
> Reduce recommissioning overhead
> -------------------------------
>
>                 Key: HDFS-4075
>                 URL: https://issues.apache.org/jira/browse/HDFS-4075
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.23.4, 2.0.2-alpha
>            Reporter: Kihwal Lee
>            Assignee: Kihwal Lee
>            Priority: Critical
>         Attachments: hdfs-4075.patch
>
>
> When datanodes are recommissioned, {BlockManager#processOverReplicatedBlocksOnReCommission()} is called for each rejoined node and excess blocks are added to the invalidate list. The problem is this is done while the namesystem write lock is held.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira