You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-issues@hadoop.apache.org by "Ming Ma (JIRA)" <ji...@apache.org> on 2014/08/01 01:12:39 UTC

[jira] [Updated] (HDFS-6425) Large postponedMisreplicatedBlocks has impact on blockReport latency

     [ https://issues.apache.org/jira/browse/HDFS-6425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ming Ma updated HDFS-6425:
--------------------------

    Summary: Large postponedMisreplicatedBlocks has impact on blockReport latency  (was: reset postponedMisreplicatedBlocks and postponedMisreplicatedBlocksCount when NN becomes active)

> Large postponedMisreplicatedBlocks has impact on blockReport latency
> --------------------------------------------------------------------
>
>                 Key: HDFS-6425
>                 URL: https://issues.apache.org/jira/browse/HDFS-6425
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Ming Ma
>            Assignee: Ming Ma
>         Attachments: HDFS-6425.patch
>
>
> Sometimes we have large number of over replicates when NN fails over. When the new active NN took over, over replicated blocks will be put to postponedMisreplicatedBlocks until all DNs for that block aren't stale anymore.
> We have a case where NNs flip flop. Before postponedMisreplicatedBlocks became empty, NN fail over again and again. So postponedMisreplicatedBlocks just kept increasing until the cluster is stable. 
> In addition, large postponedMisreplicatedBlocks could make rescanPostponedMisreplicatedBlocks slow. rescanPostponedMisreplicatedBlocks takes write lock. So it could slow down the block report processing.



--
This message was sent by Atlassian JIRA
(v6.2#6252)