[jira] Updated: (HADOOP-5012) addStoredBlock should take into account corrupted blocks when determining excesses

["Brian Bockelman (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HADOOP-5012?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brian Bockelman updated HADOOP-5012:
------------------------------------

    Attachment: hadoop-5012.patch

Patch for problem as described.

I do have an example of this problem happening in a logfile if someone's interested.

> addStoredBlock should take into account corrupted blocks when determining excesses
> ----------------------------------------------------------------------------------
>
>                 Key: HADOOP-5012
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5012
>             Project: Hadoop Core
>          Issue Type: Bug
>            Reporter: Brian Bockelman
>         Attachments: hadoop-5012.patch
>
>
> I found another source of corruption on our cluster.
> 0) Three replicas of a block exist
> 1) One is recognized as corrupt (3 reps total)
> 2) Namenode decides to create a new replica.  Replication done and addStoredBlock is called (4 reps total)
> 3) There are too many replicas, so processOverReplicatedBlock is called by addStoredBlock.
> 4) processOverReplicatedBlock is called, and it decides do invalidate the newly created replica.  [Oddly enough, it decides to invalidate the newly created one instead of the one in the corrupted replicas map!]
> 5) We are in the same state as (1) -- 3 replicas total, 1 of which is still bad.
> I believe we can fix this easily -- change numCurrentReplica variable to take into account the number of corrupt replicas.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.