[jira] [Updated] (HDFS-8827) Erasure Coding: When namenode processes over replicated striped block, NPE will be occur in ReplicationMonitor

["Takuya Fukudome (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-8827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Takuya Fukudome updated HDFS-8827:
----------------------------------
    Attachment: HDFS-8827.1.patch

Thanks for the comment, [~zhz]! I attached an initial patch which added a unit test occurs this issue. It processes a small EC file which doesn't have full internal blocks and its internal blocks are over replicated.
If I understood correctly, when some indices of internal blocks are missing and internal blocks are over replicated, {{BlockPlacementPolicyDefault#chooseReplicaToDelete}} will return null. I think the cause is the {{excessTypes}} in {{chooseExcessReplicasStriped}} is empty during the process of such blocks.

> Erasure Coding: When namenode processes over replicated striped block, NPE will be occur in ReplicationMonitor
> --------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-8827
>                 URL: https://issues.apache.org/jira/browse/HDFS-8827
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Takuya Fukudome
>            Assignee: Takuya Fukudome
>         Attachments: HDFS-8827.1.patch, processing-over-replica-npe.log
>
>
> In our test cluster, when namenode processed over replicated striped blocks, null pointer exception(NPE) occurred. This happened under below situation: 1) some datanodes shutdown. 2) namenode recovers block group which lost internal blocks. 3) restart the stopped datanodes. 4) namenode processes over replicated striped blocks. 5) NPE occurs
> I think BlockPlacementPolicyDefault#chooseReplicaToDelete will return null in this situation which causes this NPE problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)