[jira] [Commented] (HDFS-17227) EC: Fix bug in choosing targets when racks is not enough.

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-17227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18028495#comment-18028495 ] 

ASF GitHub Bot commented on HDFS-17227:
---------------------------------------

github-actions[bot] commented on PR #6192:
URL: https://github.com/apache/hadoop/pull/6192#issuecomment-3383635780

   We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
   If you feel like this was a mistake, or you would like to continue working on it, please feel free to re-open it and ask for a committer to remove the stale tag and review again.
   Thanks all for your contribution.




> EC: Fix bug in choosing targets when racks is not enough.
> ---------------------------------------------------------
>
>                 Key: HDFS-17227
>                 URL: https://issues.apache.org/jira/browse/HDFS-17227
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Shuyan Zhang
>            Assignee: Shuyan Zhang
>            Priority: Major
>              Labels: pull-request-available
>
> *Bug description*
> If,
> 1. There is a striped block blockinfo1, which has an excess replica on datanodeA.
> 2. blockinfo1 has an internal block that needs to be reconstruction.
> 3. The number of racks is less than the number of internal blocks of blockinfo1.
> Then, NN may choose datanodeA to reconstruct the internal block, resulting in two internal blocks of blockinfo1 on datanodeA, causing confusion. 
> *Root cause and solution*
> When we use `BlockPlacementPolicyRackFaultTolerant` for choosing targets and the racks is insufficient, `chooseEvenlyFromRemainingRacks` will be called. Currently, `chooseEvenlyFromRemainingRacks` calls `chooseOnce`, `chooseOnce` use `newExcludeNodes` as parameter instead of `excludedNodes`. When we choose targets for reconstructing internal blocks, 'newExcludeNodes' only includes datanodes that contain live replicas, and does not include datanodes that have excess replicas. This may result in datanodes with excess replicas are chosen.
> I think we do not need to use `newExcludeNodes`, just pass `excludedNodes` to `chooseOnce`.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org