[jira] [Resolved] (HDFS-17218) NameNode should process time out excess redundancy blocks

["Xiaoqiao He (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-17218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Xiaoqiao He resolved HDFS-17218.
--------------------------------
    Fix Version/s: 3.4.0
     Hadoop Flags: Reviewed
       Resolution: Fixed

> NameNode should process time out excess redundancy blocks
> ---------------------------------------------------------
>
>                 Key: HDFS-17218
>                 URL: https://issues.apache.org/jira/browse/HDFS-17218
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namanode
>            Reporter: Haiyang Hu
>            Assignee: Haiyang Hu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.4.0
>
>         Attachments: image-2023-10-12-15-52-52-336.png
>
>
> Currently found that DN will lose all pending DNA_INVALIDATE blocks if it restarts.
> *Root case*
> Current DN enables asynchronously deletion, it have many pending deletion blocks in memory.
> when DN restarts, these cached blocks may be lost. it causes some blocks in the excess map in the namenode to be leaked and this will result in many blocks having more replicas then expected.
> *solution*
> NameNode add logic to handle excess redundant block timeouts to resolve current issue.
> If NN determines that the excess redundancy block in DN has timed out and re-adds it to Invalidates.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org