[jira] [Resolved] (HDFS-17297) The NameNode should remove block from the BlocksMap if the block is marked as deleted.

["Takanobu Asanuma (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-17297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Takanobu Asanuma resolved HDFS-17297.
-------------------------------------
    Fix Version/s: 3.4.0
       Resolution: Fixed

> The NameNode should remove block from the BlocksMap if the block is marked as deleted.
> --------------------------------------------------------------------------------------
>
>                 Key: HDFS-17297
>                 URL: https://issues.apache.org/jira/browse/HDFS-17297
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namanode
>            Reporter: Haiyang Hu
>            Assignee: Haiyang Hu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.4.0
>
>
> When call internalReleaseLease method:
> {code:java}
> boolean internalReleaseLease(
> ...
> int minLocationsNum = 1;
> if (lastBlock.isStriped()) {
>   minLocationsNum = ((BlockInfoStriped) lastBlock).getRealDataBlockNum();
> }
> if (uc.getNumExpectedLocations() < minLocationsNum &&
>     lastBlock.getNumBytes() == 0) {
>   // There is no datanode reported to this block.
>   // may be client have crashed before writing data to pipeline.
>   // This blocks doesn't need any recovery.
>   // We can remove this block and close the file.
>   pendingFile.removeLastBlock(lastBlock);
>   finalizeINodeFileUnderConstruction(src, pendingFile,
>       iip.getLatestSnapshotId(), false); 
> ...
> }
> {code}
>  if the condition `uc.getNumExpectedLocations() < minLocationsNum && lastBlock.getNumBytes() == 0` is met during the execution of UNDER_RECOVERY logic, the block is removed from the block list in the inode file and marked as deleted. 
> However it is not removed from the BlocksMap, it may cause memory leak.
> Therefore it is necessary to remove the block from the BlocksMap at this point as well.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org