[jira] [Created] (HDFS-17050) Erasure coding: invalidate duplicated block when actual block numbers less than data blocks plus parity blocks.

["farmmamba (Jira)" <ji...@apache.org>]

farmmamba created HDFS-17050:
--------------------------------

             Summary: Erasure coding: invalidate duplicated block when actual block numbers less than data blocks plus parity blocks.
                 Key: HDFS-17050
                 URL: https://issues.apache.org/jira/browse/HDFS-17050
             Project: Hadoop HDFS
          Issue Type: Bug
    Affects Versions: 3.3.2, 3.4.0
            Reporter: farmmamba


Currently, I found a strange phenomenon mentioned in HDFS-17047.

When triggered FBR manually or automatically, we found some warn log like below:
{code:java}
2023-06-14 16:29:36,432 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036578646784_59354864 moved to storageType DISK on node datanode12:50010
2023-06-14 16:29:36,477 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036578646784_59354864 moved to storageType DISK on node datanode12:50010{code}
The above logs print the same storedBlock two times. After diving into logs, I found that there exist two blocks of a same block group due to some unknown reasons. And one of the two blocks is also exists in other datanode. But fsck did not print the duplicated replicas info.

additional information： the file is 3MB+,  we use RS-6-3-1024K, so the fsck only print seven blocks information. But indeed, we have eight blocks and one of them is a duplicated block.

 

The reason why print above logs is that:

In BlockManager#addStoredBlock method, because a datanode has two blocks of the same block group, the AddBlockResult would be REPLACED.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org