You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2023/06/16 07:31:00 UTC
[jira] [Commented] (HDFS-17050) Erasure coding: fix bug for invalidating duplicated block when two ec block at the same datanode but different storage.
[ https://issues.apache.org/jira/browse/HDFS-17050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733373#comment-17733373 ]
ASF GitHub Bot commented on HDFS-17050:
---------------------------------------
hfutatzhanghb opened a new pull request, #5753:
URL: https://github.com/apache/hadoop/pull/5753
### Description of PR
Please see HDFS-17050.
### How was this patch tested?
add a UT
> Erasure coding: fix bug for invalidating duplicated block when two ec block at the same datanode but different storage.
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-17050
> URL: https://issues.apache.org/jira/browse/HDFS-17050
> Project: Hadoop HDFS
> Issue Type: Bug
> Affects Versions: 3.4.0, 3.3.2
> Reporter: farmmamba
> Assignee: farmmamba
> Priority: Major
>
> Currently, I found a strange phenomenon mentioned in HDFS-17047.
> When triggered FBR manually or automatically, we found some warn log like below:
> {code:java}
> 2023-06-14 16:29:36,432 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036578646784_59354864 moved to storageType DISK on node datanode12:50010
> 2023-06-14 16:29:36,477 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036578646784_59354864 moved to storageType DISK on node datanode12:50010{code}
> The above logs print the same storedBlock two times. After diving into logs, I found that there exist two blocks of a same block group due to some unknown reasons. And one of the two blocks is also exists in other datanode. But fsck did not print the duplicated replicas info.
> additional information： the file is 3MB+, we use RS-6-3-1024K, so the fsck only print seven blocks information. But indeed, we have eight blocks and one of them is a duplicated block.
>
> The reason why print above logs is that:
> In BlockManager#addStoredBlock method, because a datanode has two blocks of the same block group, the AddBlockResult would be REPLACED.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org