You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "farmmamba (Jira)" <ji...@apache.org> on 2023/06/13 10:27:00 UTC
[jira] [Updated] (HDFS-17047) BlockManager#addStoredBlock should log storage id when AddBlockResult is REPLACED
[ https://issues.apache.org/jira/browse/HDFS-17047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
farmmamba updated HDFS-17047:
-----------------------------
Description:
Recently, we found some logs in active namenode frequently like belows:
{code:java}
2023-06-12 05:34:09,821 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
2023-06-12 05:34:09,892 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
2023-06-12 11:34:07,932 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
2023-06-12 11:34:08,027 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
2023-06-12 17:34:08,742 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
2023-06-12 17:34:08,813 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
2023-06-12 23:34:09,752 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
2023-06-12 23:34:09,812 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
2023-06-13 05:34:08,065 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
2023-06-13 05:34:08,144 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
2023-06-13 11:34:08,638 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
2023-06-13 11:34:08,681 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010{code}
All logs have the same ec block id : blk_-9223372036614126544_57136788 and printed every 6 hours(FBR interval of our cluster).
To figure out what happened, I think we should also log storage id here.
> BlockManager#addStoredBlock should log storage id when AddBlockResult is REPLACED
> ---------------------------------------------------------------------------------
>
> Key: HDFS-17047
> URL: https://issues.apache.org/jira/browse/HDFS-17047
> Project: Hadoop HDFS
> Issue Type: Wish
> Reporter: farmmamba
> Priority: Minor
>
> Recently, we found some logs in active namenode frequently like belows:
>
> {code:java}
> 2023-06-12 05:34:09,821 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
> 2023-06-12 05:34:09,892 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
> 2023-06-12 11:34:07,932 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
> 2023-06-12 11:34:08,027 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
> 2023-06-12 17:34:08,742 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
> 2023-06-12 17:34:08,813 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
> 2023-06-12 23:34:09,752 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
> 2023-06-12 23:34:09,812 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
> 2023-06-13 05:34:08,065 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
> 2023-06-13 05:34:08,144 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
> 2023-06-13 11:34:08,638 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010
> 2023-06-13 11:34:08,681 WARN BlockStateChange: BLOCK* addStoredBlock: block blk_-9223372036614126544_57136788 moved to storageType DISK on node datanode1:50010{code}
>
>
> All logs have the same ec block id : blk_-9223372036614126544_57136788 and printed every 6 hours(FBR interval of our cluster).
> To figure out what happened, I think we should also log storage id here.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org