You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "wuchang (Jira)" <ji...@apache.org> on 2024/07/10 07:36:00 UTC
[jira] [Updated] (HDFS-17568) [Decommission]Show Aggregated Reason for Why Low-Redundancy Block is Skipped for Reconstruction
[ https://issues.apache.org/jira/browse/HDFS-17568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
wuchang updated HDFS-17568:
---------------------------
Summary: [Decommission]Show Aggregated Reason for Why Low-Redundancy Block is Skipped for Reconstruction (was: [Decommission]Show Info Log for Repeated Useless refreshNode Operation)
> [Decommission]Show Aggregated Reason for Why Low-Redundancy Block is Skipped for Reconstruction
> -----------------------------------------------------------------------------------------------
>
> Key: HDFS-17568
> URL: https://issues.apache.org/jira/browse/HDFS-17568
> Project: Hadoop HDFS
> Issue Type: Improvement
> Reporter: wuchang
> Priority: Major
> Labels: pull-request-available
>
> The troubleshooting for decommission is very hard even in DEBUG mode.
> In somehow cases when decommission has taken a lot of time but we are not sure whether it is still in-progress, we run {{-refreshNodes}} to try to re-trigger the decommission(In fact it is useless).
> Then we check the NameNode logs, but it is a pity that we cannot find any useful log about whether or not our {{refreshNodes}} subcommand has taken any effect or not.
> So, my change is:
> * I changed this critical log 's level from TRACE to INFO since this log is not a repeatedly log and it gives administrator critical information for about what happened.
> {code:java}
> } else {
> LOG.info("startDecommission: Node {} in {}, nothing to do.",
> node, node.getAdminState());
> }{code}
> * When the Reconstruction which is trigged by node decommission is skipped, we want to know the reason. The reason is divided to 3 catagories
> ## No source node is available
> ## No Target node is available
> ## ReconstructionWork is built but validation failed
> I put these reasons in a single enum {{{}ReconstructionSkipReason{}}}. In DEBUG mode, the reason will be aggregated and showed to users.
> The log aggregated by {{ReconstructionSkipReason}} is as below:
> {code:java}
> 2024-07-10 02:59:09,707 [Thread-990] DEBUG blockmanagement.BlockManager: Block blk_3_0 is not scheduled for reconstruction since: [ source node or storage unavailable on node [DISK]storageID_0_3:NORMAL:127.0.0.1:9866. Detail : [stored replica state is corrupt or excess] source node or storage unavailable on node [DISK]storageID_1_3:NORMAL:127.0.0.1:9866. Detail : [replica is already decommissioned] ]{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org