[jira] [Created] (HDFS-17949) Add optional FedBalance content summary verification

["QI, Jiale (Jira)" <ji...@apache.org>]

QI, Jiale created HDFS-17949:
--------------------------------

             Summary: Add optional FedBalance content summary verification
                 Key: HDFS-17949
                 URL: https://issues.apache.org/jira/browse/HDFS-17949
             Project: Hadoop HDFS
          Issue Type: Improvement
            Reporter: QI, Jiale


FedBalance currently finishes after the final DistCp succeeds, but it does not provide an optional built-in verification step to compare the source and destination tree at a coarse content-summary level.

This issue proposes adding an optional FedBalance verification phase after the final DistCp succeeds. The verification is disabled by default and can be enabled through CLI or configuration. When enabled, it compares the source and destination ContentSummary directory count, file count, and length after excluding snapshot content.

This gives operators a lightweight built-in correctness check without changing the default FedBalance behavior.

Proposed changes:
- Add an optional VERIFY stage to DistCpProcedure.
- Add a FedBalanceVerifier helper based on ContentSummary.
- Add a CLI option and configuration key to enable verification.
- Add tests for successful verification and mismatch detection.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org