[jira] [Commented] (HDFS-17937) Limit fsimage transfer concurrency during checkpoint with multiple NameNodes.

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HDFS-17937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18090503#comment-18090503 ] 

ASF GitHub Bot commented on HDFS-17937:
---------------------------------------

huangzhaobo99 opened a new pull request, #8559:
URL: https://github.com/apache/hadoop/pull/8559

   ### Description of PR
   JIRA: https://issues.apache.org/jira/browse/HDFS-17937
   
   This patch adds **dfs.namenode.checkpoint.parallel.upload.max-threads** to cap fsimage upload concurrency in StandbyCheckpointer when parallel upload is enabled.
   The limit is needed to avoid excessive simultaneous fsimage uploads in multi-NameNode HA deployments, which can otherwise increase CPU/network/disk pressure and impact checkpoint/upload stability.
   Effective concurrency is now min(remoteNNAddresses.size(), max-threads), with config/docs updates in DFSConfigKeys, CheckpointConf, and hdfs-default.xml, plus validation that max-threads > 0.
   
   ### How was this patch tested?
   Added TestStandbyCheckpoints#testCheckpointParallelUploadRespectsMaxThreads UT to verify observed upload concurrency does not exceed configured max threads.




> Limit fsimage transfer concurrency during checkpoint with multiple NameNodes.
> -----------------------------------------------------------------------------
>
>                 Key: HDFS-17937
>                 URL: https://issues.apache.org/jira/browse/HDFS-17937
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Zhaobo Huang
>            Assignee: Zhaobo Huang
>            Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org