[jira] [Created] (HDFS-15253) Set default throttle value on dfs.image.transfer.bandwidthPerSec

["Karthik Palanisamy (Jira)" <ji...@apache.org>]

Karthik Palanisamy created HDFS-15253:
-----------------------------------------

             Summary: Set default throttle value on dfs.image.transfer.bandwidthPerSec
                 Key: HDFS-15253
                 URL: https://issues.apache.org/jira/browse/HDFS-15253
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: namenode
            Reporter: Karthik Palanisamy
            Assignee: Karthik Palanisamy


The default value dfs.image.transfer.bandwidthPerSec is set to 0 so it can use maximum available bandwidth for fsimage transfers during checkpoint. I think we should throttle this. Many users were experienced namenode failover when transferring large image size along with fsimage replication on dfs.namenode.name.dir. eg. >25Gb.  

Thought to increase,

dfs.image.transfer.bandwidthPerSec=52428800. (50 MB/s)

dfs.namenode.checkpoint.txns=2000000 (Default is 1M, good to avoid frequent checkpoint. However, the default checkpoint runs every 6 hours once)

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org