[GitHub] [hadoop] cndaimin commented on pull request #4101: HDFS-16519. Add throttler to EC reconstruction

[GitBox <gi...@apache.org>]

cndaimin commented on pull request #4101:
URL: https://github.com/apache/hadoop/pull/4101#issuecomment-1081368937


   @tomscut Thanks for your review. Yes, background reconstruction is necessary. With missing block(s) of block group, client has to read `dataUnits` blocks in block group to decode the target data. This causes `dataUnits` times of network traffic and corresponding cpu costs.
   In our product cluster, we encountered backgroud reconstruction exhausts the network traffic of DN node. This leads to the failure of client read/write. The speed of read/write and reconstruction is in conflict, we can not have both of them actually. And in practice, we set throttler of EC reconstruction to half of available network traffic, for example, 500 MB/s to 1GB/s NIC. In this PR, we set the default value to 0, which indicates there is no limit to reconstruction at all, to be sure there is no side effect to upgrade.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org