[jira] [Created] (HDFS-12412) Remove ErasureCodingWorker.stripedReadPool

["Lei (Eddy) Xu (JIRA)" <ji...@apache.org>]

Lei (Eddy) Xu created HDFS-12412:
------------------------------------

             Summary: Remove ErasureCodingWorker.stripedReadPool
                 Key: HDFS-12412
                 URL: https://issues.apache.org/jira/browse/HDFS-12412
             Project: Hadoop HDFS
          Issue Type: Improvement
          Components: erasure-coding
    Affects Versions: 3.0.0-alpha3
            Reporter: Lei (Eddy) Xu
            Assignee: Lei (Eddy) Xu


In {{ErasureCodingWorker}}, it uses {{stripedReconstructionPool}} to schedule the EC recovery tasks, while uses {{stripedReadPool}} for the reader threads in each recovery task.  We only need one of them to throttle the speed of recovery process, because each EC recovery task has a fix number of source readers (i.e., 3 for RS(3,2)). And because of the findings in HDFS-12044, the speed of EC recovery can be throttled by {{strippedReconstructionPool}} with {{xmitsInProgress}}. 

Moreover, keeping {{stripedReadPool}} makes customer difficult to understand and calculate the right balance between {{dfs.datanode.ec.reconstruction.stripedread.threads}}, {{dfs.datanode.ec.reconstruction.stripedblock.threads.size}} and {{maxReplicationStreams}}.  For example, a small {{stripread.threads}} (comparing to which {{reconstruction.threads.size}} implies), will unnecessarily limit the speed of recovery, which leads to larger MTTR. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org