[jira] [Updated] (HDFS-9303) Balancer slowly with too many small file blocks

["Lin Yiqun (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-9303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Lin Yiqun updated HDFS-9303:
----------------------------
    Status: Patch Available  (was: Open)

The solution of this problem is to limit the small source blocks and give a limit value then filter some small blocks.This will be increase the every balance iteration time but the balancer's velocity is faster.Attach a init patch.

> Balancer slowly with too many small file blocks
> -----------------------------------------------
>
>                 Key: HDFS-9303
>                 URL: https://issues.apache.org/jira/browse/HDFS-9303
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: balancer & mover
>    Affects Versions: 2.7.1
>            Reporter: Lin Yiqun
>            Assignee: Lin Yiqun
>
> In the recent hadoop release versions I found that balance operation is always so slowly, even though I upgrade the version.When I analyse balancer log, I found that in every balance iteration,it use only 4 to 5 minutes, and is a short time.And The most important is that the most of being moving blocks is small blocks ,and the size is smaller than 1M.And this is a main reason of the low effective of balance operation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)