You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-issues@hadoop.apache.org by "Kuhu Shukla (JIRA)" <ji...@apache.org> on 2015/10/28 03:14:27 UTC

[jira] [Updated] (HDFS-9318) considerLoad factor can be improved

     [ https://issues.apache.org/jira/browse/HDFS-9318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kuhu Shukla updated HDFS-9318:
------------------------------
    Attachment: HDFS-9318-v1.patch

The simpler way is to make this threshold check have a configurable factor that multiplies to the loadAverage instead of the fixed 2x.

> considerLoad factor can be improved
> -----------------------------------
>
>                 Key: HDFS-9318
>                 URL: https://issues.apache.org/jira/browse/HDFS-9318
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.6.0
>            Reporter: Kuhu Shukla
>            Assignee: Kuhu Shukla
>         Attachments: HDFS-9318-v1.patch
>
>
> Currently considerLoad avoids choosing nodes that are too active, so it helps level the HDFS load across the cluster. Under normal conditions, this is desired. However, when a cluster has a large percentage of nearly full nodes, this can make it difficult to find good targets because the placement policy wants to avoid the full nodes, but considerLoad wants to avoid the busy less-full nodes.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)