[jira] [Updated] (HDFS-9560) Fair AvailableSpaceVolumeChoosingPolicy

["BELUGA BEHR (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-9560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

BELUGA BEHR updated HDFS-9560:
------------------------------
    Attachment: AvailableSpaceVolumeChoosingPolicy.java

> Fair AvailableSpaceVolumeChoosingPolicy
> ---------------------------------------
>
>                 Key: HDFS-9560
>                 URL: https://issues.apache.org/jira/browse/HDFS-9560
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: BELUGA BEHR
>            Priority: Minor
>         Attachments: AvailableSpaceVolumeChoosingPolicy.java
>
>
> I took a look at AvailableSpaceVolumeChoosingPolicy.  It seems a bit overkill and includes some configuration items that seem a bit arbitrary with no real clear guidance on how to effectively use them:
> _dfs.datanode.available-space-volume-choosing-policy.balanced-space-preference-fraction_
> _dfs.datanode.available-space-volume-choosing-policy.balanced-space-threshold_
> I have created an alternative implementation that does not require any external configuration, is thread-safe, and requires no synchronization.
> "Weighted Randomized Ordering"
> http://stackoverflow.com/questions/23971365/weighted-randomized-ordering
> Conceptually, a dart-board is constructed of several wedges, each wedge represents a disk volume.  The more available space that a volume has relative to the other volumes, the larger its wedge.  Then, a dart is thrown at the board and whichever wedge(volume) the dart lands on, that wedge is assigned the incoming block.
> Over time, the wedges balance and all have an equal chance of being "hit."



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)