[jira] Commented: (HADOOP-26) DFS node choice doesn't take available space into account effectively

["Mike Cafarella (JIRA)" <ji...@apache.org>]

    [ http://issues.apache.org/jira/browse/HADOOP-26?page=comments#action_12365457 ] 

Mike Cafarella commented on HADOOP-26:
--------------------------------------


  OK, the above comment is a little misleading.

  We used to have a difference space-management system in place, which would set
certain nodes off-limits if they didn't have enough space.  Now we choose nodes probabilistically,
with more weight given to nodes with more space.

  That's probably a little too cute than is necessary.  We should just have hard limits on how
much a given node will take before it complains and never receives further node allocations.

> DFS node choice doesn't take available space into account effectively
> ---------------------------------------------------------------------
>
>          Key: HADOOP-26
>          URL: http://issues.apache.org/jira/browse/HADOOP-26
>      Project: Hadoop
>         Type: Bug
>   Components: dfs
>     Reporter: Mike Cafarella
>     Assignee: Mike Cafarella

>
>   We used to have node-allocation be sensitive to available disk space.  It turned out that
> this was a little accident-prone and hard to debug on then-available machines, so I removed it.
>   DFS is mature enough now that this needs to come back and work properly.  A node with less
> than X bytes should never accept a new block for allocation.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira