[jira] [Created] (YARN-1799) Enhance LocalDirAllocator in NM to consider DiskMaxUtilization cutoff

["Sunil G (JIRA)" <ji...@apache.org>]

Sunil G created YARN-1799:
-----------------------------

             Summary: Enhance LocalDirAllocator in NM to consider DiskMaxUtilization cutoff
                 Key: YARN-1799
                 URL: https://issues.apache.org/jira/browse/YARN-1799
             Project: Hadoop YARN
          Issue Type: Sub-task
          Components: nodemanager
    Affects Versions: 2.3.0
            Reporter: Sunil G


LocalDirAllocator provides paths for all tasks for its local write.
This considers the good list of directories which are selected by the HealthCheck mechamnism in LocalDirsHandlerService

getLocalPathForWrite() considers whether input demand "size" can meet the "capacity" in "lastAccessed" directory.
If more tasks asks for path from LocalDirAllocator, then it is possible that the allocation is done based on the current disk availability at that given time.
But this path would have earlier given to some other tasks to write and they may be sequentially doing writing.

It is better to check for an upper cutoff for disk availability



--
This message was sent by Atlassian JIRA
(v6.2#6252)