[jira] [Comment Edited] (MAPREDUCE-311) JobClient should use multiple volumes as hadoop.tmp.dir

["Allen Wittenauer (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MAPREDUCE-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14070802#comment-14070802 ] 

Allen Wittenauer edited comment on MAPREDUCE-311 at 7/22/14 8:11 PM:
---------------------------------------------------------------------

It's likely too late to change hadoop.tmp.dir.  But this is still an issue.  Debating opening a new JIRA (under YARN) that states the problem but not a solution so that hadoop.tmp.dir is left alone.


was (Author: aw):
It's likely too late to change hadoop.tmp.dir.  But this is still an issue.  Debating opening a new JIRA that states the problem but not a solution so that hadoop.tmp.dir is left alone.

> JobClient should use multiple volumes as hadoop.tmp.dir
> -------------------------------------------------------
>
>                 Key: MAPREDUCE-311
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-311
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>         Environment: All
>            Reporter: Milind Bhandarkar
>
> Currently, hadoop.tmp.dir configuration variable allows specification of only a single directory to be used as scratch space. In particular, on the job launcher nodes with multiple volumes, this fails the entire job if the tmp.dir is somehow unusable. When the job launcher nodes have multiple volumes, the tmp space availability can be improved by using multiple volumes (either randomly or in round-robin.) The code for choosing a volume from a comma-separated list of multiple volumes is already there for mapred.local.dir etc. That needs to be used by job client as well.



--
This message was sent by Atlassian JIRA
(v6.2#6252)