You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Hitesh Shah (Commented) (JIRA)" <ji...@apache.org> on 2011/11/18 01:47:52 UTC
[jira] [Commented] (MAPREDUCE-3426) uber-jobs tried to write
outputs into wrong dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13152513#comment-13152513 ]
Hitesh Shah commented on MAPREDUCE-3426:
----------------------------------------
Map task tries to write output into top-level directory instead of user-specific appcache:
2011-11-17 00:16:38,196 WARN [uber-SubtaskRunner] org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext: /grid/0/dev/hrt_mr/tmp/mapred-local is not writable
org.apache.hadoop.util.DiskChecker$DiskErrorException: directory is not writable: /grid/0/dev/hrt_mr/tmp/mapred-local
at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:99)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.confChanged(LocalDirAllocator.java:272)
at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:331)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:150)
at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:131)
at org.apache.hadoop.mapred.MROutputFiles.getSpillFileForWrite(MROutputFiles.java:147)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1503)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1397)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:649)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:716)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:328)
at org.apache.hadoop.mapred.LocalContainerLauncher$SubtaskRunner.runSubtask(LocalContainerLauncher.java:294)
at org.apache.hadoop.mapred.LocalContainerLauncher$SubtaskRunner.run(LocalContainerLauncher.java:224)
at java.lang.Thread.run(Thread.java:662)
> uber-jobs tried to write outputs into wrong dir
> -----------------------------------------------
>
> Key: MAPREDUCE-3426
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3426
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: mrv2
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Hitesh Shah
> Fix For: 0.23.1
>
>
> Incorrect setup of the uber tasks causes tasks to try to write intermidiate outputs into dirs that the user does not have permissions to write to on a secure cluster.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira