You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Sandy Ryza (JIRA)" <ji...@apache.org> on 2013/03/06 03:22:12 UTC
[jira] [Commented] (MAPREDUCE-5047) keep.failed.task.files=true
causes job failure on secure clusters
[ https://issues.apache.org/jira/browse/MAPREDUCE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13594251#comment-13594251 ]
Sandy Ryza commented on MAPREDUCE-5047:
---------------------------------------
localizeConfiguration is needed in the tasktracker in order to set task-specific configuration options, but split.info does not need to be created at that time. The patch moves the action of writing out split.info into a new writeFilesRequiredForRerun method. This method is called by the Child, but not by the tasktracker.
Tested on a pseudo distributed cluster and on a secure distributed cluster that the permissions error no longer shows up and that split.info is still written out to the correct location.
> keep.failed.task.files=true causes job failure on secure clusters
> -----------------------------------------------------------------
>
> Key: MAPREDUCE-5047
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5047
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: task, tasktracker
> Affects Versions: 1.1.1
> Reporter: Sandy Ryza
> Assignee: Sandy Ryza
> Attachments: MAPREDUCE-5047.patch
>
>
> To support IsolationRunner, split info is written to local directories. This occurs inside MapTask#localizeConfiguration, which is called both tasktracker and by the child JVM. On a secure cluster, the tasktacker's attempt to write it fails, because the tasktracker does not have permission to write to the user's directory. It is likely that the call to localizeConfiguration in the tasktracker can be removed.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira