You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Yuan LUO (Jira)" <ji...@apache.org> on 2019/09/03 11:55:00 UTC
[jira] [Commented] (MAPREDUCE-7227) Fix job staging directory
residual problem in a big yarn cluster composed of multiple independent
hdfs clusters
[ https://issues.apache.org/jira/browse/MAPREDUCE-7227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16921365#comment-16921365 ]
Yuan LUO commented on MAPREDUCE-7227:
-------------------------------------
HADOOP-MAPREDUCE-7227.005.patch for 3.3.0
> Fix job staging directory residual problem in a big yarn cluster composed of multiple independent hdfs clusters
> ---------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-7227
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7227
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, mrv2
> Affects Versions: 2.6.0, 2.7.0, 3.1.2
> Reporter: Yuan LUO
> Assignee: Yuan LUO
> Priority: Major
> Attachments: 1.png, 2.png, HADOOP-MAPREDUCE-7227.001.patch, HADOOP-MAPREDUCE-7227.002.patch, HADOOP-MAPREDUCE-7227.003.patch, HADOOP-MAPREDUCE-7227.004.patch, HADOOP-MAPREDUCE-7227.005.patch
>
>
> Our yarn cluster is made up of some independent hdfs cluster, the 'default.FS' in every hdfs cluster is different.when user submit job to yarn cluster, if the 'default.FS' between client and nodemanager is inconsistent, then the job staging dir can't be cleanup by AppMaster. Because it will produce two job staging dirs in our conditions by client and appmaster. So we can modify AppMaster through client's ‘default.FS’ to create job staging dir.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org