You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Alejandro Abdelnur (JIRA)" <ji...@apache.org> on 2012/08/15 21:05:38 UTC
[jira] [Commented] (MAPREDUCE-4421) Remove dependency on deployed
MR jars
[ https://issues.apache.org/jira/browse/MAPREDUCE-4421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13435420#comment-13435420 ]
Alejandro Abdelnur commented on MAPREDUCE-4421:
-----------------------------------------------
This seems more a general YARN AM issue than a MR issue.
How about tweaking YARN so a client can specify a {{yarn.am.lib.path}} property, which would be a FileSystem accessible to all NMs (typically HDFS)? Then YARN would use that similarly to distributed cache entries, localizing all JARs before bootstrapping the AM & tasks.
> Remove dependency on deployed MR jars
> -------------------------------------
>
> Key: MAPREDUCE-4421
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4421
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Affects Versions: 2.0.0-alpha
> Reporter: Arun C Murthy
> Assignee: Arun C Murthy
>
> Currently MR AM depends on MR jars being deployed on all nodes via implicit dependency on YARN_APPLICATION_CLASSPATH.
> We should stop adding mapreduce jars to YARN_APPLICATION_CLASSPATH and, probably, just rely on adding a shaded MR jar along with job.jar to the dist-cache.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira