You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Jacob Metcalf (JIRA)" <ji...@apache.org> on 2012/06/13 19:55:42 UTC
[jira] [Commented] (MAPREDUCE-2884) tmpjars not working when
default filesystem mismatches between client and server
[ https://issues.apache.org/jira/browse/MAPREDUCE-2884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13294589#comment-13294589 ]
Jacob Metcalf commented on MAPREDUCE-2884:
------------------------------------------
Would this also affect -libjars on Hadoop 2 (Cloudera's CDH4) when running using YARN in pseudo-distributed mode?
- Referring to: http://www.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job/
- With Hadoop 1 (Cloudera's CDH3) I had been packaging all my dependencies into the jar option (2). This did not work when I upgraded to Hadoop 2.
- Reading the update on the article: I concluded that option(2) has been deprecated.
- So I attempted to use option (1) -libjars and this did not appear to work either.
- I resorted to option (3) copying jars around manually and this works.
Debugging I saw that the ToolRunner was populating "tmpjars" with the contents of -libjars so concluded that this issue may be related.
> tmpjars not working when default filesystem mismatches between client and server
> --------------------------------------------------------------------------------
>
> Key: MAPREDUCE-2884
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2884
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv1
> Affects Versions: 0.23.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Critical
> Fix For: 0.24.0
>
>
> One of the HBase tests is failing which tries to add a local file to the distributed cache using the "tmpjars" configuration variable. The first half of the distributedcache setup decides not to copy it to the JT, because the JT is apparently using the same filesystem, but the second half of distributedcache setup tries to check timestamps on a different filesystem where the file does not exist.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira