You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Ramya Sunil (JIRA)" <ji...@apache.org> on 2011/08/02 00:33:48 UTC
[jira] [Created] (MAPREDUCE-2763) IllegalArgumentException while
using the dist cache
IllegalArgumentException while using the dist cache
---------------------------------------------------
Key: MAPREDUCE-2763
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2763
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Fix For: 0.23.0
IllegalArgumentException is seen while using distributed cache to cache some files and custom jars in classpath.
A simple way to reproduce this error is by using a streaming job:
hadoop jar hadoop-streaming.jar -libjars file://<path to custom jar> -input <path to input file> -output out -mapper "cat" -reducer NONE -cacheFile hdfs://<path to some file>#linkname
This is a regression introduced and the same command works fine on 0.20.x
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2763) IllegalArgumentException while
using the dist cache
Posted by "Ramya Sunil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13073835#comment-13073835 ]
Ramya Sunil commented on MAPREDUCE-2763:
----------------------------------------
Below is the complete stacktrace:
{noformat}
Exception in thread "main" java.lang.IllegalArgumentException: Invalid specification for distributed-cache artifacts of type FILE : #uris=1 #timestamps=2 #visibilities=2
at org.apache.hadoop.mapred.YARNRunner.parseDistributedCacheArtifacts(YARNRunner.java:411)
at org.apache.hadoop.mapred.YARNRunner.setupDistributedCache(YARNRunner.java:392)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:234)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:388)
at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1064)
at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1061)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1094)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1061)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:534)
at org.apache.hadoop.streaming.StreamJob.submitAndMonitorJob(StreamJob.java:1010)
at org.apache.hadoop.streaming.StreamJob.run(StreamJob.java:133)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:192)
{noformat}
> IllegalArgumentException while using the dist cache
> ---------------------------------------------------
>
> Key: MAPREDUCE-2763
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2763
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Fix For: 0.23.0
>
>
> IllegalArgumentException is seen while using distributed cache to cache some files and custom jars in classpath.
> A simple way to reproduce this error is by using a streaming job:
> hadoop jar hadoop-streaming.jar -libjars file://<path to custom jar> -input <path to input file> -output out -mapper "cat" -reducer NONE -cacheFile hdfs://<path to some file>#linkname
> This is a regression introduced and the same command works fine on 0.20.x
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2763) IllegalArgumentException while
using the dist cache
Posted by "Ramya Sunil (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ramya Sunil resolved MAPREDUCE-2763.
------------------------------------
Resolution: Fixed
I see this issue being fixed in the latest code base. Hence resolving this.
> IllegalArgumentException while using the dist cache
> ---------------------------------------------------
>
> Key: MAPREDUCE-2763
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2763
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 0.23.0
> Reporter: Ramya Sunil
> Fix For: 0.23.0
>
>
> IllegalArgumentException is seen while using distributed cache to cache some files and custom jars in classpath.
> A simple way to reproduce this error is by using a streaming job:
> hadoop jar hadoop-streaming.jar -libjars file://<path to custom jar> -input <path to input file> -output out -mapper "cat" -reducer NONE -cacheFile hdfs://<path to some file>#linkname
> This is a regression introduced and the same command works fine on 0.20.x
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira