You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Jason Lowe (JIRA)" <ji...@apache.org> on 2013/11/11 23:36:18 UTC
[jira] [Commented] (HADOOP-10091) Job with a har archive as input
fails on 0.23
[ https://issues.apache.org/jira/browse/HADOOP-10091?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13819521#comment-13819521 ]
Jason Lowe commented on HADOOP-10091:
-------------------------------------
Sample backtrace from a pig job:
{noformat}
org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Wrong FS:
har://hdfs-x:x/x.har/x,
expected: hdfs://x:x
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:288)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:473)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:490)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:385)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1215)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1264)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1215)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:336)
at org.apache.hadoop.mapreduce.lib.jobcontrol.JobControl.run(JobControl.java:231)
at java.lang.Thread.run(Thread.java:722)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:260)
Caused by: java.lang.IllegalArgumentException: Wrong FS:
har://hdfs-x:x/x.har/x,
expected: hdfs://x:x
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:582)
at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:155)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:76)
at org.apache.hadoop.hdfs.DistributedFileSystem$1.<init>(DistributedFileSystem.java:416)
at org.apache.hadoop.hdfs.DistributedFileSystem.listLocatedStatus(DistributedFileSystem.java:409)
at org.apache.hadoop.fs.FileSystem.listLocatedStatus(FileSystem.java:1654)
at org.apache.hadoop.fs.FilterFileSystem.listLocatedStatus(FilterFileSystem.java:208)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:259)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:335)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:274)
... 13 more
{noformat}
> Job with a har archive as input fails on 0.23
> ---------------------------------------------
>
> Key: HADOOP-10091
> URL: https://issues.apache.org/jira/browse/HADOOP-10091
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.23.10
> Reporter: Jason Lowe
> Assignee: Jason Lowe
> Priority: Blocker
>
> Attempting to run a MapReduce job with a har as input fails. Sample stacktrace to follow. We need to backport the fix for HADOOP-10003 to branch-0.23.
--
This message was sent by Atlassian JIRA
(v6.1#6144)