You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by "Gabriel Reid (JIRA)" <ji...@apache.org> on 2013/01/02 20:30:13 UTC
[jira] [Commented] (CRUNCH-138) Not on right filesystem in
SequentialFileNamingScheme.getSequentialFileName
[ https://issues.apache.org/jira/browse/CRUNCH-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13542325#comment-13542325 ]
Gabriel Reid commented on CRUNCH-138:
-------------------------------------
I've committed this to master, thanks a lot for catching it!
> Not on right filesystem in SequentialFileNamingScheme.getSequentialFileName
> ---------------------------------------------------------------------------
>
> Key: CRUNCH-138
> URL: https://issues.apache.org/jira/browse/CRUNCH-138
> Project: Crunch
> Issue Type: Bug
> Components: IO
> Affects Versions: 0.4.0
> Reporter: Daithi O Crualaoich
> Attachments: CRUNCH-138.patch
>
>
> Exception on writing output to S3. Needs the same as in CRUNCH-47.
> 2013-01-02 16:16:43,264 INFO org.apache.hadoop.fs.s3native.NativeS3FileSystem (Thread-27): Making dir 's3://my-bucket/out' in S3
> 2013-01-02 16:16:43,300 ERROR org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl (Thread-27): Error in run loop
> java.lang.IllegalArgumentException: This file system object (hdfs://10.226.229.178:9000) does not support access to the request path 's3://my-bucket/out' You possibly called FileSystem.get(conf) when you should have called FileSystem.get(uri, conf) to obtain a file system supporting your path.
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:384)
> at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:129)
> at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:269)
> at org.apache.crunch.io.SequentialFileNamingScheme.getSequentialFileName(SequentialFileNamingScheme.java:46)
> at org.apache.crunch.io.SequentialFileNamingScheme.getReduceOutputName(SequentialFileNamingScheme.java:40)
> at org.apache.crunch.impl.mr.exec.CrunchJob.getDestFile(CrunchJob.java:101)
> at org.apache.crunch.impl.mr.exec.CrunchJob.handleMultiPaths(CrunchJob.java:75)
> at org.apache.crunch.impl.mr.exec.CrunchJob.checkRunningState(CrunchJob.java:114)
> at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchControlledJob.checkState(CrunchControlledJob.java:278)
> at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.checkRunningJobs(CrunchJobControl.java:227)
> at org.apache.crunch.hadoop.mapreduce.lib.jobcontrol.CrunchJobControl.run(CrunchJobControl.java:277)
> at java.lang.Thread.run(Thread.java:662)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira