You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "François Le Lay (JIRA)" <ji...@apache.org> on 2013/10/26 16:11:31 UTC
[jira] [Commented] (MAPREDUCE-968) NPE in distcp encountered when
placing _logs directory on S3FileSystem
[ https://issues.apache.org/jira/browse/MAPREDUCE-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13806105#comment-13806105 ]
François Le Lay commented on MAPREDUCE-968:
-------------------------------------------
Hi, I am encountering this issue in 1.0.4
{code}
hadoop@ip-172-21-1-200 ~]$ hadoop distcp -i hdfs://172.21.1.200:9000/user s3://my-empty-bucket/
Warning: $HADOOP_HOME is deprecated.
13/10/26 14:08:11 INFO tools.DistCp: srcPaths=[hdfs://172.21.1.200:9000/user]
13/10/26 14:08:11 INFO tools.DistCp: destPath=s3://my-empty-bucket/
With failures, global counters are inaccurate; consider running with -i
Copy failed: java.lang.NullPointerException
at org.apache.hadoop.fs.s3.S3FileSystem.makeAbsolute(S3FileSystem.java:121)
at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:332)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:768)
at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1048)
at org.apache.hadoop.tools.DistCp.copy(DistCp.java:666)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)
{code}
> NPE in distcp encountered when placing _logs directory on S3FileSystem
> ----------------------------------------------------------------------
>
> Key: MAPREDUCE-968
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-968
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.1
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Fix For: 0.21.0
>
> Attachments: MAPREDUCE-968.patch
>
>
> If distcp is pointed to an empty S3 bucket as the destination for an s3:// filesystem transfer, it will fail with the following exception
> Copy failed: java.lang.NullPointerException
> at org.apache.hadoop.fs.s3.S3FileSystem.makeAbsolute(S3FileSystem.java:121)
> at org.apache.hadoop.fs.s3.S3FileSystem.getFileStatus(S3FileSystem.java:332)
> at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:633)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1005)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:884)
--
This message was sent by Atlassian JIRA
(v6.1#6144)