You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-dev@hadoop.apache.org by "Arpit Gupta (Created) (JIRA)" <ji...@apache.org> on 2011/09/27 06:43:12 UTC
[jira] [Created] (MAPREDUCE-3100) distcp with an hftp destination
url fails if the destination directory does not exist
distcp with an hftp destination url fails if the destination directory does not exist
-------------------------------------------------------------------------------------
Key: MAPREDUCE-3100
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3100
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: distcp
Affects Versions: 0.20.205.0
Reporter: Arpit Gupta
bash-3.2$ bin/hadoop --config HADOOP_CONF_DIR distcp -i hftp://NN_HOST:50070/input_path hftp://NN_HOST:50070/output_path
11/09/27 04:36:01 INFO tools.DistCp: srcPaths=[hftp://hftp://NN_HOST:50070/input_path]
11/09/27 04:36:01 INFO tools.DistCp: destPath=hftp://hftp://NN_HOST:50070/output_path
11/09/27 04:36:02 INFO tools.DistCp: hftp://hftp://NN_HOST:50070/output_path does not exist.
With failures, global counters are inaccurate; consider running with -i
Copy failed: java.io.IOException: Not supported
at org.apache.hadoop.hdfs.HftpFileSystem.mkdirs(HftpFileSystem.java:558)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1066)
at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1170)
at org.apache.hadoop.tools.DistCp.copy(DistCp.java:666)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)
The same command works if instead of hftp default filesystem (hdfs) is used. It creates the dir if it does not exist. We should do the same for hftp. I also suspect that we have this issue with webhdfs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-3100) distcp with an hftp destination
url fails if the destination directory does not exist
Posted by "Arpit Gupta (Resolved) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arpit Gupta resolved MAPREDUCE-3100.
------------------------------------
Resolution: Invalid
hftp is read only
> distcp with an hftp destination url fails if the destination directory does not exist
> -------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3100
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3100
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: distcp
> Affects Versions: 0.20.205.0
> Reporter: Arpit Gupta
>
> bash-3.2$ bin/hadoop --config HADOOP_CONF_DIR distcp -i hftp://NN_HOST:50070/input_path hftp://NN_HOST:50070/output_path
> 11/09/27 04:36:01 INFO tools.DistCp: srcPaths=[hftp://hftp://NN_HOST:50070/input_path]
> 11/09/27 04:36:01 INFO tools.DistCp: destPath=hftp://hftp://NN_HOST:50070/output_path
> 11/09/27 04:36:02 INFO tools.DistCp: hftp://hftp://NN_HOST:50070/output_path does not exist.
> With failures, global counters are inaccurate; consider running with -i
> Copy failed: java.io.IOException: Not supported
> at org.apache.hadoop.hdfs.HftpFileSystem.mkdirs(HftpFileSystem.java:558)
> at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1066)
> at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1170)
> at org.apache.hadoop.tools.DistCp.copy(DistCp.java:666)
> at org.apache.hadoop.tools.DistCp.run(DistCp.java:881)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.hadoop.tools.DistCp.main(DistCp.java:908)
> The same command works if instead of hftp default filesystem (hdfs) is used. It creates the dir if it does not exist. We should do the same for hftp. I also suspect that we have this issue with webhdfs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira