[jira] Commented: (HADOOP-862) Add handling of s3 to CopyFile tool

["stack@archive.org (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HADOOP-862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12462676 ] 

stack@archive.org commented on HADOOP-862:
------------------------------------------

Attached is first cut at adding s3 handling to CopyFiles.

Here's list of changes:

+ Allow hdfs or dfs URI schemes (Used to be dfs only).
+ Changed the usage message so filesystem is generic URI (rather than namenode:port | local).
+ getFileSysName was removed.  Use Filesystem.get with fs URI instead.
+ getMapCount:  Moved duplicated code for figuring number of maps here.
+ toURI: Added. Have (duplicated) tests of URIness go via here instead.
+ CopyFilesReducer: Removed two instances.  Does nothing.
+ Added testing of URIness to members of file-of-source URIs.
+ Minor javadoc and formatting changes.

Its lightly tested.

> Add handling of s3 to CopyFile tool
> -----------------------------------
>
>                 Key: HADOOP-862
>                 URL: https://issues.apache.org/jira/browse/HADOOP-862
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: util
>    Affects Versions: 0.10.0
>            Reporter: stack@archive.org
>            Priority: Minor
>         Attachments: copyfiles-s3.diff
>
>
> CopyFile is a useful tool for doing bulk copies.  It doesn't have handling for the recently added s3 filesystem.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: https://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira