You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Doug Cutting (JIRA)" <ji...@apache.org> on 2006/07/19 09:40:14 UTC
[jira] Commented: (HADOOP-369) Added ability to copy all part-files
into one output file
[ http://issues.apache.org/jira/browse/HADOOP-369?page=comments#action_12422050 ]
Doug Cutting commented on HADOOP-369:
-------------------------------------
I don't see why this needs to be a new generic FileSystem method. Can't it simply be a FileUtil method?
Alternately, couldn't we just modify DFSShell.cat(String) to, when it's argument is a directory, recursively descend and copy all contained file content to standard output? Then one could just 'bin/hadoop dfs cat foo > bar' to append all content under 'foo' into a local file 'bar'. Would that work for you?
> Added ability to copy all part-files into one output file
> ---------------------------------------------------------
>
> Key: HADOOP-369
> URL: http://issues.apache.org/jira/browse/HADOOP-369
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.4.0
> Reporter: Johan Oskarson
> Priority: Trivial
> Attachments: copymerge.patch
>
>
> Since we use the hadoop output in non-hadoop applications it's nice to be able to merge the part-files into one output file on the local filesystem.
> So I've added a dfsshell feature that streams from all files in a directory to one output file.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira