You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Eli Collins (JIRA)" <ji...@apache.org> on 2010/02/04 04:46:27 UTC
[jira] Created: (HDFS-950) Add concat to FsShell
Add concat to FsShell
---------------------
Key: HDFS-950
URL: https://issues.apache.org/jira/browse/HDFS-950
Project: Hadoop HDFS
Issue Type: New Feature
Components: tools
Reporter: Eli Collins
Would be nice if concat (HDFS-222) was exposed up to FsShell so users don't have to use hadoop jar.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-950) Add concat to FsShell
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12829843#action_12829843 ]
Jakob Homan commented on HDFS-950:
----------------------------------
I share Owen's concern. The bigger question is that how to expose specific capabilities of the implementing filesystems to the command line. In this case, no other file system supports this operation as it is defined and implemented. It may not be a good idea to provide this ability on the general interface to the dfs.
> Add concat to FsShell
> ---------------------
>
> Key: HDFS-950
> URL: https://issues.apache.org/jira/browse/HDFS-950
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: tools
> Reporter: Eli Collins
>
> Would be nice if concat (HDFS-222) was exposed up to FsShell so users don't have to use hadoop jar.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HDFS-950) Add concat to FsShell
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12829425#action_12829425 ]
Owen O'Malley commented on HDFS-950:
------------------------------------
I'm not a fan of this idea. The vencat operation is very limited and meant to be used in very specific contexts. In particular, it is only available in HDFS with files that are exactly the same block size and owned by the same owner.
> Add concat to FsShell
> ---------------------
>
> Key: HDFS-950
> URL: https://issues.apache.org/jira/browse/HDFS-950
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: tools
> Reporter: Eli Collins
>
> Would be nice if concat (HDFS-222) was exposed up to FsShell so users don't have to use hadoop jar.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (HDFS-950) Add concat to FsShell
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins resolved HDFS-950.
------------------------------
Resolution: Won't Fix
The specific context seems to be a fairly general problem--if you want to transfer a large file in parallel you can't because HDFS doesn't support writing at arbitrary offsets (which is a reasonable limitation) so doing multiple transfers and concatenating the results is a reasonable workaround. We're already exposing file system specific APIs through hadoop fs (eg setrep). Perhaps we should make fs file system agnostic and add hadoop hdfs for those.
In any case, it's not a big limitation to make people use the jar, just makes it more inconvenient, so I'll close this one out.
> Add concat to FsShell
> ---------------------
>
> Key: HDFS-950
> URL: https://issues.apache.org/jira/browse/HDFS-950
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: tools
> Reporter: Eli Collins
>
> Would be nice if concat (HDFS-222) was exposed up to FsShell so users don't have to use hadoop jar.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.