You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Alejandro Abdelnur (Created) (JIRA)" <ji...@apache.org> on 2012/01/24 19:58:40 UTC
[jira] [Created] (HDFS-2833) Add GETMERGE operation to httpfs
Add GETMERGE operation to httpfs
--------------------------------
Key: HDFS-2833
URL: https://issues.apache.org/jira/browse/HDFS-2833
Project: Hadoop HDFS
Issue Type: New Feature
Affects Versions: 0.23.1
Reporter: Alejandro Abdelnur
Assignee: Alejandro Abdelnur
Fix For: 0.23.1
Add to a convenience operation GETMERGE to httpfs.
This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2833) Add GETMERGE operation to httpfs
Posted by "Eli Collins (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HDFS-2833:
------------------------------
Resolution: Won't Fix
Fix Version/s: (was: 0.23.3)
Status: Resolved (was: Patch Available)
Closing as won't fix, Tucu, feel free to re-open if you disagree.
> Add GETMERGE operation to httpfs
> --------------------------------
>
> Key: HDFS-2833
> URL: https://issues.apache.org/jira/browse/HDFS-2833
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 0.23.1
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Attachments: HDFS-2833.patch
>
>
> Add to a convenience operation GETMERGE to httpfs.
> This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream.
> It would have the same semantics as the 'hadoop fs -getmerge' command.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2833) Add GETMERGE operation to httpfs
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192454#comment-13192454 ]
Hadoop QA commented on HDFS-2833:
---------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12511705/HDFS-2833.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 21 warning messages.
-1 javac. The applied patch generated 23 javac compiler warnings (more than the trunk's current 22 warnings).
+1 eclipse:eclipse. The patch built with eclipse:eclipse.
-1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1805//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1805//artifact/trunk/hadoop-hdfs-project/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1805//console
This message is automatically generated.
> Add GETMERGE operation to httpfs
> --------------------------------
>
> Key: HDFS-2833
> URL: https://issues.apache.org/jira/browse/HDFS-2833
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 0.23.1
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.1
>
> Attachments: HDFS-2833.patch
>
>
> Add to a convenience operation GETMERGE to httpfs.
> This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream.
> It would have the same semantics as the 'hadoop fs -getmerge' command.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2833) Add GETMERGE operation to httpfs
Posted by "Alejandro Abdelnur (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HDFS-2833:
-------------------------------------
Status: Patch Available (was: Open)
> Add GETMERGE operation to httpfs
> --------------------------------
>
> Key: HDFS-2833
> URL: https://issues.apache.org/jira/browse/HDFS-2833
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 0.23.1
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.1
>
> Attachments: HDFS-2833.patch
>
>
> Add to a convenience operation GETMERGE to httpfs.
> This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream.
> It would have the same semantics as the 'hadoop fs -getmerge' command.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2833) Add GETMERGE operation to httpfs
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated HDFS-2833:
--------------------------------
Fix Version/s: (was: 0.23.1)
0.23.3
> Add GETMERGE operation to httpfs
> --------------------------------
>
> Key: HDFS-2833
> URL: https://issues.apache.org/jira/browse/HDFS-2833
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 0.23.1
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.3
>
> Attachments: HDFS-2833.patch
>
>
> Add to a convenience operation GETMERGE to httpfs.
> This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream.
> It would have the same semantics as the 'hadoop fs -getmerge' command.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2833) Add GETMERGE operation to httpfs
Posted by "Alejandro Abdelnur (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HDFS-2833:
-------------------------------------
Description:
Add to a convenience operation GETMERGE to httpfs.
This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream.
It would have the same semantics as the 'hadoop fs -getmerge' command.
was:
Add to a convenience operation GETMERGE to httpfs.
This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream.
> Add GETMERGE operation to httpfs
> --------------------------------
>
> Key: HDFS-2833
> URL: https://issues.apache.org/jira/browse/HDFS-2833
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 0.23.1
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.1
>
>
> Add to a convenience operation GETMERGE to httpfs.
> This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream.
> It would have the same semantics as the 'hadoop fs -getmerge' command.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2833) Add GETMERGE operation to httpfs
Posted by "Alejandro Abdelnur (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192373#comment-13192373 ]
Alejandro Abdelnur commented on HDFS-2833:
------------------------------------------
The new operation would be GETMERGE and it would support 2 optional parameters 'fileseparator=' and filter=' (a globfilter).
> Add GETMERGE operation to httpfs
> --------------------------------
>
> Key: HDFS-2833
> URL: https://issues.apache.org/jira/browse/HDFS-2833
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 0.23.1
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.1
>
>
> Add to a convenience operation GETMERGE to httpfs.
> This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-2833) Add GETMERGE operation to httpfs
Posted by "Alejandro Abdelnur (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HDFS-2833:
-------------------------------------
Attachment: HDFS-2833.patch
> Add GETMERGE operation to httpfs
> --------------------------------
>
> Key: HDFS-2833
> URL: https://issues.apache.org/jira/browse/HDFS-2833
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 0.23.1
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.1
>
> Attachments: HDFS-2833.patch
>
>
> Add to a convenience operation GETMERGE to httpfs.
> This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream.
> It would have the same semantics as the 'hadoop fs -getmerge' command.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2833) Add GETMERGE operation to httpfs
Posted by "Eli Collins (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13195002#comment-13195002 ]
Eli Collins commented on HDFS-2833:
-----------------------------------
getmerge is an FsShell API, not a FileSystem/Context API. I think we should keep HttpFs as a FileSystem/Context proxy. Otherwise, eg HttpFS won't be compatible with WebHDFS, we'll end up putting in the kitchen sink, etc.
Sounds like what the user really wants is a rest API for FsShell.
> Add GETMERGE operation to httpfs
> --------------------------------
>
> Key: HDFS-2833
> URL: https://issues.apache.org/jira/browse/HDFS-2833
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 0.23.1
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.1
>
> Attachments: HDFS-2833.patch
>
>
> Add to a convenience operation GETMERGE to httpfs.
> This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream.
> It would have the same semantics as the 'hadoop fs -getmerge' command.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-2833) Add GETMERGE operation to httpfs
Posted by "Alejandro Abdelnur (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13192475#comment-13192475 ]
Alejandro Abdelnur commented on HDFS-2833:
------------------------------------------
-1 findbugs is unrelated
-1 javadoc is unrelated (the warnings in httpfs are already accounted for, its seems a javadoc issue)
-1 javac is warning in a testcase
> Add GETMERGE operation to httpfs
> --------------------------------
>
> Key: HDFS-2833
> URL: https://issues.apache.org/jira/browse/HDFS-2833
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 0.23.1
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Fix For: 0.23.1
>
> Attachments: HDFS-2833.patch
>
>
> Add to a convenience operation GETMERGE to httpfs.
> This will simplify for external system accessing over HTTP to consume the output of an MR job a single stream.
> It would have the same semantics as the 'hadoop fs -getmerge' command.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira