You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Aaron Kimball (JIRA)" <ji...@apache.org> on 2010/02/05 01:02:27 UTC
[jira] Created: (HADOOP-6541) An Interactive Hadoop FS shell
An Interactive Hadoop FS shell
------------------------------
Key: HADOOP-6541
URL: https://issues.apache.org/jira/browse/HADOOP-6541
Project: Hadoop Common
Issue Type: New Feature
Reporter: Aaron Kimball
Assignee: Aaron Kimball
Attachments: HADOOP-6541.patch
A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6541:
----------------------------------
Status: Open (was: Patch Available)
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12829867#action_12829867 ]
Todd Lipcon commented on HADOOP-6541:
-------------------------------------
+1 on the idea, though have not looked at the code. I saw it in action on Aaron's computer and it seems very useful.
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830411#action_12830411 ]
Jakob Homan commented on HADOOP-6541:
-------------------------------------
Haven't looked at the code, but wanted to amend my previous comment to include the caveat that whatever changes are made cannot in any way change the behavior of the current commands, lest we break backwards compatibility. I don't see why this would happen, but such a restriction would help ease any not-yet-expressed resistance.
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6541:
----------------------------------
Attachment: HADOOP-6541.patch
Alex Loddengaard and I wrote a simple command shell that uses jline (a readline implementation) to create a simple shell that includes some handy features like tab completion for HDFS. This supports access to the various FsShell commands in an interactive fashion.
There's a definite need for something like this - executing several hadoop fs ... commands in a bash script or in a regular interactive shell is very time-consuming due to the overhead of starting Java for each such command. Putting all the commands into a single JVM instance is a major win.
When using Hadoop from outside of Java, programs that wish to interact with
the DFS may need to run several commands through the hadoop fs ... interface, which is
very slow.
This shell:
* Supports interactive use of HDFS (with a reasonable notion of a "current directory", etc)
* Supports executing scripts of several commands
External programs could write a script of several DFS operations and batch them up for execution in a single Java process.
This changes some of the methods (e.g., ls()) from FsShell to be public instead of package-public so that their code can be reused here.
We tested CmdShell by running the various commands locally, as well as testing short scripts of the commands included together. Some basic functionality unit tests are also included in this patch.
If a committer could please re-open this issue and take a look at the attached code, I'd appreciate it.
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6541:
----------------------------------
Status: Patch Available (was: Open)
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831074#action_12831074 ]
Aaron Kimball commented on HADOOP-6541:
---------------------------------------
Allen,
You can indeed shell escape with {{! command}}. No notion of a local working dir, unfortunately. (Next task: lcd and lls commands!)
Currently you can't launch a Hadoop job within this shell. I think that's another patch / separate issue -- dealing with the classloader for a jar has enough subtleties associated with it that I punted for the first version. Eventually this shell should be able to do all the {{hadoop jar}}, {{hadoop job}}, {{mradmin/dfsadmin}} etc commands too, but there's a lot of functionality to implement here. All separate tasks in my mind.
In order to establish this code base, my first cut of this system is just focused on implementing filesystem access.
You're right that 'rmr' is a hack. A better argument parsing system is probably necessary moving forward to support things like 'rm -[rf]', 'test', etc. Right now all arguments are treated as filenames in commands like rm; supporting various flags would require using CommandLineParser and other fun. Not impossible, but not first-cut material. Also, jline's tab-completor makes it a bit tricky to handle command-specific {{-arguments}}. I'll need to poke around more to figure out if/how that can be done.
As for cd taking a URL -- technically it already does. But the current design initializes a FileContext and uses its setWorkingDirectory() method to move around, and then uses FsShell to do some operations like deletes (which require globbing). FsShell doesn't allow reinitialization of the FileSystem, so this code checks for changes in the default FS and reinitializes the structures it uses. We should probably refactor FsShell soon to allow it to work on an arbitrary FileContext instead of the one it instantiates at construction time (or instantiate a new FsShell for each operation? That seems unnecessary). But again, refactoring FsShell I think is beyond the scope of this issue.
All of these are great ideas but I think that they deserve separate tasks. There are enough discussion points as to how to implement each of these that I think a single thread would be confusing. Also getting this committed first would allow different people to attack these problems in parallel if they'd like. I'll file these follow-up tasks after this is resolved. Of course, feel free to file your own in the meantime if you'd like :)
Thanks for the suggestions!
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6541:
----------------------------------
Status: Patch Available (was: Open)
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.4.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6541:
----------------------------------
Attachment: HADOOP-6541.2.patch
Here's a new patch that puts this code in {{org.apache.hadoop.shell}}. Running '{{bin/hadoop shell}}' will get you to the prompt.
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6541:
----------------------------------
Attachment: HADOOP-6541.4.patch
I think the -1 javadoc warnings was based on javadoc errors in FsShell. New patch that corrects these.
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.4.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830433#action_12830433 ]
Hadoop QA commented on HADOOP-6541:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12435049/HADOOP-6541.3.patch
against trunk revision 906388.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 14 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/338/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/338/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/338/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/338/console
This message is automatically generated.
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12928644#action_12928644 ]
Hadoop QA commented on HADOOP-6541:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12435198/HADOOP-6541.4.patch
against trunk revision 1031422.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://hudson.apache.org/hudson/job/PreCommit-HADOOP-Build/62//console
This message is automatically generated.
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.4.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831085#action_12831085 ]
Hadoop QA commented on HADOOP-6541:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12435198/HADOOP-6541.4.patch
against trunk revision 907549.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 2 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/343/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/343/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/343/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/343/console
This message is automatically generated.
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.4.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831089#action_12831089 ]
Allen Wittenauer commented on HADOOP-6541:
------------------------------------------
OK, Thanks.
[Hmm. I wonder how much duplication there is with grunt.]
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.4.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830341#action_12830341 ]
Aaron Kimball commented on HADOOP-6541:
---------------------------------------
Jakob,
I'm happy to put this in mainline; I just figured that contrib might be a path of less resistance :) but if there's momentum for it, I'll put together a patch that integrates it a bit more closely, and post that in this module's place.
There's relatively little (if any?) code duplication though. This actually calls methods of FsShell to do operations like move, rm, ls, etc, which already handle globbing on their own. The vast majority of code in this module is specific to interacting with the user, parsing arguments, tab completion, maintaining 'exit status' from commands, etc.
FsShell does deserve a pretty broad refactoring, but I think that's out-of-scope for this issue.
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831046#action_12831046 ]
Allen Wittenauer commented on HADOOP-6541:
------------------------------------------
Cool. It looks like it has a shell escape.
How does one launch a job from within the shell?
Should rm be smart enough to take r and f as parameters? [The fact that rmr is a separate command has always been a mistake in my mind.]
Should there be a test operator? This would be especially useful for -f functionality.
set fs.defaultFS=hdfs://nn.example.com/ --- feels like shades of VMS. :p Any reason cd couldn't take a URL?
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Jeff Hammerbacher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12831365#action_12831365 ]
Jeff Hammerbacher commented on HADOOP-6541:
-------------------------------------------
bq. [Hmm. I wonder how much duplication there is with grunt.]
We all asked this question. Many users only have Common + HDFS + MR, and it was a Hackathon, and the feature was deemed useful enough by all involved to move forward.
It would be nice to have a general shell in Common, or as a separate Apache project; in fact, Carl Steinbach proposed such a thing over at https://issues.apache.org/jira/browse/HIVE-987.
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.4.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830419#action_12830419 ]
Hadoop QA commented on HADOOP-6541:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12435043/HADOOP-6541.2.patch
against trunk revision 906388.
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 14 new or modified tests.
-1 javadoc. The javadoc tool appears to have generated 1 warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 5 new Findbugs warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/336/testReport/
Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/336/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/336/artifact/trunk/build/test/checkstyle-errors.html
Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/336/console
This message is automatically generated.
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6541:
----------------------------------
Status: Open (was: Patch Available)
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.4.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Jakob Homan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12829958#action_12829958 ]
Jakob Homan commented on HADOOP-6541:
-------------------------------------
I really like this idea and jline would be a good way to do it, but I'm not sure a separate contrib module is the way to go. FsShell is definitely due for a refactoring/improvement, and this might be the opportunity to do it. Aaron's comments re: speed issues are correct. In addition, there's no getting around that our current command line tools are rather clunky and could be improved within a shell context. This is a pain point for our users, both new and experienced. But since this is true, why not go all the way and improve FsShell with these features rather than creating a contrib module that has some code duplication and will have to be maintained separately?
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6541:
----------------------------------
Attachment: HADOOP-6541.3.patch
new patch to nail down some findbugs warnings.
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.3.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Allen Wittenauer (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12830198#action_12830198 ]
Allen Wittenauer commented on HADOOP-6541:
------------------------------------------
+1 on stopping the contrib madness for things that should just be in mainline (i'm looking at you, scheduler people)
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (HADOOP-6541) An Interactive Hadoop FS shell
Posted by "Aaron Kimball (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron Kimball updated HADOOP-6541:
----------------------------------
Status: Patch Available (was: Open)
> An Interactive Hadoop FS shell
> ------------------------------
>
> Key: HADOOP-6541
> URL: https://issues.apache.org/jira/browse/HADOOP-6541
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Aaron Kimball
> Assignee: Aaron Kimball
> Attachments: HADOOP-6541.2.patch, HADOOP-6541.patch
>
>
> A shell that allows the user to execute multiple filesystem operations in a single JVM instance at a prompt.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.