You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by "Johan Oskarson (JIRA)" <ji...@apache.org> on 2006/05/16 13:18:05 UTC
[jira] Created: (HADOOP-222) Set replication from dfsshell
Set replication from dfsshell
-----------------------------
Key: HADOOP-222
URL: http://issues.apache.org/jira/browse/HADOOP-222
Project: Hadoop
Type: New Feature
Components: dfs
Versions: 0.3
Reporter: Johan Oskarson
Priority: Trivial
Added ability to set replication for a directory/file from the command line.
Not heavily tested...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-222) Set replication from dfsshell
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-222?page=comments#action_12412249 ]
Konstantin Shvachko commented on HADOOP-222:
--------------------------------------------
This is very useful.
Some things that I have noticed.
- Usage info might be useful.
Right now it just throws ArrayIndexOutOfBoundsException if you do not specify the path.
Or some other exception if you forget to provide the replication number. Try
hadoop dfs -setrep
hadoop dfs -setrep 3
hadoop dfs -setrep /user
- I think the 2 commands should be combined into one like
hadoop dfs -setrep [-R] <path>
- Should we use abbreviations like "setrep" or full names?
see HADOOP-226 (4) about necessity for a general DFSShell convention.
> Set replication from dfsshell
> -----------------------------
>
> Key: HADOOP-222
> URL: http://issues.apache.org/jira/browse/HADOOP-222
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Versions: 0.3
> Reporter: Johan Oskarson
> Priority: Trivial
> Attachments: setrep.patch
>
> Added ability to set replication for a directory/file from the command line.
> Not heavily tested...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-222) Set replication from dfsshell
Posted by "Johan Oskarson (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-222?page=comments#action_12412317 ]
Johan Oskarson commented on HADOOP-222:
---------------------------------------
Yeah, I know there are issues like that, this is mostly a quick hack I needed to set the replication in order to save space.
My question is, should I fix them or should I wait for a complete DFSShell rehaul with Commons CLI?
I currently do not have time to rewrite the DFSShell myself, but perhaps if there's been no attempt made in a few weeks I might give it a try.
You're absolutely right that they should be combined into one, also related to the whole issue with proper command line parsing etc.
Same thing with -ls and -lsr I suppose.
Thanks for the comments.
> Set replication from dfsshell
> -----------------------------
>
> Key: HADOOP-222
> URL: http://issues.apache.org/jira/browse/HADOOP-222
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Versions: 0.3
> Reporter: Johan Oskarson
> Priority: Trivial
> Attachments: setrep.patch
>
> Added ability to set replication for a directory/file from the command line.
> Not heavily tested...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-222) Set replication from dfsshell
Posted by "Johan Oskarson (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-222?page=comments#action_12413746 ]
Johan Oskarson commented on HADOOP-222:
---------------------------------------
I finally had some time to fix the issues in the previous patch.
Posting new patch that shouldn't throw any nasty exceptions if the user doesn't provide all the needed input
> Set replication from dfsshell
> -----------------------------
>
> Key: HADOOP-222
> URL: http://issues.apache.org/jira/browse/HADOOP-222
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Versions: 0.3
> Reporter: Johan Oskarson
> Priority: Trivial
> Attachments: setrep.patch
>
> Added ability to set replication for a directory/file from the command line.
> Not heavily tested...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-222) Set replication from dfsshell
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-222?page=comments#action_12412407 ]
Konstantin Shvachko commented on HADOOP-222:
--------------------------------------------
Johan, I would vote for your patch if you fix it.
> Set replication from dfsshell
> -----------------------------
>
> Key: HADOOP-222
> URL: http://issues.apache.org/jira/browse/HADOOP-222
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Versions: 0.3
> Reporter: Johan Oskarson
> Priority: Trivial
> Attachments: setrep.patch
>
> Added ability to set replication for a directory/file from the command line.
> Not heavily tested...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-222) Set replication from dfsshell
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-222?page=comments#action_12413863 ]
Konstantin Shvachko commented on HADOOP-222:
--------------------------------------------
+1
> Set replication from dfsshell
> -----------------------------
>
> Key: HADOOP-222
> URL: http://issues.apache.org/jira/browse/HADOOP-222
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Versions: 0.3
> Reporter: Johan Oskarson
> Priority: Trivial
> Attachments: setrep-v2.patch, setrep.patch
>
> Added ability to set replication for a directory/file from the command line.
> Not heavily tested...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-222) Set replication from dfsshell
Posted by "Johan Oskarson (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-222?page=all ]
Johan Oskarson updated HADOOP-222:
----------------------------------
Attachment: setrep.patch
> Set replication from dfsshell
> -----------------------------
>
> Key: HADOOP-222
> URL: http://issues.apache.org/jira/browse/HADOOP-222
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Versions: 0.3
> Reporter: Johan Oskarson
> Priority: Trivial
> Attachments: setrep.patch
>
> Added ability to set replication for a directory/file from the command line.
> Not heavily tested...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-222) Set replication from dfsshell
Posted by "Johan Oskarson (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-222?page=all ]
Johan Oskarson updated HADOOP-222:
----------------------------------
Attachment: setrep-v2.patch
New version with error handling etc
> Set replication from dfsshell
> -----------------------------
>
> Key: HADOOP-222
> URL: http://issues.apache.org/jira/browse/HADOOP-222
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Versions: 0.3
> Reporter: Johan Oskarson
> Priority: Trivial
> Attachments: setrep-v2.patch, setrep.patch
>
> Added ability to set replication for a directory/file from the command line.
> Not heavily tested...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Closed: (HADOOP-222) Set replication from dfsshell
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-222?page=all ]
Doug Cutting closed HADOOP-222:
-------------------------------
> Set replication from dfsshell
> -----------------------------
>
> Key: HADOOP-222
> URL: http://issues.apache.org/jira/browse/HADOOP-222
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Versions: 0.3.0
> Reporter: Johan Oskarson
> Priority: Trivial
> Fix For: 0.3.0
> Attachments: setrep-v2.patch, setrep.patch
>
> Added ability to set replication for a directory/file from the command line.
> Not heavily tested...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
[jira] Resolved: (HADOOP-222) Set replication from dfsshell
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-222?page=all ]
Doug Cutting resolved HADOOP-222:
---------------------------------
Fix Version: 0.3
Resolution: Fixed
I just committed this. Thanks, Johan!
> Set replication from dfsshell
> -----------------------------
>
> Key: HADOOP-222
> URL: http://issues.apache.org/jira/browse/HADOOP-222
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Versions: 0.3
> Reporter: Johan Oskarson
> Priority: Trivial
> Fix For: 0.3
> Attachments: setrep-v2.patch, setrep.patch
>
> Added ability to set replication for a directory/file from the command line.
> Not heavily tested...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira
Re: [jira] Commented: (HADOOP-222) Set replication from dfsshell
Posted by Konstantin Shvachko <sh...@yahoo-inc.com>.
The bug number is
[jira] Created: (HADOOP-231) DFSShell Improvement wish list
It has a link on this and other related issues.
Better to keep them in one place, as Doug proposed.
Eric Baldeschwieler wrote:
> Good ideas, you should file a new bug on this.
>
> On May 18, 2006, at 3:32 AM, Andrzej Bialecki (JIRA) wrote:
>
>> [ http://issues.apache.org/jira/browse/HADOOP-222?
>> page=comments#action_12412326 ]
>>
>> Andrzej Bialecki commented on HADOOP-222:
>> ------------------------------------------
>>
>> Other things that I would love to see added to DFSShell (to make it
>> more shell-like). They are not such a big deal to implement, but
>> they would increase the usefulness of this class tremendously (note:
>> this is orthogonal to the issue of using Commons CLI, which I'd love
>> to see too):
>>
>> * globbing and multi-argument cp/mv/rm/ls, using the standard shell
>> metacharacters
>>
>> * interactive mode: ability to issue several commands in the same
>> session, while keeping track of the context (such as pwd)
>>
>> * scripts: nothing fancy, just the ability to execute several
>> commands recorded in a text file.
>>
>> The scriptability and interactive mode, and a host of other useful
>> features we can throw in cheaply by using Rhino (http://
>> www.mozilla.org/rhino/shell.html).
>>
>>> Set replication from dfsshell
>>> -----------------------------
>>>
>>> Key: HADOOP-222
>>> URL: http://issues.apache.org/jira/browse/HADOOP-222
>>> Project: Hadoop
>>> Type: New Feature
>>
>>
>>> Components: dfs
>>> Versions: 0.3
>>> Reporter: Johan Oskarson
>>> Priority: Trivial
>>> Attachments: setrep.patch
>>>
>>> Added ability to set replication for a directory/file from the
>>> command line.
>>> Not heavily tested...
>>
>>
>> --
>> This message is automatically generated by JIRA.
>> -
>> If you think it was sent incorrectly contact one of the administrators:
>> http://issues.apache.org/jira/secure/Administrators.jspa
>> -
>> For more information on JIRA, see:
>> http://www.atlassian.com/software/jira
>>
>
>
>
Re: [jira] Commented: (HADOOP-222) Set replication from dfsshell
Posted by Eric Baldeschwieler <er...@yahoo-inc.com>.
Good ideas, you should file a new bug on this.
On May 18, 2006, at 3:32 AM, Andrzej Bialecki (JIRA) wrote:
> [ http://issues.apache.org/jira/browse/HADOOP-222?
> page=comments#action_12412326 ]
>
> Andrzej Bialecki commented on HADOOP-222:
> ------------------------------------------
>
> Other things that I would love to see added to DFSShell (to make it
> more shell-like). They are not such a big deal to implement, but
> they would increase the usefulness of this class tremendously
> (note: this is orthogonal to the issue of using Commons CLI, which
> I'd love to see too):
>
> * globbing and multi-argument cp/mv/rm/ls, using the standard shell
> metacharacters
>
> * interactive mode: ability to issue several commands in the same
> session, while keeping track of the context (such as pwd)
>
> * scripts: nothing fancy, just the ability to execute several
> commands recorded in a text file.
>
> The scriptability and interactive mode, and a host of other useful
> features we can throw in cheaply by using Rhino (http://
> www.mozilla.org/rhino/shell.html).
>
>> Set replication from dfsshell
>> -----------------------------
>>
>> Key: HADOOP-222
>> URL: http://issues.apache.org/jira/browse/HADOOP-222
>> Project: Hadoop
>> Type: New Feature
>
>> Components: dfs
>> Versions: 0.3
>> Reporter: Johan Oskarson
>> Priority: Trivial
>> Attachments: setrep.patch
>>
>> Added ability to set replication for a directory/file from the
>> command line.
>> Not heavily tested...
>
> --
> This message is automatically generated by JIRA.
> -
> If you think it was sent incorrectly contact one of the
> administrators:
> http://issues.apache.org/jira/secure/Administrators.jspa
> -
> For more information on JIRA, see:
> http://www.atlassian.com/software/jira
>
[jira] Commented: (HADOOP-222) Set replication from dfsshell
Posted by "Andrzej Bialecki (JIRA)" <ji...@apache.org>.
[ http://issues.apache.org/jira/browse/HADOOP-222?page=comments#action_12412326 ]
Andrzej Bialecki commented on HADOOP-222:
------------------------------------------
Other things that I would love to see added to DFSShell (to make it more shell-like). They are not such a big deal to implement, but they would increase the usefulness of this class tremendously (note: this is orthogonal to the issue of using Commons CLI, which I'd love to see too):
* globbing and multi-argument cp/mv/rm/ls, using the standard shell metacharacters
* interactive mode: ability to issue several commands in the same session, while keeping track of the context (such as pwd)
* scripts: nothing fancy, just the ability to execute several commands recorded in a text file.
The scriptability and interactive mode, and a host of other useful features we can throw in cheaply by using Rhino (http://www.mozilla.org/rhino/shell.html).
> Set replication from dfsshell
> -----------------------------
>
> Key: HADOOP-222
> URL: http://issues.apache.org/jira/browse/HADOOP-222
> Project: Hadoop
> Type: New Feature
> Components: dfs
> Versions: 0.3
> Reporter: Johan Oskarson
> Priority: Trivial
> Attachments: setrep.patch
>
> Added ability to set replication for a directory/file from the command line.
> Not heavily tested...
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
http://www.atlassian.com/software/jira