You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Prashant Sharma (Created) (JIRA)" <ji...@apache.org> on 2011/09/30 18:20:45 UTC
[jira] [Created] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Docs and Scripts for setting up single node MRV2 cluster.
----------------------------------------------------------
Key: MAPREDUCE-3131
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Reporter: Prashant Sharma
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Attachment: MAPREDUCE-3131.patch
Task Completed !.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Assignee: (was: Prashant Sharma)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 96h
> Remaining Estimate: 72h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120246#comment-13120246 ]
Prashant Sharma commented on MAPREDUCE-3131:
--------------------------------------------
Well above config worked for running multiple datanodes. But for yarn
<property>
<name>yarn.nodemanager.address</name>
<value>0.0.0.0:0</value>
<description>the nodemanagers bind to this port</description>
</property>
Setting port to 0 here has no effect it always tries to bind to 4344 :(
I am sorry I am saying this without properly digging the code but i was getting bind exceptions. while starting more than one yarn NMs
So is it like this feature hasn't been built into yarn yet? In the second case I don't mind doing so.
Thanks.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Status: Patch Available (was: Open)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Attachment: MAPREDUCE-3131.patch
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Status: Patch Available (was: Open)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Status: Patch Available (was: Open)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Subroto Sanyal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118682#comment-13118682 ]
Subroto Sanyal commented on MAPREDUCE-3131:
-------------------------------------------
hi Prashant,
The existing documentation:
1) ~hadoop-trunk/BUILDING.txt
2) ~hadoop-trunk/hadoop-mapreduce-project/INSTALL
clearly tells a user:
a) How to build the project
b) How to get the distributable?
c) How to run the Yarn Cluster?
IMO the existing documentation and scripts are enough.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Reporter: Prashant Sharma
> Fix For: 0.23.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119931#comment-13119931 ]
Prashant Sharma commented on MAPREDUCE-3131:
--------------------------------------------
Hi Arun,
I am not very old with hadoop So I have two questions.
1. Why do we need more than one datanode and nodeManager in a single node setup.?
2. Is there a standard documentation to do it? Because I was getting bind exceptions while trying to start two datanodes. Is it configurable somewhere? Or Can I figure it out myself.
I have already improved the scripts with some enhancements. And was thinking having debug also as an option in it will make them more useful. Again It would be helpful if you can refer me to some standard doc for that.
Thanks
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131478#comment-13131478 ]
Hadoop QA commented on MAPREDUCE-3131:
--------------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12499821/MAPREDUCE-3131.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+0 tests included. The patch appears to be a documentation patch that doesn't require tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 findbugs. The patch appears to introduce 160 new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1080//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1080//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1080//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1080//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1080//console
This message is automatically generated.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 96h
> Remaining Estimate: 72h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Status: Open (was: Patch Available)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Arun C Murthy (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123885#comment-13123885 ]
Arun C Murthy commented on MAPREDUCE-3131:
------------------------------------------
bq. Why do we need more than one datanode and nodeManager in a single node setup.?
Prashanth - we already have docs/scripts for running all of Hadoop on a single cluster - they are the standard bin/hdfs & bin/yarn scripts.
----
Now, one possible enhancement is to go through the default values of configs and check which ones cause it to not work properly on a single node *out of the box*.
If there is none, we declare victory.
If there are some we cannot make default (for e.g. they break tests etc.) we can then check in a conf/singlenode-hdfs-site.xml and conf/singlenode-yarn-site.xml.
----
Another enhancement is to check in configs & scripts with which people can run multiple DataNodes/NodeManagers on a single node so that folks get a feel for a 'real' cluster on a single machine to test/develop/experience.
----
I'm happy to have you do either or both of the above.
However, checking in another set of scripts to run a single node install with 1 NN/DN/RM/NN is just a maintenance overhead.
Makes sense?
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 48h
> Remaining Estimate: 120h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123908#comment-13123908 ]
Prashant Sharma commented on MAPREDUCE-3131:
--------------------------------------------
Arun,
You are right, Adding another set of scripts should have some value more than just starting a cluster without much hassle.
I made these scripts because it made sense for me and gave me the feel of all possible options I can break the cluster. And I had all weird(possibly) Ideas like trying out multiple "Single node" Cluster on the same machine and trying out variety of things(like distcp).
So, In order to bring in value with a new feature. I have already implemented following(I have not submitted the patch though) :
*To setup the cluster home in one step
{noformat}
./run.sh prepare
{noformat}
*Start stop and number of NM or Datanodes.
{noformat}
./run.sh -NM <No of Nodemanager daemons> -D <No of datanode daemons> [start|afresh|stop|kill]
{noformat}
*Debugging and changing between or choosing between daemons for debugging would be breeze.
{noformat}
./run.sh -NM <port no> -RM <port no> -DN <port no> -NN <portno> -JH <port no> debug
{noformat}
I will soon submit the patch with much better code quality. I understand the hassle of maintaining a set of badly written code (Specially scripts). Also I have already written some default configs ideal for single node cluster. I liked the naming conventions.
Now do you see some value.? I would like to have you people throwing in ideas. :)
Thanks
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 48h
> Remaining Estimate: 120h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work stopped] (MAPREDUCE-3131) Docs and Scripts for setting
up single node MRV2 cluster.
Posted by "Prashant Sharma (Work stopped) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Work on MAPREDUCE-3131 stopped by Prashant Sharma.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 96h
> Remaining Estimate: 72h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Status: Patch Available (was: Open)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3131:
-------------------------------------
Status: Open (was: Patch Available)
Prashanth, this is great progress. Sorry, for being late on this one.
Thinking a bit more...
# It would be really nice to not require a new 'run.sh' script. Can we just get bin/hadoop-daemon.sh and bin/yarn-daemon.sh to start/stop multiple DNs/NMs on the same node? Then we can just document this in our site-docs.
# Can we add the -debug functionality to all daemons?
Basically, I'm trying to avoid maintain another set of scripts (run.sh) and put everything in the main scripts/configs...
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 96h
> Remaining Estimate: 72h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13130479#comment-13130479 ]
Prashant Sharma commented on MAPREDUCE-3131:
--------------------------------------------
DEPENDS on https://issues.apache.org/jira/browse/MAPREDUCE-3211
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 48h
> Remaining Estimate: 120h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Status: In Progress (was: Patch Available)
A new feature : Options for debugging daemons.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 48h
> Remaining Estimate: 120h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3131:
-------------------------------------
Issue Type: Improvement (was: Sub-task)
Parent: (was: MAPREDUCE-2890)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 48h
> Remaining Estimate: 120h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work logged] (MAPREDUCE-3131) Docs and Scripts for setting
up single node MRV2 cluster.
Posted by "Prashant Sharma (Work logged) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel\#worklog-{worklog.getId()} ]
Prashant Sharma logged work on MAPREDUCE-3131:
----------------------------------------------
Author: Prashant Sharma
Created on: 20/Oct/11 08:04
Start Date: 20/Oct/11 08:04
Worklog Time Spent: 48h
Issue Time Tracking
-------------------
Worklog Id: (was: 12234)
Time Spent: 96h (was: 48h)
Remaining Estimate: 72h (was: 120h)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 96h
> Remaining Estimate: 72h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Description: Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
Target Version/s: 0.24.0
Affects Version/s: 0.24.0
Fix Version/s: (was: 0.23.0)
0.24.0
Labels: documentation hadoop (was: )
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Status: Open (was: Patch Available)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Attachment: MAPREDUCE-3131.patch
New Improved.
Improved documentation for setting up single Node Cluster using scripts.
Waiting for reviews.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 96h
> Remaining Estimate: 72h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Attachment: MAPREDUCE-3131.patch
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Mahadev konar (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mahadev konar reassigned MAPREDUCE-3131:
----------------------------------------
Assignee: Prashant Sharma
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13142088#comment-13142088 ]
Hadoop QA commented on MAPREDUCE-3131:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12501949/MAPREDUCE-3131.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+0 tests included. The patch appears to be a documentation patch that doesn't require tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1239//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/1239//console
This message is automatically generated.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 96h
> Remaining Estimate: 72h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123911#comment-13123911 ]
Prashant Sharma commented on MAPREDUCE-3131:
--------------------------------------------
Edit in step 2
{noformat}
./run.sh -NM <No of Nodemanager daemons> -D <No of datanode daemons> [start|afresh|stop|kill|status]
{noformat}
Added status option. Currently work in progress. Status will check pid files of process and querry system about the status of those processes. Would be useful in case you run multiple clusters.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 48h
> Remaining Estimate: 120h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Attachment: MAPREDUCE-3131.patch
New improved. This currently depends on fixing of MAPREDUCE-2986. Till then you can remove all the ephemeral port configurations and run only single daemons of all types. Or you may run multiple daemons but dont submit the jobs.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 48h
> Remaining Estimate: 120h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119147#comment-13119147 ]
Hadoop QA commented on MAPREDUCE-3131:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12497425/MAPREDUCE-3131.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+0 tests included. The patch appears to be a documentation patch that doesn't require tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/921//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/921//console
This message is automatically generated.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118167#comment-13118167 ]
Prashant Sharma commented on MAPREDUCE-3131:
--------------------------------------------
I am thinking of good and clean(and easy) way of starting of MRV2 Cluster using scripts. I will soon submit code and documentation for review. Till then take a look at http://github.com/prashantiiith/Scripts/wiki/Setup-required-to-play-with-Hadoop-MRV2
thanks
prashant
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Reporter: Prashant Sharma
> Fix For: 0.23.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122097#comment-13122097 ]
Hadoop QA commented on MAPREDUCE-3131:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12498032/MAPREDUCE-3131.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+0 tests included. The patch appears to be a documentation patch that doesn't require tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/956//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/956//console
This message is automatically generated.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Hadoop QA (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121909#comment-13121909 ]
Hadoop QA commented on MAPREDUCE-3131:
--------------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12497982/MAPREDUCE-3131.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+0 tests included. The patch appears to be a documentation patch that doesn't require tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed unit tests in .
+1 contrib tests. The patch passed contrib unit tests.
Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/955//testReport/
Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/955//console
This message is automatically generated.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Status: Patch Available (was: Open)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Subroto Sanyal (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13119967#comment-13119967 ]
Subroto Sanyal commented on MAPREDUCE-3131:
-------------------------------------------
To run two datanodes on same system update the following properties in hdfs-site.xml
{code}
<property>
<name>dfs.datanode.address</name>
<value>xxx.xxx.xxx.xxx:aaaaaa</value>
<description>
The address where the datanode server will listen to.
If the port is 0 then the server will start on a free port.
</description>
</property>
<property>
<name>dfs.datanode.ipc.address</name>
<value>xxx.xxx.xxx.xxx:zzzzzz</value>
<description>
The datanode ipc server address and port.
If the port is 0 then the server will start on a free port.
</description>
</property>
<property>
<name>dfs.datanode.http.address</name>
<value>xxx.xxx.xxx.xxx:yyyyy</value>
<description>
The datanode http server address and port.
If the port is 0 then the server will start on a free port.
</description>
</property>
{code}
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Priority: Trivial (was: Major)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118872#comment-13118872 ]
Prashant Sharma commented on MAPREDUCE-3131:
--------------------------------------------
Hi Subroto,
Thanks for reading the doc. My effort is not to make them redundant but to provide an easier and generic way of doing it with scripts. I felt the current documentation of getting hdfs to run as daemon and then configuring mapred is spread across different documents. So initially it was difficult for me to get things running and I was looking for something close to what I have done (Please review it again. I have updated it.) And that became the motivation behind putting back to the community. I would be grateful to receive any suggestion and will try to improve it with evolution of the framework.
Thanks again
Prashant
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Reporter: Prashant Sharma
> Fix For: 0.23.0
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Arun C Murthy (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Arun C Murthy updated MAPREDUCE-3131:
-------------------------------------
Status: Open (was: Patch Available)
Prashanth, thanks for taking this up.
No doubt, this is very useful for folks new to the project.
Thinking a bit more, maybe we can add a bin/single-node directory where we check-in these scripts. To make them even more useful you can provide a way to run at least 2 (or more) DataNodes and 2 (or more) NodeManagers.
You can also provide a documentation patch to hadoop-yarn-site/src/site/apt/SingleCluster.apt.vm. (mvn site:site to generate updated docs).
Thoughts?
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13131619#comment-13131619 ]
Prashant Sharma commented on MAPREDUCE-3131:
--------------------------------------------
For convenience read the wiki here.
http://github.com/ScrapCodes/Scripts/wiki/Setup-required-to-play-with-Hadoop-MRV2
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 96h
> Remaining Estimate: 72h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work logged] (MAPREDUCE-3131) Docs and Scripts for setting
up single node MRV2 cluster.
Posted by "Prashant Sharma (Work logged) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel\#worklog-{worklog.getId()} ]
Prashant Sharma logged work on MAPREDUCE-3131:
----------------------------------------------
Author: Prashant Sharma
Created on: 07/Oct/11 19:27
Start Date: 07/Oct/11 19:26
Worklog Time Spent: 48h
Issue Time Tracking
-------------------
Worklog Id: (was: 12135)
Time Spent: 48h
Remaining Estimate: 120h (was: 168h)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 48h
> Remaining Estimate: 120h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Attachment: MAPREDUCE-3131.patch
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Attachment: (was: MAPREDUCE-3131.patch)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 48h
> Remaining Estimate: 120h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Attachment: (was: MAPREDUCE-3131.patch)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 48h
> Remaining Estimate: 120h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13121856#comment-13121856 ]
Prashant Sharma commented on MAPREDUCE-3131:
--------------------------------------------
Please ignore the above comment. There was a problem with using ephemeral ports for the property "mapreduce.shuffle.port" which got fixed after an svn update. :)
Now the task is complete with all features.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Status: Open (was: Patch Available)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Remaining Estimate: 168h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Prashant Sharma updated MAPREDUCE-3131:
---------------------------------------
Status: Patch Available (was: Open)
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 96h
> Remaining Estimate: 72h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3131) Docs and Scripts for setting up
single node MRV2 cluster.
Posted by "Prashant Sharma (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3131?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145044#comment-13145044 ]
Prashant Sharma commented on MAPREDUCE-3131:
--------------------------------------------
Hadoop Mapreduce has many users exploiting its usefulness and simplicity. Some of them are students(or non industrial users) and need it the power of mapreduce for solving their research problems or more commonly even more trivial problems(like generating indexes or just run WordCount ;) ).
So was thinking a bit further - if I can make it (very) easy to setup a trivial(unsecured and with default parameters) cluster(Created by sharing desktops resources - A typical scenario among students). Our folks will have a by pass to figuring out how to setup.
There are two approaches to this.
1. make a tar and copy it around. or( Rsync the cluster home)
2. use deb/rpm approach.
deb and rpm setup will be more ideal but a bit nontrivial to have and will make steps more cumbersome. But doing without package management simplifies "a few" things. And since users of our trivial clusters are less particular about the "Ideal" approach I propose the later.
> Docs and Scripts for setting up single node MRV2 cluster.
> ----------------------------------------------------------
>
> Key: MAPREDUCE-3131
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3131
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.24.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
> Priority: Trivial
> Labels: documentation, hadoop
> Fix For: 0.24.0
>
> Attachments: MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch, MAPREDUCE-3131.patch
>
> Original Estimate: 168h
> Time Spent: 96h
> Remaining Estimate: 72h
>
> Scripts to run a single node cluster with a default configuration. Takes care of running all the daemons including hdfs and yarn.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira