You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Arun C Murthy (JIRA)" <ji...@apache.org> on 2011/08/28 07:03:38 UTC
[jira] [Issue Comment Edited] (MAPREDUCE-2889) Add docs for writing
new application frameworks
[ https://issues.apache.org/jira/browse/MAPREDUCE-2889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13092425#comment-13092425 ]
Arun C Murthy edited comment on MAPREDUCE-2889 at 8/28/11 5:03 AM:
-------------------------------------------------------------------
Some info to help folks get started:
It should be fairly trivial to ask for n resources, launch containers and then get them to run.
Some of the tests like [TestFifoScheduler|http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java?view=markup], and the mock [Application|http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/Application.java?view=markup] used for testing should help give you an idea too.
The interfaces you'd most like be concerned with are:
[ClientRMProtocol|http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java?view=markup] - Protocol for job-client (launching program from gateway) to talk to RM to launch a new application i.e. application master itself
[AMRMProtocol|http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/AMRMProtocol.java?view=markup] - Protocol for AM to use to register/stop itself and get resources from the Scheduler
[ContainerManager|http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ContainerManager.java?view=markup] - Protocol for AM to talk to NodeManager to start/stop containers
Sharad's presentation at the HUG on writing a custom AM: [hadoop_contributors_meet_07_01_2011.pdf|https://issues.apache.org/jira/secure/attachment/12485267/hadoop_contributors_meet_07_01_2011.pdf]
was (Author: acmurthy):
Some info to help folks get started:
It should be fairly trivial to ask for n resources, launch containers and then get them to run.
Some of the tests like [TestFifoScheduler|http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/TestFifoScheduler.java?view=markup], and the mock [Application|http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/Application.java?view=markup] used for testing should help give you an idea too.
The interfaces you'd most like be concerned with are:
[ClientRMProtocol|http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ClientRMProtocol.java?view=markup] - Protocol for job-client (launching program from gateway) to talk to RM to launch a new application i.e. application master itself
[AMRMProtocol|http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/AMRMProtocol.java?view=markup] - Protocol for AM to use to register/stop itself and get resources from the Scheduler
[ContainerManager|http://svn.apache.org/viewvc/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/ContainerManager.java?view=markup] - Protocol for AM to talk to NodeManager to start/stop containers
Sharad's presentation at the HUG on writing a custom AM: hadoop_contributors_meet_07_01_2011.pdf
> Add docs for writing new application frameworks
> -----------------------------------------------
>
> Key: MAPREDUCE-2889
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2889
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: documentation, mrv2
> Affects Versions: 0.23.0
> Reporter: Arun C Murthy
> Priority: Critical
> Fix For: 0.23.0
>
>
> We need to add docs for writing new application frameworks, including examples, javadocs and sample apps.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira