You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-issues@hadoop.apache.org by "Hudson (Commented) (JIRA)" <ji...@apache.org> on 2011/10/01 00:23:45 UTC
[jira] [Commented] (MAPREDUCE-3050) YarnScheduler needs to expose
Resource Usage Information
[ https://issues.apache.org/jira/browse/MAPREDUCE-3050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13118515#comment-13118515 ]
Hudson commented on MAPREDUCE-3050:
-----------------------------------
Integrated in Hadoop-Hdfs-trunk-Commit #1072 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/1072/])
MAPREDUCE-3050. Add ability to get resource usage information for applications and nodes. Contributed by Robert Evans.
acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1177859
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/QueueMetrics.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerAppReport.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerNodeReport.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/YarnScheduler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestFifoScheduler.java
> YarnScheduler needs to expose Resource Usage Information
> --------------------------------------------------------
>
> Key: MAPREDUCE-3050
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3050
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager
> Affects Versions: 0.23.0, 0.24.0
> Reporter: Robert Joseph Evans
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MR-3050.txt, MR-3050.txt, MR-3050.txt, MR-3050.txt, MR-3050.txt, MR-3050.txt, MR-3050.txt
>
>
> Before the recent refactor The nodes had information in them about how much resources they were using. This information is not hidden inside SchedulerNode. Similarly resource usage information about an application, or in aggregate is only available through the Scheduler and there is not interface to pull it out.
> We need to expose APIs to get Resource and Container information from the scheduler, in aggregate across the entire cluster, per application, per node, and ideally also per queue if applicable (although there are no JIRAs I am aware of that need this right now).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira