You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-commits@hadoop.apache.org by ji...@apache.org on 2015/05/12 00:54:56 UTC
[22/50] [abbrv] hadoop git commit: YARN-1050. Document the Fair
Scheduler REST API. (Kenji Kikushima and Roman Shaposhnik via kasha)
YARN-1050. Document the Fair Scheduler REST API. (Kenji Kikushima and Roman Shaposhnik via kasha)
Project: http://git-wip-us.apache.org/repos/asf/hadoop/repo
Commit: http://git-wip-us.apache.org/repos/asf/hadoop/commit/96473bdc
Tree: http://git-wip-us.apache.org/repos/asf/hadoop/tree/96473bdc
Diff: http://git-wip-us.apache.org/repos/asf/hadoop/diff/96473bdc
Branch: refs/heads/HDFS-7240
Commit: 96473bdc2b090c13708fd467fd626621ef1d47eb
Parents: 59995ce
Author: Karthik Kambatla <ka...@apache.org>
Authored: Fri May 8 16:31:05 2015 -0700
Committer: Karthik Kambatla <ka...@apache.org>
Committed: Fri May 8 16:31:14 2015 -0700
----------------------------------------------------------------------
hadoop-yarn-project/CHANGES.txt | 2 +
.../src/site/markdown/ResourceManagerRest.md | 306 ++++++++++++++++++-
2 files changed, 307 insertions(+), 1 deletion(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/hadoop/blob/96473bdc/hadoop-yarn-project/CHANGES.txt
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/CHANGES.txt b/hadoop-yarn-project/CHANGES.txt
index 02226db..9b75474 100644
--- a/hadoop-yarn-project/CHANGES.txt
+++ b/hadoop-yarn-project/CHANGES.txt
@@ -209,6 +209,8 @@ Release 2.8.0 - UNRELEASED
YARN-2331. Distinguish shutdown during supervision vs. shutdown for
rolling upgrade. (Jason Lowe via xgong)
+ YARN-1050. Document the Fair Scheduler REST API. (Kenji Kikushima and Roman Shaposhnik via kasha)
+
OPTIMIZATIONS
YARN-3339. TestDockerContainerExecutor should pull a single image and not
http://git-wip-us.apache.org/repos/asf/hadoop/blob/96473bdc/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md
----------------------------------------------------------------------
diff --git a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md
index b1591bb..d5df8ba 100644
--- a/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md
+++ b/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/ResourceManagerRest.md
@@ -278,7 +278,7 @@ Response Body:
Cluster Scheduler API
---------------------
-A scheduler resource contains information about the current scheduler configured in a cluster. It currently supports both the Fifo and Capacity Scheduler. You will get different information depending on which scheduler is configured so be sure to look at the type information.
+A scheduler resource contains information about the current scheduler configured in a cluster. It currently supports the Fifo, Capacity and Fair Scheduler. You will get different information depending on which scheduler is configured so be sure to look at the type information.
### URI
@@ -984,6 +984,310 @@ Response Body:
</scheduler>
```
+### Fair Scheduler API
+
+### Elements of the *schedulerInfo* object
+
+| Item | Data Type | Description |
+|:---- |:---- |:---- |
+| type | string | Scheduler type - fairScheduler |
+| rootQueue | The root queue object | A collection of root queue resources |
+
+### Elements of the root queue object
+
+| Item | Data Type | Description |
+|:---- |:---- |:---- |
+| maxApps | int | The maximum number of applications the queue can have |
+| minResources | A single resource object | The configured minimum resources that are guaranteed to the queue |
+| maxResources | A single resource object | The configured maximum resources that are allowed to the queue |
+| usedResources | A single resource object | The sum of resources allocated to containers within the queue |
+| fairResources | A single resource object | The queue's fair share of resources |
+| clusterResources | A single resource object | The capacity of the cluster |
+| queueName | string | The name of the queue |
+| schedulingPolicy | string | The name of the scheduling policy used by the queue |
+| childQueues | array of queues(JSON)/queue objects(XML) | A collection of sub-queue information |
+
+### Elements of the queues object for a Leaf queue - contains all elements in parent plus the following
+
+| Item | Data Type | Description |
+|:---- |:---- |:---- |
+| type | string | type of the queue - fairSchedulerLeafQueueInfo |
+| numActiveApps | int | The number of active applications in this queue |
+| numPendingApps | int | The number of pending applications in this queue |
+
+### Elements of the resource object for resourcesUsed in queues
+
+| Item | Data Type | Description |
+|:---- |:---- |:---- |
+| memory | int | The amount of memory used (in MB) |
+| vCores | int | The number of virtual cores |
+
+#### Response Examples
+
+**JSON response**
+
+HTTP Request:
+
+ GET http://<rm http address:port>/ws/v1/cluster/scheduler
+
+Response Header:
+
+ HTTP/1.1 200 OK
+ Content-Type: application/json
+ Transfer-Encoding: chunked
+ Server: Jetty(6.1.26)
+
+Response Body:
+
+```json
+{
+ "scheduler": {
+ "schedulerInfo": {
+ "rootQueue": {
+ "childQueues": [
+ {
+ "clusterResources": {
+ "memory": 8192,
+ "vCores": 8
+ },
+ "fairResources": {
+ "memory": 0,
+ "vCores": 0
+ },
+ "maxApps": 2147483647,
+ "maxResources": {
+ "memory": 8192,
+ "vCores": 8
+ },
+ "minResources": {
+ "memory": 0,
+ "vCores": 0
+ },
+ "numActiveApps": 0,
+ "numPendingApps": 0,
+ "queueName": "root.default",
+ "schedulingPolicy": "fair",
+ "type": "fairSchedulerLeafQueueInfo",
+ "usedResources": {
+ "memory": 0,
+ "vCores": 0
+ }
+ },
+ {
+ "childQueues": {
+ "clusterResources": {
+ "memory": 8192,
+ "vCores": 8
+ },
+ "fairResources": {
+ "memory": 10000,
+ "vCores": 0
+ },
+ "maxApps": 2147483647,
+ "maxResources": {
+ "memory": 8192,
+ "vCores": 8
+ },
+ "minResources": {
+ "memory": 5000,
+ "vCores": 0
+ },
+ "numActiveApps": 0,
+ "numPendingApps": 0,
+ "queueName": "root.sample_queue.sample_sub_queue",
+ "schedulingPolicy": "fair",
+ "type": [
+ "fairSchedulerLeafQueueInfo"
+ ],
+ "usedResources": {
+ "memory": 0,
+ "vCores": 0
+ }
+ },
+ "clusterResources": {
+ "memory": 8192,
+ "vCores": 8
+ },
+ "fairResources": {
+ "memory": 10000,
+ "vCores": 0
+ },
+ "maxApps": 50,
+ "maxResources": {
+ "memory": 8192,
+ "vCores": 0
+ },
+ "minResources": {
+ "memory": 10000,
+ "vCores": 0
+ },
+ "queueName": "root.sample_queue",
+ "schedulingPolicy": "fair",
+ "usedResources": {
+ "memory": 0,
+ "vCores": 0
+ }
+ }
+ ],
+ "clusterResources": {
+ "memory": 8192,
+ "vCores": 8
+ },
+ "fairResources": {
+ "memory": 8192,
+ "vCores": 8
+ },
+ "maxApps": 2147483647,
+ "maxResources": {
+ "memory": 8192,
+ "vCores": 8
+ },
+ "minResources": {
+ "memory": 0,
+ "vCores": 0
+ },
+ "queueName": "root",
+ "schedulingPolicy": "fair",
+ "usedResources": {
+ "memory": 0,
+ "vCores": 0
+ }
+ },
+ "type": "fairScheduler"
+ }
+ }
+}
+```
+
+**XML response**
+
+HTTP Request:
+
+ GET http://<rm http address:port>/ws/v1/cluster/scheduler
+ Accept: application/xml
+
+Response Header:
+
+ HTTP/1.1 200 OK
+ Content-Type: application/xml
+ Content-Length: 2321
+ Server: Jetty(6.1.26)
+
+Response Body:
+
+```xml
+<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
+<scheduler>
+ <schedulerInfo xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="fairScheduler">
+ <rootQueue>
+ <maxApps>2147483647</maxApps>
+ <minResources>
+ <memory>0</memory>
+ <vCores>0</vCores>
+ </minResources>
+ <maxResources>
+ <memory>8192</memory>
+ <vCores>8</vCores>
+ </maxResources>
+ <usedResources>
+ <memory>0</memory>
+ <vCores>0</vCores>
+ </usedResources>
+ <fairResources>
+ <memory>8192</memory>
+ <vCores>8</vCores>
+ </fairResources>
+ <clusterResources>
+ <memory>8192</memory>
+ <vCores>8</vCores>
+ </clusterResources>
+ <queueName>root</queueName>
+ <schedulingPolicy>fair</schedulingPolicy>
+ <childQueues xsi:type="fairSchedulerLeafQueueInfo">
+ <maxApps>2147483647</maxApps>
+ <minResources>
+ <memory>0</memory>
+ <vCores>0</vCores>
+ </minResources>
+ <maxResources>
+ <memory>8192</memory>
+ <vCores>8</vCores>
+ </maxResources>
+ <usedResources>
+ <memory>0</memory>
+ <vCores>0</vCores>
+ </usedResources>
+ <fairResources>
+ <memory>0</memory>
+ <vCores>0</vCores>
+ </fairResources>
+ <clusterResources>
+ <memory>8192</memory>
+ <vCores>8</vCores>
+ </clusterResources>
+ <queueName>root.default</queueName>
+ <schedulingPolicy>fair</schedulingPolicy>
+ <numPendingApps>0</numPendingApps>
+ <numActiveApps>0</numActiveApps>
+ </childQueues>
+ <childQueues>
+ <maxApps>50</maxApps>
+ <minResources>
+ <memory>10000</memory>
+ <vCores>0</vCores>
+ </minResources>
+ <maxResources>
+ <memory>8192</memory>
+ <vCores>0</vCores>
+ </maxResources>
+ <usedResources>
+ <memory>0</memory>
+ <vCores>0</vCores>
+ </usedResources>
+ <fairResources>
+ <memory>10000</memory>
+ <vCores>0</vCores>
+ </fairResources>
+ <clusterResources>
+ <memory>8192</memory>
+ <vCores>8</vCores>
+ </clusterResources>
+ <queueName>root.sample_queue</queueName>
+ <schedulingPolicy>fair</schedulingPolicy>
+ <childQueues xsi:type="fairSchedulerLeafQueueInfo">
+ <maxApps>2147483647</maxApps>
+ <minResources>
+ <memory>5000</memory>
+ <vCores>0</vCores>
+ </minResources>
+ <maxResources>
+ <memory>8192</memory>
+ <vCores>8</vCores>
+ </maxResources>
+ <usedResources>
+ <memory>0</memory>
+ <vCores>0</vCores>
+ </usedResources>
+ <fairResources>
+ <memory>10000</memory>
+ <vCores>0</vCores>
+ </fairResources>
+ <clusterResources>
+ <memory>8192</memory>
+ <vCores>8</vCores>
+ </clusterResources>
+ <queueName>root.sample_queue.sample_sub_queue</queueName>
+ <schedulingPolicy>fair</schedulingPolicy>
+ <numPendingApps>0</numPendingApps>
+ <numActiveApps>0</numActiveApps>
+ </childQueues>
+ </childQueues>
+ </rootQueue>
+ </schedulerInfo>
+</scheduler>
+```
+
+
Cluster Applications API
------------------------