You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-commits@hadoop.apache.org by tu...@apache.org on 2013/01/18 01:37:46 UTC
svn commit: r1434991 - in
/hadoop/common/branches/branch-2/hadoop-yarn-project: CHANGES.txt
hadoop-yarn/hadoop-yarn-site/src/site/apt/FairScheduler.apt.vm
Author: tucu
Date: Fri Jan 18 00:37:46 2013
New Revision: 1434991
URL: http://svn.apache.org/viewvc?rev=1434991&view=rev
Log:
YARN-331. Fill in missing fair scheduler documentation. (sandyr via tucu)
Modified:
hadoop/common/branches/branch-2/hadoop-yarn-project/CHANGES.txt
hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/FairScheduler.apt.vm
Modified: hadoop/common/branches/branch-2/hadoop-yarn-project/CHANGES.txt
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/CHANGES.txt?rev=1434991&r1=1434990&r2=1434991&view=diff
==============================================================================
--- hadoop/common/branches/branch-2/hadoop-yarn-project/CHANGES.txt (original)
+++ hadoop/common/branches/branch-2/hadoop-yarn-project/CHANGES.txt Fri Jan 18 00:37:46 2013
@@ -82,6 +82,8 @@ Release 2.0.3-alpha - Unreleased
YARN-170. Change NodeManager stop to be reentrant. (Sandy Ryza via vinodkv)
+ YARN-331. Fill in missing fair scheduler documentation. (sandyr via tucu)
+
OPTIMIZATIONS
BUG FIXES
Modified: hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/FairScheduler.apt.vm
URL: http://svn.apache.org/viewvc/hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/FairScheduler.apt.vm?rev=1434991&r1=1434990&r2=1434991&view=diff
==============================================================================
--- hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/FairScheduler.apt.vm (original)
+++ hadoop/common/branches/branch-2/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/FairScheduler.apt.vm Fri Jan 18 00:37:46 2013
@@ -132,20 +132,45 @@ Hadoop MapReduce Next Generation - Fair
* Whether to use the username associated with the allocation as the default
queue name, in the event that a queue name is not specified. If this is set
to "false" or unset, all jobs have a shared default queue, called "default".
+ Defaults to true.
* <<<yarn.scheduler.fair.preemption>>>
* Whether to use preemption. Note that preemption is experimental in the current
- version.
+ version. Defaults to false.
* <<<yarn.scheduler.fair.sizebasedweight>>>
* Whether to assign shares to individual apps based on their size, rather than
- providing an equal share to all apps regardless of size.
+ providing an equal share to all apps regardless of size. Defaults to false.
* <<<yarn.scheduler.fair.assignmultiple>>>
- * Whether to allow multiple container assignments in one heartbeat.
+ * Whether to allow multiple container assignments in one heartbeat. Defaults
+ to false.
+
+ * <<<yarn.scheduler.fair.max.assign>>>
+
+ * If assignmultiple is true, the maximum amount of containers that can be
+ assigned in one heartbeat. Defaults to -1, which sets no limit.
+
+ * <<<locality.threshold.node>>>
+
+ * For applications that request containers on particular nodes, the number of
+ scheduling opportunities since the last container assignment to wait before
+ accepting a placement on another node. Expressed as a float between 0 and 1,
+ which, as a fraction of the cluster size, is the number of scheduling
+ opportunities to pass up. The default value of -1.0 means don't pass up any
+ scheduling opportunities.
+
+ * <<<locality.threshold.rack>>>
+
+ * For applications that request containers on particular racks, the number of
+ scheduling opportunities since the last container assignment to wait before
+ accepting a placement on another rack. Expressed as a float between 0 and 1,
+ which, as a fraction of the cluster size, is the number of scheduling
+ opportunities to pass up. The default value of -1.0 means don't pass up any
+ scheduling opportunities.
Allocation file format
@@ -166,6 +191,14 @@ Allocation file format
* schedulingMode: either "fifo" or "fair" depending on the in-queue scheduling
policy desired
+ * aclSubmitApps: a list of users that can submit apps to the queue. A (default)
+ value of "*" means that any users can submit apps. A queue inherits the ACL of
+ its parent, so if a queue2 descends from queue1, and user1 is in queue1's ACL,
+ and user2 is in queue2's ACL, then both users may submit to queue2.
+
+ * minSharePreemptionTimeout: number of seconds the queue is under its minimum share
+ before it will try to preempt containers to take resources from other queues.
+
* <<User elements>>, which represent settings governing the behavior of individual
users. They can contain a single property: maxRunningApps, a limit on the
number of running apps for a particular user.
@@ -173,6 +206,10 @@ Allocation file format
* <<A userMaxAppsDefault element>>, which sets the default running app limit
for any users whose limit is not otherwise specified.
+ * <<A fairSharePreemptionTimeout element>>, number of seconds a queue is under
+ its fair share before it will try to preempt containers to take resources from
+ other queues.
+
An example allocation file is given here:
---