You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by gi...@apache.org on 2019/08/13 20:28:29 UTC
[mesos-site] branch asf-site updated: Updated the website built
from mesos SHA: 11a7e7153.
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/mesos-site.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 999caae Updated the website built from mesos SHA: 11a7e7153.
999caae is described below
commit 999caae4902338275b0fe35c7f5abbc50673b7e3
Author: jenkins <bu...@apache.org>
AuthorDate: Tue Aug 13 20:28:23 2019 +0000
Updated the website built from mesos SHA: 11a7e7153.
---
content/api/latest/c++/functions_func_w.html | 4 +-
content/api/latest/c++/functions_u.html | 6 +-
content/api/latest/c++/functions_w.html | 2 +-
.../api/latest/c++/hierarchical_8hpp_source.html | 2 +-
.../latest/operator-http-api/index.html | 65 +--
content/documentation/latest/quota/index.html | 533 +++++++--------------
content/documentation/operator-http-api/index.html | 65 +--
content/documentation/quota/index.html | 533 +++++++--------------
8 files changed, 416 insertions(+), 794 deletions(-)
diff --git a/content/api/latest/c++/functions_func_w.html b/content/api/latest/c++/functions_func_w.html
index 0948e01..006d7a5 100644
--- a/content/api/latest/c++/functions_func_w.html
+++ b/content/api/latest/c++/functions_func_w.html
@@ -147,7 +147,7 @@
: <a class="el" href="classDuration.html#ae98a411bf78d0ab537c021bf3dbd01b0">Duration</a>
</li>
<li>Weeks()
-: <a class="el" href="classWeeks.html#a2606052f27e4baecbf6eb75bd695a034">Weeks</a>
+: <a class="el" href="classWeeks.html#a01338098b8fc98f665af954f34fe520b">Weeks</a>
</li>
<li>when()
: <a class="el" href="classprocess_1_1StateMachine.html#ab5ecde2413bc4dc62d5b47584940d1ca">process::StateMachine< State ></a>
@@ -165,7 +165,7 @@
: <a class="el" href="classos_1_1WindowsFD.html#a43994eeb484a426990af657f691f66e0">os::WindowsFD</a>
</li>
<li>WindowsSocketError()
-: <a class="el" href="classWindowsSocketError.html#a4a81a41e85b87bb39141743f6a99b978">WindowsSocketError</a>
+: <a class="el" href="classWindowsSocketError.html#af866e904d4e353613312fc82b24ac5ab">WindowsSocketError</a>
</li>
<li>withdraw()
: <a class="el" href="classzookeeper_1_1LeaderContender.html#a8b9c11cee8ab52cdd9cd8cd3991f6c69">zookeeper::LeaderContender</a>
diff --git a/content/api/latest/c++/functions_u.html b/content/api/latest/c++/functions_u.html
index 162eaef..bb0e1a5 100644
--- a/content/api/latest/c++/functions_u.html
+++ b/content/api/latest/c++/functions_u.html
@@ -317,11 +317,13 @@
, <a class="el" href="classmesos_1_1internal_1_1slave_1_1XfsDiskIsolatorProcess.html#ab2351a69f1f7ed476aa6f11e9401055a">mesos::internal::slave::XfsDiskIsolatorProcess</a>
, <a class="el" href="classmesos_1_1internal_1_1StatusUpdateManagerProcess.html#a1658b4adb952dc5e20465e31d572633b">mesos::internal::StatusUpdateManagerProcess< IDType, CheckpointType, UpdateType ></a>
, <a class="el" href="classmesos_1_1slave_1_1Isolator.html#aa4e5910588131613e4c10903282dc252">mesos::slave::Isolator</a>
-, <a class="el" href="classprocess_1_1Clock.html#a196836d438ff28617159c9e682be3656">process::Clock</a>
</li>
<li>Update
: <a class="el" href="classprocess_1_1Clock.html#a1e639da11c2a00b3bffd3391d4a9412f">process::Clock</a>
</li>
+<li>update()
+: <a class="el" href="classprocess_1_1Clock.html#a196836d438ff28617159c9e682be3656">process::Clock</a>
+</li>
<li>updateAllocation()
: <a class="el" href="classmesos_1_1allocator_1_1Allocator.html#aa8ee61bea88a926c920e909571f836dd">mesos::allocator::Allocator</a>
, <a class="el" href="classmesos_1_1internal_1_1master_1_1allocator_1_1internal_1_1HierarchicalAllocatorProcess.html#a257182b5b97d3fdbeb397e5814639b28">mesos::internal::master::allocator::internal::HierarchicalAllocatorProcess</a>
@@ -461,7 +463,7 @@
, <a class="el" href="structprocess_1_1http_1_1Request.html#aba3024fe3e1028d19f5fb4e92519cb56">process::http::Request</a>
</li>
<li>URL()
-: <a class="el" href="structprocess_1_1http_1_1URL.html#a6f6a9c959c9bb1519f1c29192aa4a407">process::http::URL</a>
+: <a class="el" href="structprocess_1_1http_1_1URL.html#a0405a5eae13705f0cfe7b78e2928bc5f">process::http::URL</a>
</li>
<li>us()
: <a class="el" href="classDuration.html#ab4bc8981d3ac26e3880e57e19f7f7f92">Duration</a>
diff --git a/content/api/latest/c++/functions_w.html b/content/api/latest/c++/functions_w.html
index 646cc36..d5502dc 100644
--- a/content/api/latest/c++/functions_w.html
+++ b/content/api/latest/c++/functions_w.html
@@ -178,7 +178,7 @@
: <a class="el" href="classDuration.html#ae98a411bf78d0ab537c021bf3dbd01b0">Duration</a>
</li>
<li>Weeks()
-: <a class="el" href="classWeeks.html#a2606052f27e4baecbf6eb75bd695a034">Weeks</a>
+: <a class="el" href="classWeeks.html#a01338098b8fc98f665af954f34fe520b">Weeks</a>
</li>
<li>weight
: <a class="el" href="structmesos_1_1internal_1_1master_1_1allocator_1_1DRFSorter_1_1Node.html#a5c40fe9adceaa47f96a3e06f6c2ba5de">mesos::internal::master::allocator::DRFSorter::Node</a>
diff --git a/content/api/latest/c++/hierarchical_8hpp_source.html b/content/api/latest/c++/hierarchical_8hpp_source.html
index 957eff5..e1ea9e3 100644
--- a/content/api/latest/c++/hierarchical_8hpp_source.html
+++ b/content/api/latest/c++/hierarchical_8hpp_source.html
@@ -52,7 +52,7 @@
<div class="title">hierarchical.hpp</div> </div>
</div><!--header-->
<div class="contents">
-<a href="hierarchical_8hpp.html">Go to the documentation of this file.</a><div class="fragment"><div class="line"><a name="l00001"></a><span class="lineno"> 1</span> <span class="comment">// Licensed to the Apache Software Foundation (ASF) under one</span></div><div class="line"><a name="l00002"></a><span class="lineno"> 2</span> <span class="comment">// or more contributor license agreements. See the NOTICE file</span></div><div class="line"><a name="l00003"></a><span c [...]
+<a href="hierarchical_8hpp.html">Go to the documentation of this file.</a><div class="fragment"><div class="line"><a name="l00001"></a><span class="lineno"> 1</span> <span class="comment">// Licensed to the Apache Software Foundation (ASF) under one</span></div><div class="line"><a name="l00002"></a><span class="lineno"> 2</span> <span class="comment">// or more contributor license agreements. See the NOTICE file</span></div><div class="line"><a name="l00003"></a><span c [...]
<div class="ttc" id="namespacemesos_1_1internal_1_1log_1_1protocol_html_afa485608d261b11b9b4c619b4b4d6e28"><div class="ttname"><a href="namespacemesos_1_1internal_1_1log_1_1protocol.html#afa485608d261b11b9b4c619b4b4d6e28">mesos::internal::log::protocol::recover</a></div><div class="ttdeci">Protocol< RecoverRequest, RecoverResponse > recover</div></div>
<div class="ttc" id="namespaceprocess_1_1ID_html_aeb11a48c9def1da169d8455a30d0ee39"><div class="ttname"><a href="namespaceprocess_1_1ID.html#aeb11a48c9def1da169d8455a30d0ee39">process::ID::generate</a></div><div class="ttdeci">std::string generate(const std::string &prefix="")</div><div class="ttdoc">Returns &#39;prefix(N)&#39; where N represents the number of instances where the same prefix (wrt...</div></div>
<div class="ttc" id="structmesos_1_1internal_1_1master_1_1allocator_1_1internal_1_1Framework_html_aeb78b13f721de0f3d6c945a5a954df0d"><div class="ttname"><a href="structmesos_1_1internal_1_1master_1_1allocator_1_1internal_1_1Framework.html#aeb78b13f721de0f3d6c945a5a954df0d">mesos::internal::master::allocator::internal::Framework::roles</a></div><div class="ttdeci">std::set< std::string > roles</div><div class="ttdef"><b>Definition:</b> hierarchical.hpp:87</div></div>
diff --git a/content/documentation/latest/operator-http-api/index.html b/content/documentation/latest/operator-http-api/index.html
index d2adf45..aa3cf47 100644
--- a/content/documentation/latest/operator-http-api/index.html
+++ b/content/documentation/latest/operator-http-api/index.html
@@ -2468,21 +2468,10 @@ Content-Type: application/json
"configs" : [
{
"role": "dev",
- "guarantees": {
- "cpus": {
- "value": 1.0
- },
- "mem": {
- "value": 1024.0
- }
- },
"limits": {
- "cpus": {
- "value": 2.0
- },
- "mem": {
- "value": 2048.0
- }
+ "cpus": { "value": 2.0 },
+ "mem": { "value": 2048.0 },
+ "disk": { "value": 4096.0 }
}
}
]
@@ -2509,34 +2498,28 @@ Content-Type: application/json
Accept: application/json
{
- "type": "UPDATE_QUOTA",
- "update_quota": {
- "force": false,
- "quota_configs": [
- {
- "role": "dev",
- "guarantees": {
- "cpus": { "value": 1.0 },
- "mem": { "value": 1024.0 }
- },
- "limits": {
- "cpus": { "value": 2.0 },
- "mem": { "value": 2048.0 }
- }
- },
- {
- "role": "test",
- "guarantees": {
- "cpus": { "value": 1.0 },
- "mem": { "value": 1024.0 }
- },
- "limits": {
- "cpus": { "value": 2.0 },
- "mem": {"value": 2048.0 }
- }
+ "type": "UPDATE_QUOTA",
+ "update_quota": {
+ "force": false,
+ "quota_configs": [
+ {
+ "role": "dev",
+ "limits": {
+ "cpus": { "value": 10 },
+ "mem": { "value": 2048 },
+ "disk": { "value": 4096 }
}
- ]
- }
+ },
+ {
+ "role": "test",
+ "limits": {
+ "cpus": { "value": 1 },
+ "mem": { "value": 256 },
+ "disk": { "value": 512 }
+ }
+ }
+ ]
+ }
}
UPDATE_QUOTA HTTP Response:
diff --git a/content/documentation/latest/quota/index.html b/content/documentation/latest/quota/index.html
index 463525b..5f5cf32 100644
--- a/content/documentation/latest/quota/index.html
+++ b/content/documentation/latest/quota/index.html
@@ -107,408 +107,235 @@
<div class="col-md-8">
<h1>Quota</h1>
-<p>A problem when running multiple frameworks on Mesos is that the default
-<a href="https://www.cs.berkeley.edu/~alig/papers/drf.pdf">wDRF allocator</a> may offer
-resources to frameworks even though they are beyond their fair share (e.g., when
-no other framework currently accepts these resources). There is no mechanism to
-lay away resources for future consumption in an entire cluster. Even though
-<a href="/documentation/latest/./reservation/">dynamic reservations</a> allow both operators and frameworks
-to dynamically reserve resources on particular Mesos agents, it does not solve
-the above problem because any individual agents might fail.</p>
-
-<p>Mesos 0.27 introduced support for <em>quotas</em>, which are a mechanism for
-guaranteeing that a <a href="/documentation/latest/./roles/">role</a> will receive a certain minimum amount of
-resources. A quota specifies a <em>minimum</em> amount of resources that the role is
-guaranteed to receive (unless the total resources in the cluster are less than
-the configured quota resources, which often indicates a misconfiguration). Of
-course, the role might receive offers for more resources than its configured
-quota, as dictated by the default Mesos DRF allocation scheme.</p>
-
-<p>Quotas are somewhat similar to <a href="/documentation/latest/./reservation/">reserved resources</a>, but they
-have several differences. Most importantly, quota resources are not tied to a
-particular agent—that is, a quota ensures that a role will receive a certain
-amount of resources somewhere in the cluster, whereas reservations can be used
-to assign specific resources at a particular agent to a given role. Quotas can
-only be configured by operators, via the HTTP endpoint described below; dynamic
-reservations can be made by frameworks, provided the framework’s principal is
-<a href="/documentation/latest/./authorization/">authorized</a> to make reservations.</p>
-
-<p>Note that reserved resources are considered to satisfy a role’s quota. For
-example, if a role has been assigned a quota of 4 CPUs and also has 2 reserved
-resources at a particular agent, the role has a minimum guarantee of 4 CPUs, not
-6.</p>
-
-<h2>Terminology</h2>
-
-<p>For the purpose of this document, an “Operator” is a person, tool, or script
-that manages the Mesos cluster.</p>
-
-<p>In computer science, a “quota” usually refers to one of the following:</p>
+<p>When multiple users are sharing a cluster, the operator may want to set limits
+on how many resources each user can use. Quota addresses this need and allows
+operators to set these limits on a per-role basis.</p>
<ul>
-<li>A minimal guarantee.</li>
-<li>A maximal limit.</li>
-<li>A pair of both.</li>
+<li><a href="#supported-resources">Supported Resources</a></li>
+<li><a href="#updating-quotas">Setting Quotas</a></li>
+<li><a href="#viewing-quotas">Viewing Quotas</a></li>
+<li><a href="#deprecated-quota-guarantees">Deprecated: Quota Guarantees</a></li>
+<li><a href="#implementation-notes">Implementation Notes</a></li>
</ul>
-<p>In Mesos, a quota is a <strong>guaranteed</strong> resource allocation that a role may rely
-on; in other words, a minimum share a role is entitled to receive.</p>
+<h2>Supported Resources</h2>
-<p><strong>NOTE:</strong> The built-in wDRF allocator extends this contract, and based on the
-definition above, treats quota as the pair of both the minimal and maximal share
-a role is entitled to receive. See
-<a href="#allocatorEnforcement">wDRF implementation notes</a> for details.</p>
-
-<h2>Motivation and Limitations</h2>
-
-<p>Consider the following scenarios in a Mesos cluster to better understand which
-use-cases are supported by quota, and which are not.</p>
-
-<h3>Scenario 1: Greedy Framework</h3>
-
-<p>There are two frameworks in a cluster, each are subscribed to their own distinct
-role, and both roles have equal weights: framework fA subscribed to role rA and
-framework fB subscribed to role rB. There is a single resource available in the
-cluster: 100 CPUs. fA consumes 10 CPUs and is idle (declines resource offers),
-while fB is greedy and accepts all offers it gets, hogging the remaining 90
-CPUs. Without quota, though fA’s fair share is 50 CPUs it will not be able to
-make use of additional the 40 CPUs until some of fB’s tasks terminate.</p>
-
-<h3>Scenario 2: Resources for a new Framework</h3>
-
-<p>A greedy framework fB subscribed to role rB is currently the only framework in
-the cluster and it uses all available resources—100 CPUs. If a new framework
-fA subscribed to role rA joins the cluster, it will not receive its fair share
-of the cluster resources (50 CPUs) until some of fB’s tasks terminate.</p>
-
-<p>To deal with Scenario 2, quota by itself is not a sufficient solution as it
-would be set after fB has started using all resources. Instead Scenario 2
-requires either always keeping a pool of resources which are not offered or
-introducing preemption for running tasks.</p>
-
-<h2>Operator HTTP Endpoint</h2>
-
-<p>The master <a href="/documentation/latest/./endpoints/master/quota/">/quota</a> HTTP endpoint enables operators
-to configure quotas. The endpoint currently offers a REST-like interface and
-supports the following operations:</p>
+<p>The following resources have quota support:</p>
<ul>
-<li><a href="#setRequest">Setting</a> a new quota with POST.</li>
-<li><a href="#removeRequest">Removing</a> an existing quota with DELETE.</li>
-<li><a href="#statusRequest">Querying</a> the currently set quota with GET.</li>
+<li><code>cpus</code></li>
+<li><code>mem</code></li>
+<li><code>disk</code></li>
+<li><code>gpus</code></li>
+<li>any custom resource of type <code>SCALAR</code></li>
</ul>
-<p>Currently it is not possible to update previously configured quotas. This means
-in order to update a quota for a given role, the operator has to remove the
-existing quota and then set a new one.</p>
-
-<p>The endpoint can optionally use authentication and authorization. See
-<a href="/documentation/latest/./authentication/">authentication guide</a> for details.</p>
-
-<p><a name="setRequest"></a></p>
-
-<h3>Set</h3>
-
-<p>The operator can set a new quota by sending an HTTP POST request to the <code>/quota</code>
-endpoint.</p>
-
-<p>An example request to the quota endpoint could look like this (using the JSON
-file below):</p>
-
-<pre><code> $ curl -d @quota.json -X POST http://<master-ip>:<port>/quota
-</code></pre>
-
-<p>For example to set a quota of 12 CPUs and 6144 MB of RAM for <code>role1</code> the operator
-can use the following <code>quota.json</code>:</p>
-
-<pre><code> {
- "role": "role1",
- "guarantee": [
- {
- "name": "cpus",
- "type": "SCALAR",
- "scalar": { "value": 12 }
- },
- {
- "name": "mem",
- "type": "SCALAR",
- "scalar": { "value": 6144 }
- }
- ]
- }
-</code></pre>
-
-<p>A set request is only valid for roles for which no quota is currently set.
-However if the master is configured without an explicit
-<a href="/documentation/latest/./roles/">role whitelist</a>, a set request can introduce new roles.</p>
-
-<p>In order to bypass the <a href="#capacityHeuristic">capacity heuristic</a> check the
-operator should set an optional <code>force</code> field:</p>
-
-<pre><code> {
- "force": true,
- "role": "role1",
- ...
- }
-</code></pre>
-
-<p>The operator will receive one of the following HTTP response codes:</p>
+<p>The following resources do not have quota support:</p>
<ul>
-<li><code>200 OK</code>: Success (the set request was successful).</li>
-<li><code>400 BadRequest</code>: Invalid arguments (e.g., quota for role exists, invalid
-JSON, non-scalar resources included).</li>
-<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
-<li><code>403 Forbidden</code>: Unauthorized request.</li>
-<li><code>409 Conflict</code>: The capacity heuristic check failed due to insufficient
-resources.</li>
+<li><code>ports</code></li>
+<li>any custom resource of type <code>RANGES</code> or <code>SET</code></li>
</ul>
-<p><a name="removeRequest"></a></p>
-
-<h3>Remove</h3>
-
-<p>The operator can remove a previously set quota by sending an HTTP DELETE request
-to the <code>/quota/<role></code> endpoint. For example the following command removes
-a previously set quota for <code>role1</code>:</p>
-
-<pre><code> $ curl -X DELETE http://<master-ip>:<port>/quota/role1
+<h2>Updating Quotas</h2>
+
+<p>By default, every role has no resource limits. To modify the resource limits
+for one or more roles, the v1 API <code>UPDATE_QUOTA</code> call is used. Note that this
+call applies the update in an all-or-nothing manner, so that if one of the
+role’s quota updates is invalid or unauthorized, the entire request will not
+go through.</p>
+
+<p>Example:</p>
+
+<pre><code>curl --request POST \
+ --url http://<master-ip>:<master-port>/api/v1/ \
+ --header 'Content-Type: application/json' \
+ --data '{
+ "type": "UPDATE_QUOTA",
+ "update_quota": {
+ "force": false,
+ "quota_configs": [
+ {
+ "role": "dev",
+ "limits": {
+ "cpus": { "value": 10 },
+ "mem": { "value": 2048 },
+ "disk": { "value": 4096 }
+ }
+ },
+ {
+ "role": "test",
+ "limits": {
+ "cpus": { "value": 1 },
+ "mem": { "value": 256 },
+ "disk": { "value": 512 }
+ }
+ }
+ ]
+ }
+ }'
</code></pre>
-<p>The operator will receive one of the following HTTP response codes:</p>
-
<ul>
-<li><code>200 OK</code>: Success (the remove request was successful).</li>
-<li><code>400 BadRequest</code>: Invalid arguments (e.g., removing a quota for a role which
-does not have any quota set).</li>
-<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
-<li><code>403 Forbidden</code>: Unauthorized request.</li>
+<li>Note that the request will be denied if the current quota consumption is above
+the provided limit. This check can be overriden by setting <code>force</code> to <code>true</code>.</li>
+<li>Note that the master will attempt to rescind a sufficient number of offers to
+ensure that the role cannot exceed its limits.</li>
</ul>
-<p><a name="statusRequest"></a></p>
-
-<h3>Status</h3>
-
-<p>The operator can query the configured quotas by sending an HTTP GET request
-to the <code>/quota</code> endpoint.</p>
+<h2>Viewing Quotas</h2>
-<pre><code> $ curl -X GET http://<master-ip>:<port>/quota
-</code></pre>
+<h3>Web UI</h3>
-<p>The response message body includes a JSON representation of the current quota
-status for role(s) which principal is authorized to query quota status (if
-authorization is enabled). For example:</p>
-
-<pre><code> {
- "infos": [
- {
- "role": "role1",
- "guarantee": [
- {
- "name": "cpus",
- "role": "*",
- "type": "SCALAR",
- "scalar": { "value": 12 }
- },
- {
- "name": "mem",
- "role": "*",
- "type": "SCALAR",
- "scalar": { "value": 6144 }
- }
- ]
- }
- ]
- }
-</code></pre>
+<p>The ‘Roles’ tab in the web ui displays resource accounting information for all
+known roles. This includes the configured quota and the quota consumption.</p>
-<p>The operator will receive one of the following HTTP response codes:</p>
+<h3>API</h3>
-<ul>
-<li><code>200 OK</code>: Success.</li>
-<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
-</ul>
+<p>There are several endpoints for viewing quota related information.</p>
+<p>The v1 API <code>GET_QUOTA</code> call will return the quota configuration:</p>
-<p><strong>NOTE:</strong> If the principal is not authorized to query quota status for certain
-role(s), the result will not include corresponding quota information.</p>
-
-<h2>How Does It Work?</h2>
-
-<p>There are several stages in the lifetime of a quota issued by operator. First
-the <a href="#requestProcessing">quota set request is handled by the master</a>, after that
-the <a href="#allocatorEnforcement">allocator enforces the quota</a>.
-Quotas can be <a href="#removeProcessing">removed</a> by the operator.</p>
-
-<p>It is important to understand that the enforcement of quota depends on
-the allocator being used. A custom allocator could choose to handle quota in its
-own way or even to ignore quota.
-<strong>The following section assumes the default
-<a href="https://www.cs.berkeley.edu/~alig/papers/drf.pdf">wDRF</a> allocator is used.</strong></p>
-
-<p>Be aware that setting quota may affect other frameworks in the cluster,
-because resources will be laid away and not offered to other frameworks.
-Also note, that quota is only applicable for scalar resources (e.g., it is not
-possible to set quota for port resources).</p>
-
-<p><a name="requestProcessing"></a></p>
-
-<h2>Quota Request Processing</h2>
-
-<p>When an operator submits a quota set request via the master <code>/quota</code> HTTP
-endpoint, the following steps are triggered:</p>
-
-<ol>
-<li><a href="/documentation/latest/./authentication/">Authenticate</a> the HTTP request.</li>
-<li>Parse and validate the request.
-See <a href="#setRequest">description of possible error codes</a>.</li>
-<li><a href="/documentation/latest/./authentication/">Authorize</a> the HTTP request if authorization is enabled.</li>
-<li>Run the <a href="#capacityHeuristic">capacity heuristic</a> if not disabled by
-<a href="#setRequest">the <code>force</code> flag</a>.</li>
-<li>Reliably store quota. See <a href="#failover">details on failover recovery</a>.</li>
-<li><a href="#rescindOffers">Rescind outstanding offers</a>.</li>
-</ol>
-
-
-<p><a name="removeProcessing"></a>
-The quota remove request processing is simpler and triggers the following steps:</p>
-
-<ol>
-<li><a href="/documentation/latest/./authentication/">Authenticate</a> the HTTP request.</li>
-<li>Validate the request.
-See <a href="#removeRequest">description of potential error codes</a>.</li>
-<li><a href="/documentation/latest/./authentication/">Authorize</a> the HTTP request if authorization is enabled.</li>
-<li>Reliably remove quota.</li>
-</ol>
-
-
-<p><a name="capacityHeuristic"></a></p>
-
-<h3>Capacity Heuristic Check</h3>
-
-<p>Misconfigured quota can render a cluster into a state where no offers are made
-to any frameworks. For example imagine an operator setting quota 1000 CPUs
-for a role <code>prosuction</code> (note the typo) in a cluster with a total capacity of
-100 CPUs. In that case after the quota is accepted by the master, no offers will
- be made to any framework in any actual role (including <code>production</code>).</p>
-
-<p>In order to prevent such extreme situations, the Mesos Master employs a capacity
-heuristic check that ensures a quota set request can reasonably be satisfied
-given the total cluster capacity. This heuristic tests whether the total quota,
-including the new request, does not exceed the sum of total
-non-statically-reserved cluster resources, i.e. the following inequality holds:</p>
-
-<pre><code> total resources - statically reserved >= total quota + quota request
+<pre><code>$ curl --request POST \
+ --url http://<master-ip>:<master-port>/api/v1/ \
+ --header 'Content-Type: application/json' \
+ --header 'Accept: application/json' \
+ --data '{ "type": "GET_QUOTA" }'
</code></pre>
-<p>Please be advised that even if there are enough resources at the moment of
-this check, agents may terminate at any time, rendering the cluster incapable
-of satisfying the configured quotas.</p>
-
-<p>A <a href="#setRequest"><code>force</code> flag</a> can be set to bypass this check. For example, this
-flag can be useful when the operator would like to configure a quota that
-exceeds the current cluster capacity, but they know that additional cluster
-resources will be added shortly.</p>
-
-<p><a name="rescindOffers"></a></p>
-
-<h3>Rescinding Outstanding Offers</h3>
-
-<p>When setting a new quota, the master rescinds outstanding offers. This avoids
-situations where the quota request cannot be satisfied by the remaining
-unoffered resources, but there are enough resources tied up in outstanding
-offers to frameworks that have not accepted them yet. Hence, we rescind
-outstanding offers with the following rules:</p>
-
-<ul>
-<li>Rescind at least as many resources as there are in the quota request.</li>
-<li>If at least one offer is to be rescinded from an agent, all offers from this
-agent are rescinded. This is done in order to make the potential offer bigger,
-which increases the chances that a quota'ed framework will be able to use the
-offer.</li>
-<li>Rescind offers from at least as many agents as there are frameworks subscribed
-to the role for which quota is being set. This enables (but does not guarantee,
-due to fair sharing) each framework subscribed to the role to receive an offer.</li>
-</ul>
+<p>Response:</p>
+
+<pre><code>{
+ "type": "GET_QUOTA",
+ "get_quota": {
+ "status": {
+ "infos": [
+ {
+ "configs" : [
+ {
+ "role": "dev",
+ "limits": {
+ "cpus": { "value": 10.0 },
+ "mem": { "value": 2048.0 },
+ "disk": { "value": 4096.0 }
+ }
+ },
+ {
+ "role": "test",
+ "limits": {
+ "cpus": { "value": 1.0 },
+ "mem": { "value": 256.0 },
+ "disk": { "value": 512.0 }
+ }
+ }
+ ]
+ }
+ ]
+ }
+ }
+}
+</code></pre>
+<p>To also view the quota consumption, use the <code>/roles</code> endpoint:</p>
-<p><a name="allocatorEnforcement"></a></p>
+<pre><code>$ curl http://<master-ip>:<master-port>/roles
+</code></pre>
-<h2>Enforcement by wDRF Allocator</h2>
+<p>Response</p>
+
+<pre><code>{
+ "roles": [
+ {
+ "name": "dev",
+ "weight": 1.0,
+ "quota":
+ {
+ "role": "dev",
+ "limit": {
+ "cpus": 10.0,
+ "mem": 2048.0,
+ "disk": 4096.0
+ },
+ "consumed": {
+ "cpus": 2.0,
+ "mem": 1024.0,
+ "disk": 2048.0
+ }
+ },
+ "allocated": {
+ "cpus": 2.0,
+ "mem": 1024.0,
+ "disk": 2048.0
+ },
+ "offered": {},
+ "reserved": {
+ "cpus": 2.0,
+ "mem": 1024.0,
+ "disk": 2048.0
+ },
+ "frameworks": []
+ }
+ ]
+}
+</code></pre>
-<p>The wDRF allocator first allocates (or lays away if offers are declined)
-resources to roles with quota set. Once all quotas are satisfied, it proceeds
-with the standard wDRF for all remaining roles.</p>
+<h3>Quota Consumption</h3>
-<p><strong>NOTE:</strong> A quota'ed role may not be allocated any unreserved non-revocable
-resources beyond its quota guarantee. If frameworks in the quota'ed role have
-not opted for revocable resources, they may stop getting offers once quota for
-the role is satisfied. In this case setting quota to any value that is less than
-the role’s fair share may reduce the amount of resources offered to this role.</p>
+<p>A role’s quota consumption consists of its allocations and reservations.
+In other words, even if reservations are not allocated, they are included
+in the quota consumption. Offered resources are not charged against quota.</p>
-<p><strong>NOTE:</strong> Currently quota guarantee also serves as quota limit, i.e. once quota
-for the role is satisfied, no further resources will be offered to the role
-except those reserved for the role. This behavior aims to mitigate the absence
-of quota limit and will be changed in future releases.</p>
+<h3>Metrics</h3>
-<p><strong>NOTE:</strong> Prior to Mesos 1.5, the allocator would allocate the entire available
-resources on an agent (including the resources that the role has no quota for)
-when trying to satisfy the quota of a role i.e. the quota allocation was
-coarse-grained. This has been fixed in Mesos 1.5. Specially, the allocator
-follows the steps below when allocating resources on an agent to a quota
-role:</p>
+<p>The following metric keys are exposed for quota:</p>
<ul>
-<li>Resources that this role has quota for are allocated up to the quota guarantee.</li>
-<li>Resources that this role has no quota for (including non-scalar resources) are
-allocated if (1) these resources are not needed for other roles' unsatisfied quotas
-and (2) this role is also getting some other resources on the same agent to meet its
-quota or reservation. The second condition is used to reduce fragmentation.</li>
+<li><code>allocator/mesos/quota/roles/<role>/resources/<resource>/guarantee</code></li>
+<li><code>allocator/mesos/quota/roles/<role>/resources/<resource>/limit</code></li>
+<li>A quota consumption metric will be added via
+<a href="https://issues.apache.org/jira/browse/MESOS-9123">MESOS-9123</a>.</li>
</ul>
-<p>If there are multiple frameworks subscribed to a role with quota set, the
-standard wDRF algorithm determines offer precedence amongst these frameworks.</p>
-
-<p>The default wDRF allocator considers only non-revocable resources as applicable
-towards quota.</p>
+<h2>Deprecated: Quota Guarantees</h2>
-<p><a name="failover"></a></p>
+<p>Prior to Mesos 1.9, the quota related APIs only exposed quota “guarantees”
+which ensured a minimum amount of resources would be available to a role.
+Setting guarantees also set implicit quota limits. In Mesos 1.9+, quota
+limits are now exposed directly per the above documentation.</p>
-<h2>Failover</h2>
+<p>Quota guarantees are now deprecated in favor of using only quota limits.
+Enforcement of quota guarantees required that Mesos holds back enough
+resources to meet all of the unsatisfied quota guarantees. Since Mesos is
+moving towards an optimistic offer model (to improve multi-role / multi-
+scheduler scalability, see
+<a href="https://issues.apache.org/jira/browse/MESOS-1607">MESOS-1607</a>), it will
+become no longer possible to enforce quota guarantees by holding back
+resources. In such a model, quota limits are simple to enforce, but quota
+guarantees would require a complex “effective limit” propagation model to
+leave space for unsatisfied guarantees.</p>
-<p>If there is at least one role with quota set, the master failover recovery
-changes significantly. The reason for this is that during the recovery there is
-a period of time when not all agents have registered with the Master. Therefore
-it is impossible to reason about whether all quotas are satisfied.</p>
-
-<p>To address this issue, if upon recovery any previously set quota are detected,
-the allocator enters recovery mode, during which the allocator
-<em>does not issue offers</em>. The recovery mode—and therefore offer
-suspension—ends when either:</p>
-
-<ul>
-<li>A certain amount of agents reregister (by default 80% of agents known before
-the failover), or</li>
-<li>a timeout expires (by default 10 minutes).</li>
-</ul>
+<p>For these reasons, quota guarantees, while still functional in Mesos 1.9,
+are now deprecated. A combination of limits and priority based preemption
+will be simpler in an optimistic offer model.</p>
+<p>For documentation on quota guarantees, please see the previous
+documentation: <a href="https://github.com/apache/mesos/blob/1.8.0/docs/quota.md">https://github.com/apache/mesos/blob/1.8.0/docs/quota.md</a></p>
-<h2>Current Limitations</h2>
+<h2>Implementation Notes</h2>
<ul>
-<li>The quota set request does not allow specifying the granularity of the
-requested resources (e.g., 10 CPUs on a single node).</li>
-<li>The quota set request does not allow to specify constraints (e.g., 2*5 cpus
-on disjoint nodes for an HA like setup).</li>
-<li>Quota is not allowed for the default role ‘*’ (see
-<a href="https://issues.apache.org/jira/browse/MESOS-3938">MESOS-3938</a>).</li>
-<li>Currently it is not possible to update previously configured quotas. See
-<a href="#setRequest">quota set request</a> for details.</li>
+<li>Quota is not supported for the default <code>*</code> role.</li>
+<li>Quota is not supported on nested roles (e.g. <code>eng/prod</code>).</li>
+<li>During failover, in order to correctly enforce limits, the allocator
+will be paused and will not issue offers until at least 80% agents
+re-register or 10 minutes elapses. These parameters will be made
+configurable: <a href="https://issues.apache.org/jira/browse/MESOS-4073">MESOS-4073</a></li>
</ul>
diff --git a/content/documentation/operator-http-api/index.html b/content/documentation/operator-http-api/index.html
index 5476150..d0dd145 100644
--- a/content/documentation/operator-http-api/index.html
+++ b/content/documentation/operator-http-api/index.html
@@ -2468,21 +2468,10 @@ Content-Type: application/json
"configs" : [
{
"role": "dev",
- "guarantees": {
- "cpus": {
- "value": 1.0
- },
- "mem": {
- "value": 1024.0
- }
- },
"limits": {
- "cpus": {
- "value": 2.0
- },
- "mem": {
- "value": 2048.0
- }
+ "cpus": { "value": 2.0 },
+ "mem": { "value": 2048.0 },
+ "disk": { "value": 4096.0 }
}
}
]
@@ -2509,34 +2498,28 @@ Content-Type: application/json
Accept: application/json
{
- "type": "UPDATE_QUOTA",
- "update_quota": {
- "force": false,
- "quota_configs": [
- {
- "role": "dev",
- "guarantees": {
- "cpus": { "value": 1.0 },
- "mem": { "value": 1024.0 }
- },
- "limits": {
- "cpus": { "value": 2.0 },
- "mem": { "value": 2048.0 }
- }
- },
- {
- "role": "test",
- "guarantees": {
- "cpus": { "value": 1.0 },
- "mem": { "value": 1024.0 }
- },
- "limits": {
- "cpus": { "value": 2.0 },
- "mem": {"value": 2048.0 }
- }
+ "type": "UPDATE_QUOTA",
+ "update_quota": {
+ "force": false,
+ "quota_configs": [
+ {
+ "role": "dev",
+ "limits": {
+ "cpus": { "value": 10 },
+ "mem": { "value": 2048 },
+ "disk": { "value": 4096 }
}
- ]
- }
+ },
+ {
+ "role": "test",
+ "limits": {
+ "cpus": { "value": 1 },
+ "mem": { "value": 256 },
+ "disk": { "value": 512 }
+ }
+ }
+ ]
+ }
}
UPDATE_QUOTA HTTP Response:
diff --git a/content/documentation/quota/index.html b/content/documentation/quota/index.html
index faf9009..b034617 100644
--- a/content/documentation/quota/index.html
+++ b/content/documentation/quota/index.html
@@ -107,408 +107,235 @@
<div class="col-md-8">
<h1>Quota</h1>
-<p>A problem when running multiple frameworks on Mesos is that the default
-<a href="https://www.cs.berkeley.edu/~alig/papers/drf.pdf">wDRF allocator</a> may offer
-resources to frameworks even though they are beyond their fair share (e.g., when
-no other framework currently accepts these resources). There is no mechanism to
-lay away resources for future consumption in an entire cluster. Even though
-<a href="/documentation/latest/./reservation/">dynamic reservations</a> allow both operators and frameworks
-to dynamically reserve resources on particular Mesos agents, it does not solve
-the above problem because any individual agents might fail.</p>
-
-<p>Mesos 0.27 introduced support for <em>quotas</em>, which are a mechanism for
-guaranteeing that a <a href="/documentation/latest/./roles/">role</a> will receive a certain minimum amount of
-resources. A quota specifies a <em>minimum</em> amount of resources that the role is
-guaranteed to receive (unless the total resources in the cluster are less than
-the configured quota resources, which often indicates a misconfiguration). Of
-course, the role might receive offers for more resources than its configured
-quota, as dictated by the default Mesos DRF allocation scheme.</p>
-
-<p>Quotas are somewhat similar to <a href="/documentation/latest/./reservation/">reserved resources</a>, but they
-have several differences. Most importantly, quota resources are not tied to a
-particular agent—that is, a quota ensures that a role will receive a certain
-amount of resources somewhere in the cluster, whereas reservations can be used
-to assign specific resources at a particular agent to a given role. Quotas can
-only be configured by operators, via the HTTP endpoint described below; dynamic
-reservations can be made by frameworks, provided the framework’s principal is
-<a href="/documentation/latest/./authorization/">authorized</a> to make reservations.</p>
-
-<p>Note that reserved resources are considered to satisfy a role’s quota. For
-example, if a role has been assigned a quota of 4 CPUs and also has 2 reserved
-resources at a particular agent, the role has a minimum guarantee of 4 CPUs, not
-6.</p>
-
-<h2>Terminology</h2>
-
-<p>For the purpose of this document, an “Operator” is a person, tool, or script
-that manages the Mesos cluster.</p>
-
-<p>In computer science, a “quota” usually refers to one of the following:</p>
+<p>When multiple users are sharing a cluster, the operator may want to set limits
+on how many resources each user can use. Quota addresses this need and allows
+operators to set these limits on a per-role basis.</p>
<ul>
-<li>A minimal guarantee.</li>
-<li>A maximal limit.</li>
-<li>A pair of both.</li>
+<li><a href="#supported-resources">Supported Resources</a></li>
+<li><a href="#updating-quotas">Setting Quotas</a></li>
+<li><a href="#viewing-quotas">Viewing Quotas</a></li>
+<li><a href="#deprecated-quota-guarantees">Deprecated: Quota Guarantees</a></li>
+<li><a href="#implementation-notes">Implementation Notes</a></li>
</ul>
-<p>In Mesos, a quota is a <strong>guaranteed</strong> resource allocation that a role may rely
-on; in other words, a minimum share a role is entitled to receive.</p>
+<h2>Supported Resources</h2>
-<p><strong>NOTE:</strong> The built-in wDRF allocator extends this contract, and based on the
-definition above, treats quota as the pair of both the minimal and maximal share
-a role is entitled to receive. See
-<a href="#allocatorEnforcement">wDRF implementation notes</a> for details.</p>
-
-<h2>Motivation and Limitations</h2>
-
-<p>Consider the following scenarios in a Mesos cluster to better understand which
-use-cases are supported by quota, and which are not.</p>
-
-<h3>Scenario 1: Greedy Framework</h3>
-
-<p>There are two frameworks in a cluster, each are subscribed to their own distinct
-role, and both roles have equal weights: framework fA subscribed to role rA and
-framework fB subscribed to role rB. There is a single resource available in the
-cluster: 100 CPUs. fA consumes 10 CPUs and is idle (declines resource offers),
-while fB is greedy and accepts all offers it gets, hogging the remaining 90
-CPUs. Without quota, though fA’s fair share is 50 CPUs it will not be able to
-make use of additional the 40 CPUs until some of fB’s tasks terminate.</p>
-
-<h3>Scenario 2: Resources for a new Framework</h3>
-
-<p>A greedy framework fB subscribed to role rB is currently the only framework in
-the cluster and it uses all available resources—100 CPUs. If a new framework
-fA subscribed to role rA joins the cluster, it will not receive its fair share
-of the cluster resources (50 CPUs) until some of fB’s tasks terminate.</p>
-
-<p>To deal with Scenario 2, quota by itself is not a sufficient solution as it
-would be set after fB has started using all resources. Instead Scenario 2
-requires either always keeping a pool of resources which are not offered or
-introducing preemption for running tasks.</p>
-
-<h2>Operator HTTP Endpoint</h2>
-
-<p>The master <a href="/documentation/latest/./endpoints/master/quota/">/quota</a> HTTP endpoint enables operators
-to configure quotas. The endpoint currently offers a REST-like interface and
-supports the following operations:</p>
+<p>The following resources have quota support:</p>
<ul>
-<li><a href="#setRequest">Setting</a> a new quota with POST.</li>
-<li><a href="#removeRequest">Removing</a> an existing quota with DELETE.</li>
-<li><a href="#statusRequest">Querying</a> the currently set quota with GET.</li>
+<li><code>cpus</code></li>
+<li><code>mem</code></li>
+<li><code>disk</code></li>
+<li><code>gpus</code></li>
+<li>any custom resource of type <code>SCALAR</code></li>
</ul>
-<p>Currently it is not possible to update previously configured quotas. This means
-in order to update a quota for a given role, the operator has to remove the
-existing quota and then set a new one.</p>
-
-<p>The endpoint can optionally use authentication and authorization. See
-<a href="/documentation/latest/./authentication/">authentication guide</a> for details.</p>
-
-<p><a name="setRequest"></a></p>
-
-<h3>Set</h3>
-
-<p>The operator can set a new quota by sending an HTTP POST request to the <code>/quota</code>
-endpoint.</p>
-
-<p>An example request to the quota endpoint could look like this (using the JSON
-file below):</p>
-
-<pre><code> $ curl -d @quota.json -X POST http://<master-ip>:<port>/quota
-</code></pre>
-
-<p>For example to set a quota of 12 CPUs and 6144 MB of RAM for <code>role1</code> the operator
-can use the following <code>quota.json</code>:</p>
-
-<pre><code> {
- "role": "role1",
- "guarantee": [
- {
- "name": "cpus",
- "type": "SCALAR",
- "scalar": { "value": 12 }
- },
- {
- "name": "mem",
- "type": "SCALAR",
- "scalar": { "value": 6144 }
- }
- ]
- }
-</code></pre>
-
-<p>A set request is only valid for roles for which no quota is currently set.
-However if the master is configured without an explicit
-<a href="/documentation/latest/./roles/">role whitelist</a>, a set request can introduce new roles.</p>
-
-<p>In order to bypass the <a href="#capacityHeuristic">capacity heuristic</a> check the
-operator should set an optional <code>force</code> field:</p>
-
-<pre><code> {
- "force": true,
- "role": "role1",
- ...
- }
-</code></pre>
-
-<p>The operator will receive one of the following HTTP response codes:</p>
+<p>The following resources do not have quota support:</p>
<ul>
-<li><code>200 OK</code>: Success (the set request was successful).</li>
-<li><code>400 BadRequest</code>: Invalid arguments (e.g., quota for role exists, invalid
-JSON, non-scalar resources included).</li>
-<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
-<li><code>403 Forbidden</code>: Unauthorized request.</li>
-<li><code>409 Conflict</code>: The capacity heuristic check failed due to insufficient
-resources.</li>
+<li><code>ports</code></li>
+<li>any custom resource of type <code>RANGES</code> or <code>SET</code></li>
</ul>
-<p><a name="removeRequest"></a></p>
-
-<h3>Remove</h3>
-
-<p>The operator can remove a previously set quota by sending an HTTP DELETE request
-to the <code>/quota/<role></code> endpoint. For example the following command removes
-a previously set quota for <code>role1</code>:</p>
-
-<pre><code> $ curl -X DELETE http://<master-ip>:<port>/quota/role1
+<h2>Updating Quotas</h2>
+
+<p>By default, every role has no resource limits. To modify the resource limits
+for one or more roles, the v1 API <code>UPDATE_QUOTA</code> call is used. Note that this
+call applies the update in an all-or-nothing manner, so that if one of the
+role’s quota updates is invalid or unauthorized, the entire request will not
+go through.</p>
+
+<p>Example:</p>
+
+<pre><code>curl --request POST \
+ --url http://<master-ip>:<master-port>/api/v1/ \
+ --header 'Content-Type: application/json' \
+ --data '{
+ "type": "UPDATE_QUOTA",
+ "update_quota": {
+ "force": false,
+ "quota_configs": [
+ {
+ "role": "dev",
+ "limits": {
+ "cpus": { "value": 10 },
+ "mem": { "value": 2048 },
+ "disk": { "value": 4096 }
+ }
+ },
+ {
+ "role": "test",
+ "limits": {
+ "cpus": { "value": 1 },
+ "mem": { "value": 256 },
+ "disk": { "value": 512 }
+ }
+ }
+ ]
+ }
+ }'
</code></pre>
-<p>The operator will receive one of the following HTTP response codes:</p>
-
<ul>
-<li><code>200 OK</code>: Success (the remove request was successful).</li>
-<li><code>400 BadRequest</code>: Invalid arguments (e.g., removing a quota for a role which
-does not have any quota set).</li>
-<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
-<li><code>403 Forbidden</code>: Unauthorized request.</li>
+<li>Note that the request will be denied if the current quota consumption is above
+the provided limit. This check can be overriden by setting <code>force</code> to <code>true</code>.</li>
+<li>Note that the master will attempt to rescind a sufficient number of offers to
+ensure that the role cannot exceed its limits.</li>
</ul>
-<p><a name="statusRequest"></a></p>
-
-<h3>Status</h3>
-
-<p>The operator can query the configured quotas by sending an HTTP GET request
-to the <code>/quota</code> endpoint.</p>
+<h2>Viewing Quotas</h2>
-<pre><code> $ curl -X GET http://<master-ip>:<port>/quota
-</code></pre>
+<h3>Web UI</h3>
-<p>The response message body includes a JSON representation of the current quota
-status for role(s) which principal is authorized to query quota status (if
-authorization is enabled). For example:</p>
-
-<pre><code> {
- "infos": [
- {
- "role": "role1",
- "guarantee": [
- {
- "name": "cpus",
- "role": "*",
- "type": "SCALAR",
- "scalar": { "value": 12 }
- },
- {
- "name": "mem",
- "role": "*",
- "type": "SCALAR",
- "scalar": { "value": 6144 }
- }
- ]
- }
- ]
- }
-</code></pre>
+<p>The ‘Roles’ tab in the web ui displays resource accounting information for all
+known roles. This includes the configured quota and the quota consumption.</p>
-<p>The operator will receive one of the following HTTP response codes:</p>
+<h3>API</h3>
-<ul>
-<li><code>200 OK</code>: Success.</li>
-<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
-</ul>
+<p>There are several endpoints for viewing quota related information.</p>
+<p>The v1 API <code>GET_QUOTA</code> call will return the quota configuration:</p>
-<p><strong>NOTE:</strong> If the principal is not authorized to query quota status for certain
-role(s), the result will not include corresponding quota information.</p>
-
-<h2>How Does It Work?</h2>
-
-<p>There are several stages in the lifetime of a quota issued by operator. First
-the <a href="#requestProcessing">quota set request is handled by the master</a>, after that
-the <a href="#allocatorEnforcement">allocator enforces the quota</a>.
-Quotas can be <a href="#removeProcessing">removed</a> by the operator.</p>
-
-<p>It is important to understand that the enforcement of quota depends on
-the allocator being used. A custom allocator could choose to handle quota in its
-own way or even to ignore quota.
-<strong>The following section assumes the default
-<a href="https://www.cs.berkeley.edu/~alig/papers/drf.pdf">wDRF</a> allocator is used.</strong></p>
-
-<p>Be aware that setting quota may affect other frameworks in the cluster,
-because resources will be laid away and not offered to other frameworks.
-Also note, that quota is only applicable for scalar resources (e.g., it is not
-possible to set quota for port resources).</p>
-
-<p><a name="requestProcessing"></a></p>
-
-<h2>Quota Request Processing</h2>
-
-<p>When an operator submits a quota set request via the master <code>/quota</code> HTTP
-endpoint, the following steps are triggered:</p>
-
-<ol>
-<li><a href="/documentation/latest/./authentication/">Authenticate</a> the HTTP request.</li>
-<li>Parse and validate the request.
-See <a href="#setRequest">description of possible error codes</a>.</li>
-<li><a href="/documentation/latest/./authentication/">Authorize</a> the HTTP request if authorization is enabled.</li>
-<li>Run the <a href="#capacityHeuristic">capacity heuristic</a> if not disabled by
-<a href="#setRequest">the <code>force</code> flag</a>.</li>
-<li>Reliably store quota. See <a href="#failover">details on failover recovery</a>.</li>
-<li><a href="#rescindOffers">Rescind outstanding offers</a>.</li>
-</ol>
-
-
-<p><a name="removeProcessing"></a>
-The quota remove request processing is simpler and triggers the following steps:</p>
-
-<ol>
-<li><a href="/documentation/latest/./authentication/">Authenticate</a> the HTTP request.</li>
-<li>Validate the request.
-See <a href="#removeRequest">description of potential error codes</a>.</li>
-<li><a href="/documentation/latest/./authentication/">Authorize</a> the HTTP request if authorization is enabled.</li>
-<li>Reliably remove quota.</li>
-</ol>
-
-
-<p><a name="capacityHeuristic"></a></p>
-
-<h3>Capacity Heuristic Check</h3>
-
-<p>Misconfigured quota can render a cluster into a state where no offers are made
-to any frameworks. For example imagine an operator setting quota 1000 CPUs
-for a role <code>prosuction</code> (note the typo) in a cluster with a total capacity of
-100 CPUs. In that case after the quota is accepted by the master, no offers will
- be made to any framework in any actual role (including <code>production</code>).</p>
-
-<p>In order to prevent such extreme situations, the Mesos Master employs a capacity
-heuristic check that ensures a quota set request can reasonably be satisfied
-given the total cluster capacity. This heuristic tests whether the total quota,
-including the new request, does not exceed the sum of total
-non-statically-reserved cluster resources, i.e. the following inequality holds:</p>
-
-<pre><code> total resources - statically reserved >= total quota + quota request
+<pre><code>$ curl --request POST \
+ --url http://<master-ip>:<master-port>/api/v1/ \
+ --header 'Content-Type: application/json' \
+ --header 'Accept: application/json' \
+ --data '{ "type": "GET_QUOTA" }'
</code></pre>
-<p>Please be advised that even if there are enough resources at the moment of
-this check, agents may terminate at any time, rendering the cluster incapable
-of satisfying the configured quotas.</p>
-
-<p>A <a href="#setRequest"><code>force</code> flag</a> can be set to bypass this check. For example, this
-flag can be useful when the operator would like to configure a quota that
-exceeds the current cluster capacity, but they know that additional cluster
-resources will be added shortly.</p>
-
-<p><a name="rescindOffers"></a></p>
-
-<h3>Rescinding Outstanding Offers</h3>
-
-<p>When setting a new quota, the master rescinds outstanding offers. This avoids
-situations where the quota request cannot be satisfied by the remaining
-unoffered resources, but there are enough resources tied up in outstanding
-offers to frameworks that have not accepted them yet. Hence, we rescind
-outstanding offers with the following rules:</p>
-
-<ul>
-<li>Rescind at least as many resources as there are in the quota request.</li>
-<li>If at least one offer is to be rescinded from an agent, all offers from this
-agent are rescinded. This is done in order to make the potential offer bigger,
-which increases the chances that a quota'ed framework will be able to use the
-offer.</li>
-<li>Rescind offers from at least as many agents as there are frameworks subscribed
-to the role for which quota is being set. This enables (but does not guarantee,
-due to fair sharing) each framework subscribed to the role to receive an offer.</li>
-</ul>
+<p>Response:</p>
+
+<pre><code>{
+ "type": "GET_QUOTA",
+ "get_quota": {
+ "status": {
+ "infos": [
+ {
+ "configs" : [
+ {
+ "role": "dev",
+ "limits": {
+ "cpus": { "value": 10.0 },
+ "mem": { "value": 2048.0 },
+ "disk": { "value": 4096.0 }
+ }
+ },
+ {
+ "role": "test",
+ "limits": {
+ "cpus": { "value": 1.0 },
+ "mem": { "value": 256.0 },
+ "disk": { "value": 512.0 }
+ }
+ }
+ ]
+ }
+ ]
+ }
+ }
+}
+</code></pre>
+<p>To also view the quota consumption, use the <code>/roles</code> endpoint:</p>
-<p><a name="allocatorEnforcement"></a></p>
+<pre><code>$ curl http://<master-ip>:<master-port>/roles
+</code></pre>
-<h2>Enforcement by wDRF Allocator</h2>
+<p>Response</p>
+
+<pre><code>{
+ "roles": [
+ {
+ "name": "dev",
+ "weight": 1.0,
+ "quota":
+ {
+ "role": "dev",
+ "limit": {
+ "cpus": 10.0,
+ "mem": 2048.0,
+ "disk": 4096.0
+ },
+ "consumed": {
+ "cpus": 2.0,
+ "mem": 1024.0,
+ "disk": 2048.0
+ }
+ },
+ "allocated": {
+ "cpus": 2.0,
+ "mem": 1024.0,
+ "disk": 2048.0
+ },
+ "offered": {},
+ "reserved": {
+ "cpus": 2.0,
+ "mem": 1024.0,
+ "disk": 2048.0
+ },
+ "frameworks": []
+ }
+ ]
+}
+</code></pre>
-<p>The wDRF allocator first allocates (or lays away if offers are declined)
-resources to roles with quota set. Once all quotas are satisfied, it proceeds
-with the standard wDRF for all remaining roles.</p>
+<h3>Quota Consumption</h3>
-<p><strong>NOTE:</strong> A quota'ed role may not be allocated any unreserved non-revocable
-resources beyond its quota guarantee. If frameworks in the quota'ed role have
-not opted for revocable resources, they may stop getting offers once quota for
-the role is satisfied. In this case setting quota to any value that is less than
-the role’s fair share may reduce the amount of resources offered to this role.</p>
+<p>A role’s quota consumption consists of its allocations and reservations.
+In other words, even if reservations are not allocated, they are included
+in the quota consumption. Offered resources are not charged against quota.</p>
-<p><strong>NOTE:</strong> Currently quota guarantee also serves as quota limit, i.e. once quota
-for the role is satisfied, no further resources will be offered to the role
-except those reserved for the role. This behavior aims to mitigate the absence
-of quota limit and will be changed in future releases.</p>
+<h3>Metrics</h3>
-<p><strong>NOTE:</strong> Prior to Mesos 1.5, the allocator would allocate the entire available
-resources on an agent (including the resources that the role has no quota for)
-when trying to satisfy the quota of a role i.e. the quota allocation was
-coarse-grained. This has been fixed in Mesos 1.5. Specially, the allocator
-follows the steps below when allocating resources on an agent to a quota
-role:</p>
+<p>The following metric keys are exposed for quota:</p>
<ul>
-<li>Resources that this role has quota for are allocated up to the quota guarantee.</li>
-<li>Resources that this role has no quota for (including non-scalar resources) are
-allocated if (1) these resources are not needed for other roles' unsatisfied quotas
-and (2) this role is also getting some other resources on the same agent to meet its
-quota or reservation. The second condition is used to reduce fragmentation.</li>
+<li><code>allocator/mesos/quota/roles/<role>/resources/<resource>/guarantee</code></li>
+<li><code>allocator/mesos/quota/roles/<role>/resources/<resource>/limit</code></li>
+<li>A quota consumption metric will be added via
+<a href="https://issues.apache.org/jira/browse/MESOS-9123">MESOS-9123</a>.</li>
</ul>
-<p>If there are multiple frameworks subscribed to a role with quota set, the
-standard wDRF algorithm determines offer precedence amongst these frameworks.</p>
-
-<p>The default wDRF allocator considers only non-revocable resources as applicable
-towards quota.</p>
+<h2>Deprecated: Quota Guarantees</h2>
-<p><a name="failover"></a></p>
+<p>Prior to Mesos 1.9, the quota related APIs only exposed quota “guarantees”
+which ensured a minimum amount of resources would be available to a role.
+Setting guarantees also set implicit quota limits. In Mesos 1.9+, quota
+limits are now exposed directly per the above documentation.</p>
-<h2>Failover</h2>
+<p>Quota guarantees are now deprecated in favor of using only quota limits.
+Enforcement of quota guarantees required that Mesos holds back enough
+resources to meet all of the unsatisfied quota guarantees. Since Mesos is
+moving towards an optimistic offer model (to improve multi-role / multi-
+scheduler scalability, see
+<a href="https://issues.apache.org/jira/browse/MESOS-1607">MESOS-1607</a>), it will
+become no longer possible to enforce quota guarantees by holding back
+resources. In such a model, quota limits are simple to enforce, but quota
+guarantees would require a complex “effective limit” propagation model to
+leave space for unsatisfied guarantees.</p>
-<p>If there is at least one role with quota set, the master failover recovery
-changes significantly. The reason for this is that during the recovery there is
-a period of time when not all agents have registered with the Master. Therefore
-it is impossible to reason about whether all quotas are satisfied.</p>
-
-<p>To address this issue, if upon recovery any previously set quota are detected,
-the allocator enters recovery mode, during which the allocator
-<em>does not issue offers</em>. The recovery mode—and therefore offer
-suspension—ends when either:</p>
-
-<ul>
-<li>A certain amount of agents reregister (by default 80% of agents known before
-the failover), or</li>
-<li>a timeout expires (by default 10 minutes).</li>
-</ul>
+<p>For these reasons, quota guarantees, while still functional in Mesos 1.9,
+are now deprecated. A combination of limits and priority based preemption
+will be simpler in an optimistic offer model.</p>
+<p>For documentation on quota guarantees, please see the previous
+documentation: <a href="https://github.com/apache/mesos/blob/1.8.0/docs/quota.md">https://github.com/apache/mesos/blob/1.8.0/docs/quota.md</a></p>
-<h2>Current Limitations</h2>
+<h2>Implementation Notes</h2>
<ul>
-<li>The quota set request does not allow specifying the granularity of the
-requested resources (e.g., 10 CPUs on a single node).</li>
-<li>The quota set request does not allow to specify constraints (e.g., 2*5 cpus
-on disjoint nodes for an HA like setup).</li>
-<li>Quota is not allowed for the default role ‘*’ (see
-<a href="https://issues.apache.org/jira/browse/MESOS-3938">MESOS-3938</a>).</li>
-<li>Currently it is not possible to update previously configured quotas. See
-<a href="#setRequest">quota set request</a> for details.</li>
+<li>Quota is not supported for the default <code>*</code> role.</li>
+<li>Quota is not supported on nested roles (e.g. <code>eng/prod</code>).</li>
+<li>During failover, in order to correctly enforce limits, the allocator
+will be paused and will not issue offers until at least 80% agents
+re-register or 10 minutes elapses. These parameters will be made
+configurable: <a href="https://issues.apache.org/jira/browse/MESOS-4073">MESOS-4073</a></li>
</ul>