You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by ka...@apache.org on 2016/02/01 03:49:28 UTC
svn commit: r1727886 [8/16] - in /mesos/site: ./ publish/
publish/assets/img/documentation/ publish/community/ publish/documentation/
publish/documentation/allocation-module/
publish/documentation/app-framework-development-guide/
publish/documentation/...
Added: mesos/site/publish/documentation/latest/quota/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/quota/index.html?rev=1727886&view=auto
==============================================================================
--- mesos/site/publish/documentation/latest/quota/index.html (added)
+++ mesos/site/publish/documentation/latest/quota/index.html Mon Feb 1 02:49:25 2016
@@ -0,0 +1,466 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title></title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
+ <link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml">
+
+ <link href="../../../assets/css/main.css" media="screen" rel="stylesheet" type="text/css" />
+
+
+
+ <!-- Google Analytics Magic -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-20226872-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <!-- magical breadcrumbs -->
+ <div class="topnav">
+ <ul class="breadcrumb">
+ <li>
+ <div class="dropdown">
+ <a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a>
+ <ul class="dropdown-menu" role="menu" aria-labelledby="dLabel">
+ <li><a href="http://www.apache.org">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ </li>
+ <li><a href="http://mesos.apache.org">Apache Mesos</a></li>
+
+
+ <li><a href="/documentation
+/">Documentation
+</a></li>
+
+
+ </ul><!-- /breadcrumb -->
+ </div>
+
+ <!-- navbar excitement -->
+ <div class="navbar navbar-static-top" role="navigation">
+ <div class="navbar-inner">
+ <div class="container">
+ <a href="/" class="logo"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo" /></a>
+ <div class="nav-collapse">
+ <ul class="nav nav-pills navbar-right">
+ <li><a href="/gettingstarted/">Getting Started</a></li>
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Community</a></li>
+ </ul>
+ </div>
+ </div>
+ </div>
+ </div><!-- /.navbar -->
+
+ <div class="container">
+
+ <div class="row-fluid">
+ <div class="col-md-4">
+ <h4>If you're new to Mesos</h4>
+ <p>See the <a href="/gettingstarted/">getting started</a> page for more information about downloading, building, and deploying Mesos.</p>
+
+ <h4>If you'd like to get involved or you're looking for support</h4>
+ <p>See our <a href="/community/">community</a> page for more details.</p>
+ </div>
+ <div class="col-md-8">
+ <h1>Quota</h1>
+
+<p>A problem when running multiple frameworks on Mesos is that the default
+<a href="https://www.cs.berkeley.edu/~alig/papers/drf.pdf">wDRF allocator</a> may offer
+resources to frameworks even though they are beyond their fair share (e.g., when
+no other framework currently accepts these resources). There is no mechanism to
+lay away resources for future consumption in an entire cluster. Even though
+<a href="/documentation/latest/reservation/">dynamic reservations</a> allow both operators and frameworks
+to dynamically reserve resources on particular Mesos agents, it does not solve
+the above problem because any individual agents might fail.</p>
+
+<p>Since version 0.27.0 Mesos provides a mechanism to reserve a certain amount of
+resources in an entire cluster, i.e. not tied to a particular agent. <em>Quota</em>
+allows operators to lay away resources for a given <a href="/documentation/latest/roles/">role</a>. These
+resources cannot be hijacked by other roles and are guaranteed to be available
+for the role if there are enough resources in the cluster. Quota can be viewed
+as a cluster-wide dynamic reservation available for operators only.</p>
+
+<h1>Terminology</h1>
+
+<p>For the purpose of this document, an âOperatorâ is a person, tool, or script
+that manages the Mesos cluster.</p>
+
+<p>In computer science, a âquotaâ usually refers to one of the following:
+* A minimal guarantee.
+* A maximal limit.
+* A pair of both.</p>
+
+<p>In Mesos we understand quota as a <strong>guaranteed</strong> resource allocation that a role
+may rely on; in other words, a minimum share a role is entitled to receive.</p>
+
+<h1>Motivation and Limitations</h1>
+
+<p>Consider the following scenarios in a Mesos cluster to better understand what
+use-cases are supported by quota (and what are not).</p>
+
+<h2>Scenario 1: Greedy Framework</h2>
+
+<p>There are two frameworks in a cluster, each running in a separate role with
+equal weights: framework fA in role rA and framework fB in role rB. There is a
+single resource available in the cluster: 100 CPUs. fA consumes 10 CPUs and is
+idle (declines resource offers), while fB is greedy and accepts all offers it gets,
+hogging the remaining 90 CPUs. Without quota, though fAâs fair share is 50 CPUs
+it will not be able to make use of additional the 40 CPUs until some of fBâs
+tasks terminate.</p>
+
+<h2>Scenario 2: Resources for a new Framework</h2>
+
+<p>A greedy framework fB in role rB is currently the only framework in the cluster
+and it uses all available resources—100 CPUs. If a new framework fA in role rA
+joins the cluster, it will not receive its fair share of the cluster resources
+(50 CPUs) until some of fBâs tasks terminate.</p>
+
+<p>To deal with Scenario 2, quota by itself is not a sufficient solution as it
+would be set after fB has started using all resources. Instead Scenario 2
+requires either always keeping a pool of resources which are not offered or
+introducing preemption for running tasks.</p>
+
+<h1>Operator HTTP Endpoint</h1>
+
+<p>The master <code>/quota</code> HTTP endpoint enables operators to configure quotas. The
+endpoint currently offers a REST-like interface and supports the
+following operations:</p>
+
+<ul>
+<li><a href="#setRequest">Setting</a> a new quota with POST.</li>
+<li><a href="#removeRequest">Removing</a> an existing quota with DELETE.</li>
+<li><a href="#statusRequest">Querying</a> the currently set quota with GET.</li>
+</ul>
+
+
+<p>Currently it is not possible to update previously configured quotas. This means
+in order to update a quota for a given role, the operator has to remove the
+existing quota and then set a new one.</p>
+
+<p>The endpoint can optionally use authentication and authorization. See the
+<a href="/documentation/latest/authentication/">authentication guide</a> for details.</p>
+
+<p><a name="setRequest"></a></p>
+
+<h2>Set</h2>
+
+<p>The operator can set a new quota by sending an HTTP POST request to the <code>/quota</code>
+endpoint.</p>
+
+<p>An example request to the quota endpoint could look like this (using the JSON
+definitions below):</p>
+
+<pre><code>$ curl -d jsonMessageBody -X POST http://<master-ip>:<port>/quota
+</code></pre>
+
+<p>For example to set a quota of 12 CPUs and 6144 MB of RAM for <code>role1</code> the operator
+can use the following <code>jsonMessageBody</code>:</p>
+
+<pre><code> {
+ "role": "role1",
+ "guarantee": [
+ {
+ "name": "cpus",
+ "type": "SCALAR",
+ "scalar": { "value": 12 }
+ },
+ {
+ "name": "mem",
+ "type": "SCALAR",
+ "scalar": { "value": 6144 }
+ }
+ ]
+ }
+</code></pre>
+
+<p>A set request is only valid for roles for which no quota is currently set.
+However if the master is configured without an explicit
+<a href="/documentation/latest/roles/">role whitelist</a> a set request can introduce new roles.</p>
+
+<p>In order to bypass the <a href="#capacityHeuristic">capacity heuristic</a> check the
+operator should set an optional <code>force</code> field:</p>
+
+<pre><code> {
+ "force": true,
+ "role": "role1",
+ ...
+ }
+</code></pre>
+
+<p>The operator will receive one of the following HTTP response codes:</p>
+
+<ul>
+<li><code>200 OK</code>: Success (the set request was successful).</li>
+<li><code>400 BadRequest</code>: Invalid arguments (e.g., quota for role exists, invalid
+JSON, non-scalar resources included).</li>
+<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
+<li><code>403 Forbidden</code>: Unauthorized request.</li>
+<li><code>409 Conflict</code>: The capacity heuristic check failed due to insufficient
+resources.</li>
+</ul>
+
+
+<p><a name="removeRequest"></a></p>
+
+<h2>Remove</h2>
+
+<p>The operator can remove a previously set quota by sending an HTTP DELETE request
+to the <code>/quota/<role></code> endpoint. For example the following command removes
+a previously set quota for <code>role1</code>:</p>
+
+<pre><code>$ curl -X DELETE http://<master-ip>:<port>/quota/role1
+</code></pre>
+
+<p>The operator will receive one of the following HTTP response codes:</p>
+
+<ul>
+<li><code>200 OK</code>: Success (the remove request was successful).</li>
+<li><code>400 BadRequest</code>: Invalid arguments (e.g., removing a quota for a role which
+does not have any quota set).</li>
+<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
+<li><code>403 Forbidden</code>: Unauthorized request.</li>
+</ul>
+
+
+<p><a name="statusRequest"></a></p>
+
+<h2>Status</h2>
+
+<p>The operator can query the configured quotas by sending a HTTP GET request
+to the <code>/quota</code> endpoint.</p>
+
+<pre><code>$ curl -X GET http://<master-ip>:<port>/quota
+</code></pre>
+
+<p>The response message body includes a JSON representation of the current quota
+status, for example:</p>
+
+<pre><code> {
+ "infos": [
+ {
+ "role": "role1",
+ "guarantee": [
+ {
+ "name": "cpus",
+ "role": "*",
+ "type": "SCALAR",
+ "scalar": { "value": 12 }
+ },
+ {
+ "name": "mem",
+ "role": "*",
+ "type": "SCALAR",
+ "scalar": { "value": 6144 }
+ }
+ ]
+ }
+ ]
+ }
+</code></pre>
+
+<p>The operator will receive one of the following HTTP response codes:</p>
+
+<ul>
+<li><code>200 OK</code>: Success.</li>
+<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
+</ul>
+
+
+<h1>How does it work?</h1>
+
+<p>There are several stages in the lifetime of a quota issued by operator. First
+the <a href="#requestProcessing">quota set request is handled by the master</a>, after that
+the <a href="#allocatorEnforcement">allocator enforces the quota</a>.
+Quotas can be <a href="#removeProcessing">removed</a> by the operator.</p>
+
+<p>It is important to understand that the enforcement of quota depends on
+the allocator being used. A custom allocator could choose to handle quota in its
+own way or even to ignore quota.
+<strong>The following section assumes the default
+<a href="https://www.cs.berkeley.edu/~alig/papers/drf.pdf">wDRF</a> allocator is used.</strong></p>
+
+<p>Be aware that setting quota may affect other frameworks in the cluster,
+because resources will be laid away and not offered to other frameworks.
+Also note, that quota is only applicable for scalar resources (e.g., it is not
+possible to set quota for port resources).</p>
+
+<p><a name="requestProcessing"></a></p>
+
+<h2>Quota Request Processing</h2>
+
+<p>When an operator submits a quota set request via the master <code>/quota</code> HTTP
+endpoint, the following steps are triggered:</p>
+
+<ol>
+<li><a href="/documentation/latest/authentication/">Authenticate</a> the HTTP request.</li>
+<li>Parse and validate the request.
+See <a href="#setRequest">description of possible error codes</a>.</li>
+<li><a href="/documentation/latest/authentication/">Authorize</a> the HTTP request if authorization is enabled.</li>
+<li>Run the <a href="#capacityHeuristic">capacity heuristic</a> if not disabled by
+<a href="#setRequest">the <code>force</code> flag</a>.</li>
+<li>Reliably store quota. See <a href="#failover">details on failover recovery</a>.</li>
+<li><a href="#rescindOffers">Rescind outstanding offers</a>.</li>
+</ol>
+
+
+<p><a name="removeProcessing"></a>
+The quota remove request processing is simpler and triggers the following steps:</p>
+
+<ol>
+<li><a href="/documentation/latest/authentication/">Authenticate</a> the HTTP request.</li>
+<li>Validate the request.
+See <a href="#removeRequest">description of potential error codes</a>.</li>
+<li><a href="/documentation/latest/authentication/">Authorize</a> the HTTP request if authorization is enabled.</li>
+<li>Reliably remove quota.</li>
+</ol>
+
+
+<p><a name="capacityHeuristic"></a></p>
+
+<h3>Capacity Heuristic Check</h3>
+
+<p>Misconfigured quota can render a cluster into a state where no offers are made
+to any frameworks. For example imagine an operator setting quota 1000 CPUs
+for a role <code>prosuction</code> (note the typo) in a cluster with a total capacity of
+100 CPUs. In that case after the quota is accepted by the master, no offers will
+ be made to any framework in any actual role (including <code>production</code>).</p>
+
+<p>In order to prevent such extreme situations, the Mesos Master employs a capacity
+heuristic check that ensures a quota set request can reasonably be satisfied
+given the total cluster capacity. This heuristic tests whether the total quota,
+including the new request, does not exceed the sum of total
+non-statically-reserved cluster resources, i.e. the following inequality holds:</p>
+
+<pre><code>total resources - statically reserved >= total quota + quota request
+</code></pre>
+
+<p>Please be advised that even if there are enough resources at the moment of
+this check, agents may terminate at any time, rendering the cluster incapable
+of satisfying the configured quotas.</p>
+
+<p>A <a href="#setRequest"><code>force</code> flag</a> can be set to bypass this check. For example, this
+flag can be useful when the operator would like to configure a quota that
+exceeds the current cluster capacity, but they know that additional cluster
+resources will be added shortly.</p>
+
+<p><a name="rescindOffers"></a></p>
+
+<h3>Rescinding Outstanding Offers</h3>
+
+<p>When setting a new quota, the master rescinds outstanding offers. This avoids
+situations where the quota request cannot be satisfied by the remaining
+unoffered resources, but there are enough resources tied up in outstanding
+offers to frameworks that have not accepted them yet. Hence, we rescind
+outstanding offers with the following rules:</p>
+
+<ul>
+<li>Rescind at least as many resources as there are in the quota request.</li>
+<li>If at least one offer is to be rescinded from an agent, all offers from this
+agent are rescinded. This is done in order to make the potential offer bigger,
+which increases the chances that a quota'ed framework will be able to use the
+offer.</li>
+<li>Rescind offers from at least as many agents as there are frameworks in the
+role for which quota is being set. This enables (but does not guarantee, due
+to fair sharing) each framework in the role to receive an offer.</li>
+</ul>
+
+
+<p><a name="allocatorEnforcement"></a></p>
+
+<h2>Enforcement by wDRF Allocator</h2>
+
+<p>The wDRF allocator first allocates (or lays away if offers are declined)
+resources to framework in roles with quota set. Once all quotas are
+satisfied, it proceeds with the standard wDRF for all frameworks.</p>
+
+<p><strong>NOTE:</strong> A quota'ed role may not be allocated any unreserved non-revocable
+ resources beyond its quota guarantee. If frameworks in the quota'ed
+ role have not opted for revocable resources, they may stop getting
+ offers once quota for the role is satisfied. In this case setting
+ quota to any value that is less than the role’s fair share may reduce
+ the amount of resources offered to this role.</p>
+
+<p><strong>NOTE:</strong> In Mesos 0.27 quota guarantee also serves as quota limit, i.e. once
+ quota for the role is satisfied, frameworks in this role will not
+ be offered any resources except those reserved for the role. This
+ behavior aims to mitigate the absence of quota limit and will be
+ changed in future releases.</p>
+
+<p>If there are multiple frameworks in a role with quota set, the standard wDRF
+algorithm determines framework priority inside this role.</p>
+
+<p>The default wDRF allocator considers only non-revocable resources as applicable
+towards quota.</p>
+
+<p><a name="failover"></a></p>
+
+<h2>Failover</h2>
+
+<p>If there is at least one role with quota set, the master failover recovery
+changes significantly. The reason for this is that during the recovery there is
+a period of time when not all agents have registered with the Master. Therefore
+it is impossible to reason about whether all quotas are satisfied.</p>
+
+<p>To address this issue, if upon recovery any previously set quota are detected,
+the allocator enters recovery mode, during which the allocator
+<em>does not issue offers</em>. The recovery mode—and therefore offer
+suspension—ends when either:</p>
+
+<ul>
+<li>A certain amount of agents reregister (by default 80% of agents known before
+the failover), or</li>
+<li>a timeout expires (by default 10 minutes).</li>
+</ul>
+
+
+<h1>Current Limitations</h1>
+
+<ul>
+<li>The quota set request does not allow specifying the granularity of the
+requested resources (e.g. 10 CPUs on a single node).</li>
+<li>The quota set request does not allow to specify constraints (e.g. 2*5 cpus on
+disjoint nodes for an HA like setup).</li>
+<li>Quota is not allowed for the default role â*â (see MESOS-3938).</li>
+<li>Currently it is not possible to update previously configured quotas. See
+<a href="#setRequest">quota set request</a> for details.</li>
+</ul>
+
+
+ </div>
+</div>
+
+
+ <hr>
+
+ <!-- footer -->
+ <div class="footer">
+ <p>© 2012-2015 <a href="http://apache.org">The Apache Software Foundation</a>.
+ Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation.<p>
+ </div><!-- /footer -->
+
+ </div> <!-- /container -->
+
+ <!-- JS -->
+ <script src="//code.jquery.com/jquery-1.11.0.min.js" type="text/javascript"></script>
+ <script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js" type="text/javascript"></script>
+ </body>
+</html>
Modified: mesos/site/publish/documentation/latest/release-guide/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/release-guide/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/release-guide/index.html (original)
+++ mesos/site/publish/documentation/latest/release-guide/index.html Mon Feb 1 02:49:25 2016
@@ -162,6 +162,7 @@ the master, slave, and configure flags.<
a live cluster from the previous release version to this release version.</p></li>
<li><p>If this is a major release, please ensure that user documentation has been
added for any new features.</p></li>
+<li><p>Make sure that for any updates of the API, specifically the scheduler API, the public mesos protobuf definitions are part of both, <code>include/mesos</code> as well as <code>include/mesos/v1</code>. NOTE: This might actually demand code updates if any omissions were identified.</p></li>
</ol>
@@ -170,7 +171,11 @@ added for any new features.</p></li>
<ol>
<li><p>Ensure that you can build and pass all the tests.</p>
-<pre><code> $ sudo make -j3 distcheck
+<pre><code> $ sudo make -j<cores> distcheck
+</code></pre></li>
+<li><p>Run the benchmarks and compare with the previous release for any performance regressions:</p>
+
+<pre><code> $ make bench -j<cores> GTEST_FILTER="*BENCHMARK*"
</code></pre></li>
<li><p>First tag the required SHA locally.</p>
@@ -192,7 +197,7 @@ It is recommended to use the <code>suppo
You may need to modify your local copy of tag.sh for it to complete successfully.</p></blockquote></li>
<li><p>It is not uncommon to release multiple release candidates, with increasing release candidate
version, if there are bugs found.</p></li>
-<li><p>Update to the <em>next</em> Mesos version in <code>configure.ac</code>: change <code>AC_INIT([mesos], [X.Y.Z]))</code> and commit.</p></li>
+<li><p>Update to the <em>next</em> Mesos version in <code>configure.ac</code>: change <code>AC_INIT([mesos], [X.Y.Z]))</code>, as well as in <code>CMakeLists.txt</code>: change <code>set(MESOS_MAJOR_VERSION X)</code>, <code>set(MESOS_MINOR_VERSION Y)</code>, <code>set(MESOS_PATCH_VERSION Z)</code> and then commit.</p></li>
</ol>
Modified: mesos/site/publish/documentation/latest/reservation/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/reservation/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/reservation/index.html (original)
+++ mesos/site/publish/documentation/latest/reservation/index.html Mon Feb 1 02:49:25 2016
@@ -107,7 +107,7 @@ We start the slave like so:</p>
<p>We now have 8 CPUs and 4096 MB of RAM reserved for <code>ads</code> on this slave.</p>
<p><strong>CAVEAT:</strong> In order to modify a static reservation, the operator must drain and
- restart the slave with the new configuration specifed in the
+ restart the slave with the new configuration specified in the
<code>--resources</code> flag.</p>
<p><strong>NOTE:</strong> This feature is supported for backwards compatibility.
@@ -117,32 +117,46 @@ We start the slave like so:</p>
<h2>Dynamic Reservation (since 0.23.0)</h2>
-<p>As mentioned in <a href="#static-reservation-since-0140">Static Reservation</a>, specifying the
-reserved resources via the <code>--resources</code> flag makes the reservation static.
+<p>As mentioned in <a href="#static-reservation-since-0140">Static Reservation</a>, specifying
+the reserved resources via the <code>--resources</code> flag makes the reservation static.
This is, statically reserved resources cannot be reserved for another role nor
be unreserved. Dynamic Reservation enables operators and authorized frameworks
to reserve and unreserve resources post slave-startup.</p>
<p>We require a <code>principal</code> from the operator or framework in order to
-authenticate/authorize the operations. <a href="/documentation/latest/authorization/">Authorization</a> is
-specified via the existing ACL mechanism. (<em>Coming Soon</em>)</p>
+authenticate/authorize the operations. Permissions are specified via the
+existing ACL mechanism. To use authorization with reserve/unreserve operations,
+the Mesos master must be configured with the desired ACLs. For more information,
+see the <a href="/documentation/latest/authorization/">authorization documentation</a>.</p>
<ul>
<li><code>Offer::Operation::Reserve</code> and <code>Offer::Operation::Unreserve</code> messages are
available for <strong>frameworks</strong> to send back via the <code>acceptOffers</code> API as a
response to a resource offer.</li>
-<li><code>/reserve</code> and <code>/unreserve</code> HTTP endpoints are available for <strong>operators</strong>
-to manage dynamic reservations through the master. (<em>Coming Soon</em>).</li>
+<li><code>/reserve</code> and <code>/unreserve</code> HTTP endpoints allow <strong>operators</strong> to manage
+dynamic reservations through the master. NOTE: As of 0.27.0, these endpoints
+cannot be used when HTTP authentication is disabled due to the current
+implementation. This will change in version 0.28.0.</li>
</ul>
<p>In the following sections, we will walk through examples of each of the
interfaces described above.</p>
-<h3><code>Offer::Operation::Reserve</code></h3>
+<p>Note that if two dynamic reservations are made for resources at a single slave,
+the reservations will be combined by adding together the resources reserved by
+each request. Similarly, “partial” unreserve operations are allowed: an
+unreserve operation can release only some of the resources at a slave that have
+been reserved for a given role. In this case, the unreserved resources will be
+subtracted from the previous reservation, and any remaining resources will still
+be reserved.</p>
-<p>A framework is able to reserve resources through the resource offer cycle.
-Suppose we receive a resource offer with 12 CPUs and 6144 MB of RAM unreserved.</p>
+<h3>Framework Scheduler API</h3>
+
+<h4><code>Offer::Operation::Reserve</code></h4>
+
+<p>A framework can reserve resources through the resource offer cycle. Suppose we
+receive a resource offer with 12 CPUs and 6144 MB of RAM unreserved.</p>
<pre><code> {
"id": <offer_id>,
@@ -168,7 +182,7 @@ Suppose we receive a resource offer with
<p>We can reserve 8 CPUs and 4096 MB of RAM by sending the following
<code>Offer::Operation</code> message. <code>Offer::Operation::Reserve</code> has a <code>resources</code> field
-which we specify with the resources to be reserved. We need to expicitly set
+which we specify with the resources to be reserved. We need to explicitly set
the <code>role</code> and <code>principal</code> fields with the framework’s role and principal.</p>
<pre><code> {
@@ -198,7 +212,8 @@ the <code>role</code> and <code>principa
}
</code></pre>
-<p>The subsequent resource offer will <strong>contain</strong> the following reserved resources:</p>
+<p>If the reservation is successful, a subsequent resource offer will contain the
+following reserved resources:</p>
<pre><code> {
"id": <offer_id>,
@@ -228,13 +243,14 @@ the <code>role</code> and <code>principa
}
</code></pre>
-<h3><code>Offer::Operation::Unreserve</code></h3>
+<h4><code>Offer::Operation::Unreserve</code></h4>
-<p>A framework is able to unreserve resources through the resource offer cycle.
+<p>A framework can unreserve resources through the resource offer cycle.
In <a href="#offeroperationreserve">Offer::Operation::Reserve</a>, we reserved 8 CPUs
-and 4096 MB of RAM for our <code>role</code>. The master will continue to offer these
-resources to our <code>role</code>. Suppose we would like to unreserve these resources.
-First, we receive a resource offer (copy/pasted from above):</p>
+and 4096 MB of RAM on a particular slave for our <code>role</code>. The master will
+continue to only offer these resources to our <code>role</code>. Suppose we would like to
+unreserve these resources. First, we receive a resource offer (copy/pasted
+from above):</p>
<pre><code> {
"id": <offer_id>,
@@ -264,9 +280,9 @@ First, we receive a resource offer (copy
}
</code></pre>
-<p>We unreserve the 8 CPUs and 4096 MB of RAM by sending the following
+<p>We can unreserve the 8 CPUs and 4096 MB of RAM by sending the following
<code>Offer::Operation</code> message. <code>Offer::Operation::Unreserve</code> has a <code>resources</code> field
-which we specify with the resources to be unreserved.</p>
+which we can use to specify the resources to be unreserved.</p>
<pre><code> {
"type": Offer::Operation::UNRESERVE,
@@ -297,34 +313,44 @@ which we specify with the resources to b
<p>The unreserved resources may now be offered to other frameworks.</p>
-<h3><code>/reserve</code> (since 0.25.0)</h3>
+<h3>Operator HTTP Endpoints</h3>
-<p>Suppose we want to reserve 8 CPUs and 4096 MB of RAM for the <code>ads</code> role on
-a slave with id=<code><slave_id></code>. We send an HTTP POST request to the <code>/reserve</code>
-HTTP endpoint like so:</p>
+<p>As described above, dynamic reservations can be made by a framework scheduler,
+typically in response to a resource offer. However, dynamic reservations can
+also be created and deleted by sending HTTP requests to the <code>/reserve</code> and
+<code>/unreserve</code> endpoints, respectively. This capability is intended for use by
+operators and administrative tools.</p>
+
+<h4><code>/reserve</code> (since 0.25.0)</h4>
+
+<p>Suppose we want to reserve 8 CPUs and 4096 MB of RAM for the <code>ads</code> role on a
+slave with id=<code><slave_id></code> (note that it is up to the user to find the ID of the
+slave that hosts the desired resources; the request will fail if sufficient
+unreserved resources cannot be found on the slave). We send an HTTP POST request
+to the <code>/reserve</code> HTTP endpoint like so:</p>
<pre><code> $ curl -i \
-u <operator_principal>:<password> \
-d slaveId=<slave_id> \
- -d resources='[ \
- { \
- "name": "cpus", \
- "type": "SCALAR", \
- "scalar": { "value": 8 }, \
- "role": "ads", \
- "reservation": { \
- "principal": <operator_principal> \
- } \
- }, \
- { \
- "name": "mem", \
- "type": "SCALAR", \
- "scalar": { "value": 4096 }, \
- "role": "ads", \
- "reservation": { \
- "principal": <operator_principal> \
- } \
- } \
+ -d resources='[
+ {
+ "name": "cpus",
+ "type": "SCALAR",
+ "scalar": { "value": 8 },
+ "role": "ads",
+ "reservation": {
+ "principal": <operator_principal>
+ }
+ },
+ {
+ "name": "mem",
+ "type": "SCALAR",
+ "scalar": { "value": 4096 },
+ "role": "ads",
+ "reservation": {
+ "principal": <operator_principal>
+ }
+ }
]' \
-X POST http://<ip>:<port>/master/reserve
</code></pre>
@@ -332,14 +358,15 @@ HTTP endpoint like so:</p>
<p>The user receives one of the following HTTP responses:</p>
<ul>
-<li><code>200 OK</code>: Success</li>
-<li><code>400 BadRequest</code>: Invalid arguments (e.g. missing parameters).</li>
-<li><code>401 Unauthorized</code>: Unauthorized request.</li>
+<li><code>200 OK</code>: Success (the requested resources have been reserved).</li>
+<li><code>400 BadRequest</code>: Invalid arguments (e.g., missing parameters).</li>
+<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
+<li><code>403 Forbidden</code>: Unauthorized request.</li>
<li><code>409 Conflict</code>: Insufficient resources to satisfy the reserve operation.</li>
</ul>
-<h3><code>/unreserve</code> (since 0.25.0)</h3>
+<h4><code>/unreserve</code> (since 0.25.0)</h4>
<p>Suppose we want to unreserve the resources that we dynamically reserved above.
We can send an HTTP POST request to the <code>/unreserve</code> HTTP endpoint like so:</p>
@@ -347,25 +374,25 @@ We can send an HTTP POST request to the
<pre><code> $ curl -i \
-u <operator_principal>:<password> \
-d slaveId=<slave_id> \
- -d resources='[ \
- { \
- "name": "cpus", \
- "type": "SCALAR", \
- "scalar": { "value": 8 }, \
- "role": "ads", \
- "reservation": { \
- "principal": <operator_principal> \
- } \
- }, \
- { \
- "name": "mem", \
- "type": "SCALAR", \
- "scalar": { "value": 4096 }, \
- "role": "ads", \
- "reservation": { \
- "principal": <operator_principal> \
- } \
- } \
+ -d resources='[
+ {
+ "name": "cpus",
+ "type": "SCALAR",
+ "scalar": { "value": 8 },
+ "role": "ads",
+ "reservation": {
+ "principal": <operator_principal>
+ }
+ },
+ {
+ "name": "mem",
+ "type": "SCALAR",
+ "scalar": { "value": 4096 },
+ "role": "ads",
+ "reservation": {
+ "principal": <operator_principal>
+ }
+ }
]' \
-X POST http://<ip>:<port>/master/unreserve
</code></pre>
@@ -373,10 +400,11 @@ We can send an HTTP POST request to the
<p>The user receives one of the following HTTP responses:</p>
<ul>
-<li><code>200 OK</code>: Success</li>
-<li><code>400 BadRequest</code>: Invalid arguments (e.g. missing parameters).</li>
-<li><code>401 Unauthorized</code>: Unauthorized request.</li>
-<li><code>409 Conflict</code>: Insufficient resources to satisfy unreserve operation.</li>
+<li><code>200 OK</code>: Success (the requested resources have been unreserved).</li>
+<li><code>400 BadRequest</code>: Invalid arguments (e.g., missing parameters).</li>
+<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
+<li><code>403 Forbidden</code>: Unauthorized request.</li>
+<li><code>409 Conflict</code>: Insufficient resources to satisfy the unreserve operation.</li>
</ul>
Modified: mesos/site/publish/documentation/latest/roles/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/roles/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/roles/index.html (original)
+++ mesos/site/publish/documentation/latest/roles/index.html Mon Feb 1 02:49:25 2016
@@ -92,7 +92,7 @@ resources are offered to frameworks. Som
<li>arranging for all the resources on a particular agent to only be offered to a
particular framework.</li>
<li>dividing a cluster between two organizations: resources assigned for use by
-organization <em>A</em> will only be offered to that frameworks that have registered
+organization <em>A</em> will only be offered to frameworks that have registered
using organization <em>A</em>’s role.</li>
<li>ensuring that <a href="/documentation/latest/persistent-volume/">persistent volumes</a> created by one
framework are not offered to frameworks registered with a different role.</li>
@@ -101,14 +101,28 @@ framework are not offered to frameworks
</ul>
-<h2>Defining roles</h2>
+<h2>Roles and access control</h2>
-<p>The set of legal roles is configured statically, when the Mesos master is
-started. The <code>--roles</code> command-line argument specifies a comma-separated list of
-role names. To change the set of roles, the Mesos master must be restarted.</p>
-
-<p>Note that you should take care to ensure that all Mesos masters are configured
-to use the same set of roles.</p>
+<p>There are two ways to control which roles a framework is allowed to register
+as. First, ACLs can be used to specify which framework principals can register
+as which roles. For more information, see the <a href="/documentation/latest/authorization/">authorization</a>
+documentation.</p>
+
+<p>Second, a <em>role whitelist</em> can be configured by passing the <code>--roles</code> flag to
+the Mesos master at startup. This flag specifies a comma-separated list of role
+names. If the whitelist is specified, only roles that appear in the whitelist
+can be used. To change the whitelist, the Mesos master must be restarted. Note
+that in a high-availability deployment of Mesos, you should take care to ensure
+that all Mesos masters are configured with the same whitelist.</p>
+
+<p>In Mesos 0.26 and earlier, you should typically configure <em>both</em> ACLs and the
+whitelist, because in these versions of Mesos, any role that does not appear in
+the whitelist cannot be used.</p>
+
+<p>In Mesos 0.27, this behavior has changed: if <code>--roles</code> is not specified, the
+whitelist permits <em>any role name</em> to be used. Hence, in Mesos 0.27, the
+recommended practice is to only use ACLs to define which roles can be used; the
+<code>--roles</code> command-line flag is deprecated.</p>
<h2>Associating frameworks with roles</h2>
@@ -123,9 +137,26 @@ start the framework. How to do this depe
framework you’re using; for example, Marathon takes a <code>--mesos_role</code>
command-line flag.</p>
-<p>As an administrator, you can use ACLs to specify which framework principals can
-register as which roles. For more information, see the
-<a href="/documentation/latest/authorization/">authorization</a> documentation.</p>
+<p><a id="roles-multiple-frameworks"></a></p>
+
+<h3>Multiple frameworks in the same role</h3>
+
+<p>Multiple frameworks can use the same role. This can be useful: for example, one
+framework can create a persistent volume and write data to it. Once the task
+that writes data to the persistent volume has finished, the volume will be
+offered to other frameworks in the same role; this might give a second
+(“consumer”) framework the opportunity to launch a task that reads the data
+produced by the first (“producer”) framework.</p>
+
+<p>However, configuring multiple frameworks to use the same role should be done
+with caution, because all the frameworks will have access to any resources that
+have been reserved for that role. For example, if a framework stores sensitive
+information on a persistent volume, that volume might be offered to a different
+framework in the same role. Similarly, if one framework creates a persistent
+volume, another framework in the same role might “steal” the volume and use it
+to launch a task of its own. In general, multiple frameworks sharing the same
+role should be prepared to collaborate with one another to ensure that
+role-specific resources are used appropriately.</p>
<h2>Associating resources with roles</h2>
@@ -150,6 +181,18 @@ unreserving the resource, e.g., using th
endpoint). Similarly, persistent volumes cannot be created on unreserved
resources.</p>
+<h2>Invalid role</h2>
+
+<p>A role name must be a valid directory name, so it cannot:</p>
+
+<ul>
+<li>Be an empty string</li>
+<li>Be <code>.</code> or <code>..</code></li>
+<li>Start with <code>-</code></li>
+<li>Contain any slash, backspace, or whitespace character</li>
+</ul>
+
+
<h2>Roles and resource allocation</h2>
<p>By default, the Mesos master uses Dominant Resource Fairness (DRF) to allocate
Modified: mesos/site/publish/documentation/latest/sandbox/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/sandbox/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/sandbox/index.html (original)
+++ mesos/site/publish/documentation/latest/sandbox/index.html Mon Feb 1 02:49:25 2016
@@ -257,7 +257,7 @@ After the delay, the files are deleted.<
<p>Additionally, according to the <code>--disk_watch_interval</code> agent flag, files
scheduled for garbage collection are pruned based on the available disk and
the <code>--gc_disk_headroom</code> agent flag.
-See <a href="/documentation/latest/configuration/#gc_disk_headroom">the formula here</a>.</p>
+See <a href="configuration.md#gc_disk_headroom">the formula here</a>.</p>
</div>
</div>
Modified: mesos/site/publish/documentation/latest/scheduler-http-api/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/scheduler-http-api/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/scheduler-http-api/index.html (original)
+++ mesos/site/publish/documentation/latest/scheduler-http-api/index.html Mon Feb 1 02:49:25 2016
@@ -100,6 +100,8 @@
<p>The following calls are currently accepted by the master. The canonical source of this information is <a href="https://github.com/apache/mesos/blob/master/include/mesos/v1/scheduler/scheduler.proto">scheduler.proto</a> (NOTE: The protobuf definitions are subject to change before the beta API is finalized). Note that when sending JSON encoded Calls, schedulers should encode raw bytes in Base64 and strings in UTF-8.</p>
+<p><a id="recordio-response-format"></a></p>
+
<h3>RecordIO response format</h3>
<p>The response returned from the <code>SUBSCRIBE</code> call (see <a href="#subscribe">below</a> is encoded in RecordIO format, which essentially prepends to a single record (either JSON or serialized Protobuf) its length in bytes, followed by a newline and then the data:</p>
@@ -139,7 +141,9 @@
<p>Network intermediaries e.g. proxies are free to change the chunk boundaries and this should not have any effect on the recipient application (scheduler layer). We wanted a way to delimit/encode two events for JSON/Protobuf responses consistently and RecordIO format allowed us to do that.</p>
-<h3><a id="subscribe"></a>SUBSCRIBE</h3>
+<p><a id="subscribe"></a></p>
+
+<h3>SUBSCRIBE</h3>
<p>This is the first step in the communication process between the scheduler and the master. This is also to be considered as subscription to the “/scheduler” events stream.</p>
@@ -440,7 +444,7 @@ HTTP/1.1 202 Accepted
<h3>OFFERS</h3>
-<p>Sent by the master whenever there are new resources that can be offered to the framework. Each offer corresponds to a set of resources on a agent. Until the scheduler ‘Accept’s or 'Decline’s an offer the resources are considered allocated to the scheduler, unless the offer is otherwise rescinded, e.g. due to a lost agent or <code>--offer_timeout</code>.</p>
+<p>Sent by the master whenever there are new resources that can be offered to the framework. Each offer corresponds to a set of resources on an agent. Until the scheduler ‘Accept’s or 'Decline’s an offer the resources are considered allocated to the scheduler, unless the offer is otherwise rescinded, e.g. due to a lost agent or <code>--offer_timeout</code>.</p>
<pre><code>OFFERS Event (JSON)
<event-length>
@@ -515,7 +519,7 @@ HTTP/1.1 202 Accepted
<h3>FAILURE</h3>
-<p>Sent by the master when a agent is removed from the cluster (e.g., failed health checks) or when an executor is terminated. Note that, this event coincides with receipt of terminal <code>UPDATE</code> events for any active tasks belonging to the agent or executor and receipt of <code>RESCIND</code> events for any outstanding offers belonging to the agent. Note that there is no guaranteed order between the <code>FAILURE</code>, <code>UPDATE</code> and <code>RESCIND</code> events.</p>
+<p>Sent by the master when an agent is removed from the cluster (e.g., failed health checks) or when an executor is terminated. Note that, this event coincides with receipt of terminal <code>UPDATE</code> events for any active tasks belonging to the agent or executor and receipt of <code>RESCIND</code> events for any outstanding offers belonging to the agent. Note that there is no guaranteed order between the <code>FAILURE</code>, <code>UPDATE</code> and <code>RESCIND</code> events.</p>
<pre><code>FAILURE Event (JSON)
@@ -576,7 +580,7 @@ HTTP/1.1 202 Accepted
<h2>Master detection</h2>
-<p>Mesos has a high-availability mode that uses multiple Mesos masters; one active master (called the leader or leading master) and several standbys in case it fails. The masters elect the leader, with ZooKeeper coordinating the election. For more details please refer to the <a href="/documentation/latest//documentation/latest/high-availability/">documentation</a>.</p>
+<p>Mesos has a high-availability mode that uses multiple Mesos masters; one active master (/documentation/latest/called the leader or leading master) and several standbys in case it fails. The masters elect the leader, with ZooKeeper coordinating the election. For more details please refer to the <a href="high-availability/">documentation</a>.</p>
<p>Schedulers are expected to make HTTP requests to the leading master. If requests are made to a non-leading master a “HTTP 307 Temporary Redirect” will be received with the “Location” header pointing to the leading master.</p>
Modified: mesos/site/publish/documentation/latest/slave-recovery/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/slave-recovery/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/slave-recovery/index.html (original)
+++ mesos/site/publish/documentation/latest/slave-recovery/index.html Mon Feb 1 02:49:25 2016
@@ -95,7 +95,7 @@
<h2>How does it work?</h2>
-<p>Slave recovery works by having the slave checkpoint enough information (e.g., Task Info, Executor Info, Status Updates) about the running tasks and executors to local disk. Once the slave <strong><em>and</em></strong> the framework(s) enable checkpointing, any subsequent slave restarts would recover the checkpointed information and reconnect with the executors. Note that if the host running the slave process is rebooted all the executors/tasks are killed.</p>
+<p>Slave recovery works by having the slave checkpoint enough information (e.g., Task Info, Executor Info, Status Updates) about the running tasks and executors to local disk. Once a framework enables checkpointing, any subsequent slave restarts would recover the checkpointed information and reconnect with the executors. Note that if the host running the slave process is rebooted all the executors/tasks are killed.</p>
<blockquote><p>NOTE: To enable recovery the framework should explicitly request checkpointing.
Alternatively, a framework that doesn’t want the disk i/o overhead of checkpointing can opt out of checkpointing.</p></blockquote>
@@ -161,9 +161,6 @@ Therefore, it is highly recommended to a
<p>As part of this feature, <code>FrameworkInfo</code> has been updated to include an optional <code>checkpoint</code> field. A framework that would like to opt in to checkpointing should set <code>FrameworkInfo.checkpoint=True</code> before registering with the master.</p>
-<blockquote><p>NOTE: Frameworks that have enabled checkpointing will only get offers from checkpointing slaves. So, before setting <code>checkpoint=True</code> on FrameworkInfo, ensure that there are slaves in your cluster that have enabled checkpointing.
-Because, if there are no checkpointing slaves, the framework would not get any offers and hence cannot launch any tasks/executors!</p></blockquote>
-
<h2>Known issues with <code>systemd</code> and POSIX isolation</h2>
<p>There is a known issue when using <code>systemd</code> to launch the <code>mesos-slave</code> while also using only <code>posix</code> isolation mechanisms that prevents tasks from recovering. The problem is that the default <a href="http://www.freedesktop.org/software/systemd/man/systemd.kill.html">KillMode</a> for systemd processes is <code>cgroup</code> and hence all child processes are killed when the slave stops. Explicitly setting <code>KillMode</code> to <code>process</code> allows the executors to survive and reconnect.</p>
Modified: mesos/site/publish/documentation/latest/submitting-a-patch/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/submitting-a-patch/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/submitting-a-patch/index.html (original)
+++ mesos/site/publish/documentation/latest/submitting-a-patch/index.html Mon Feb 1 02:49:25 2016
@@ -115,7 +115,7 @@
<ol>
<li>To find a shepherd, email the dev mailing list (include a link to your JIRA issue). You can also try asking by adding a comment to your JIRA issue.</li>
-<li>You can also find a shepherd by asking the developers on IRC (in the <a href="irc://irc.freenode.net/mesos">mesos channel</a> on <a href="https://freenode.net">Freenode</a>). You can find the current list of committers <a href="/documentation/latest/committers/">here</a>: a developer that has previously worked on the component you are modifying might be a good candidate shepherd.</li>
+<li>You can also find a shepherd by asking the developers on IRC (/documentation/latest/in the <a href="irc://irc.freenode.net/mesos">mesos channel</a> on <a href="https://freenode.net">Freenode</a>). You can find the current list of committers <a href="committers/">here</a>: a developer that has previously worked on the component you are modifying might be a good candidate shepherd.</li>
</ol>
</li>
</ol>
@@ -154,11 +154,12 @@
<li>To execute a single unit test (helpful when trying to debug a test case failure), use something like: <code>make check GTEST_FILTER="HTTPTest.Delete"</code>.</li>
</ol>
</li>
-<li><p>Divide your change into one or more Git commits. Each commit should represent a single logical (atomic) change to the Mesos source code: this makes your changes easier to review. For more information, see the <a href="/documentation/latest/effective-code-reviewing/">reviewer guidelines</a>.</p>
+<li><p>Divide your change into one or more Git commits. Each commit should represent a single logical (/documentation/latest/atomic) change to the Mesos source code: this makes your changes easier to review. For more information, see the <a href="effective-code-reviewing/">reviewer guidelines</a>.</p>
<ol>
<li>Try to avoid including other, unrelated cleanups (e.g., typo fixes or style nits) in the same commit that makes functional changes. While typo fixes are great, including them in the same commit as functional changes makes the commit history harder to read.</li>
<li>Developers often make incremental commits to save their progress when working on a change, and then “rewrite history” (e.g., using <code>git rebase -i</code>) to create a clean set of commits once the change is ready to be reviewed.</li>
+<li>Commit messages should be in past tense. The first sentence should summarize the change; it should start with a capital letter, not exceed 72 characters and end in a period.</li>
</ol>
</li>
<li><p>Make sure to pull in any changes that have been committed to master branch. Using Git, do this via something like:</p>
Modified: mesos/site/publish/documentation/latest/tools/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/tools/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/tools/index.html (original)
+++ mesos/site/publish/documentation/latest/tools/index.html Mon Feb 1 02:49:25 2016
@@ -103,7 +103,7 @@
<p>If you want to hack on Mesos or write a new framework, these tools will help.</p>
<ul>
-<li><a href="/documentation/latest/clang-format/">clang-format</a> to automatically apply some of the style rules dictated by the <a href="/documentation/latest/c++-style-guide/">Mesos C++ Style Guide</a>.</li>
+<li><a href="/documentation/latest/clang-format.md">clang-format</a> to automatically apply some of the style rules dictated by the <a href="c++-style-guide/">Mesos C++ Style Guide</a>.</li>
<li><a href="https://github.com/mesosphere/mesos-go">Go Bindings and Examples</a> Write a Mesos framework in Go! Comes with an example scheduler and executor.</li>
<li><a href="https://github.com/mesosphere/scala-sbt-mesos-framework.g8">Mesos Framework giter8 Template</a> This is a giter8 template. The result of applying this template is a bare-bones Apache Mesos framework in Scala using SBT for builds and Vagrant for testing on a singleton cluster.</li>
<li><a href="https://gist.github.com/guenter/7471695">Scala Hello World</a> A simple Mesos “Hello World”: downloads and starts a web server on every node in the cluster.</li>
Modified: mesos/site/publish/documentation/latest/upgrades/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/upgrades/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/upgrades/index.html (original)
+++ mesos/site/publish/documentation/latest/upgrades/index.html Mon Feb 1 02:49:25 2016
@@ -83,7 +83,19 @@
<div class="col-md-8">
<h1>Upgrading Mesos</h1>
-<p>This document serves as a guide for users who wish to upgrade an existing mesos cluster. Some versions require particular upgrade techniques when upgrading a running cluster. Some upgrades will have incompatible changes.</p>
+<p>This document serves as a guide for users who wish to upgrade an existing Mesos cluster. Some versions require particular upgrade techniques when upgrading a running cluster. Some upgrades will have incompatible changes.</p>
+
+<h2>Upgrading from 0.26.x to 0.27.x</h2>
+
+<ul>
+<li><p>Mesos 0.27 introduces the concept of <em>implicit roles</em>. In previous releases, configuring roles required specifying a static whitelist of valid role names on master startup (/documentation/latest/via the <code>--roles</code> flag). In Mesos 0.27, if <code>--roles</code> is omitted, <em>any</em> role name can be used; controlling which principals are allowed to register as which roles should be done using <a href="authorization/">ACLs</a>. The role whitelist functionality is still supported but is deprecated.</p></li>
+<li><p>The Allocator API has changed due to the introduction of implicit roles. Custom allocator implementations will need to be updated. See <a href="https://issues.apache.org/jira/browse/MESOS-4000">MESOS-4000</a> for more information.</p></li>
+<li><p>The <code>executorLost</code> callback in the Scheduler interface will now be called whenever the slave detects termination of a custom executor. This callback was never called in previous versions, so please make sure any framework schedulers can now safely handle this callback. Note that this callback may not be reliably delivered.</p></li>
+<li><p>The isolator <code>prepare</code> interface has been changed slightly. Instead of keeping adding parameters to the <code>prepare</code> interface, we decide to use a protobuf (<code>ContainerConfig</code>). Also, we renamed <code>ContainerPrepareInfo</code> to <code>ContainerLaunchInfo</code> to better capture the purpose of this struct. See <a href="https://issues.apache.org/jira/browse/MESOS-4240">MESOS-4240</a> and <a href="https://issues.apache.org/jira/browse/MESOS-4282">MESOS-4282</a> for more information. If you are an isolator module writer, you will have to adjust your isolator module according to the new interface and re-compile with 0.27.</p></li>
+<li><p>ACLs.shutdown_frameworks has been deprecated in favor of the new ACLs.teardown_frameworks. This affects the <code>--acls</code> master flag for the local authorizer.</p></li>
+<li><p>Reserved resources are now accounted for in the DRF role sorter. Previously unaccounted reservations will influence the weighted DRF sorter. If role weights were explicitly set, they may need to be adjusted in order to account for the reserved resources in the cluster.</p></li>
+</ul>
+
<h2>Upgrading from 0.25.x to 0.26.x</h2>
@@ -210,7 +222,7 @@ Please refer to the metrics/snapshot end
}
</code></pre>
-<p><strong>NOTE</strong> All Mesos arguments can now be passed using file:// to read them out of a file (either an absolute or relative path). The –credentials, –whitelist, and any flags that expect JSON backed arguments (such as –modules) behave as before, although support for just passing a absolute path for any JSON flags rather than file:// has been deprecated and will produce a warning (and the absolute path behavior will be removed in a future release).</p>
+<p><strong>NOTE</strong> All Mesos arguments can now be passed using file:// to read them out of a file (either an absolute or relative path). The –credentials, –whitelist, and any flags that expect JSON backed arguments (such as –modules) behave as before, although support for just passing an absolute path for any JSON flags rather than file:// has been deprecated and will produce a warning (and the absolute path behavior will be removed in a future release).</p>
<p>In order to upgrade a running cluster:</p>
Added: mesos/site/publish/documentation/latest/versioning/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/versioning/index.html?rev=1727886&view=auto
==============================================================================
--- mesos/site/publish/documentation/latest/versioning/index.html (added)
+++ mesos/site/publish/documentation/latest/versioning/index.html Mon Feb 1 02:49:25 2016
@@ -0,0 +1,222 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title></title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
+ <link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml">
+
+ <link href="../../../assets/css/main.css" media="screen" rel="stylesheet" type="text/css" />
+
+
+
+ <!-- Google Analytics Magic -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-20226872-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <!-- magical breadcrumbs -->
+ <div class="topnav">
+ <ul class="breadcrumb">
+ <li>
+ <div class="dropdown">
+ <a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a>
+ <ul class="dropdown-menu" role="menu" aria-labelledby="dLabel">
+ <li><a href="http://www.apache.org">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ </li>
+ <li><a href="http://mesos.apache.org">Apache Mesos</a></li>
+
+
+ <li><a href="/documentation
+/">Documentation
+</a></li>
+
+
+ </ul><!-- /breadcrumb -->
+ </div>
+
+ <!-- navbar excitement -->
+ <div class="navbar navbar-static-top" role="navigation">
+ <div class="navbar-inner">
+ <div class="container">
+ <a href="/" class="logo"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo" /></a>
+ <div class="nav-collapse">
+ <ul class="nav nav-pills navbar-right">
+ <li><a href="/gettingstarted/">Getting Started</a></li>
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Community</a></li>
+ </ul>
+ </div>
+ </div>
+ </div>
+ </div><!-- /.navbar -->
+
+ <div class="container">
+
+ <div class="row-fluid">
+ <div class="col-md-4">
+ <h4>If you're new to Mesos</h4>
+ <p>See the <a href="/gettingstarted/">getting started</a> page for more information about downloading, building, and deploying Mesos.</p>
+
+ <h4>If you'd like to get involved or you're looking for support</h4>
+ <p>See our <a href="/community/">community</a> page for more details.</p>
+ </div>
+ <div class="col-md-8">
+ <h1>Mesos Versioning</h1>
+
+<p>The Mesos API and release versioning policy gives operators and developers clear guidelines on:</p>
+
+<ul>
+<li>Making modifications to the existing APIs without affecting backward compatibility.</li>
+<li>How long a Mesos API will be supported.</li>
+<li>Upgrading the Mesos installation across release versions.</li>
+</ul>
+
+
+<p>API versioning was introduced in Mesos 0.24.0 and this scheme only applies to Mesos 1.0.0 and higher.</p>
+
+<h2>Terminology</h2>
+
+<ul>
+<li><strong>Release Versioning</strong>: This refers to the version of Mesos that is being released. It is of the form <strong>Mesos X.Y.Z</strong> (X is the major version, Y is the minor version, and Z is the patch version).</li>
+<li><strong>API Versioning</strong>: This refers to the version of the Mesos API. It is of the form <strong>vX</strong> (X is the major version).</li>
+</ul>
+
+
+<h2>How does it work?</h2>
+
+<p>The Mesos APIs (constituting Scheduler, Executor, Internal, Operator/Admin APIs) will have a version in the URL. The versioned URL will have a prefix of <strong><code>/api/vN</code></strong> where “N” is the version of the API. The “/api” prefix is chosen to distinguish API resources from Web UI paths.</p>
+
+<p>Examples:</p>
+
+<ul>
+<li>http://localhost:5050/api/v1/scheduler : Scheduler HTTP API hosted by the master.</li>
+<li>http://localhost:5051/api/v1/executor : Executor HTTP API hosted by the agent.</li>
+</ul>
+
+
+<p>A given Mesos installation might host multiple versions of the same API i.e., Scheduler API v1 and/or v2 etc.</p>
+
+<h3>API version vs Release version</h3>
+
+<ul>
+<li>To keep things simple, the stable version of the API will correspond to the major release version of Mesos.
+
+<ul>
+<li>For example, v1 of the API will be supported by Mesos release versions 1.0.0, 1.4.0, 1.20.0 etc.</li>
+</ul>
+</li>
+<li>vN version of the API might also be supported by release versions of N-1 series but the vN API is not considered stable until the last release version of N-1 series.</li>
+<li>For example, v2 of the API might be introduced in Mesos 1.12.0 release but it is only considered stable in Mesos 1.21.0 release if it is the last release of â1â series. Note that all Mesos 1.x.y versions will still support v1 of the API.</li>
+<li>The API version is only bumped if we need to make a backwards <a href="#api-compatibility">incompatible</a> API change. We will strive to support a given API version for at least a year.</li>
+<li>The deprecation clock for vN-1 API will start as soon as we release âN.0.0â version of Mesos. We will strive to give enough time (e.g., 6 months) for frameworks/operators to upgrade to vN API before we stop supporting vN-1 API.</li>
+<li>Minor release version is bumped roughly on a monthly cycle to give a cadence of new features to users.</li>
+</ul>
+
+
+<p>NOTE: Presently, for “0.X.Y” releases i.e. till we reach “1.0.0”, we wait for at least 6 monthly releases before deprecating functionality. So, functionality that has been deprecated in 0.26.0 can be safely removed in 0.32.0.</p>
+
+<h3>API Compatibility</h3>
+
+<p>The API compatibility is determined by the corresponding protobuf guarantees.</p>
+
+<p>As an example, the following are considered “backwards compatible” changes for Scheduler API:</p>
+
+<ul>
+<li>Adding new types of Calls i.e., new types of HTTP requests to “/scheduler”.</li>
+<li>Adding new optional fields to existing requests to “/scheduler”.</li>
+<li>Adding new types of Events i.e., new types of chunks streamed on “/scheduler”.</li>
+<li>Adding new header fields to chunked response streamed on “/scheduler”.</li>
+<li>Adding new fields (or changing the order of fields) to chunksâ body streamed on “/scheduler”.</li>
+<li>Adding new API resources (e.g., “/foobar”).</li>
+</ul>
+
+
+<p>The following are considered backwards incompatible changes for Scheduler API:</p>
+
+<ul>
+<li>Adding new required fields to existing requests to “/scheduler”.</li>
+<li>Renaming/removing fields from existing requests to “/scheduler”.</li>
+<li>Renaming/removing fields from chunks streamed on “/scheduler”.</li>
+<li>Renaming/removing existing Calls.</li>
+</ul>
+
+
+<h2>Upgrades</h2>
+
+<ul>
+<li>The master and agent are typically compatible as long as they are running the same major version.</li>
+<li>For example, 1.3.0 master is compatible with 1.13.0 agent.</li>
+<li>In rare cases, we might require the master and agent to go through a specific minor version for upgrades.</li>
+<li>For example, we might require that a 1.1.0 master (and/or agent) be upgraded to 1.8.0 before it can be upgraded to 1.9.0 or later versions.</li>
+</ul>
+
+
+<p>The detailed information about upgrading to a particular Mesos version would be posted <a href="/documentation/latest/upgrades/">here</a>.</p>
+
+<h3>Implementation Details</h3>
+
+<p>Most APIs in Mesos accept protobuf messages with a corresponding JSON field mapping. To support multiple versions of the API, we decoupled the versioned protobufs backing the API from the âinternalâ protobufs used by the Mesos code.</p>
+
+<p>For example, the protobufs for the v1 Scheduler API are located at:</p>
+
+<pre><code>include/mesos/v1/scheduler/scheduler.proto
+
+package mesos.v1.scheduler;
+option java_package = "org.apache.mesos.v1.scheduler";
+option java_outer_classname = "Protos";
+â¦
+</code></pre>
+
+<p>The corresponding internal protobufs for the Scheduler API are located at:</p>
+
+<pre><code>include/mesos/scheduler/scheduler.proto
+
+package mesos.scheduler;
+option java_package = "org.apache.mesos.scheduler";
+option java_outer_classname = "Protos";
+â¦
+</code></pre>
+
+<p>The users of the API send requests (and receive responses) based on the versioned protobufs. We implemented <a href="https://github.com/apache/mesos/blob/master/src/internal/evolve.hpp">evolve</a>/<a href="https://github.com/apache/mesos/blob/master/src/internal/devolve.hpp">devolve</a> converters that can convert protobufs from any supported version to the internal protobuf and vice versa.</p>
+
+<p>Internally, message passing between various Mesos components would use the internal unversioned protobufs. When sending response (if any) back to the user of the API, the unversioned protobuf would be converted back to a versioned protobuf.</p>
+
+ </div>
+</div>
+
+
+ <hr>
+
+ <!-- footer -->
+ <div class="footer">
+ <p>© 2012-2015 <a href="http://apache.org">The Apache Software Foundation</a>.
+ Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation.<p>
+ </div><!-- /footer -->
+
+ </div> <!-- /container -->
+
+ <!-- JS -->
+ <script src="//code.jquery.com/jquery-1.11.0.min.js" type="text/javascript"></script>
+ <script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js" type="text/javascript"></script>
+ </body>
+</html>
Added: mesos/site/publish/documentation/logging/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/logging/index.html?rev=1727886&view=auto
==============================================================================
--- mesos/site/publish/documentation/logging/index.html (added)
+++ mesos/site/publish/documentation/logging/index.html Mon Feb 1 02:49:25 2016
@@ -0,0 +1,306 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title></title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
+ <link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml">
+
+ <link href="../../assets/css/main.css" media="screen" rel="stylesheet" type="text/css" />
+
+
+
+ <!-- Google Analytics Magic -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-20226872-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <!-- magical breadcrumbs -->
+ <div class="topnav">
+ <ul class="breadcrumb">
+ <li>
+ <div class="dropdown">
+ <a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a>
+ <ul class="dropdown-menu" role="menu" aria-labelledby="dLabel">
+ <li><a href="http://www.apache.org">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ </li>
+ <li><a href="http://mesos.apache.org">Apache Mesos</a></li>
+
+
+ <li><a href="/documentation
+/">Documentation
+</a></li>
+
+
+ </ul><!-- /breadcrumb -->
+ </div>
+
+ <!-- navbar excitement -->
+ <div class="navbar navbar-static-top" role="navigation">
+ <div class="navbar-inner">
+ <div class="container">
+ <a href="/" class="logo"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo" /></a>
+ <div class="nav-collapse">
+ <ul class="nav nav-pills navbar-right">
+ <li><a href="/gettingstarted/">Getting Started</a></li>
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Community</a></li>
+ </ul>
+ </div>
+ </div>
+ </div>
+ </div><!-- /.navbar -->
+
+ <div class="container">
+
+ <div class="row-fluid">
+ <div class="col-md-4">
+ <h4>If you're new to Mesos</h4>
+ <p>See the <a href="/gettingstarted/">getting started</a> page for more information about downloading, building, and deploying Mesos.</p>
+
+ <h4>If you'd like to get involved or you're looking for support</h4>
+ <p>See our <a href="/community/">community</a> page for more details.</p>
+ </div>
+ <div class="col-md-8">
+ <h1>Logging</h1>
+
+<p>Mesos handles the logs of each Mesos component differently depending on the
+degree of control Mesos has over the source code of the component.</p>
+
+<p>Roughly, these categories are:</p>
+
+<ul>
+<li><a href="#Internal">Internal</a> - Master and Agent.</li>
+<li><a href="#Containers">Containers</a> - Executors and Tasks.</li>
+<li>External - Components launched outside of Mesos, like
+Frameworks and <a href="/documentation/latest/high-availability/">ZooKeeper</a>. These are expected to
+implement their own logging solution.</li>
+</ul>
+
+
+<h2><a name="Internal"></a>Internal</h2>
+
+<p>The Mesos Master and Agent use the
+<a href="https://github.com/google/glog">Google’s logging library</a>.
+For information regarding the command-line options used to configure this
+library, see the <a href="/documentation/latest/configuration/">configuration documentation</a>.
+Google logging options that are not explicitly mentioned there can be
+configured via environment variables.</p>
+
+<p>Both Master and Agent also expose an HTTP endpoint which temporarily toggles
+verbose logging:</p>
+
+<pre><code>POST <ip:port>/logging/toggle?level=[1|2|3]&duration=VALUE
+</code></pre>
+
+<p>The effect is analogous to setting the <code>GLOG_v</code> environment variable prior
+to starting the Master/Agent, except the logging level will revert to the
+original level after the given duration.</p>
+
+<h2><a name="Containers"></a>Containers</h2>
+
+<p>For background, see <a href="/documentation/latest/containerizer/">the containerizer documentation</a>.</p>
+
+<p>Mesos does not assume any structured logging for entities running inside
+containers. Instead, Mesos will store the stdout and stderr of containers
+into plain files (“stdout” and “stderr”) located inside
+<a href="sandbox.md#where-is-it">the sandbox</a>.</p>
+
+<p>In some cases, the default Container logger behavior of Mesos is not ideal:</p>
+
+<ul>
+<li>Logging may not be standardized across containers.</li>
+<li>Logs are not easily aggregated.</li>
+<li>Log file sizes are not managed. Given enough time, the “stdout” and “stderr”
+files can fill up the Agent’s disk.</li>
+</ul>
+
+
+<h2><code>ContainerLogger</code> Module</h2>
+
+<p>The <code>ContainerLogger</code> module was introduced in Mesos 0.27.0 and aims to address
+the shortcomings of the default logging behavior for containers. The module
+can be used to change how Mesos redirects the stdout and stderr of containers.</p>
+
+<p>The <a href="https://github.com/apache/mesos/blob/master/include/mesos/slave/container_logger.hpp">interface for a <code>ContainerLogger</code> can be found here</a>.</p>
+
+<p>Mesos comes with two <code>ContainerLogger</code> modules:</p>
+
+<ul>
+<li>The <code>SandboxContainerLogger</code> implements the existing logging behavior as
+a <code>ContainerLogger</code>. This is the default behavior.</li>
+<li>The <code>LogrotateContainerLogger</code> addresses the problem of unbounded log file
+sizes.</li>
+</ul>
+
+
+<h3><code>LogrotateContainerLogger</code></h3>
+
+<p>The <code>LogrotateContainerLogger</code> constrains the total size of a container’s
+stdout and stderr files. The module does this by rotating log files based
+on the parameters to the module. When a log file reaches its specified
+maximum size, it is renamed by appending a <code>.N</code> to the end of the filename,
+where <code>N</code> increments each rotation. Older log files are deleted when the
+specified maximum number of files is reached.</p>
+
+<h4>Invoking the module</h4>
+
+<p>The <code>LogrotateContainerLogger</code> can be loaded by specifying the library
+<code>liblogrotate_container_logger.so</code> in the
+<a href="modules.md#Invoking"><code>--modules</code> flag</a> when starting the Agent and by
+setting the <code>--container_logger</code> Agent flag to
+<code>org_apache_mesos_LogrotateContainerLogger</code>.</p>
+
+<h4>Module parameters</h4>
+
+<table class="table table-striped">
+ <thead>
+ <tr>
+ <th width="30%">
+ Key
+ </th>
+ <th>
+ Explanation
+ </th>
+ </tr>
+ </thead>
+
+ <tr>
+ <td>
+ <code>max_stdout_size</code>/<code>max_stderr_size</code>
+ </td>
+ <td>
+ Maximum size, in bytes, of a single stdout/stderr log file.
+ When the size is reached, the file will be rotated.
+
+ Defaults to 10 MB. Minimum size of 1 (memory) page, usually around 4 KB.
+ </td>
+ </tr>
+
+ <tr>
+ <td>
+ <code>logrotate_stdout_options</code>/
+ <code>logrotate_stderr_options</code>
+ </td>
+ <td>
+ Additional config options to pass into <code>logrotate</code> for stdout.
+ This string will be inserted into a <code>logrotate</code> configuration
+ file. i.e. For "stdout":
+ <pre>
+/path/to/stdout {
+ [logrotate_stdout_options]
+ size [max_stdout_size]
+}</pre>
+ NOTE: The <code>size</code> option will be overriden by this module.
+ </td>
+ </tr>
+
+ <tr>
+ <td>
+ <code>launcher_dir</code>
+ </td>
+ <td>
+ Directory path of Mesos binaries.
+ The <code>LogrotateContainerLogger</code> will find the
+ <code>mesos-logrotate-logger</code> binary under this directory.
+
+ Defaults to <code>/usr/local/libexec/mesos</code>.
+ </td>
+ </tr>
+
+ <tr>
+ <td>
+ <code>logrotate_path</code>
+ </td>
+ <td>
+ If specified, the <code>LogrotateContainerLogger</code> will use the
+ specified <code>logrotate</code> instead of the system's
+ <code>logrotate</code>. If <code>logrotate</code> is not found, then
+ the module will exit with an error.
+ </td>
+ </tr>
+</table>
+
+
+<h4>How it works</h4>
+
+<ol>
+<li>Every time a container starts up, the <code>LogrotateContainerLogger</code>
+starts up companion subprocesses of the <code>mesos-logrotate-logger</code> binary.</li>
+<li>The module instructs Mesos to redirect the container’s stdout/stderr
+to the <code>mesos-logrotate-logger</code>.</li>
+<li>As the container outputs to stdout/stderr, <code>mesos-logrotate-logger</code> will
+pipe the output into the “stdout”/“stderr” files. As the files grow,
+<code>mesos-logrotate-logger</code> will call <code>logrotate</code> to keep the files strictly
+under the configured maximum size.</li>
+<li>When the container exits, <code>mesos-logrotate-logger</code> will finish logging before
+exiting as well.</li>
+</ol>
+
+
+<p>The <code>LogrotateContainerLogger</code> is designed to be resilient across Agent
+failover. If the Agent process dies, any instances of <code>mesos-logrotate-logger</code>
+will continue to run.</p>
+
+<h3>Writing a Custom <code>ContainerLogger</code></h3>
+
+<p>For basics on module writing, see <a href="/documentation/latest/modules/">the modules documentation</a>.</p>
+
+<p>There are several caveats to consider when designing a new <code>ContainerLogger</code>:</p>
+
+<ul>
+<li>Logging by the <code>ContainerLogger</code> should be resilient to Agent failover.
+If the Agent process dies (which includes the <code>ContainerLogger</code> module),
+logging should continue. This is usually achieved by using subprocesses.</li>
+<li>When containers shut down, the <code>ContainerLogger</code> is not explicitly notified.
+Instead, encountering <code>EOF</code> in the container’s stdout/stderr signifies
+that the container has exited. This provides a stronger guarantee that the
+<code>ContainerLogger</code> has seen all the logs before exiting itself.</li>
+<li>The <code>ContainerLogger</code> should not assume that containers have been launched
+with any specific <code>ContainerLogger</code>. The Agent may be restarted with a
+different <code>ContainerLogger</code>.</li>
+<li>Each <a href="/documentation/latest/containerizer/">containerizer</a> running on an Agent uses its own
+instance of the <code>ContainerLogger</code>. This means more than one <code>ContainerLogger</code>
+may be running in a single Agent. However, each Agent will only run a single
+type of <code>ContainerLogger</code>.</li>
+</ul>
+
+
+ </div>
+</div>
+
+
+ <hr>
+
+ <!-- footer -->
+ <div class="footer">
+ <p>© 2012-2015 <a href="http://apache.org">The Apache Software Foundation</a>.
+ Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation.<p>
+ </div><!-- /footer -->
+
+ </div> <!-- /container -->
+
+ <!-- JS -->
+ <script src="//code.jquery.com/jquery-1.11.0.min.js" type="text/javascript"></script>
+ <script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js" type="text/javascript"></script>
+ </body>
+</html>
Modified: mesos/site/publish/documentation/maintenance/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/maintenance/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/maintenance/index.html (original)
+++ mesos/site/publish/documentation/maintenance/index.html Mon Feb 1 02:49:25 2016
@@ -146,7 +146,7 @@ resources or resource preemption.</p>
<p>Maintenance primitives were introduced in Mesos 0.25.0. Several machine
maintenance modes were also introduced. Those modes are illustrated below.</p>
-<p><img src="/assets/img/documentation/maintenance-primitives-modes.png" alt="Maintenance mode transitions" /></p>
+<p><img src="images/maintenance-primitives-modes.png" alt="Maintenance mode transitions" /></p>
<p>All mode transitions must be initiated by the operator. Mesos will not
change the mode of any machine, regardless of the estimate provided in
Modified: mesos/site/publish/documentation/markdown-style-guide/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/markdown-style-guide/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/markdown-style-guide/index.html (original)
+++ mesos/site/publish/documentation/markdown-style-guide/index.html Mon Feb 1 02:49:25 2016
@@ -137,60 +137,55 @@ Continued longer note.
<p>Files and path references should be specified as follows:</p>
-<pre><code class="{.text}">Remember you can also use the `file:///path/to/file` or `/path/to/file`
+<pre><code class="{.txt}">Remember you can also use the `file:///path/to/file` or `/path/to/file`
</code></pre>
<h2>Tables</h2>
<p>In order to avoid problems with markdown formatting we should specify tables in html directly:</p>
-<p>~~~{.html}</p>
-
-<table class="table table-striped">
- <thead>
- <tr>
- <th width="30%">
+<pre><code><table class="table table-striped">
+ <thead>
+ <tr>
+ <th width="30%">
Flag
- </th>
- <th>
+ </th>
+ <th>
Explanation
- </th>
- </thead>
- <tr>
- <td>
+ </th>
+ </thead>
+ <tr>
+ <td>
--ip=VALUE
- </td>
- <td>
+ </td>
+ <td>
IP address to listen on
- </td>
- </tr>
- <tr>
- <td>
+ </td>
+ </tr>
+ <tr>
+ <td>
--[no-]help
- </td>
- <td>
+ </td>
+ <td>
Prints this help message (default: false)
+ </td>
+ </tr>
+</table>
+</code></pre>
- </td>
- </tr>
-</table>
-
-
-<pre><code>
-
-## Indendation and Whitespace
+<h2>Indendation and Whitespace</h2>
-We use no extra indentation in markdown files.
+<p>We use no extra indentation in markdown files.
We have one new line after section headings and two blank lines
-in between sections.
-</code></pre>
+in between sections.</p>
-<p>… end of previous section.</p>
+<pre><code class="{.txt}">... end of previous section.
-<h2>New Section</h2>
-<p>Beginning of new section ….
-~~~</p>
+## New Section
+
+Beginning of new section ....
+</code></pre>
</div>
</div>