You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by mp...@apache.org on 2016/03/08 06:37:26 UTC
svn commit: r1734022 [4/6] - in /mesos/site/publish: ./ blog/
blog/mesos-0-27-1-released/ blog/mesos-0-27-2-released/ documentation/
documentation/allocation-module/
documentation/app-framework-development-guide/ documentation/architecture/
documentati...
Modified: mesos/site/publish/documentation/latest/scheduler-http-api/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/scheduler-http-api/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/scheduler-http-api/index.html (original)
+++ mesos/site/publish/documentation/latest/scheduler-http-api/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Scheduler HTTP API</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -98,7 +98,7 @@
<h2>Calls</h2>
-<p>The following calls are currently accepted by the master. The canonical source of this information is <a href="https://github.com/apache/mesos/blob/master/include/mesos/v1/scheduler/scheduler.proto">scheduler.proto</a> (NOTE: The protobuf definitions are subject to change before the beta API is finalized). Note that when sending JSON encoded Calls, schedulers should encode raw bytes in Base64 and strings in UTF-8.</p>
+<p>The following calls are currently accepted by the master. The canonical source of this information is <a href="https://github.com/apache/mesos/blob/master/include/mesos/v1/scheduler/scheduler.proto">scheduler.proto</a> (NOTE: The protobuf definitions are subject to change before the beta API is finalized). Note that when sending JSON encoded Calls, schedulers should encode raw bytes in Base64 and strings in UTF-8. All non-<code>SUBSCRIBE</code> calls should include the <code>Mesos-Stream-Id</code> header, explained in the <a href="#subscribe"><code>SUBSCRIBE</code></a> section. <code>SUBSCRIBE</code> calls should never include the <code>Mesos-Stream-Id</code> header.</p>
<p><a id="recordio-response-format"></a></p>
@@ -147,7 +147,7 @@
<p>This is the first step in the communication process between the scheduler and the master. This is also to be considered as subscription to the “/scheduler” events stream.</p>
-<p>To subscribe with the master, the scheduler sends a HTTP POST request with encoded <code>SUBSCRIBE</code> message with the required FrameworkInfo. Note that if “subscribe.framework_info.id” is not set, master considers the scheduler as a new one and subscribes it by assigning it a FrameworkID. The HTTP response is a stream with RecordIO encoding, with the first event being <code>SUBSCRIBED</code> event (see details in <strong>Events</strong> section).</p>
+<p>To subscribe with the master, the scheduler sends a HTTP POST request with encoded <code>SUBSCRIBE</code> message with the required FrameworkInfo. Note that if “subscribe.framework_info.id” is not set, master considers the scheduler as a new one and subscribes it by assigning it a FrameworkID. The HTTP response is a stream with RecordIO encoding, with the first event being <code>SUBSCRIBED</code> event (see details in <strong>Events</strong> section). The response also includes the <code>Mesos-Stream-Id</code> header, which is used by the master to uniquely identify the subscribed scheduler instance. This stream ID header should be included in all subsequent non-<code>SUBSCRIBE</code> calls sent over this subscription connection to the master. The value of <code>Mesos-Stream-Id</code> is guaranteed to be equal to or less than 128 bytes in length.</p>
<pre><code>SUBSCRIBE Request (JSON):
@@ -165,9 +165,7 @@ Connection: close
"framework_info" : {
"user" : "foo",
"name" : "Example HTTP Framework"
- },
-
- "force" : true
+ }
}
}
@@ -176,6 +174,7 @@ HTTP/1.1 200 OK
Content-Type: application/json
Transfer-Encoding: chunked
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
<event length>
{
@@ -188,7 +187,8 @@ Transfer-Encoding: chunked
<more events>
</code></pre>
-<p>Alternatively, if “subscribe.framework_info.id” is set, master considers this a request from an already subscribed scheduler reconnecting after a disconnection (e.g., due to failover or network disconnection) and responds with <code>SUBSCRIBED</code> event containing the same FrameworkID. The “subscribe.force” field describes how the master reacts when multiple scheduler instances (with the same framework id) attempt to subscribe with the master at the same time (e.g., due to network partition). See the semantics in <strong>Disconnections</strong> section below.</p>
+<p>Alternatively, if “subscribe.framework_info.id” is set, master considers this a request from an already subscribed scheduler reconnecting after a disconnection (e.g., due to master/scheduler failover or network disconnection) and responds
+with a <code>SUBSCRIBED</code> event. For further details, see the <strong>Disconnections</strong> section below.</p>
<p>NOTE: In the old version of the API, (re-)registered callbacks also included MasterInfo, which contained information about the master the driver currently connected to. With the new API, since schedulers explicitly subscribe with the leading master (see details below in <strong>Master Detection</strong> section), it’s not relevant anymore.</p>
@@ -196,6 +196,8 @@ Transfer-Encoding: chunked
<p>Scheduler must make additional HTTP requests to the “/scheduler” endpoint only after it has opened a persistent connection to it by sending a <code>SUBSCRIBE</code> request and received a <code>SUBSCRIBED</code> response. Calls made without subscription will result in a “403 Forbidden” instead of a “202 Accepted” response. A scheduler might also receive a “400 Bad Request” response if the HTTP request is malformed (e.g., malformed HTTP headers).</p>
+<p>Note that the <code>Mesos-Stream-Id</code> header should <strong>never</strong> be included with a <code>SUBSCRIBE</code> call; the master will always provide a new unique stream ID for each subscription.</p>
+
<h3>TEARDOWN</h3>
<p>Sent by the scheduler when it wants to tear itself down. When Mesos receives this request it will shut down all executors (and consequently kill tasks) and remove persistent volumes (if requested). It then removes the framework and closes all open connections from this scheduler to the Master.</p>
@@ -205,6 +207,7 @@ POST /api/v1/scheduler HTTP/1.1
Host: masterhost:5050
Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
{
"framework_id" : {"value" : "12220-3440-12532-2345"},
@@ -224,6 +227,7 @@ POST /api/v1/scheduler HTTP/1.1
Host: masterhost:5050
Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
{
"framework_id" : {"value" : "12220-3440-12532-2345"},
@@ -251,6 +255,7 @@ POST /api/v1/scheduler HTTP/1.1
Host: masterhost:5050
Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
{
"framework_id" : {"value" : "12220-3440-12532-2345"},
@@ -277,6 +282,7 @@ POST /api/v1/scheduler HTTP/1.1
Host: masterhost:5050
Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
{
"framework_id" : {"value" : "12220-3440-12532-2345"},
@@ -296,6 +302,7 @@ POST /api/v1/scheduler HTTP/1.1
Host: masterhost:5050
Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
{
"framework_id" : {"value" : "12220-3440-12532-2345"},
@@ -319,6 +326,7 @@ POST /api/v1/scheduler HTTP/1.1
Host: masterhost:5050
Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
{
"framework_id" : {"value" : "12220-3440-12532-2345"},
@@ -342,6 +350,7 @@ POST /api/v1/scheduler HTTP/1.1
Host: masterhost:5050
Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
{
"framework_id" : {"value" : "12220-3440-12532-2345"},
@@ -366,6 +375,7 @@ POST /api/v1/scheduler HTTP/1.1
Host: masterhost:5050
Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
{
"framework_id" : {"value" : "12220-3440-12532-2345"},
@@ -392,6 +402,7 @@ POST /api/v1/scheduler HTTP/1.1
Host: masterhost:5050
Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
{
"framework_id" : {"value" : "12220-3440-12532-2345"},
@@ -416,6 +427,7 @@ POST /api/v1/scheduler HTTP/1.1
Host: masterhost:5050
Content-Type: application/json
+Mesos-Stream-Id: 130ae4e3-6b13-4ef4-baa9-9f2e85c3e9af
{
"framework_id" : {"value" : "12220-3440-12532-2345"},
@@ -568,15 +580,16 @@ HTTP/1.1 202 Accepted
<p>NOTE: To force shutdown a framework before the framework timeout elapses (e.g., during framework development and testing), either the framework can send <code>TEARDOWN</code> call (part of Scheduler API) or an operator can use the “/master/teardown” endpoint (part of Operator API).</p>
-<p>If the scheduler realizes that its subscription connection to “/scheduler” is broken, it should attempt to open a new persistent connection to the
-“/scheduler” (on possibly new master based on the result of master detection) and resubscribe. It should not send new non-subscribe HTTP requests to “/scheduler” unless it gets a <code>SUBSCRIBED</code> event; such requests will result in “403 Forbidden”.</p>
+<p>If the scheduler realizes that its subscription connection to “/scheduler” is broken or the master has changed (e.g., via ZooKeeper) it should resubscribe (using a backoff strategy). This is done by sending a <code>SUBSCRIBE</code> request (with framework ID set) on a <strong>new</strong> persistent connection to the “/scheduler” endpoint on the (possibly new) master. It should not send new non-subscribe HTTP requests to “/scheduler” unless it receives a <code>SUBSCRIBED</code> event; such requests will result in “403 Forbidden”.</p>
<p>If the master does not realize that the subscription connection is broken, but the scheduler realizes it, the scheduler might open a new persistent connection to
-“/scheduler” via <code>SUBSCRIBE</code>. In this case, the semantics depend on the value of <code>subscribe.force</code>. If set to true, master closes the existing subscription connection and allows subscription on the new connection. If set to false, the new connection attempt is disallowed in favor of the existing connection. The invariant here is that, only one persistent subscription connection for a given FrameworkID is allowed on the master. For HA schedulers, it is recommended that a scheduler instance set <code>subscribe.force</code> to true only when it just got elected and set it to false for all subsequent reconnection attempts (e.g, due to disconnection or master failover).</p>
+“/scheduler” via <code>SUBSCRIBE</code>. In this case, the master closes the existing subscription connection and allows subscription on the new connection. The invariant here is that only one persistent subscription connection for a given framework ID is allowed on the master.</p>
+
+<p>The master uses the <code>Mesos-Stream-Id</code> header to distinguish scheduler instances from one another. In the case of highly-available schedulers with multiple instances, this can prevent unwanted behavior in certain failure scenarios. Each unique <code>Mesos-Stream-Id</code> is valid only for the life of a single subscription connection. Each response to a <code>SUBSCRIBE</code> request contains a <code>Mesos-Stream-Id</code>, and this ID must be included with all subsequent non-subscribe calls sent over that subscription connection. Whenever a new subscription connection is established, a new stream ID is generated and should be used for the life of that connection.</p>
<h3>Network partitions</h3>
-<p>In the case of a network partition, the subscription connection between the scheduler and master might not necessarily break. To be able to detect this scenario, master periodically (e.g., 15s) sends <code>HEARTBEAT</code> events (similar in vein to Twitter’s Streaming API). If a scheduler doesn’t receive a bunch (e.g., 5) of these heartbeats within a time window, it should immediately disconnect and try to re-subscribe. It is highly recommended for schedulers to use an exponential backoff strategy (e.g., upto a maximum of 15s) to avoid overwhelming the master while reconnecting. Schedulers can use a similar timeout (e.g., 75s) for receiving responses to any HTTP requests.</p>
+<p>In the case of a network partition, the subscription connection between the scheduler and master might not necessarily break. To be able to detect this scenario, master periodically (e.g., 15s) sends <code>HEARTBEAT</code> events (similar to Twitter’s Streaming API). If a scheduler doesn’t receive a bunch (e.g., 5) of these heartbeats within a time window, it should immediately disconnect and try to resubscribe. It is highly recommended for schedulers to use an exponential backoff strategy (e.g., up to a maximum of 15s) to avoid overwhelming the master while reconnecting. Schedulers can use a similar timeout (e.g., 75s) for receiving responses to any HTTP requests.</p>
<h2>Master detection</h2>
Modified: mesos/site/publish/documentation/latest/slave-recovery/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/slave-recovery/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/slave-recovery/index.html (original)
+++ mesos/site/publish/documentation/latest/slave-recovery/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Slave Recovery</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -83,83 +83,77 @@
<div class="col-md-8">
<h1>Slave Recovery</h1>
-<p>Slave recovery is a feature of Mesos that allows:</p>
+<p>If the <code>mesos-slave</code> process on a host exits (perhaps due to a Mesos bug or
+because the operator kills the process while <a href="/documentation/latest/./upgrades/">upgrading Mesos</a>),
+any executors/tasks that were being managed by the <code>mesos-slave</code> process will
+continue to run. When <code>mesos-slave</code> is restarted, the operator can control how
+those old executors/tasks are handled:</p>
<ol>
-<li>Executors/tasks to keep running when the slave process is down and</li>
-<li>Allows a restarted slave process to reconnect with running executors/tasks on the slave.</li>
+<li>By default, all the executors/tasks that were being managed by the old
+<code>mesos-slave</code> process are killed.</li>
+<li>If a framework enabled <em>checkpointing</em> when it registered with the master,
+any executors belonging to that framework can reconnect to the new
+<code>mesos-slave</code> process and continue running uninterrupted.</li>
</ol>
-<p>Mesos slave could be restarted for an upgrade or due to a crash. This feature is introduced in <strong><em>0.14.0</em></strong> release.</p>
+<p>Hence, enabling framework checkpointing enables tasks to tolerate Mesos slave
+upgrades and unexpected <code>mesos-slave</code> crashes without experiencing any
+downtime.</p>
-<h2>How does it work?</h2>
+<p>Slave recovery works by having the slave <em>checkpoint</em> information (e.g., Task
+Info, Executor Info, Status Updates) about the tasks and executors it is
+managing to local disk. If a framework enables checkpointing, any subsequent
+slave restarts will recover the checkpointed information and reconnect with any
+executors that are still running.</p>
-<p>Slave recovery works by having the slave checkpoint enough information (e.g., Task Info, Executor Info, Status Updates) about the running tasks and executors to local disk. Once a framework enables checkpointing, any subsequent slave restarts would recover the checkpointed information and reconnect with the executors. Note that if the host running the slave process is rebooted all the executors/tasks are killed.</p>
+<p>Note that if the operating system on the slave is rebooted, all executors and
+tasks running on the host are killed and are not automatically restarted when
+the host comes back up.</p>
-<blockquote><p>NOTE: To enable recovery the framework should explicitly request checkpointing.
-Alternatively, a framework that doesn’t want the disk i/o overhead of checkpointing can opt out of checkpointing.</p></blockquote>
+<h2>Framework Configuration</h2>
-<h2>Enabling slave checkpointing</h2>
+<p>A framework can control whether its executors will be recovered by setting the <code>checkpoint</code> flag in its <code>FrameworkInfo</code> when registering with the master. Enabling this feature results in increased I/O overhead at each slave that runs tasks launched by the framework. By default, frameworks do <strong>not</strong> checkpoint their state.</p>
-<blockquote><p>NOTE: From Mesos 0.22.0 slave checkpointing will be automatically enabled for all slaves.</p></blockquote>
+<h2>Slave Configuration</h2>
-<p>As part of this feature, 4 new flags were added to the slave.</p>
+<p>Three <a href="/documentation/latest/./configuration/">configuration flags</a> control the recovery behavior of a Mesos slave:</p>
<ul>
-<li><p><code>checkpoint</code> : Whether to checkpoint slave and frameworks information
- to disk [Default: true].</p>
+<li><p><code>strict</code>: Whether to do slave recovery in strict mode [Default: true].</p>
<ul>
-<li>This enables a restarted slave to recover status updates and reconnect
-with (–recover=reconnect) or kill (–recover=cleanup) old executors.
-
-<blockquote><p>NOTE: From Mesos 0.22.0 this flag will be removed as it will be enabled for all slaves.</p></blockquote></li>
-</ul>
-</li>
-<li><p><code>strict</code> : Whether to do recovery in strict mode [Default: true].</p>
-
-<ul>
-<li>If strict=true, any and all recovery errors are considered fatal.</li>
+<li>If strict=true, all recovery errors are considered fatal.</li>
<li>If strict=false, any errors (e.g., corruption in checkpointed data) during recovery are
ignored and as much state as possible is recovered.</li>
</ul>
</li>
-<li><p><code>recover</code> : Whether to recover status updates and reconnect with old executors [Default: reconnect].</p>
+<li><p><code>recover</code>: Whether to recover status updates and reconnect with old executors [Default: reconnect].</p>
<ul>
-<li>If recover=reconnect, Reconnect with any old live executors.</li>
-<li>If recover=cleanup, Kill any old live executors and exit.
-Use this option when doing an incompatible slave or executor upgrade!).
+<li>If recover=reconnect, reconnect with any old live executors, provided the executor’s framework enabled checkpointing.</li>
+<li>If recover=cleanup, kill any old live executors and exit. Use this option when doing an incompatible slave or executor upgrade!
<blockquote><p>NOTE: If no checkpointing information exists, no recovery is performed
and the slave registers with the master as a new slave.</p></blockquote></li>
</ul>
</li>
-<li><p><code>recovery_timeout</code> : Amount of time allotted for the slave to recover [Default: 15 mins].</p>
+<li><p><code>recovery_timeout</code>: Amount of time allotted for the slave to recover [Default: 15 mins].</p>
<ul>
<li>If the slave takes longer than <code>recovery_timeout</code> to recover, any executors that are waiting to
-reconnect to the slave will self-terminate.
-
-<blockquote><p>NOTE: This flag is only applicable when <code>--checkpoint</code> is enabled.</p></blockquote></li>
+reconnect to the slave will self-terminate.</li>
</ul>
</li>
</ul>
<blockquote><p>NOTE: If none of the frameworks have enabled checkpointing,
-executors/tasks of frameworks die when the slave dies and are not recovered.</p></blockquote>
+the executors and tasks running at a slave die when the slave dies
+and are not recovered.</p></blockquote>
-<p>A restarted slave should re-register with master within a timeout (currently, 75s). If the slave takes longer
-than this timeout to re-register, the master shuts down the slave, which in turn shuts down any live executors/tasks.
-Therefore, it is highly recommended to automate the process of restarting a slave (e.g, using <a href="http://mmonit.com/monit/">monit</a>).</p>
-
-<p><strong>For the complete list of slave options: ./mesos-slave.sh –help</strong></p>
-
-<h2>Enabling framework checkpointing</h2>
-
-<p>As part of this feature, <code>FrameworkInfo</code> has been updated to include an optional <code>checkpoint</code> field. A framework that would like to opt in to checkpointing should set <code>FrameworkInfo.checkpoint=True</code> before registering with the master.</p>
+<p>A restarted slave should re-register with master within a timeout (75 seconds by default: see the <code>--max_slave_ping_timeouts</code> and <code>--slave_ping_timeout</code> <a href="/documentation/latest/./configuration/">configuration flags</a>). If the slave takes longer than this timeout to re-register, the master shuts down the slave, which in turn will shutdown any live executors/tasks. Therefore, it is highly recommended to automate the process of restarting a slave (e.g., using a process supervisor such as <a href="http://mmonit.com/monit/">monit</a> or <code>systemd</code>).</p>
<h2>Known issues with <code>systemd</code> and POSIX isolation</h2>
@@ -174,10 +168,6 @@ KillMode=process
<blockquote><p>NOTE: There are also known issues with using <code>systemd</code> and raw <code>cgroups</code> based isolation, for now the suggested non-Posix isolation mechanism is to use Docker containerization.</p></blockquote>
-<h2>Upgrading to 0.14.0</h2>
-
-<p>If you want to upgrade a running Mesos cluster to 0.14.0 to take advantage of slave recovery please follow the <a href="/documentation/latest/./upgrades/">upgrade instructions</a>.</p>
-
</div>
</div>
Modified: mesos/site/publish/documentation/latest/ssl/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/ssl/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/ssl/index.html (original)
+++ mesos/site/publish/documentation/latest/ssl/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - SSL in Mesos</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
Modified: mesos/site/publish/documentation/latest/submitting-a-patch/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/submitting-a-patch/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/submitting-a-patch/index.html (original)
+++ mesos/site/publish/documentation/latest/submitting-a-patch/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Submitting a Patch</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -81,7 +81,7 @@
<p>See our <a href="/community/">community</a> page for more details.</p>
</div>
<div class="col-md-8">
- <h1>Submitting a patch</h1>
+ <h1>Submitting a Patch</h1>
<ul>
<li>You’ve fixed a bug or added a feature and want to contribute it. AWESOME!</li>
Modified: mesos/site/publish/documentation/latest/testing-patterns/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/testing-patterns/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/testing-patterns/index.html (original)
+++ mesos/site/publish/documentation/latest/testing-patterns/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Testing Patterns</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
Modified: mesos/site/publish/documentation/latest/tools/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/tools/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/tools/index.html (original)
+++ mesos/site/publish/documentation/latest/tools/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Tools</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
Modified: mesos/site/publish/documentation/latest/upgrades/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/upgrades/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/upgrades/index.html (original)
+++ mesos/site/publish/documentation/latest/upgrades/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Upgrading Mesos</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -85,6 +85,26 @@
<p>This document serves as a guide for users who wish to upgrade an existing Mesos cluster. Some versions require particular upgrade techniques when upgrading a running cluster. Some upgrades will have incompatible changes.</p>
+<h2>Upgrading from 0.27.x to 0.28.x</h2>
+
+<ul>
+<li><p>Mesos 0.28 only supports three decimal digits of precision for scalar resource values. For example, frameworks can reserve “0.001” CPUs but more fine-grained reservations (e.g., “0.0001” CPUs) are no longer supported (although they did not work reliably in prior versions of Mesos anyway). Internally, resource math is now done using a fixed-point format that supports three decimal digits of precision, and then converted to/from floating point for input and output, respectively. Frameworks that do their own resource math and manipulate fractional resources may observe differences in roundoff error and numerical precision.</p></li>
+<li><p>Mesos 0.28 changes the definitions of two ACLs used for authorization. The objects of the <code>ReserveResources</code> and <code>CreateVolume</code> ACLs have been changed to <code>roles</code>. In both cases, principals can now be authorized to perform these operations for particular roles. This means that by default, a framework or operator can reserve resources/create volumes for any role. To restrict this behavior, <a href="/documentation/latest/./authorization/">ACLs can be added</a> to the master which authorize principals to reserve resources/create volumes for specified roles only. Previously, frameworks could only reserve resources for their own role; this behavior can be preserved by configuring the <code>ReserveResources</code> ACLs such that the framework’s principal is only authorized to reserve for the framework’s role. <strong>NOTE</strong> This renders existing <code>ReserveResources</code> and <code>CreateVolume</code> ACL definitions obsolete; i
f you are authorizing these operations, your ACL definitions should be updated.</p></li>
+</ul>
+
+
+<p>In order to upgrade a running cluster:</p>
+
+<ol>
+<li>Rebuild and install any modules so that upgraded masters/slaves can use them.</li>
+<li>Install the new master binaries and restart the masters.</li>
+<li>Install the new slave binaries and restart the slaves.</li>
+<li>Upgrade the schedulers by linking the latest native library / jar / egg (if necessary).</li>
+<li>Restart the schedulers.</li>
+<li>Upgrade the executors by linking the latest native library / jar / egg (if necessary).</li>
+</ol>
+
+
<h2>Upgrading from 0.26.x to 0.27.x</h2>
<ul>
@@ -97,27 +117,56 @@
</ul>
+<p>In order to upgrade a running cluster:</p>
+
+<ol>
+<li>Rebuild and install any modules so that upgraded masters/slaves can use them.</li>
+<li>Install the new master binaries and restart the masters.</li>
+<li>Install the new slave binaries and restart the slaves.</li>
+<li>Upgrade the schedulers by linking the latest native library / jar / egg (if necessary).</li>
+<li>Restart the schedulers.</li>
+<li>Upgrade the executors by linking the latest native library / jar / egg (if necessary).</li>
+</ol>
+
+
<h2>Upgrading from 0.25.x to 0.26.x</h2>
-<p><strong>NOTE</strong> The names of some TaskStatus::Reason enums have been changed. But the tag numbers remain unchanged, so it is backwards compatible. Frameworks using the new version might need to do some compile time adjustments:</p>
+<ul>
+<li><p>The names of some TaskStatus::Reason enums have been changed. But the tag numbers remain unchanged, so it is backwards compatible. Frameworks using the new version might need to do some compile time adjustments:</p>
<ul>
<li>REASON_MEM_LIMIT -> REASON_CONTAINER_LIMITATION_MEMORY</li>
<li>REASON_EXECUTOR_PREEMPTED -> REASON_CONTAINER_PREEMPTED</li>
</ul>
+</li>
+<li><p>The <code>Credential</code> protobuf has been changed. <code>Credential</code> field <code>secret</code> is now a string, it used to be bytes. This will affect framework developers and language bindings ought to update their generated protobuf with the new version. This fixes JSON based credentials file support.</p></li>
+<li><p>The <code>/state</code> endpoints on master and slave will no longer include <code>data</code> fields as part of the JSON models for <code>ExecutorInfo</code> and <code>TaskInfo</code> out of consideration for memory scalability (see <a href="https://issues.apache.org/jira/browse/MESOS-3794">MESOS-3794</a> and <a href="http://www.mail-archive.com/dev@mesos.apache.org/msg33536.html">this email thread</a>).</p>
+
+<ul>
+<li>On master, the affected <code>data</code> field was originally found via <code>frameworks[*].executors[*].data</code>.</li>
+<li>On slaves, the affected <code>data</code> field was originally found via <code>executors[*].tasks[*].data</code>.</li>
+</ul>
+</li>
+<li><p>The <code>NetworkInfo</code> protobuf has been changed. The fields <code>protocol</code> and <code>ip_address</code> are now deprecated. The new field <code>ip_addresses</code> subsumes the information provided by them.</p></li>
+</ul>
-<p><strong>NOTE</strong> The <code>Credential</code> protobuf has been changed. <code>Credential</code> field <code>secret</code> is now a string, it used to be bytes. This will affect framework developers and language bindings ought to update their generated protobuf with the new version. This fixes JSON based credentials file support.</p>
+<p>In order to upgrade a running cluster:</p>
-<p><strong>NOTE</strong> The <code>/state</code> endpoints on master and slave will no longer include <code>data</code> fields as part of the JSON models for <code>ExecutorInfo</code> and <code>TaskInfo</code> out of consideration for memory scalability (see <a href="https://issues.apache.org/jira/browse/MESOS-3794">MESOS-3794</a> and <a href="http://www.mail-archive.com/dev@mesos.apache.org/msg33536.html">this email thread</a>).
-On master, the affected <code>data</code> field was originally found via <code>frameworks[*].executors[*].data</code>.
-On slaves, the affected <code>data</code> field was originally found via <code>executors[*].tasks[*].data</code>.</p>
+<ol>
+<li>Rebuild and install any modules so that upgraded masters/slaves can use them.</li>
+<li>Install the new master binaries and restart the masters.</li>
+<li>Install the new slave binaries and restart the slaves.</li>
+<li>Upgrade the schedulers by linking the latest native library / jar / egg (if necessary).</li>
+<li>Restart the schedulers.</li>
+<li>Upgrade the executors by linking the latest native library / jar / egg (if necessary).</li>
+</ol>
-<p><strong>NOTE</strong> The <code>NetworkInfo</code> protobuf has been changed. The fields <code>protocol</code> and <code>ip_address</code> are now deprecated. The new field <code>ip_addresses</code> subsumes the information provided by them.</p>
<h2>Upgrading from 0.24.x to 0.25.x</h2>
-<p><strong>NOTE</strong> The following endpoints will be deprecated in favor of new endpoints. Both versions will be available in 0.25 but the deprecated endpoints will be removed in a subsequent release.</p>
+<ul>
+<li><p>The following endpoints will be deprecated in favor of new endpoints. Both versions will be available in 0.25 but the deprecated endpoints will be removed in a subsequent release.</p>
<p>For master endpoints:</p>
@@ -130,8 +179,8 @@ On slaves, the affected <code>data</code
<p>For slave endpoints:</p>
<ul>
-<li>/state.json becomes /state</li>
-<li>/monitor/statistics.json becomes /monitor/statistics</li>
+<li>/state.json becomes /stateÏ</li>
+<li>/monitor/statistics.json becomes /monitor/statisticsÏ</li>
</ul>
@@ -143,143 +192,141 @@ On slaves, the affected <code>data</code
<li>/files/download.json becomes /files/download</li>
<li>/files/read.json becomes /files/read</li>
</ul>
+</li>
+<li><p>The C++/Java/Python scheduler bindings have been updated. In particular, the driver can make a suppressOffers() call to stop receiving offers (until reviveOffers() is called).</p></li>
+</ul>
-<p><strong>NOTE</strong> The C++/Java/Python scheduler bindings have been updated. In particular, the driver can make a suppressOffers() call to stop receiving offers (until reviveOffers() is called).</p>
-
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Rebuild and install any modules so that upgraded masters/slaves can use them.</li>
<li>Install the new master binaries and restart the masters.</li>
<li>Install the new slave binaries and restart the slaves.</li>
<li>Upgrade the schedulers by linking the latest native library / jar / egg (if necessary).</li>
<li>Restart the schedulers.</li>
<li>Upgrade the executors by linking the latest native library / jar / egg (if necessary).</li>
-</ul>
+</ol>
<h2>Upgrading from 0.23.x to 0.24.x</h2>
-<p><strong>NOTE</strong> Support for live upgrading a driver based scheduler to HTTP based (experimental) scheduler has been added.</p>
+<ul>
+<li><p>Support for live upgrading a driver based scheduler to HTTP based (experimental) scheduler has been added.</p></li>
+<li><p>Master now publishes its information in ZooKeeper in JSON (instead of protobuf). Make sure schedulers are linked against >= 0.23.0 libmesos before upgrading the master.</p></li>
+</ul>
-<p><strong>NOTE</strong> Master now publishes its information in ZooKeeper in JSON (instead of protobuf). Make sure schedulers are linked against >= 0.23.0 libmesos before upgrading the master.</p>
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Rebuild and install any modules so that upgraded masters/slaves can use them.</li>
<li>Install the new master binaries and restart the masters.</li>
<li>Install the new slave binaries and restart the slaves.</li>
<li>Upgrade the schedulers by linking the latest native library / jar / egg (if necessary).</li>
<li>Restart the schedulers.</li>
<li>Upgrade the executors by linking the latest native library / jar / egg (if necessary).</li>
-</ul>
+</ol>
<h2>Upgrading from 0.22.x to 0.23.x</h2>
-<p><strong>NOTE</strong> The ‘stats.json’ endpoints for masters and slaves have been removed. Please use the ‘metrics/snapshot’ endpoints instead.</p>
-
-<p><strong>NOTE</strong> The ‘/master/shutdown’ endpoint is deprecated in favor of the new ‘/master/teardown’ endpoint.</p>
-
-<p><strong>NOTE</strong> In order to enable decorator modules to remove metadata (environment variables or labels), we changed the meaning of the return value for decorator hooks in Mesos 0.23.0. Please refer to the modules documentation for more details.</p>
-
-<p><strong>NOTE</strong> Slave ping timeouts are now configurable on the master via <code>--slave_ping_timeout</code> and <code>--max_slave_ping_timeouts</code>. Slaves should be upgraded to 0.23.x before changing these flags.</p>
-
-<p><strong>NOTE</strong> A new scheduler driver API, <code>acceptOffers</code>, has been introduced. This is a more general version of the <code>launchTasks</code> API, which allows the scheduler to accept an offer and specify a list of operations (Offer.Operation) to perform using the resources in the offer. Currently, the supported operations include LAUNCH (launching tasks), RESERVE (making dynamic reservations), UNRESERVE (releasing dynamic reservations), CREATE (creating persistent volumes) and DESTROY (releasing persistent volumes). Similar to the <code>launchTasks</code> API, any unused resources will be considered declined, and the specified filters will be applied on all unused resources.</p>
+<ul>
+<li><p>The ‘stats.json’ endpoints for masters and slaves have been removed. Please use the ‘metrics/snapshot’ endpoints instead.</p></li>
+<li><p>The ‘/master/shutdown’ endpoint is deprecated in favor of the new ‘/master/teardown’ endpoint.</p></li>
+<li><p>In order to enable decorator modules to remove metadata (environment variables or labels), we changed the meaning of the return value for decorator hooks in Mesos 0.23.0. Please refer to the modules documentation for more details.</p></li>
+<li><p>Slave ping timeouts are now configurable on the master via <code>--slave_ping_timeout</code> and <code>--max_slave_ping_timeouts</code>. Slaves should be upgraded to 0.23.x before changing these flags.</p></li>
+<li><p>A new scheduler driver API, <code>acceptOffers</code>, has been introduced. This is a more general version of the <code>launchTasks</code> API, which allows the scheduler to accept an offer and specify a list of operations (Offer.Operation) to perform using the resources in the offer. Currently, the supported operations include LAUNCH (launching tasks), RESERVE (making dynamic reservations), UNRESERVE (releasing dynamic reservations), CREATE (creating persistent volumes) and DESTROY (releasing persistent volumes). Similar to the <code>launchTasks</code> API, any unused resources will be considered declined, and the specified filters will be applied on all unused resources.</p></li>
+<li><p>The Resource protobuf has been extended to include more metadata for supporting persistence (DiskInfo), dynamic reservations (ReservationInfo) and oversubscription (RevocableInfo). You must not combine two Resource objects if they have different metadata.</p></li>
+</ul>
-<p><strong>NOTE</strong> The Resource protobuf has been extended to include more metadata for supporting persistence (DiskInfo), dynamic reservations (ReservationInfo) and oversubscription (RevocableInfo). You must not combine two Resource objects if they have different metadata.</p>
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Rebuild and install any modules so that upgraded masters/slaves can use them.</li>
<li>Install the new master binaries and restart the masters.</li>
<li>Install the new slave binaries and restart the slaves.</li>
<li>Upgrade the schedulers by linking the latest native library / jar / egg (if necessary).</li>
<li>Restart the schedulers.</li>
<li>Upgrade the executors by linking the latest native library / jar / egg (if necessary).</li>
-</ul>
+</ol>
<h2>Upgrading from 0.21.x to 0.22.x</h2>
-<p><strong>NOTE</strong> Slave checkpoint flag has been removed as it will be enabled for all
+<ul>
+<li><p>Slave checkpoint flag has been removed as it will be enabled for all
slaves. Frameworks must still enable checkpointing during registration to take advantage
-of checkpointing their tasks.</p>
-
-<p><strong>NOTE</strong> The stats.json endpoints for masters and slaves have been deprecated.
-Please refer to the metrics/snapshot endpoint.</p>
+of checkpointing their tasks.</p></li>
+<li><p>The stats.json endpoints for masters and slaves have been deprecated.
+Please refer to the metrics/snapshot endpoint.</p></li>
+<li><p>The C++/Java/Python scheduler bindings have been updated. In particular, the driver can be constructed with an additional argument that specifies whether to use implicit driver acknowledgements. In <code>statusUpdate</code>, the <code>TaskStatus</code> now includes a UUID to make explicit acknowledgements possible.</p></li>
+<li><p>The Authentication API has changed slightly in this release to support additional authentication mechanisms. The change from ‘string’ to ‘bytes’ for AuthenticationStartMessage.data has no impact on C++ or the over-the-wire representation, so it only impacts pure language bindings for languages like Java and Python that use different types for UTF-8 strings vs. byte arrays.</p>
-<p><strong>NOTE</strong> The C++/Java/Python scheduler bindings have been updated. In particular, the driver can be constructed with an additional argument that specifies whether to use implicit driver acknowledgements. In <code>statusUpdate</code>, the <code>TaskStatus</code> now includes a UUID to make explicit acknowledgements possible.</p>
-
-<p><strong>NOTE</strong>: The Authentication API has changed slightly in this release to support additional authentication mechanisms. The change from ‘string’ to ‘bytes’ for AuthenticationStartMessage.data has no impact on C++ or the over-the-wire representation, so it only impacts pure language bindings for languages like Java and Python that use different types for UTF-8 strings vs. byte arrays.</p>
-
-<pre><code>message AuthenticationStartMessage {
- required string mechanism = 1;
- optional bytes data = 2;
-}
-</code></pre>
+<p> message AuthenticationStartMessage {
+ required string mechanism = 1;
+ optional bytes data = 2;
+ }</p></li>
+<li><p>All Mesos arguments can now be passed using file:// to read them out of a file (either an absolute or relative path). The –credentials, –whitelist, and any flags that expect JSON backed arguments (such as –modules) behave as before, although support for just passing an absolute path for any JSON flags rather than file:// has been deprecated and will produce a warning (and the absolute path behavior will be removed in a future release).</p></li>
+</ul>
-<p><strong>NOTE</strong> All Mesos arguments can now be passed using file:// to read them out of a file (either an absolute or relative path). The –credentials, –whitelist, and any flags that expect JSON backed arguments (such as –modules) behave as before, although support for just passing an absolute path for any JSON flags rather than file:// has been deprecated and will produce a warning (and the absolute path behavior will be removed in a future release).</p>
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Install the new master binaries and restart the masters.</li>
<li>Install the new slave binaries and restart the slaves.</li>
-<li>Upgrade the schedulers:
-
-<ul>
-<li>For Java schedulers, link the new native library against the new JAR. The JAR contains API changes per the <strong>NOTE</strong> above. A 0.21.0 JAR will work with a 0.22.0 libmesos. A 0.22.0 JAR will work with a 0.21.0 libmesos if explicit acks are not being used. 0.22.0 and 0.21.0 are inter-operable at the protocol level between the master and the scheduler.</li>
+<li>Upgrade the schedulers:</li>
+<li>For Java schedulers, link the new native library against the new JAR. The JAR contains API above changes. A 0.21.0 JAR will work with a 0.22.0 libmesos. A 0.22.0 JAR will work with a 0.21.0 libmesos if explicit acks are not being used. 0.22.0 and 0.21.0 are inter-operable at the protocol level between the master and the scheduler.</li>
<li>For Python schedulers, upgrade to use a 0.22.0 egg. If constructing <code>MesosSchedulerDriverImpl</code> with <code>Credentials</code>, your code must be updated to pass the <code>implicitAcknowledgements</code> argument before <code>Credentials</code>. You may run a 0.21.0 Python scheduler against a 0.22.0 master, and vice versa.</li>
-</ul>
-</li>
<li>Restart the schedulers.</li>
<li>Upgrade the executors by linking the latest native library / jar / egg.</li>
-</ul>
+</ol>
<h2>Upgrading from 0.20.x to 0.21.x</h2>
-<p><strong>NOTE</strong> Disabling slave checkpointing has been deprecated; the slave –checkpoint flag has been deprecated and will be removed in a future release.</p>
+<ul>
+<li>Disabling slave checkpointing has been deprecated; the slave –checkpoint flag has been deprecated and will be removed in a future release.</li>
+</ul>
+
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Install the new master binaries and restart the masters.</li>
<li>Install the new slave binaries and restart the slaves.</li>
<li>Upgrade the schedulers by linking the latest native library (mesos jar upgrade not necessary).</li>
<li>Restart the schedulers.</li>
<li>Upgrade the executors by linking the latest native library and mesos jar (if necessary).</li>
-</ul>
+</ol>
<h2>Upgrading from 0.19.x to 0.20.x.</h2>
-<p><strong>NOTE</strong>: The Mesos API has been changed slightly in this release. The CommandInfo has been changed (see below), which makes launching a command more flexible. The ‘value’ field has been changed from <em>required</em> to <em>optional</em>. However, it will not cause any issue during the upgrade (since the existing schedulers always set this field).</p>
-
-<pre><code>message CommandInfo {
- ...
- // There are two ways to specify the command:
- // 1) If 'shell == true', the command will be launched via shell
- // (i.e., /bin/sh -c 'value'). The 'value' specified will be
- // treated as the shell command. The 'arguments' will be ignored.
- // 2) If 'shell == false', the command will be launched by passing
- // arguments to an executable. The 'value' specified will be
- // treated as the filename of the executable. The 'arguments'
- // will be treated as the arguments to the executable. This is
- // similar to how POSIX exec families launch processes (i.e.,
- // execlp(value, arguments(0), arguments(1), ...)).
- optional bool shell = 6 [default = true];
- optional string value = 3;
- repeated string arguments = 7;
- ...
-}
-</code></pre>
+<ul>
+<li><p>The Mesos API has been changed slightly in this release. The CommandInfo has been changed (see below), which makes launching a command more flexible. The ‘value’ field has been changed from <em>required</em> to <em>optional</em>. However, it will not cause any issue during the upgrade (since the existing schedulers always set this field).</p>
-<p><strong>NOTE</strong>: The Python bindings are also changing in this release. There are now sub-modules which allow you to use either the interfaces and/or the native driver.</p>
+<pre><code> message CommandInfo {
+ ...
+ // There are two ways to specify the command:
+ // 1) If 'shell == true', the command will be launched via shell
+ // (i.e., /bin/sh -c 'value'). The 'value' specified will be
+ // treated as the shell command. The 'arguments' will be ignored.
+ // 2) If 'shell == false', the command will be launched by passing
+ // arguments to an executable. The 'value' specified will be
+ // treated as the filename of the executable. The 'arguments'
+ // will be treated as the arguments to the executable. This is
+ // similar to how POSIX exec families launch processes (i.e.,
+ // execlp(value, arguments(0), arguments(1), ...)).
+ optional bool shell = 6 [default = true];
+ optional string value = 3;
+ repeated string arguments = 7;
+ ...
+ }
+</code></pre></li>
+<li><p>The Python bindings are also changing in this release. There are now sub-modules which allow you to use either the interfaces and/or the native driver.</p>
<ul>
<li><code>import mesos.native</code> for the native drivers</li>
@@ -289,58 +336,61 @@ Please refer to the metrics/snapshot end
<p>To ensure a smooth upgrade, we recommend to upgrade your python framework and executor first. You will be able to either import using the new configuration or the old. Replace the existing imports with something like the following:</p>
-<pre><code>try:
- from mesos.native import MesosExecutorDriver, MesosSchedulerDriver
- from mesos.interface import Executor, Scheduler
- from mesos.interface import mesos_pb2
-except ImportError:
- from mesos import Executor, MesosExecutorDriver, MesosSchedulerDriver, Scheduler
- import mesos_pb2
-</code></pre>
+<p> try:
+ from mesos.native import MesosExecutorDriver, MesosSchedulerDriver
+ from mesos.interface import Executor, Scheduler
+ from mesos.interface import mesos_pb2
+ except ImportError:
+ from mesos import Executor, MesosExecutorDriver, MesosSchedulerDriver, Scheduler
+ import mesos_pb2</p></li>
+<li><p>If you’re using a pure language binding, please ensure that it sends status update acknowledgements through the master before upgrading.</p></li>
+</ul>
-<p><strong>NOTE</strong>: If you’re using a pure language binding, please ensure that it sends status update acknowledgements through the master before upgrading.</p>
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Install the new master binaries and restart the masters.</li>
<li>Install the new slave binaries and restart the slaves.</li>
<li>Upgrade the schedulers by linking the latest native library (install the latest mesos jar and python egg if necessary).</li>
<li>Restart the schedulers.</li>
<li>Upgrade the executors by linking the latest native library (install the latest mesos jar and python egg if necessary).</li>
-</ul>
+</ol>
<h2>Upgrading from 0.18.x to 0.19.x.</h2>
-<p><strong>NOTE</strong>: There are new required flags on the master (<code>--work_dir</code> and <code>--quorum</code>) to support the <em>Registrar</em> feature, which adds replicated state on the masters.</p>
+<ul>
+<li><p>There are new required flags on the master (<code>--work_dir</code> and <code>--quorum</code>) to support the <em>Registrar</em> feature, which adds replicated state on the masters.</p></li>
+<li><p>No required upgrade ordering across components.</p></li>
+</ul>
-<p><strong>NOTE</strong>: No required upgrade ordering across components.</p>
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Install the new master binaries and restart the masters.</li>
<li>Install the new slave binaries and restart the slaves.</li>
<li>Upgrade the schedulers by linking the latest native library (mesos jar upgrade not necessary).</li>
<li>Restart the schedulers.</li>
<li>Upgrade the executors by linking the latest native library and mesos jar (if necessary).</li>
-</ul>
+</ol>
<h2>Upgrading from 0.17.0 to 0.18.x.</h2>
-<p>In order to upgrade a running cluster:</p>
+<ul>
+<li>This upgrade requires a system reboot for slaves that use Linux cgroups for isolation.</li>
+</ul>
-<p><strong>NOTE</strong>: This upgrade requires a system reboot for slaves that use Linux cgroups for isolation.</p>
-<ul>
+<p>In order to upgrade a running cluster:</p>
+
+<ol>
<li>Install the new master binaries and restart the masters.</li>
<li>Upgrade the schedulers by linking the latest native library and mesos jar (if necessary).</li>
<li>Restart the schedulers.</li>
-<li>Install the new slave binaries then perform one of the following two steps, depending on if cgroups isolation is used:
-
-<ul>
+<li>Install the new slave binaries then perform one of the following two steps, depending on if cgroups isolation is used:</li>
<li>[no cgroups]
<ul>
@@ -354,106 +404,119 @@ except ImportError:
<li>The suggested configuration is to mount a tmpfs filesystem to /sys/fs/cgroup and to let the slave mount the required controllers. However, the slave will also use previously mounted controllers if they are appropriately mounted under “–cgroups_hierarchy”.</li>
<li>It has been observed that unmounting and remounting of cgroups from the single to separate configuration is unreliable and a reboot into the new configuration is strongly advised. Restart the slaves after reboot.</li>
<li>The “–cgroups_hierarchy” now defaults to “/sys/fs/cgroup”. The “–cgroups_root” flag default remains “mesos”.</li>
-<li> The “–isolation” flag has changed and “cgroups” has been deprecated in favor of “cgroups/cpu,cgroups/mem”.</li>
+<li>The “–isolation” flag has changed and “cgroups” has been deprecated in favor of “cgroups/cpu,cgroups/mem”.</li>
<li>The “–cgroup_subsystems” flag is no longer required and will be ignored.</li>
</ul>
</li>
-</ul>
-</li>
<li>Upgrade the executors by linking the latest native library and mesos jar (if necessary).</li>
-</ul>
+</ol>
<h2>Upgrading from 0.16.0 to 0.17.0.</h2>
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Install the new master binaries and restart the masters.</li>
<li>Upgrade the schedulers by linking the latest native library and mesos jar (if necessary).</li>
<li>Restart the schedulers.</li>
<li>Install the new slave binaries and restart the slaves.</li>
<li>Upgrade the executors by linking the latest native library and mesos jar (if necessary).</li>
-</ul>
+</ol>
<h2>Upgrading from 0.15.0 to 0.16.0.</h2>
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Install the new master binaries and restart the masters.</li>
<li>Upgrade the schedulers by linking the latest native library and mesos jar (if necessary).</li>
<li>Restart the schedulers.</li>
<li>Install the new slave binaries and restart the slaves.</li>
<li>Upgrade the executors by linking the latest native library and mesos jar (if necessary).</li>
-</ul>
+</ol>
<h2>Upgrading from 0.14.0 to 0.15.0.</h2>
+<ul>
+<li>Schedulers should implement the new <code>reconcileTasks</code> driver method.</li>
+<li>Schedulers should call the new <code>MesosSchedulerDriver</code> constructor that takes <code>Credential</code> to authenticate.</li>
+<li>–authentication=false (default) allows both authenticated and unauthenticated frameworks to register.</li>
+</ul>
+
+
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Install the new master binaries.</li>
<li>Restart the masters with –credentials pointing to credentials of the framework(s).</li>
-<li>NOTE: –authentication=false (default) allows both authenticated and unauthenticated frameworks to register.</li>
<li>Install the new slave binaries and restart the slaves.</li>
<li>Upgrade the executors by linking the latest native library and mesos jar (if necessary).</li>
<li>Upgrade the schedulers by linking the latest native library and mesos jar (if necessary).</li>
-<li>NOTE: Schedulers should implement the new <code>reconcileTasks</code> driver method.</li>
-<li>Schedulers should call the new <code>MesosSchedulerDriver</code> constructor that takes <code>Credential</code> to authenticate.</li>
-<li>Restart the schedulers.</li>
-<li>Restart the masters with –authentication=true.</li>
-<li>NOTE: After the restart unauthenticated frameworks <em>will not</em> be allowed to register.</li>
-</ul>
+<li>Restart the schedulers.
+Restart the masters with –authentication=true.</li>
+</ol>
+
+<p>NOTE: After the restart unauthenticated frameworks <em>will not</em> be allowed to register.</p>
<h2>Upgrading from 0.13.0 to 0.14.0.</h2>
+<ul>
+<li>/vars endpoint has been removed.</li>
+</ul>
+
+
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Install the new master binaries and restart the masters.</li>
-<li>NOTE: /vars endpoint has been removed.</li>
<li>Upgrade the executors by linking the latest native library and mesos jar (if necessary).</li>
<li>Install the new slave binaries.</li>
<li>Restart the slaves after adding –checkpoint flag to enable checkpointing.</li>
-<li>NOTE: /vars endpoint has been removed.</li>
<li>Upgrade the schedulers by linking the latest native library and mesos jar (if necessary).</li>
<li>Set FrameworkInfo.checkpoint in the scheduler if checkpointing is desired (recommended).</li>
<li>Restart the schedulers.</li>
<li>Restart the masters (to get rid of the cached FrameworkInfo).</li>
<li>Restart the slaves (to get rid of the cached FrameworkInfo).</li>
-</ul>
+</ol>
<h2>Upgrading from 0.12.0 to 0.13.0.</h2>
+<ul>
+<li>cgroups_hierarchy_root slave flag is renamed as cgroups_hierarchy</li>
+</ul>
+
+
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Install the new master binaries and restart the masters.</li>
<li>Upgrade the schedulers by linking the latest native library and mesos jar (if necessary).</li>
<li>Restart the schedulers.</li>
<li>Install the new slave binaries.</li>
-<li>NOTE: cgroups_hierarchy_root slave flag is renamed as cgroups_hierarchy</li>
<li>Restart the slaves.</li>
<li>Upgrade the executors by linking the latest native library and mesos jar (if necessary).</li>
-</ul>
+</ol>
<h2>Upgrading from 0.11.0 to 0.12.0.</h2>
+<ul>
+<li>If you are a framework developer, you will want to examine the new ‘source’ field in the ExecutorInfo protobuf. This will allow you to take further advantage of the resource monitoring.</li>
+</ul>
+
+
<p>In order to upgrade a running cluster:</p>
-<ul>
+<ol>
<li>Install the new slave binaries and restart the slaves.</li>
<li>Install the new master binaries and restart the masters.</li>
-</ul>
-
+</ol>
-<p>If you are a framework developer, you will want to examine the new ‘source’ field in the ExecutorInfo protobuf. This will allow you to take further advantage of the resource monitoring.</p>
</div>
</div>
Modified: mesos/site/publish/documentation/latest/versioning/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/latest/versioning/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/latest/versioning/index.html (original)
+++ mesos/site/publish/documentation/latest/versioning/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Versioning</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
Modified: mesos/site/publish/documentation/logging/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/logging/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/logging/index.html (original)
+++ mesos/site/publish/documentation/logging/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Logging</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -106,8 +106,8 @@ library, see the <a href="/documentation
Google logging options that are not explicitly mentioned there can be
configured via environment variables.</p>
-<p>Both Master and Agent also expose an HTTP endpoint which temporarily toggles
-verbose logging:</p>
+<p>Both Master and Agent also expose a <a href="/documentation/latest/./endpoints/logging/toggle/">/logging/toggle</a>
+HTTP endpoint which temporarily toggles verbose logging:</p>
<pre><code>POST <ip:port>/logging/toggle?level=[1|2|3]&duration=VALUE
</code></pre>
Modified: mesos/site/publish/documentation/maintenance/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/maintenance/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/maintenance/index.html (original)
+++ mesos/site/publish/documentation/maintenance/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Maintenance Primitives</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
Modified: mesos/site/publish/documentation/markdown-style-guide/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/markdown-style-guide/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/markdown-style-guide/index.html (original)
+++ mesos/site/publish/documentation/markdown-style-guide/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Markdown Style Guide</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
Modified: mesos/site/publish/documentation/mesos-containerizer/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/mesos-containerizer/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/mesos-containerizer/index.html (original)
+++ mesos/site/publish/documentation/mesos-containerizer/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Mesos Containerizer</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -156,13 +156,158 @@ specify <code>--enforce_container_disk_q
<p>The Posix Disk isolator reports disk usage for each sandbox by
periodically running the <code>du</code> command. The disk usage can be retrieved
-from the resource statistics endpoint (<code>/monitor/statistics.json</code>).</p>
+from the resource statistics endpoint (<a href="/documentation/latest/./endpoints/monitor/statistics/">/monitor/statistics</a>).</p>
<p>The interval between two <code>du</code>s can be controlled by the slave flag
<code>--container_disk_watch_interval</code>. For example,
<code>--container_disk_watch_interval=1mins</code> sets the interval to be 1
minute. The default interval is 15 seconds.</p>
+<h3>Docker Runtime Isolator</h3>
+
+<p>The Docker Runtime isolator is used for supporting runtime
+configurations from the docker image (e.g., Entrypoint/Cmd, Env,
+etc.). This isolator is tied with <code>--image_providers=docker</code>. If
+<code>--image_providers</code> contains <code>docker</code>, this isolator must be used.
+Otherwise, slave will refuse to start.</p>
+
+<p>To enable the Docker Runtime isolator, append <code>docker/runtime</code> to the
+<code>--isolation</code> flag when starting the slave.</p>
+
+<p>Currently, docker image default <code>Entrypoint</code>, <code>Cmd</code>, <code>Env</code> and
+<code>WorkingDir</code> are supported with docker runtime isolator. Users can
+specify <code>CommandInfo</code> to override the default <code>Entrypoint</code> and <code>Cmd</code>
+in the image (see below for details). The <code>CommandInfo</code> should be
+inside of either <code>TaskInfo</code> or <code>ExecutorInfo</code> (depending on running
+command task or custom executor respectively).</p>
+
+<h4>Determine the Launch Command</h4>
+
+<p>If user specifies a command in <code>CommandInfo</code>, that will override the
+default Entrypoint/Cmd in the docker image. Otherwise, we will use the
+default Entrypoint/Cmd and append arguments specified in <code>CommandInfo</code>
+accordingly. The details are explained in the following table.</p>
+
+<p>Users can specify <code>CommandInfo</code> including <code>shell</code>, <code>value</code> and
+<code>arguments</code>, which are represented in the first column of the table
+below. <code>0</code> represents <code>not specified</code>, while <code>1</code> represents
+<code>specified</code>. The first row is how <code>Entrypoint</code> and <code>Cmd</code> defined in
+the docker image. All cells in the table, except the first column and
+row, as well as cells labeled as <code>Error</code>, have the first element
+(i.e., <code>/Entrypt[0]</code>) as executable, and the rest as appending
+arguments.</p>
+
+<table class="table table-striped">
+ <tr>
+ <th></th>
+ <th>Entrypoint=0<br>Cmd=0</th>
+ <th>Entrypoint=0<br>Cmd=1</th>
+ <th>Entrypoint=1<br>Cmd=0</th>
+ <th>Entrypoint=1<br>Cmd=1</th>
+ </tr>
+ <tr>
+ <td>sh=0<br>value=0<br>argv=0</td>
+ <td>Error</td>
+ <td>/Cmd[0]<br>Cmd[1]..</td>
+ <td>/Entrypt[0]<br>Entrypy[1]..</td>
+ <td>/Entrypy[0]<br>Entrypy[1]..<br>Cmd..</td>
+ </tr>
+ <tr>
+ <td>sh=0<br>value=0<br>argv=1</td>
+ <td>Error</td>
+ <td>/Cmd[0]<br>argv</td>
+ <td>/Entrypt[0]<br>Entrypy[1]..<br>argv</td>
+ <td>/Entrypy[0]<br>Entrypy[1]..<br>argv</td>
+ </tr>
+ <tr>
+ <td>sh=0<br>value=1<br>argv=0</td>
+ <td>/value</td>
+ <td>/value</td>
+ <td>/value</td>
+ <td>/value</td>
+ </tr>
+ <tr>
+ <td>sh=0<br>value=1<br>argv=1</td>
+ <td>/value<br>argv</td>
+ <td>/value<br>argv</td>
+ <td>/value<br>argv</td>
+ <td>/value<br>argv</td>
+ </tr>
+ <tr>
+ <td>sh=1<br>value=0<br>argv=0</td>
+ <td>Error</td>
+ <td>Error</td>
+ <td>Error</td>
+ <td>Error</td>
+ </tr>
+ <tr>
+ <td>sh=1<br>value=0<br>argv=1</td>
+ <td>Error</td>
+ <td>Error</td>
+ <td>Error</td>
+ <td>Error</td>
+ </tr>
+ <tr>
+ <td>sh=1<br>value=1<br>argv=0</td>
+ <td>/bin/sh -c<br>value</td>
+ <td>/bin/sh -c<br>value</td>
+ <td>/bin/sh -c<br>value</td>
+ <td>/bin/sh -c<br>value</td>
+ </tr>
+ <tr>
+ <td>sh=1<br>value=1<br>argv=1</td>
+ <td>/bin/sh -c<br>value</td>
+ <td>/bin/sh -c<br>value</td>
+ <td>/bin/sh -c<br>value</td>
+ <td>/bin/sh -c<br>value</td>
+ </tr>
+</table>
+
+
+<h3>The <code>cgroups/net_cls</code> Isolator</h3>
+
+<p>The cgroups/net_cls isolator allows operators to provide network
+performance isolation and network segmentation for containers within
+a Mesos cluster. To enable the cgroups/net_cls isolator, append
+<code>cgroups/net_cls</code> to the <code>--isolation</code> flag when starting the slave.</p>
+
+<p>As the name suggests, the isolator enables the net_cls subsystem for
+Linux cgroups and assigns a net_cls cgroup to each container launched
+by the <code>MesosContainerizer</code>. The objective of the net_cls subsystem
+is to allow the kernel to tag packets originating from a container
+with a 32-bit handle. These handles can be used by kernel modules such
+as <code>qdisc</code> (for traffic engineering) and <code>net-filter</code> (for
+firewall) to enforce network performance and security policies
+specified by the operators. The policies, based on the net_cls
+handles, can be specified by the operators through user-space tools
+such as
+<a href="http://tldp.org/HOWTO/Traffic-Control-HOWTO/software.html#s-iproute2-tc">tc</a>
+and <a href="http://linux.die.net/man/8/iptables">iptables</a>.</p>
+
+<p>The 32-bit handle associated with a net_cls cgroup can be specified by
+writing the handle to the <code>net_cls.classid</code> file, present within the
+net_cls cgroup. The 32-bit handle is of the form <code>0xAAAABBBB</code>, and
+consists of a 16-bit primary handle 0xAAAA and a 16-bit secondary
+handle 0xBBBB. You can read more about the use cases for the primary
+and secondary handles in the <a href="https://www.kernel.org/doc/Documentation/cgroup-v1/net_cls.txt">Linux kernel documentation for
+net_cls</a>.</p>
+
+<p>By default the cgroups/net_cls isolator does not manage the net_cls
+handles, and assumes the operator is going to manage/assign these
+handles. To enable the management of net_cls handles by the
+cgroups/net_cls isolator you need to specify a 16-bit primary handle,
+of the form 0xAAAA, using the <code>--cgroups_net_cls_primary_handle</code> flag at
+slave startup.</p>
+
+<p>Once a primary handle has been specified for a slave, for each
+container the cgroups/net_cls isolator allocates a 16-bit secondary
+handle. It then assigns the 32-bit combination of the primary and
+secondary handle to the net_cls cgroup associated with the container
+by writing to <code>net_cls.classid</code>. The cgroups/net_cls isolator exposes
+the assigned net_cls handle to operators by exposing the handle as
+part of the <code>ContainerStatus</code> —associated with any task running within
+the container— in the slave’s <code>state</code> endpoint.</p>
+
</div>
</div>
Modified: mesos/site/publish/documentation/modules/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/modules/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/modules/index.html (original)
+++ mesos/site/publish/documentation/modules/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Modules</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
Modified: mesos/site/publish/documentation/monitoring/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/monitoring/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/monitoring/index.html (original)
+++ mesos/site/publish/documentation/monitoring/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Observability Metrics</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
Modified: mesos/site/publish/documentation/multiple-disk/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/multiple-disk/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/multiple-disk/index.html (original)
+++ mesos/site/publish/documentation/multiple-disk/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Multiple Disks</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
Modified: mesos/site/publish/documentation/network-monitoring/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/network-monitoring/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/network-monitoring/index.html (original)
+++ mesos/site/publish/documentation/network-monitoring/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Per-container Network Monitoring and Isolation</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -87,7 +87,7 @@
isolation. The network isolation prevents a single container from exhausting the
available network ports, consuming an unfair share of the network bandwidth or
significantly delaying packet transmission for others. Network statistics for
-each active container are published through the <code>/monitor/statistics.json</code>
+each active container are published through the <a href="/documentation/latest/./endpoints/monitor/statistics/">/monitor/statistics</a>
endpoint on the slave. The network isolation is transparent for the majority of
tasks running on a slave (those that bind to port 0 and let the kernel allocate
their port).</p>
@@ -275,7 +275,7 @@ enabled would thus be:</p>
<h2>Monitoring container network statistics</h2>
<p>Mesos exposes statistics from the Linux network stack for each container network
-on the <code>/monitor/statistics.json</code> slave endpoint.</p>
+on the <a href="/documentation/latest/./endpoints/monitor/statistics/">/monitor/statistics</a> slave endpoint.</p>
<p>From the network interface inside the container, we report the following
counters (since container creation) under the <code>statistics</code> key:</p>
@@ -384,9 +384,9 @@ for each of these elements includes:</p>
</table>
-<p>[1] <code>backlog</code> is only reported on the bloat_reduction interface</p>
+<p>[1] <code>backlog</code> is only reported on the bloat_reduction interface.</p>
-<p>[2] <code>overlimits</code> are only reported on the bw_limit interface</p>
+<p>[2] <code>overlimits</code> are only reported on the bw_limit interface.</p>
<p>[3] Currently always reported as 0 by the underlying Traffic Control element.</p>
Modified: mesos/site/publish/documentation/networking-for-mesos-managed-containers/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/networking-for-mesos-managed-containers/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/networking-for-mesos-managed-containers/index.html (original)
+++ mesos/site/publish/documentation/networking-for-mesos-managed-containers/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Networking for Mesos-Managed Containers</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -81,7 +81,7 @@
<p>See our <a href="/community/">community</a> page for more details.</p>
</div>
<div class="col-md-8">
- <h1>Networking for Mesos-managed containers</h1>
+ <h1>Networking for Mesos-Managed Containers</h1>
<p>While networking plays a key role in data center infrastructure, it is – for
now – beyond the scope of Mesos to try to address the concerns of networking
@@ -205,7 +205,7 @@ or not.</li>
<ul>
<li>The IP address(es) from TaskStatus are made available at Master’s
-state endpoint.</li>
+<a href="/documentation/latest/./endpoints/master/state/">/state</a> endpoint.</li>
<li>The TaskStatus is also forwarded to the framework to inform it of the IP
addresses.</li>
<li>When a task is killed or lost, NIM communicates with IPAM client to release
Modified: mesos/site/publish/documentation/newbie-guide/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/newbie-guide/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/newbie-guide/index.html (original)
+++ mesos/site/publish/documentation/newbie-guide/index.html Tue Mar 8 05:37:24 2016
@@ -81,7 +81,8 @@
<p>See our <a href="/community/">community</a> page for more details.</p>
</div>
<div class="col-md-8">
- <p>–</p>
+ <p>–
+title: Apache Mesos - Newbie Guide</p>
<h2>layout: documentation</h2>
Modified: mesos/site/publish/documentation/operational-guide/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/operational-guide/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/operational-guide/index.html (original)
+++ mesos/site/publish/documentation/operational-guide/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Operational Guide</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
Modified: mesos/site/publish/documentation/oversubscription/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/oversubscription/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/oversubscription/index.html (original)
+++ mesos/site/publish/documentation/oversubscription/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Oversubscription</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
Modified: mesos/site/publish/documentation/persistent-volume/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/persistent-volume/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/persistent-volume/index.html (original)
+++ mesos/site/publish/documentation/persistent-volume/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Persistent Volumes</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -108,11 +108,14 @@ regarding reservation mechanisms availab
reserving <a href="/documentation/latest/./multiple-disk/">multiple disk resources</a>.</p>
<p>Persistent volumes can be created by <strong>operators</strong> and authorized
-<strong>frameworks</strong>. We require a <code>principal</code> from the operator or framework in order
-to authenticate/authorize the operations. Permissions are specified via the
-existing ACL mechanism. To use authorization with reserve, unreserve, create,
-and destroy operations, the Mesos master must be configured with the desired
-ACLs. For more information, see the
+<strong>frameworks</strong>. By default, frameworks and operators can create volumes for any
+role and destroy any persistent volumes. <a href="/documentation/latest/./authorization/">Authorization</a>
+allows this behavior to be limited so that volumes can only be created for
+particular roles and only particular volumes can be destroyed. For these
+operations to be authorized, the framework or operator should provide a
+<code>principal</code> to identify itself. To use authorization with reserve, unreserve,
+create, and destroy operations, the Mesos master must be configured with the
+appropriate ACLs. For more information, see the
<a href="/documentation/latest/./authorization/">authorization documentation</a>.</p>
<ul>
@@ -161,36 +164,36 @@ volume information. We need to specify t
<ol>
<li>The ID for the persistent volume; this must be unique per role on each slave.</li>
<li>The non-nested relative path within the container to mount the volume.</li>
-<li>The permissions for the volume. Currently, <code>"RW"</code> is the only possible value.</li>
-</ol>
+<li><p>The permissions for the volume. Currently, <code>"RW"</code> is the only possible value.</p>
+<pre><code> {
+ "type" : Offer::Operation::CREATE,
+ "create": {
+ "volumes" : [
+ {
+ "name" : "disk",
+ "type" : "SCALAR",
+ "scalar" : { "value" : 2048 },
+ "role" : <framework_role>,
+ "reservation" : {
+ "principal" : <framework_principal>
+ },
+ "disk": {
+ "persistence": {
+ "id" : <persistent_volume_id>
+ },
+ "volume" : {
+ "container_path" : <container_path>,
+ "mode" : <mode>
+ }
+ }
+ }
+ ]
+ }
+ }
+</code></pre></li>
+</ol>
-<pre><code>{
- "type" : Offer::Operation::CREATE,
- "create": {
- "volumes" : [
- {
- "name" : "disk",
- "type" : "SCALAR",
- "scalar" : { "value" : 2048 },
- "role" : <framework_role>,
- "reservation" : {
- "principal" : <framework_principal>
- },
- "disk": {
- "persistence": {
- "id" : <persistent_volume_id>
- },
- "volume" : {
- "container_path" : <container_path>,
- "mode" : <mode>
- }
- }
- }
- ]
- }
-}
-</code></pre>
<p>If this succeeds, a subsequent resource offer will contain the following
persistent volume:</p>
@@ -333,10 +336,12 @@ by operators and administrative tools.</
<p>To use this endpoint, the operator should first ensure that a reservation for
the necessary resources has been made on the appropriate slave (e.g., by using
-the <code>/reserve</code> HTTP endpoint or by configuring a static reservation).</p>
+the <a href="/documentation/latest/./endpoints/master/reserve/">/reserve</a> HTTP endpoint or by configuring a
+static reservation).</p>
<p>To create a 512MB persistent volume for the <code>ads</code> role on a dynamically reserved
-disk resource, we can send a request like so:</p>
+disk resource, we can send an HTTP POST request to the master’s
+<a href="/documentation/latest/./endpoints/master/create-volumes/">/create-volumes</a> endpoint like so:</p>
<pre><code>curl -i \
-u <operator_principal>:<password> \
@@ -386,11 +391,12 @@ the reserved resources are located. That
delivered or creating the volumes at the slave might fail, in which case no
volumes will be created. To determine if a create operation has succeeded, the
user can examine the state of the appropriate Mesos slave (e.g., via the slave’s
-<code>/state</code> HTTP endpoint).</p>
+<a href="/documentation/latest/./endpoints/slave/state/">/state</a> HTTP endpoint).</p>
<h4><code>/destroy-volumes</code></h4>
-<p>To destroy the volume created above, we can send an HTTP POST like so:</p>
+<p>To destroy the volume created above, we can send an HTTP POST to the master’s
+<a href="/documentation/latest/./endpoints/master/destroy-volumes/">/destroy-volumes</a> endpoint like so:</p>
<pre><code>curl -i \
-u <operator_principal>:<password> \
@@ -440,7 +446,13 @@ the persistent volumes are located. That
delivered or destroying the volumes at the slave might fail, in which case no
volumes will be destroyed. To determine if a destroy operation has succeeded,
the user can examine the state of the appropriate Mesos slave (e.g., via the
-slave’s <code>/state</code> HTTP endpoint).</p>
+slave’s <a href="/documentation/latest/./endpoints/slave/state/">/state</a> HTTP endpoint).</p>
+
+<h3>Listing Persistent Volumes</h3>
+
+<p>Information about the persistent volumes at each slave in the cluster can be
+found by querying the <a href="/documentation/latest/./endpoints/master/slaves/">/slaves</a> master endpoint
+(under the <code>reserved_resources_full</code> key).</p>
<h3>Programming with Persistent Volumes</h3>
@@ -496,7 +508,7 @@ requests to reserve 2 CPUs at a single a
succeed, the result will be a single reservation of 4 CPUs. To handle this
situation, applications should be prepared for resource offers that contain
more resources than expected. Some applications may also want to detect this
-situation and unreserve an additional reserved resources that will not be
+situation and unreserve any additional reserved resources that will not be
required.</p></li>
<li><p>It often makes sense to structure application logic as a “state machine”,
where the application moves from its initial state (no reserved resources and
@@ -516,7 +528,7 @@ difficult for frameworks to determine wh
succeeded: as discussed above, frameworks need to wait for an offer that
contains the “expected” reserved resources to determine when a reservation
request has succeeded. Determining what a framework should “expect” to find in
-an offer is more difficult when multiple frameworks can be making reservations
+an offer is more difficult when multiple frameworks can make reservations
for the same role concurrently. In general, whenever multiple frameworks are
allowed to register in the same role, the operator should ensure that those
frameworks are configured to collaborate with one another when using
Modified: mesos/site/publish/documentation/powered-by-mesos/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/powered-by-mesos/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/powered-by-mesos/index.html (original)
+++ mesos/site/publish/documentation/powered-by-mesos/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Powered By Mesos</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
Modified: mesos/site/publish/documentation/presentations/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/presentations/index.html?rev=1734022&r1=1734021&r2=1734022&view=diff
==============================================================================
--- mesos/site/publish/documentation/presentations/index.html (original)
+++ mesos/site/publish/documentation/presentations/index.html Tue Mar 8 05:37:24 2016
@@ -2,7 +2,7 @@
<html>
<head>
<meta charset="utf-8">
- <title></title>
+ <title>Apache Mesos - Video and Slides of Mesos Presentations</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">