You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@mesos.apache.org by ka...@apache.org on 2016/02/01 03:49:28 UTC
svn commit: r1727886 [9/16] - in /mesos/site: ./ publish/
publish/assets/img/documentation/ publish/community/ publish/documentation/
publish/documentation/allocation-module/
publish/documentation/app-framework-development-guide/
publish/documentation/...
Added: mesos/site/publish/documentation/mesos-containerizer/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/mesos-containerizer/index.html?rev=1727886&view=auto
==============================================================================
--- mesos/site/publish/documentation/mesos-containerizer/index.html (added)
+++ mesos/site/publish/documentation/mesos-containerizer/index.html Mon Feb 1 02:49:25 2016
@@ -0,0 +1,184 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title></title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
+ <link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml">
+
+ <link href="../../assets/css/main.css" media="screen" rel="stylesheet" type="text/css" />
+
+
+
+ <!-- Google Analytics Magic -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-20226872-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <!-- magical breadcrumbs -->
+ <div class="topnav">
+ <ul class="breadcrumb">
+ <li>
+ <div class="dropdown">
+ <a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a>
+ <ul class="dropdown-menu" role="menu" aria-labelledby="dLabel">
+ <li><a href="http://www.apache.org">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ </li>
+ <li><a href="http://mesos.apache.org">Apache Mesos</a></li>
+
+
+ <li><a href="/documentation
+/">Documentation
+</a></li>
+
+
+ </ul><!-- /breadcrumb -->
+ </div>
+
+ <!-- navbar excitement -->
+ <div class="navbar navbar-static-top" role="navigation">
+ <div class="navbar-inner">
+ <div class="container">
+ <a href="/" class="logo"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo" /></a>
+ <div class="nav-collapse">
+ <ul class="nav nav-pills navbar-right">
+ <li><a href="/gettingstarted/">Getting Started</a></li>
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Community</a></li>
+ </ul>
+ </div>
+ </div>
+ </div>
+ </div><!-- /.navbar -->
+
+ <div class="container">
+
+ <div class="row-fluid">
+ <div class="col-md-4">
+ <h4>If you're new to Mesos</h4>
+ <p>See the <a href="/gettingstarted/">getting started</a> page for more information about downloading, building, and deploying Mesos.</p>
+
+ <h4>If you'd like to get involved or you're looking for support</h4>
+ <p>See our <a href="/community/">community</a> page for more details.</p>
+ </div>
+ <div class="col-md-8">
+ <h1>Mesos Containerizer</h1>
+
+<p>The MesosContainerizer provides lightweight containerization and
+resource isolation of executors using Linux-specific functionality
+such as control cgroups and namespaces. It is composable so operators
+can selectively enable different isolators.</p>
+
+<p>It also provides basic support for POSIX systems (e.g., OSX) but
+without any actual isolation, only resource usage reporting.</p>
+
+<h3>Shared Filesystem</h3>
+
+<p>The SharedFilesystem isolator can optionally be used on Linux hosts to
+enable modifications to each container’s view of the shared
+filesystem.</p>
+
+<p>The modifications are specified in the ContainerInfo included in the
+ExecutorInfo, either by a framework or by using the
+<code>--default_container_info</code> slave flag.</p>
+
+<p>ContainerInfo specifies Volumes which map parts of the shared
+filesystem (host_path) into the container’s view of the filesystem
+(container_path), as read-write or read-only. The host_path can be
+absolute, in which case it will make the filesystem subtree rooted at
+host_path also accessible under container_path for each container.
+If host_path is relative then it is considered as a directory
+relative to the executor’s work directory. The directory will be
+created and permissions copied from the corresponding directory (which
+must exist) in the shared filesystem.</p>
+
+<p>The primary use-case for this isolator is to selectively make parts of
+the shared filesystem private to each container. For example, a
+private “/tmp” directory can be achieved with <code>host_path="tmp"</code> and
+<code>container_path="/tmp"</code> which will create a directory “tmp” inside the
+executor’s work directory (mode 1777) and simultaneously mount it as
+/tmp inside the container. This is transparent to processes running
+inside the container. Containers will not be able to see the host’s
+/tmp or any other container’s /tmp.</p>
+
+<h3>Pid Namespace</h3>
+
+<p>The Pid Namespace isolator can be used to isolate each container in
+a separate pid namespace with two main benefits:</p>
+
+<ol>
+<li><p>Visibility: Processes running in the container (executor and
+descendants) are unable to see or signal processes outside the
+namespace.</p></li>
+<li><p>Clean termination: Termination of the leading process in a pid
+namespace will result in the kernel terminating all other processes
+in the namespace.</p></li>
+</ol>
+
+
+<p>The Launcher will use (2) during destruction of a container in
+preference to the freezer cgroup, avoiding known kernel issues related
+to freezing cgroups under OOM conditions.</p>
+
+<p>/proc will be mounted for containers so tools such as ‘ps’ will work
+correctly.</p>
+
+<h3>Posix Disk Isolator</h3>
+
+<p>The Posix Disk isolator provides basic disk isolation. It is able to
+report the disk usage for each sandbox and optionally enforce the disk
+quota. It can be used on both Linux and OS X.</p>
+
+<p>To enable the Posix Disk isolator, append <code>posix/disk</code> to the
+<code>--isolation</code> flag when starting the slave.</p>
+
+<p>By default, the disk quota enforcement is disabled. To enable it,
+specify <code>--enforce_container_disk_quota</code> when starting the slave.</p>
+
+<p>The Posix Disk isolator reports disk usage for each sandbox by
+periodically running the <code>du</code> command. The disk usage can be retrieved
+from the resource statistics endpoint (<code>/monitor/statistics.json</code>).</p>
+
+<p>The interval between two <code>du</code>s can be controlled by the slave flag
+<code>--container_disk_watch_interval</code>. For example,
+<code>--container_disk_watch_interval=1mins</code> sets the interval to be 1
+minute. The default interval is 15 seconds.</p>
+
+ </div>
+</div>
+
+
+ <hr>
+
+ <!-- footer -->
+ <div class="footer">
+ <p>© 2012-2015 <a href="http://apache.org">The Apache Software Foundation</a>.
+ Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation.<p>
+ </div><!-- /footer -->
+
+ </div> <!-- /container -->
+
+ <!-- JS -->
+ <script src="//code.jquery.com/jquery-1.11.0.min.js" type="text/javascript"></script>
+ <script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js" type="text/javascript"></script>
+ </body>
+</html>
Modified: mesos/site/publish/documentation/mesos-provisioner/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/mesos-provisioner/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/mesos-provisioner/index.html (original)
+++ mesos/site/publish/documentation/mesos-provisioner/index.html Mon Feb 1 02:49:25 2016
@@ -101,28 +101,29 @@
<p>We introduced a new protobuf message <code>Image</code> that describes a container filesystem image.</p>
-<p>message Image {
+<pre><code class="{.proto}">message Image {
enum Type {
APPC = 1;
DOCKER = 2;
// More Image types.
- }</p>
+ }
-<p> message Appc {
+ message Appc {
// Appc configurations.
- }</p>
+ }
-<p> message Docker {
+ message Docker {
// Docker configurations
- }</p>
+ }
-<p> required Type type = 1;</p>
+ required Type type = 1;
-<p> // Only one of the following image messages should be set to match
+ // Only one of the following image messages should be set to match
// the type.
optional Appc appc = 2;
optional Docker docker = 3;
-}</p>
+}
+</code></pre>
<p>This <code>Image</code> message type contains the type of image specification and the corresponding configurations for that type.</p>
@@ -135,12 +136,12 @@ that volume will be mounted with the ima
<h2>Setup and Agent Flags</h2>
-<p>To run the agent to enable Mesos containerizer, you must launch the agent with mesos as a containerizer option, which is the default containerizer for the mesos agent. It also has to enable linux/filesystem as part of the enabled isolators. The supported image providers can also be configured through agent flags, as well as the supported image provider backend.</p>
+<p>To run the agent to enable Mesos containerizer, you must launch the agent with mesos as a containerizer option, which is the default containerizer for the mesos agent. It also has to enable filesystem/linux as part of the enabled isolators. The supported image providers can also be configured through agent flags, as well as the supported image provider backend.</p>
<p>Mesos agent also needs to be running under linux with root permissions.</p>
<ul>
-<li>Example: <code>mesos-slave --containerizers=mesos --image_providers=appc,docker --image_provisioner_backend=copy --isolation=linux/filesystem</code></li>
+<li>Example: <code>mesos-slave --containerizers=mesos --image_providers=appc,docker --image_provisioner_backend=copy --isolation=filesystem/linux</code></li>
</ul>
@@ -165,11 +166,11 @@ Mesos agent supports multiple image prov
<p>https://github.com/docker/docker/blob/master/image/spec/v1.md</p>
-<p>Docker image provider supports two different methods of finding and pulling images: local and registry. The <code>docker_puller</code> agent flag allows the slave to choose between these methods.</p>
+<p>Docker image provider supports two different methods of finding and pulling images: local and registry. The <code>docker_registry</code> agent flag allows the slave to choose between these methods based on the URL scheme.</p>
-<p>Local puller finds docker images based on image name and tag in the host filesystem, in the directory configured by <code>docker_local_archives_dir</code>.</p>
+<p>By specifying a URL in <code>docker_registry</code> agent flag pointing to a local directory (file:///tmp/mesos/images/docker), the provisioner will use the Local puller to find docker images based on image name and tag in the host filesystem.</p>
-<p>Registry puller finds and downloads images by contacting Docker registry, and by default contacts server configured by <code>docker_registry</code> and <code>docker_registry_port</code> agent flag when no custom registry is specified.</p>
+<p>If the URL configured in <code>docker_registry</code> isn’t pointing to a local directory, it will be assumed to be a registry referring to a Docker registry. The Registry puller will find and download images by contacting Docker registry with the configured URL when no custom registry is specified.</p>
<p>Note that to run the Registry puller Mesos agent must be running with SSL enabled.</p>
@@ -196,8 +197,8 @@ backend tracked by MESOS-2971). For smal
<h2>Related Docs</h2>
-<p>For more information on the Mesos containerizer filesystem, namespace, and isolator features, visit <a href="/documentation/latest/https://github.com/apache/mesos/blob/master/docs/containerizer/">here</a>.
-For more information on launching Docker containers through the Docker containerizer, visit <a href="/documentation/latest/https://github.com/apache/mesos/blob/master/docs/docker-containerizer/">here</a>.</p>
+<p>For more information on the Mesos containerizer filesystem, namespace, and isolator features, visit <a href="/documentation/latest/mesos-containerizer/">Mesos Containerizer</a>.
+For more information on launching Docker containers through the Docker containerizer, visit <a href="/documentation/latest/docker-containerizer/">Docker Containerizer</a>.</p>
</div>
</div>
Modified: mesos/site/publish/documentation/modules/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/modules/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/modules/index.html (original)
+++ mesos/site/publish/documentation/modules/index.html Mon Feb 1 02:49:25 2016
@@ -109,7 +109,7 @@ scripting languages without forcing thos
Finally, modules provide an easy way for third parties to easily extend Mesos
without having to know all the internal details.</p>
-<h2>Invoking Mesos modules</h2>
+<h2><a name="Invoking"></a>Invoking Mesos modules</h2>
<p>The command-line flag <code>--modules</code> is available for Mesos master, slave, and
tests to specify a list of modules to be loaded and be available to the internal
Modified: mesos/site/publish/documentation/networking-for-mesos-managed-containers/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/networking-for-mesos-managed-containers/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/networking-for-mesos-managed-containers/index.html (original)
+++ mesos/site/publish/documentation/networking-for-mesos-managed-containers/index.html Mon Feb 1 02:49:25 2016
@@ -105,7 +105,7 @@ framework changes.</p>
<h2>How does it work?</h2>
-<p><img src="/assets/img/documentation/networking-architecture.png" alt="Mesos Networking Architecture" /></p>
+<p><img src="images/networking-architecture.png" alt="Mesos Networking Architecture" /></p>
<p>A key observation is that the networking support is enabled via a Mesos module
and thus the Mesos master and agents are completely oblivious of it. It is
@@ -150,7 +150,7 @@ isolation</li>
<li><p>Cleanup Module:</p>
<ul>
-<li>responsible for doing a cleanup (releasing IPs, etc.) during a Agent lost
+<li>responsible for doing a cleanup (releasing IPs, etc.) during an Agent lost
event, dormant otherwise</li>
</ul>
</li>
@@ -231,7 +231,7 @@ before the address(es) are recycled.</p>
there is an example implementation at https://github.com/mesosphere/net-modules.</p>
<p>Once the network isolation module has been built into a shared dynamic library,
-we can load it into Mesos Agent (see <a href="/documentation/latest/modules/">modules documentation</a> on
+we can load it into Mesos Agent (/documentation/latest/see <a href="modules/">modules documentation</a> on
instructions for building and loading a module).</p>
<h2>Enabling frameworks for IP-per-container capability</h2>
Added: mesos/site/publish/documentation/newbie-guide/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/newbie-guide/index.html?rev=1727886&view=auto
==============================================================================
--- mesos/site/publish/documentation/newbie-guide/index.html (added)
+++ mesos/site/publish/documentation/newbie-guide/index.html Mon Feb 1 02:49:25 2016
@@ -0,0 +1,248 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title></title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
+ <link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml">
+
+ <link href="../../assets/css/main.css" media="screen" rel="stylesheet" type="text/css" />
+
+
+
+ <!-- Google Analytics Magic -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-20226872-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <!-- magical breadcrumbs -->
+ <div class="topnav">
+ <ul class="breadcrumb">
+ <li>
+ <div class="dropdown">
+ <a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a>
+ <ul class="dropdown-menu" role="menu" aria-labelledby="dLabel">
+ <li><a href="http://www.apache.org">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ </li>
+ <li><a href="http://mesos.apache.org">Apache Mesos</a></li>
+
+
+ <li><a href="/documentation
+/">Documentation
+</a></li>
+
+
+ </ul><!-- /breadcrumb -->
+ </div>
+
+ <!-- navbar excitement -->
+ <div class="navbar navbar-static-top" role="navigation">
+ <div class="navbar-inner">
+ <div class="container">
+ <a href="/" class="logo"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo" /></a>
+ <div class="nav-collapse">
+ <ul class="nav nav-pills navbar-right">
+ <li><a href="/gettingstarted/">Getting Started</a></li>
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Community</a></li>
+ </ul>
+ </div>
+ </div>
+ </div>
+ </div><!-- /.navbar -->
+
+ <div class="container">
+
+ <div class="row-fluid">
+ <div class="col-md-4">
+ <h4>If you're new to Mesos</h4>
+ <p>See the <a href="/gettingstarted/">getting started</a> page for more information about downloading, building, and deploying Mesos.</p>
+
+ <h4>If you'd like to get involved or you're looking for support</h4>
+ <p>See our <a href="/community/">community</a> page for more details.</p>
+ </div>
+ <div class="col-md-8">
+ <p>–</p>
+
+<h2>layout: documentation</h2>
+
+<h1>Newbie Guide</h1>
+
+<h2>Introduction</h2>
+
+<p>The purpose of this document is to give an introduction and overall description of the contribution process for contributors new to Mesos. This document is intended to extend the existing documentation with a focus on on-boarding contributors to the community as quickly as possible.</p>
+
+<h1>Contribution Process Overview</h1>
+
+<p>The figure below shows a high-level process flow of the contribution delivery process for the Mesos open source distribution. Details of the process can be obtained from the [submitting-a-patch] (/documentation/latest/submitting-a-patch/) document of the Mesos distribution along with the Getting Started Guidance section of this document.</p>
+
+<p><img src="NewbieContributionOverview.jpg" alt="Newbie Document" /></p>
+
+<h2>Preparation and Build Environment Setup</h2>
+
+<p>In order to complete the contribution process you will need to obtain account authorizations and install additional tooling to build the Mesos binaries. These pre-requisites are listed below:</p>
+
+<ul>
+<li><p>Required account authorizations</p>
+
+<ul>
+<li>Apache Review Board Account</li>
+<li>Apache JIRA Account</li>
+</ul>
+</li>
+<li><p>Required installations</p>
+
+<ul>
+<li>Git</li>
+<li>RBtools</li>
+<li>GTest</li>
+<li>GLog</li>
+</ul>
+</li>
+</ul>
+
+
+<h2>Download and Build Mesos</h2>
+
+<p>In order to begin contributing you will need to download and build the latest stable release of the Mesos distribution. Documentation on the steps to complete this process is described in the Mesos document and link listed above.</p>
+
+<h2>Make Modifications</h2>
+
+<p>Make your intended modifications to the source code and then commit the changes to your local build repository. Changes to the source should also include test cases to exercise the modification. Details about this process are found in the Mesos document listed above and Getting Started section below.</p>
+
+<h2>Submit Patch</h2>
+
+<p>Once changes are completed and tested submit the patch for review. This process will take the modifications from review submission to committed changes in the Mesos distribution.</p>
+
+<h1>Getting Started Guidance</h1>
+
+<p>The following information will help provide additional guidance to execute the steps summarized in Contribution Process Overview described above as well as provide helpful hints when following the <a href="/documentation/latest/submitting-a-patch/">submitting-a-patch</a> document of the Mesos distribution.</p>
+
+<h2>Apache JIRA Account Pre-Requisite</h2>
+
+<p>Detailed steps are provided in the Mesos documentation to obtain a JIRA account. Once you have obtained a JIRA account, ensure the additional step to request addition to the list of âcontributorsâ is completed.</p>
+
+<h2>Tooling Pre-Requisites</h2>
+
+<p>There is a set of tooling listed below that is required to be installed for the contribution process. These tools will not be described in this document and it is assumed that the reader will seek alternative references to become familiar with each of these tools outside of this document.</p>
+
+<ul>
+<li><a href="https://git-scm.com/">Git Client</a></li>
+<li><a href="https://www.reviewboard.org/docs/rbtools/dev/">RBTools</a></li>
+<li><a href="https://github.com/google/googletest">GTest</a></li>
+<li><a href="https://github.com/google/glog">GLog</a></li>
+</ul>
+
+
+<h2>Core Libraries</h2>
+
+<p>There are two core libraries in Mesos: <em>stout</em> and <em>libprocess</em>. <em>stout</em> is the low level operating system abstraction tooling that should be used in place of writing your own low level tools. <em>libprocess</em> is a library used to support building compatible concurrent components (see <a href="https://www.youtube.com/watch?v=5EIiU6R_6KE">video</a> for an introduction to the <em>libprocess</em> library). New contributors to Mesos should become familiar with these libraries and utilize them where appropriate. Additional documentation can be found in the following two README files: 3rdparty/libprocess/README.md and 3rdparty/libprocess/3rdparty/stout/README.md.</p>
+
+<h2>Download and Build Mesos</h2>
+
+<p>Detailed steps are provided in the <a href="/documentation/latest/getting-started/">Mesos documentation</a> to download and build Mesos. When downloading the Mesos distribution there are 2 options to obtain the Mesos source code. For contributing, ensure that the Git option is used.</p>
+
+<h2>JIRA</h2>
+
+<p><a href="http://issues.apache.org/jira/browse/MESOS">JIRA</a> is the issue tracking system for all bug fixes and feature additions in Mesos. When contributing to Mesos, all assignments, discussions, and resolution proposals for a specific issue should be documented as comments within the JIRA item.</p>
+
+<h2>Identifying a JIRA Issue</h2>
+
+<p>To begin the contribution process, identify a Mesos JIRA issue that is currently unassigned. It is highly recommended to start contributing to beginner level issues and overtime move to advanced level issues. The JIRA issue-tracking system can be filtered based on labels. The following labels can be used to identify beginner level JIRA tickets:</p>
+
+<ul>
+<li>newbie</li>
+<li>â*newbie++â</li>
+<li>beginner</li>
+<li>beginners</li>
+</ul>
+
+
+<p>When identifying a JIRA issue to work on, it is recommended to work on items that are relevant to the next release. Selecting work items important for the next release increases the priority for reviewers during the contribution process. See the tracking ticket for the release to figure out the high priority projects or ask the release manager to guide you.</p>
+
+<h2>Assign a JIRA Issue</h2>
+
+<p>There are a couple options to assign a work item: 1) create a new JIRA issue to work on or 2) identify an existing JIRA issue item to work on as described above. Whichever option is chosen there are several steps that should be followed when assigning yourself a JIRA ticket.</p>
+
+<p>It is important to identify a shepherd before you assign a ticket to yourself. Working with a shepherd will reveal its priority at the current time. To identify a shepherd look at the <a href="/documentation/latest/committers/">maintainers</a> file to get an idea who to ask to shepherd your JIRA issue.\</p>
+
+<h2>JIRA Issue Solution Proposals</h2>
+
+<p>Once you have an assigned JIRA issue and you have identified a shepherd, it is important to discuss your proposed solution within the JIRA ticket early in the resolution process in order to get feedback from reviewers. Early discussions will help:</p>
+
+<ol>
+<li><p>ensure the solution will be scoped in a consumable fashion;</p></li>
+<li><p>eliminate duplicate work with other contributions; and</p></li>
+<li><p>alert anyone interested in following the activity and progress of the ticket.</p></li>
+</ol>
+
+
+<p>Guidelines to consider when designing a solution can be found in the <a href="/documentation/latest/effective-code-reviewing/">effective-code-reviewing</a> document.</p>
+
+<h2>Making Changes</h2>
+
+<p>After completing the solution review make the source code changes, build and successfully run relevant test cases. Guidelines for these processes can be found in the <a href="/documentation/latest/submitting-a-patch.md">submitting-a-patch</a> and [mesos-c++style-guide] (c++-style-guide/).</p>
+
+<p>When creating and running test cases pay particular attention to race conditions. One recommendation is to run system tests multiple times. Below is a sample command to repeat the test 100 times.</p>
+
+<p><em>sudo GLOG_v=1 ./bin/mesos-tests.sh –verbose –gtest_filter=â*DOCKER*â –break-on-error –gtest_shuffle –gtest_repeat=100</em></p>
+
+<h2>Submit a Patch to the Review Board</h2>
+
+<p>Submit your patch for review after you have come to an agreement with your shepherd on the proposed solution, made the modifications and successfully ran the test cases. Make sure your shepherd is added as a “reviewer” (among others) in the review. This will ensure that your reviews get processed.</p>
+
+<p>When submitting a patch for review, include all testing done in the submission documentation. Follow the detailed steps found in the <a href="/documentation/latest/submitting-a-patch/">submitting-a-patch</a> document to submit a patch for review.</p>
+
+<h1>Advanced Level JIRA Items</h1>
+
+<p>As you gain experience contributing to Mesos you may want to tackle more advanced level JIRA items. These items may touch multiple components within Mesos and/or may have a significant impact on the developer or user community. In these cases, a working group of stakeholders is formed to develop a design document. The initial formation of this working group will be part of the community communication resources, e.g. the re-occurring developer sync meetings, the developer email list, the IRC channel, etc. For reference, a contributor new to an advanced level work item can refer to the work done for the <a href="https://issues.apache.org/jira/browse/MESOS-1592">inverse offer</a> project.</p>
+
+<hr />
+
+<h1>FAQs</h1>
+
+<p>Q: Where can I find documentation about Mesos?
+A: Mesos documentation is located in the âdocsâ directory of the Mesos distribution. This documentation includes information about Mesos architecture and design, running Mesos, developing within the Mesos ecosystem and contributing to Mesos. Additionally, the latest documentation can be found here:
+<a href="http://mesos.apache.org/documentation/latest/index.html">http://mesos.apache.org/documentation/latest/index.html</a></p>
+
+<p>Q: What is a Shepherd?
+A: An identified PMC/committer that works with a contributor to help shepherd a JIRA item to completion. Shepherds should be as identified at the beginning of the design/development phase.</p>
+
+ </div>
+</div>
+
+
+ <hr>
+
+ <!-- footer -->
+ <div class="footer">
+ <p>© 2012-2015 <a href="http://apache.org">The Apache Software Foundation</a>.
+ Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation.<p>
+ </div><!-- /footer -->
+
+ </div> <!-- /container -->
+
+ <!-- JS -->
+ <script src="//code.jquery.com/jquery-1.11.0.min.js" type="text/javascript"></script>
+ <script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js" type="text/javascript"></script>
+ </body>
+</html>
Modified: mesos/site/publish/documentation/operational-guide/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/operational-guide/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/operational-guide/index.html (original)
+++ mesos/site/publish/documentation/operational-guide/index.html Mon Feb 1 02:49:25 2016
@@ -85,7 +85,7 @@
<h2>Using a process supervisor</h2>
-<p>Mesos uses a “<a href="https://en.wikipedia.org/wiki/Fail-fast">fail-fast</a>” approach to error handling: if a serious error occurs, Mesos will typically exit rather than trying to continue running in a possibly erroneous state. For example, when Mesos is configured for <a href="/documentation/latest/high-availability">high availability</a>, the leading master will abort itself when it discovers it has been partitioned away from the Zookeeper quorum. This is a safety precaution to ensure the previous leader doesn’t continue communicating in an unsafe state.</p>
+<p>Mesos uses a “<a href="/documentation/latest/https://en.wikipedia.org/wiki/Fail-fast">fail-fast</a>” approach to error handling: if a serious error occurs, Mesos will typically exit rather than trying to continue running in a possibly erroneous state. For example, when Mesos is configured for <a href="high-availability/">high availability</a>, the leading master will abort itself when it discovers it has been partitioned away from the Zookeeper quorum. This is a safety precaution to ensure the previous leader doesn’t continue communicating in an unsafe state.</p>
<p>To ensure that such failures are handled appropriately, production deployments of Mesos typically use a <em>process supervisor</em> (such as systemd or supervisord) to detect when Mesos processes exit. The supervisor can be configured to restart the failed process automatically and/or to notify the cluster operator to investigate the situation.</p>
Modified: mesos/site/publish/documentation/oversubscription/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/oversubscription/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/oversubscription/index.html (original)
+++ mesos/site/publish/documentation/oversubscription/index.html Mon Feb 1 02:49:25 2016
@@ -97,7 +97,7 @@ components: a Resource Estimator and a Q
alongside extending the existing resource allocator, resource monitor, and
mesos slave. The new components and their interactions are illustrated below.</p>
-<p><img src="/assets/img/documentation/oversubscription-overview.jpg" alt="Oversubscription overview" /></p>
+<p><img src="images/oversubscription-overview.jpg" alt="Oversubscription overview" /></p>
<h3>Resource estimation</h3>
@@ -285,6 +285,16 @@ Controller with it’s own libproces
be made. Each corrective action contains information about executor or task and
the type of action to perform.</p>
+<p>Mesos comes with a <code>noop</code> and a <code>load</code> qos controller. The <code>noop</code> controller
+does not provide any corrections, thus does not assure any quality of service
+for regular tasks. The <code>load</code> controller is ensuring the total system load
+doesn’t exceed a configurable thresholds and as a result try to avoid the cpu
+congestion on the node. If the load is above the thresholds controller evicts
+all the revocable executors. These thresholds are configurable via two module
+parameters <code>load_threshold_5min</code> and <code>load_threshold_15min</code>. They represent
+standard unix load averages in the system. 1 minute system load is ignored,
+since for oversubscription use case it can be a misleading signal.</p>
+
<pre><code class="{.proto}">message QoSCorrection {
enum Type {
KILL = 1; // Terminate an executor.
@@ -380,6 +390,37 @@ between these corrections is controlled
<p>In the example above, a fixed amount of 14 cpus will be offered as revocable
resources.</p>
+<p>The <code>load</code> qos controller is enabled as follows:</p>
+
+<pre><code>--qos_controller="org_apache_mesos_LoadQoSController"
+
+--qos_correction_interval_min="20secs"
+
+--modules='{
+ "libraries": {
+ "file": "/usr/local/lib64/libload_qos_controller.so",
+ "modules": {
+ "name": "org_apache_mesos_LoadQoSController",
+ "parameters": [
+ {
+ "key": "load_threshold_5min",
+ "value": "6"
+ },
+ {
+ "key": "load_threshold_15min",
+ "value": "4"
+ }
+ ]
+ }
+ }
+}'
+</code></pre>
+
+<p>In the example above, when standard unix system load average for 5 minutes will
+be above 6, or for 15 minutes will be above 4 then slave will evict all the
+<code>revocable</code> executors. <code>LoadQoSController</code> will be effectively run every 20
+seconds.</p>
+
<p>To install a custom resource estimator and QoS controller, please refer to the
<a href="/documentation/latest/modules/">modules documentation</a>.</p>
Modified: mesos/site/publish/documentation/persistent-volume/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/persistent-volume/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/persistent-volume/index.html (original)
+++ mesos/site/publish/documentation/persistent-volume/index.html Mon Feb 1 02:49:25 2016
@@ -96,35 +96,40 @@ that is mounted at a well-known location
<p>Persistent volumes can only be created from <strong>reserved</strong> disk resources, whether
it be statically reserved or dynamically reserved. A dynamically reserved
-persistent volume also cannot be unreserved without having explicitly destroyed
-the volume. These rules exist to limit the accidental mistakes such as:
-a persistent volume containing sensitive data being offered to other frameworks
-in the cluster.</p>
+persistent volume also cannot be unreserved without first explicitly destroying
+the volume. These rules exist to limit accidental mistakes, such as a persistent
+volume containing sensitive data being offered to other frameworks in the
+cluster.</p>
<p>Please refer to the <a href="/documentation/latest/reservation/">Reservation</a> documentation for details
regarding reservation mechanisms available in Mesos.</p>
<p>Persistent volumes can be created by <strong>operators</strong> and authorized
<strong>frameworks</strong>. We require a <code>principal</code> from the operator or framework in order
-to authenticate/authorize the operations. <a href="/documentation/latest/authorization/">Authorization</a> is
-specified via the existing ACL mechanism. (<strong><em>Coming Soon</em></strong>)</p>
+to authenticate/authorize the operations. Permissions are specified via the
+existing ACL mechanism. To use authorization with reserve, unreserve, create,
+and destroy operations, the Mesos master must be configured with the desired
+ACLs. For more information, see the
+<a href="/documentation/latest/authorization/">authorization documentation</a>.</p>
<ul>
<li><code>Offer::Operation::Create</code> and <code>Offer::Operation::Destroy</code> messages are
available for <strong>frameworks</strong> to send back via the <code>acceptOffers</code> API as a
response to a resource offer.</li>
-<li><code>/create</code> and <code>/destroy</code> HTTP endpoints are available for <strong>operators</strong>
-to manage persistent volumes through the master. (<strong><em>Coming Soon</em></strong>).</li>
+<li><code>/create-volumes</code> and <code>/destroy-volumes</code> HTTP endpoints allow
+<strong>operators</strong> to manage persistent volumes through the master.</li>
</ul>
<p>In the following sections, we will walk through examples of each of the
interfaces described above.</p>
-<h2><code>Offer::Operation::Create</code></h2>
+<h3>Framework Scheduler API</h3>
-<p>A framework is able to create volumes through the resource offer cycle.
-Suppose we receive a resource offer with 2048 MB of dynamically reserved disk.</p>
+<h4><code>Offer::Operation::Create</code></h4>
+
+<p>A framework can create volumes through the resource offer cycle. Suppose we
+receive a resource offer with 2048 MB of dynamically reserved disk.</p>
<pre><code>{
"id" : <offer_id>,
@@ -146,12 +151,12 @@ Suppose we receive a resource offer with
</code></pre>
<p>We can create a persistent volume from the 2048 MB of disk resources by sending
-the following <code>Offer::Operation</code> message via the <code>acceptOffers</code> API.
-<code>Offer::Operation::Create</code> has a <code>volumes</code> field which we specify with the
-persistent volume information. We need to specify the following:</p>
+an <code>Offer::Operation</code> message via the <code>acceptOffers</code> API.
+<code>Offer::Operation::Create</code> has a <code>volumes</code> field which specifies the persistent
+volume information. We need to specify the following:</p>
<ol>
-<li>ID of the persistent volume which needs to be unique per role on each slave.</li>
+<li>The ID for the persistent volume; this must be unique per role on each slave.</li>
<li>The non-nested relative path within the container to mount the volume.</li>
<li>The permissions for the volume. Currently, <code>"RW"</code> is the only possible value.</li>
</ol>
@@ -184,7 +189,8 @@ persistent volume information. We need t
}
</code></pre>
-<p>The subsequent resource offer will <strong>contain</strong> the following persistent volume:</p>
+<p>If this succeeds, a subsequent resource offer will contain the following
+persistent volume:</p>
<pre><code>{
"id" : <offer_id>,
@@ -214,12 +220,12 @@ persistent volume information. We need t
}
</code></pre>
-<h2><code>Offer::Operation::Destroy</code></h2>
+<h4><code>Offer::Operation::Destroy</code></h4>
-<p>A framework is able to destroy persistent volumes through the resource offer
-cycle. In <a href="#offeroperationcreate">Offer::Operation::Create</a>, we created a
-persistent volume from 2048 MB of disk resources. Mesos will not garbage-collect
-this volume until we explicitly destroy it. Suppose we would like to destroy the
+<p>A framework can destroy persistent volumes through the resource offer cycle. In
+<a href="#offeroperationcreate">Offer::Operation::Create</a>, we created a persistent
+volume from 2048 MB of disk resources. Mesos will not garbage-collect this
+volume until we explicitly destroy it. Suppose we would like to destroy the
volume we created. First, we receive a resource offer (copy/pasted from above):</p>
<pre><code>{
@@ -252,7 +258,7 @@ volume we created. First, we receive a r
<p>We destroy the persistent volume by sending the <code>Offer::Operation</code> message via
the <code>acceptOffers</code> API. <code>Offer::Operation::Destroy</code> has a <code>volumes</code> field which
-we specify the persistent volumes to be destroyed.</p>
+specifies the persistent volumes to be destroyed.</p>
<pre><code>{
"type" : Offer::Operation::DESTROY,
@@ -281,9 +287,9 @@ we specify the persistent volumes to be
}
</code></pre>
-<p>The persistent volume will be destroyed, but the disk resources will still be
-reserved. As such, the subsequent resource offer will <strong>contain</strong> the following
-reserved disk resources:</p>
+<p>If this request succeeds, the persistent volume will be destroyed but the disk
+resources will still be reserved. As such, a subsequent resource offer will
+contain the following reserved disk resources:</p>
<pre><code>{
"id" : <offer_id>,
@@ -304,13 +310,196 @@ reserved disk resources:</p>
}
</code></pre>
-<p>Note that in 0.23, even after you destroy the persistent volume, its content
-will still be on the disk. The garbage collection for persistent volumes is
-coming soon: <a href="https://issues.apache.org/jira/browse/MESOS-2408">MESOS-2048</a>.</p>
+<p>Those reserved resources can then be used as normal: e.g., they can be used to
+create another persistent volume or can be unreserved.</p>
+
+<p>Garbage collection for persistent volumes is planned but has not been
+implemented yet – <a href="https://issues.apache.org/jira/browse/MESOS-2408">MESOS-2408</a>.
+In the mean time, even after you destroy a persistent volume, its content will
+remain on disk.</p>
+
+<h3>Operator HTTP Endpoints</h3>
+
+<p>As described above, persistent volumes can be created by a framework scheduler
+as part of the resource offer cycle. Persistent volumes can also be created and
+destroyed by sending HTTP requests to the <code>/create-volumes</code> and
+<code>/destroy-volumes</code> endpoints, respectively. This capability is intended for use
+by operators and administrative tools.</p>
+
+<h4><code>/create-volumes</code></h4>
+
+<p>To use this endpoint, the operator should first ensure that a reservation for
+the necessary resources has been made on the appropriate slave (e.g., by using
+the <code>/reserve</code> HTTP endpoint or by configuring a static reservation).</p>
+
+<p>To create a 512MB persistent volume for the <code>ads</code> role on a dynamically reserved
+disk resource, we can send a request like so:</p>
+
+<pre><code>curl -i \
+ -u <operator_principal>:<password> \
+ -d slaveId=<slave_id> \
+ -d volumes='[
+ {
+ "name": "disk",
+ "type": "SCALAR",
+ "scalar": { "value": 512 },
+ "role": "ads",
+ "reservation": {
+ "principal": <operator_principal>
+ },
+ "disk": {
+ "persistence": {
+ "id" : <persistence_id>
+ },
+ "volume": {
+ "mode": "RW",
+ "container_path": <path>
+ }
+ }
+ }
+ ]' \
+ -X POST http://<ip>:<port>/master/create-volumes
+</code></pre>
+
+<p>The user receives one of the following HTTP responses:</p>
+
+<ul>
+<li><code>200 OK</code>: Success (the persistent volumes have been created).</li>
+<li><code>400 BadRequest</code>: Invalid arguments (e.g., missing parameters).</li>
+<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
+<li><code>403 Forbidden</code>: Unauthorized request.</li>
+<li><code>409 Conflict</code>: Insufficient resources to create the volumes.</li>
+</ul>
+
+
+<p>Note that a single <code>/create-volumes</code> request can create multiple persistent
+volumes, but all of the volumes must be on the same slave.</p>
+
+<h4><code>/destroy-volumes</code></h4>
+
+<p>To destroy the volume created above, we can send an HTTP POST like so:</p>
+
+<pre><code>curl -i \
+ -u <operator_principal>:<password> \
+ -d slaveId=<slave_id> \
+ -d volumes='[
+ {
+ "name": "disk",
+ "type": "SCALAR",
+ "scalar": { "value": 512 },
+ "role": "ads",
+ "reservation": {
+ "principal": <operator_principal>
+ },
+ "disk": {
+ "persistence": {
+ "id" : <persistence_id>
+ },
+ "volume": {
+ "mode": "RW",
+ "container_path": <path>
+ }
+ }
+ }
+ ]' \
+ -X POST http://<ip>:<port>/master/destroy-volumes
+</code></pre>
+
+<p>The user receives one of the following HTTP responses:</p>
+
+<ul>
+<li><code>200 OK</code>: Success (the volumes have been destroyed).</li>
+<li><code>400 BadRequest</code>: Invalid arguments (e.g., missing parameters).</li>
+<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
+<li><code>403 Forbidden</code>: Unauthorized request.</li>
+<li><code>409 Conflict</code>: Insufficient resources to destroy the volumes.</li>
+</ul>
+
+
+<p>Note that a single <code>/destroy-volumes</code> request can destroy multiple persistent
+volumes, but all of the volumes must be on the same slave.</p>
+
+<h3>Programming with Persistent Volumes</h3>
-<h3><code>/create</code> (<em>Coming Soon</em>)</h3>
+<p>Some suggestions to keep in mind when building applications that use persistent
+volumes:</p>
+
+<ul>
+<li><p>A single <code>acceptOffers</code> call can be used to both create a new dynamic
+reservation (via <code>Offer::Operation::Reserve</code>) and create a new persistent
+volume on those newly reserved resources (via <code>Offer::Operation::Create</code>).</p></li>
+<li><p>Attempts to dynamically reserve resources or create persistent volumes might
+fail—for example, because the network message containing the operation did
+not reach the master or because the master rejected the operation.
+Applications should be prepared to detect failures and correct for them (e.g.,
+by retrying the operation).</p></li>
+<li><p>When using HTTP endpoints to reserve resources or create persistent volumes,
+<em>some</em> failures can be detected by examining the HTTP response code returned
+to the client. However, it is still possible for a <code>200</code> response code to be
+returned to the client but for the associated operation to fail.</p></li>
+<li><p>When using the scheduler API, detecting that a dynamic reservation has failed
+is a little tricky: reservations do not have unique identifiers, and the Mesos
+master does not provide explicit feedback on whether a reservation request has
+succeeded or failed. Hence, framework schedulers typically use a combination
+of two techniques:</p>
+
+<ol>
+<li><p>They use timeouts to detect that a reservation request may have failed
+(because they don’t receive a resource offer containing the expected
+resources after a given period of time).</p></li>
+<li><p>To check whether a resource offer includes the effect of a dynamic
+reservation, applications <em>cannot</em> check for the presence of a “reservation
+ID” or similar value (because reservations do not have IDs). Instead,
+applications should examine the resource offer and check it contains
+sufficient reserved resources for the application’s role. If it does not,
+the application should make additional reservation requests as necessary.</p></li>
+</ol>
+</li>
+<li><p>When a scheduler issues a dynamic reservation request, the reserved resources
+might <em>not</em> be present in the next resource offer the scheduler receives.
+There are two reasons for this: first, the reservation request might fail or
+be dropped by the network, as discussed above. Second, the reservation request
+might simply be delayed, so that the next resource offer from the master will
+be issued before the reservation request is received by the master. This is
+why the text above suggests that applications wait for a timeout before
+assuming that a reservation request should be retried.</p></li>
+<li><p>A consequence of using timeouts to detect failures is that an application
+might submit more reservation requests than intended (e.g., a timeout fires
+and an application makes another reservation request; meanwhile, the original
+reservation request is also processed). Recall that two reservations for the
+same role at the same agent are “merged”: for example, role <code>foo</code> makes two
+requests to reserve 2 CPUs at a single agent and both reservation requests
+succeed, the result will be a single reservation of 4 CPUs. To handle this
+situation, applications should be prepared for resource offers that contain
+more resources than expected. Some applications may also want to detect this
+situation and unreserve an additional reserved resources that will not be
+required.</p></li>
+<li><p>It often makes sense to structure application logic as a “state machine”,
+where the application moves from its initial state (no reserved resources and
+no persistent volumes) and eventually transitions toward a single terminal
+state (necessary resources reserved and persistent volume created). As new
+events (such as timeouts and resource offers) are received, the application
+compares the event with its current state and decides what action to take
+next.</p></li>
+<li><p>Because persistent volumes are associated with roles, a volume might be
+offered to <em>any</em> of the frameworks that are registered in that role. For
+example, a persistent volume might be created by one framework and then
+offered to a different framework in the same role. This can be used to pass
+large volumes of data between frameworks in a convenient way. However, this
+behavior might also allow sensitive data created by one framework to be read
+or modified by another framework in the same role. It can also make it more
+difficult for frameworks to determine whether a dynamic reservation has
+succeeded: as discussed above, frameworks need to wait for an offer that
+contains the “expected” reserved resources to determine when a reservation
+request has succeeded. Determining what a framework should “expect” to find in
+an offer is more difficult when multiple frameworks can be making reservations
+for the same role concurrently. In general, whenever multiple frameworks are
+allowed to register in the same role, the operator should ensure that those
+frameworks are configured to collaborate with one another when using
+role-specific resources. For more information, see the discussion of
+<a href="roles.md#roles-multiple-frameworks">multiple frameworks in the same role</a>.</p></li>
+</ul>
-<h3><code>/destroy</code> (<em>Coming Soon</em>)</h3>
</div>
</div>
Modified: mesos/site/publish/documentation/powered-by-mesos/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/powered-by-mesos/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/powered-by-mesos/index.html (original)
+++ mesos/site/publish/documentation/powered-by-mesos/index.html Mon Feb 1 02:49:25 2016
@@ -94,6 +94,7 @@
<li><a href="http://www.atigeo.com">Atigeo</a></li>
<li><a href="http://www.atlassian.com">Atlassian</a></li>
<li><a href="http://www.auchan.fr">Auchan Ecommerce France</a></li>
+<li><a href="http://www.blue-yonder.com">Blue Yonder</a></li>
<li><a href="http://www.bol.com">Bol.com</a></li>
<li><a href="http://www.brandingbrand.com">Branding Brand</a></li>
<li><a href="http://www.brigade.com">Brigade</a></li>
@@ -113,8 +114,10 @@
<li><a href="http://www.ebay.com">eBay</a></li>
<li><a href="http://www.ericsson.com/">Ericsson</a></li>
<li><a href="http://www.foursquare.com">Foursquare</a></li>
+<li><a href="http://www.genouest.org">GenOuest</a></li>
<li><a href="http://www.gocardless.com">GoCardless</a></li>
<li><a href="http://www.groupon.com">Groupon</a></li>
+<li><a href="http://www.gsshop.com">GSShop</a></li>
<li><a href="http://www.gutefrage.net">Gutefrage.net</a></li>
<li><a href="https://hootsuite.com">Hootsuite</a></li>
<li><a href="http://www.hubspot.com">HubSpot</a></li>
@@ -134,6 +137,7 @@
<li><a href="http://www.motus.com">Motus</a></li>
<li><a href="http://www.netflix.com">Netflix</a></li>
<li><a href="http://www.oakmorelabs.com">Oakmore Labs</a></li>
+<li><a href="http://www.orbitz.com/">Orbitz Worldwide</a></li>
<li><a href="http://www.opencredo.com/">OpenCredo</a></li>
<li><a href="http://www.opentable.com">OpenTable</a></li>
<li><a href="https://www.hioscar.com/">Oscar Insurance</a></li>
@@ -141,6 +145,7 @@
<li><a href="http://www.pinkbike.com">Pinkbike</a></li>
<li><a href="http://www.profitstars.com">ProfitStars</a></li>
<li><a href="http://www.qubitproducts.com">Qubit</a></li>
+<li><a href="http://www.qunar.com">Qunar</a></li>
<li><a href="http://www.relateiq.com">RelateIQ</a></li>
<li><a href="http://revise.ly">Revisely</a></li>
<li><a href="http://www.sabre.com">Sabre Labs</a></li>
Added: mesos/site/publish/documentation/quota/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/quota/index.html?rev=1727886&view=auto
==============================================================================
--- mesos/site/publish/documentation/quota/index.html (added)
+++ mesos/site/publish/documentation/quota/index.html Mon Feb 1 02:49:25 2016
@@ -0,0 +1,466 @@
+<!DOCTYPE html>
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title></title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+
+ <link href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
+ <link rel="alternate" type="application/atom+xml" title="Apache Mesos Blog" href="/blog/feed.xml">
+
+ <link href="../../assets/css/main.css" media="screen" rel="stylesheet" type="text/css" />
+
+
+
+ <!-- Google Analytics Magic -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-20226872-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <!-- magical breadcrumbs -->
+ <div class="topnav">
+ <ul class="breadcrumb">
+ <li>
+ <div class="dropdown">
+ <a data-toggle="dropdown" href="#">Apache Software Foundation <span class="caret"></span></a>
+ <ul class="dropdown-menu" role="menu" aria-labelledby="dLabel">
+ <li><a href="http://www.apache.org">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ </li>
+ <li><a href="http://mesos.apache.org">Apache Mesos</a></li>
+
+
+ <li><a href="/documentation
+/">Documentation
+</a></li>
+
+
+ </ul><!-- /breadcrumb -->
+ </div>
+
+ <!-- navbar excitement -->
+ <div class="navbar navbar-static-top" role="navigation">
+ <div class="navbar-inner">
+ <div class="container">
+ <a href="/" class="logo"><img src="/assets/img/mesos_logo.png" alt="Apache Mesos logo" /></a>
+ <div class="nav-collapse">
+ <ul class="nav nav-pills navbar-right">
+ <li><a href="/gettingstarted/">Getting Started</a></li>
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Community</a></li>
+ </ul>
+ </div>
+ </div>
+ </div>
+ </div><!-- /.navbar -->
+
+ <div class="container">
+
+ <div class="row-fluid">
+ <div class="col-md-4">
+ <h4>If you're new to Mesos</h4>
+ <p>See the <a href="/gettingstarted/">getting started</a> page for more information about downloading, building, and deploying Mesos.</p>
+
+ <h4>If you'd like to get involved or you're looking for support</h4>
+ <p>See our <a href="/community/">community</a> page for more details.</p>
+ </div>
+ <div class="col-md-8">
+ <h1>Quota</h1>
+
+<p>A problem when running multiple frameworks on Mesos is that the default
+<a href="https://www.cs.berkeley.edu/~alig/papers/drf.pdf">wDRF allocator</a> may offer
+resources to frameworks even though they are beyond their fair share (e.g., when
+no other framework currently accepts these resources). There is no mechanism to
+lay away resources for future consumption in an entire cluster. Even though
+<a href="/documentation/latest/reservation/">dynamic reservations</a> allow both operators and frameworks
+to dynamically reserve resources on particular Mesos agents, it does not solve
+the above problem because any individual agents might fail.</p>
+
+<p>Since version 0.27.0 Mesos provides a mechanism to reserve a certain amount of
+resources in an entire cluster, i.e. not tied to a particular agent. <em>Quota</em>
+allows operators to lay away resources for a given <a href="/documentation/latest/roles/">role</a>. These
+resources cannot be hijacked by other roles and are guaranteed to be available
+for the role if there are enough resources in the cluster. Quota can be viewed
+as a cluster-wide dynamic reservation available for operators only.</p>
+
+<h1>Terminology</h1>
+
+<p>For the purpose of this document, an âOperatorâ is a person, tool, or script
+that manages the Mesos cluster.</p>
+
+<p>In computer science, a âquotaâ usually refers to one of the following:
+* A minimal guarantee.
+* A maximal limit.
+* A pair of both.</p>
+
+<p>In Mesos we understand quota as a <strong>guaranteed</strong> resource allocation that a role
+may rely on; in other words, a minimum share a role is entitled to receive.</p>
+
+<h1>Motivation and Limitations</h1>
+
+<p>Consider the following scenarios in a Mesos cluster to better understand what
+use-cases are supported by quota (and what are not).</p>
+
+<h2>Scenario 1: Greedy Framework</h2>
+
+<p>There are two frameworks in a cluster, each running in a separate role with
+equal weights: framework fA in role rA and framework fB in role rB. There is a
+single resource available in the cluster: 100 CPUs. fA consumes 10 CPUs and is
+idle (declines resource offers), while fB is greedy and accepts all offers it gets,
+hogging the remaining 90 CPUs. Without quota, though fAâs fair share is 50 CPUs
+it will not be able to make use of additional the 40 CPUs until some of fBâs
+tasks terminate.</p>
+
+<h2>Scenario 2: Resources for a new Framework</h2>
+
+<p>A greedy framework fB in role rB is currently the only framework in the cluster
+and it uses all available resources—100 CPUs. If a new framework fA in role rA
+joins the cluster, it will not receive its fair share of the cluster resources
+(50 CPUs) until some of fBâs tasks terminate.</p>
+
+<p>To deal with Scenario 2, quota by itself is not a sufficient solution as it
+would be set after fB has started using all resources. Instead Scenario 2
+requires either always keeping a pool of resources which are not offered or
+introducing preemption for running tasks.</p>
+
+<h1>Operator HTTP Endpoint</h1>
+
+<p>The master <code>/quota</code> HTTP endpoint enables operators to configure quotas. The
+endpoint currently offers a REST-like interface and supports the
+following operations:</p>
+
+<ul>
+<li><a href="#setRequest">Setting</a> a new quota with POST.</li>
+<li><a href="#removeRequest">Removing</a> an existing quota with DELETE.</li>
+<li><a href="#statusRequest">Querying</a> the currently set quota with GET.</li>
+</ul>
+
+
+<p>Currently it is not possible to update previously configured quotas. This means
+in order to update a quota for a given role, the operator has to remove the
+existing quota and then set a new one.</p>
+
+<p>The endpoint can optionally use authentication and authorization. See the
+<a href="/documentation/latest/authentication/">authentication guide</a> for details.</p>
+
+<p><a name="setRequest"></a></p>
+
+<h2>Set</h2>
+
+<p>The operator can set a new quota by sending an HTTP POST request to the <code>/quota</code>
+endpoint.</p>
+
+<p>An example request to the quota endpoint could look like this (using the JSON
+definitions below):</p>
+
+<pre><code>$ curl -d jsonMessageBody -X POST http://<master-ip>:<port>/quota
+</code></pre>
+
+<p>For example to set a quota of 12 CPUs and 6144 MB of RAM for <code>role1</code> the operator
+can use the following <code>jsonMessageBody</code>:</p>
+
+<pre><code> {
+ "role": "role1",
+ "guarantee": [
+ {
+ "name": "cpus",
+ "type": "SCALAR",
+ "scalar": { "value": 12 }
+ },
+ {
+ "name": "mem",
+ "type": "SCALAR",
+ "scalar": { "value": 6144 }
+ }
+ ]
+ }
+</code></pre>
+
+<p>A set request is only valid for roles for which no quota is currently set.
+However if the master is configured without an explicit
+<a href="/documentation/latest/roles/">role whitelist</a> a set request can introduce new roles.</p>
+
+<p>In order to bypass the <a href="#capacityHeuristic">capacity heuristic</a> check the
+operator should set an optional <code>force</code> field:</p>
+
+<pre><code> {
+ "force": true,
+ "role": "role1",
+ ...
+ }
+</code></pre>
+
+<p>The operator will receive one of the following HTTP response codes:</p>
+
+<ul>
+<li><code>200 OK</code>: Success (the set request was successful).</li>
+<li><code>400 BadRequest</code>: Invalid arguments (e.g., quota for role exists, invalid
+JSON, non-scalar resources included).</li>
+<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
+<li><code>403 Forbidden</code>: Unauthorized request.</li>
+<li><code>409 Conflict</code>: The capacity heuristic check failed due to insufficient
+resources.</li>
+</ul>
+
+
+<p><a name="removeRequest"></a></p>
+
+<h2>Remove</h2>
+
+<p>The operator can remove a previously set quota by sending an HTTP DELETE request
+to the <code>/quota/<role></code> endpoint. For example the following command removes
+a previously set quota for <code>role1</code>:</p>
+
+<pre><code>$ curl -X DELETE http://<master-ip>:<port>/quota/role1
+</code></pre>
+
+<p>The operator will receive one of the following HTTP response codes:</p>
+
+<ul>
+<li><code>200 OK</code>: Success (the remove request was successful).</li>
+<li><code>400 BadRequest</code>: Invalid arguments (e.g., removing a quota for a role which
+does not have any quota set).</li>
+<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
+<li><code>403 Forbidden</code>: Unauthorized request.</li>
+</ul>
+
+
+<p><a name="statusRequest"></a></p>
+
+<h2>Status</h2>
+
+<p>The operator can query the configured quotas by sending a HTTP GET request
+to the <code>/quota</code> endpoint.</p>
+
+<pre><code>$ curl -X GET http://<master-ip>:<port>/quota
+</code></pre>
+
+<p>The response message body includes a JSON representation of the current quota
+status, for example:</p>
+
+<pre><code> {
+ "infos": [
+ {
+ "role": "role1",
+ "guarantee": [
+ {
+ "name": "cpus",
+ "role": "*",
+ "type": "SCALAR",
+ "scalar": { "value": 12 }
+ },
+ {
+ "name": "mem",
+ "role": "*",
+ "type": "SCALAR",
+ "scalar": { "value": 6144 }
+ }
+ ]
+ }
+ ]
+ }
+</code></pre>
+
+<p>The operator will receive one of the following HTTP response codes:</p>
+
+<ul>
+<li><code>200 OK</code>: Success.</li>
+<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
+</ul>
+
+
+<h1>How does it work?</h1>
+
+<p>There are several stages in the lifetime of a quota issued by operator. First
+the <a href="#requestProcessing">quota set request is handled by the master</a>, after that
+the <a href="#allocatorEnforcement">allocator enforces the quota</a>.
+Quotas can be <a href="#removeProcessing">removed</a> by the operator.</p>
+
+<p>It is important to understand that the enforcement of quota depends on
+the allocator being used. A custom allocator could choose to handle quota in its
+own way or even to ignore quota.
+<strong>The following section assumes the default
+<a href="https://www.cs.berkeley.edu/~alig/papers/drf.pdf">wDRF</a> allocator is used.</strong></p>
+
+<p>Be aware that setting quota may affect other frameworks in the cluster,
+because resources will be laid away and not offered to other frameworks.
+Also note, that quota is only applicable for scalar resources (e.g., it is not
+possible to set quota for port resources).</p>
+
+<p><a name="requestProcessing"></a></p>
+
+<h2>Quota Request Processing</h2>
+
+<p>When an operator submits a quota set request via the master <code>/quota</code> HTTP
+endpoint, the following steps are triggered:</p>
+
+<ol>
+<li><a href="/documentation/latest/authentication/">Authenticate</a> the HTTP request.</li>
+<li>Parse and validate the request.
+See <a href="#setRequest">description of possible error codes</a>.</li>
+<li><a href="/documentation/latest/authentication/">Authorize</a> the HTTP request if authorization is enabled.</li>
+<li>Run the <a href="#capacityHeuristic">capacity heuristic</a> if not disabled by
+<a href="#setRequest">the <code>force</code> flag</a>.</li>
+<li>Reliably store quota. See <a href="#failover">details on failover recovery</a>.</li>
+<li><a href="#rescindOffers">Rescind outstanding offers</a>.</li>
+</ol>
+
+
+<p><a name="removeProcessing"></a>
+The quota remove request processing is simpler and triggers the following steps:</p>
+
+<ol>
+<li><a href="/documentation/latest/authentication/">Authenticate</a> the HTTP request.</li>
+<li>Validate the request.
+See <a href="#removeRequest">description of potential error codes</a>.</li>
+<li><a href="/documentation/latest/authentication/">Authorize</a> the HTTP request if authorization is enabled.</li>
+<li>Reliably remove quota.</li>
+</ol>
+
+
+<p><a name="capacityHeuristic"></a></p>
+
+<h3>Capacity Heuristic Check</h3>
+
+<p>Misconfigured quota can render a cluster into a state where no offers are made
+to any frameworks. For example imagine an operator setting quota 1000 CPUs
+for a role <code>prosuction</code> (note the typo) in a cluster with a total capacity of
+100 CPUs. In that case after the quota is accepted by the master, no offers will
+ be made to any framework in any actual role (including <code>production</code>).</p>
+
+<p>In order to prevent such extreme situations, the Mesos Master employs a capacity
+heuristic check that ensures a quota set request can reasonably be satisfied
+given the total cluster capacity. This heuristic tests whether the total quota,
+including the new request, does not exceed the sum of total
+non-statically-reserved cluster resources, i.e. the following inequality holds:</p>
+
+<pre><code>total resources - statically reserved >= total quota + quota request
+</code></pre>
+
+<p>Please be advised that even if there are enough resources at the moment of
+this check, agents may terminate at any time, rendering the cluster incapable
+of satisfying the configured quotas.</p>
+
+<p>A <a href="#setRequest"><code>force</code> flag</a> can be set to bypass this check. For example, this
+flag can be useful when the operator would like to configure a quota that
+exceeds the current cluster capacity, but they know that additional cluster
+resources will be added shortly.</p>
+
+<p><a name="rescindOffers"></a></p>
+
+<h3>Rescinding Outstanding Offers</h3>
+
+<p>When setting a new quota, the master rescinds outstanding offers. This avoids
+situations where the quota request cannot be satisfied by the remaining
+unoffered resources, but there are enough resources tied up in outstanding
+offers to frameworks that have not accepted them yet. Hence, we rescind
+outstanding offers with the following rules:</p>
+
+<ul>
+<li>Rescind at least as many resources as there are in the quota request.</li>
+<li>If at least one offer is to be rescinded from an agent, all offers from this
+agent are rescinded. This is done in order to make the potential offer bigger,
+which increases the chances that a quota'ed framework will be able to use the
+offer.</li>
+<li>Rescind offers from at least as many agents as there are frameworks in the
+role for which quota is being set. This enables (but does not guarantee, due
+to fair sharing) each framework in the role to receive an offer.</li>
+</ul>
+
+
+<p><a name="allocatorEnforcement"></a></p>
+
+<h2>Enforcement by wDRF Allocator</h2>
+
+<p>The wDRF allocator first allocates (or lays away if offers are declined)
+resources to framework in roles with quota set. Once all quotas are
+satisfied, it proceeds with the standard wDRF for all frameworks.</p>
+
+<p><strong>NOTE:</strong> A quota'ed role may not be allocated any unreserved non-revocable
+ resources beyond its quota guarantee. If frameworks in the quota'ed
+ role have not opted for revocable resources, they may stop getting
+ offers once quota for the role is satisfied. In this case setting
+ quota to any value that is less than the role’s fair share may reduce
+ the amount of resources offered to this role.</p>
+
+<p><strong>NOTE:</strong> In Mesos 0.27 quota guarantee also serves as quota limit, i.e. once
+ quota for the role is satisfied, frameworks in this role will not
+ be offered any resources except those reserved for the role. This
+ behavior aims to mitigate the absence of quota limit and will be
+ changed in future releases.</p>
+
+<p>If there are multiple frameworks in a role with quota set, the standard wDRF
+algorithm determines framework priority inside this role.</p>
+
+<p>The default wDRF allocator considers only non-revocable resources as applicable
+towards quota.</p>
+
+<p><a name="failover"></a></p>
+
+<h2>Failover</h2>
+
+<p>If there is at least one role with quota set, the master failover recovery
+changes significantly. The reason for this is that during the recovery there is
+a period of time when not all agents have registered with the Master. Therefore
+it is impossible to reason about whether all quotas are satisfied.</p>
+
+<p>To address this issue, if upon recovery any previously set quota are detected,
+the allocator enters recovery mode, during which the allocator
+<em>does not issue offers</em>. The recovery mode—and therefore offer
+suspension—ends when either:</p>
+
+<ul>
+<li>A certain amount of agents reregister (by default 80% of agents known before
+the failover), or</li>
+<li>a timeout expires (by default 10 minutes).</li>
+</ul>
+
+
+<h1>Current Limitations</h1>
+
+<ul>
+<li>The quota set request does not allow specifying the granularity of the
+requested resources (e.g. 10 CPUs on a single node).</li>
+<li>The quota set request does not allow to specify constraints (e.g. 2*5 cpus on
+disjoint nodes for an HA like setup).</li>
+<li>Quota is not allowed for the default role â*â (see MESOS-3938).</li>
+<li>Currently it is not possible to update previously configured quotas. See
+<a href="#setRequest">quota set request</a> for details.</li>
+</ul>
+
+
+ </div>
+</div>
+
+
+ <hr>
+
+ <!-- footer -->
+ <div class="footer">
+ <p>© 2012-2015 <a href="http://apache.org">The Apache Software Foundation</a>.
+ Apache Mesos, the Apache feather logo, and the Apache Mesos project logo are trademarks of The Apache Software Foundation.<p>
+ </div><!-- /footer -->
+
+ </div> <!-- /container -->
+
+ <!-- JS -->
+ <script src="//code.jquery.com/jquery-1.11.0.min.js" type="text/javascript"></script>
+ <script src="//netdna.bootstrapcdn.com/bootstrap/3.1.1/js/bootstrap.min.js" type="text/javascript"></script>
+ </body>
+</html>
Modified: mesos/site/publish/documentation/release-guide/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/release-guide/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/release-guide/index.html (original)
+++ mesos/site/publish/documentation/release-guide/index.html Mon Feb 1 02:49:25 2016
@@ -162,6 +162,7 @@ the master, slave, and configure flags.<
a live cluster from the previous release version to this release version.</p></li>
<li><p>If this is a major release, please ensure that user documentation has been
added for any new features.</p></li>
+<li><p>Make sure that for any updates of the API, specifically the scheduler API, the public mesos protobuf definitions are part of both, <code>include/mesos</code> as well as <code>include/mesos/v1</code>. NOTE: This might actually demand code updates if any omissions were identified.</p></li>
</ol>
@@ -170,7 +171,11 @@ added for any new features.</p></li>
<ol>
<li><p>Ensure that you can build and pass all the tests.</p>
-<pre><code> $ sudo make -j3 distcheck
+<pre><code> $ sudo make -j<cores> distcheck
+</code></pre></li>
+<li><p>Run the benchmarks and compare with the previous release for any performance regressions:</p>
+
+<pre><code> $ make bench -j<cores> GTEST_FILTER="*BENCHMARK*"
</code></pre></li>
<li><p>First tag the required SHA locally.</p>
@@ -192,7 +197,7 @@ It is recommended to use the <code>suppo
You may need to modify your local copy of tag.sh for it to complete successfully.</p></blockquote></li>
<li><p>It is not uncommon to release multiple release candidates, with increasing release candidate
version, if there are bugs found.</p></li>
-<li><p>Update to the <em>next</em> Mesos version in <code>configure.ac</code>: change <code>AC_INIT([mesos], [X.Y.Z]))</code> and commit.</p></li>
+<li><p>Update to the <em>next</em> Mesos version in <code>configure.ac</code>: change <code>AC_INIT([mesos], [X.Y.Z]))</code>, as well as in <code>CMakeLists.txt</code>: change <code>set(MESOS_MAJOR_VERSION X)</code>, <code>set(MESOS_MINOR_VERSION Y)</code>, <code>set(MESOS_PATCH_VERSION Z)</code> and then commit.</p></li>
</ol>
Modified: mesos/site/publish/documentation/reservation/index.html
URL: http://svn.apache.org/viewvc/mesos/site/publish/documentation/reservation/index.html?rev=1727886&r1=1727885&r2=1727886&view=diff
==============================================================================
--- mesos/site/publish/documentation/reservation/index.html (original)
+++ mesos/site/publish/documentation/reservation/index.html Mon Feb 1 02:49:25 2016
@@ -107,7 +107,7 @@ We start the slave like so:</p>
<p>We now have 8 CPUs and 4096 MB of RAM reserved for <code>ads</code> on this slave.</p>
<p><strong>CAVEAT:</strong> In order to modify a static reservation, the operator must drain and
- restart the slave with the new configuration specifed in the
+ restart the slave with the new configuration specified in the
<code>--resources</code> flag.</p>
<p><strong>NOTE:</strong> This feature is supported for backwards compatibility.
@@ -117,32 +117,46 @@ We start the slave like so:</p>
<h2>Dynamic Reservation (since 0.23.0)</h2>
-<p>As mentioned in <a href="#static-reservation-since-0140">Static Reservation</a>, specifying the
-reserved resources via the <code>--resources</code> flag makes the reservation static.
+<p>As mentioned in <a href="#static-reservation-since-0140">Static Reservation</a>, specifying
+the reserved resources via the <code>--resources</code> flag makes the reservation static.
This is, statically reserved resources cannot be reserved for another role nor
be unreserved. Dynamic Reservation enables operators and authorized frameworks
to reserve and unreserve resources post slave-startup.</p>
<p>We require a <code>principal</code> from the operator or framework in order to
-authenticate/authorize the operations. <a href="/documentation/latest/authorization/">Authorization</a> is
-specified via the existing ACL mechanism. (<em>Coming Soon</em>)</p>
+authenticate/authorize the operations. Permissions are specified via the
+existing ACL mechanism. To use authorization with reserve/unreserve operations,
+the Mesos master must be configured with the desired ACLs. For more information,
+see the <a href="/documentation/latest/authorization/">authorization documentation</a>.</p>
<ul>
<li><code>Offer::Operation::Reserve</code> and <code>Offer::Operation::Unreserve</code> messages are
available for <strong>frameworks</strong> to send back via the <code>acceptOffers</code> API as a
response to a resource offer.</li>
-<li><code>/reserve</code> and <code>/unreserve</code> HTTP endpoints are available for <strong>operators</strong>
-to manage dynamic reservations through the master. (<em>Coming Soon</em>).</li>
+<li><code>/reserve</code> and <code>/unreserve</code> HTTP endpoints allow <strong>operators</strong> to manage
+dynamic reservations through the master. NOTE: As of 0.27.0, these endpoints
+cannot be used when HTTP authentication is disabled due to the current
+implementation. This will change in version 0.28.0.</li>
</ul>
<p>In the following sections, we will walk through examples of each of the
interfaces described above.</p>
-<h3><code>Offer::Operation::Reserve</code></h3>
+<p>Note that if two dynamic reservations are made for resources at a single slave,
+the reservations will be combined by adding together the resources reserved by
+each request. Similarly, “partial” unreserve operations are allowed: an
+unreserve operation can release only some of the resources at a slave that have
+been reserved for a given role. In this case, the unreserved resources will be
+subtracted from the previous reservation, and any remaining resources will still
+be reserved.</p>
-<p>A framework is able to reserve resources through the resource offer cycle.
-Suppose we receive a resource offer with 12 CPUs and 6144 MB of RAM unreserved.</p>
+<h3>Framework Scheduler API</h3>
+
+<h4><code>Offer::Operation::Reserve</code></h4>
+
+<p>A framework can reserve resources through the resource offer cycle. Suppose we
+receive a resource offer with 12 CPUs and 6144 MB of RAM unreserved.</p>
<pre><code> {
"id": <offer_id>,
@@ -168,7 +182,7 @@ Suppose we receive a resource offer with
<p>We can reserve 8 CPUs and 4096 MB of RAM by sending the following
<code>Offer::Operation</code> message. <code>Offer::Operation::Reserve</code> has a <code>resources</code> field
-which we specify with the resources to be reserved. We need to expicitly set
+which we specify with the resources to be reserved. We need to explicitly set
the <code>role</code> and <code>principal</code> fields with the framework’s role and principal.</p>
<pre><code> {
@@ -198,7 +212,8 @@ the <code>role</code> and <code>principa
}
</code></pre>
-<p>The subsequent resource offer will <strong>contain</strong> the following reserved resources:</p>
+<p>If the reservation is successful, a subsequent resource offer will contain the
+following reserved resources:</p>
<pre><code> {
"id": <offer_id>,
@@ -228,13 +243,14 @@ the <code>role</code> and <code>principa
}
</code></pre>
-<h3><code>Offer::Operation::Unreserve</code></h3>
+<h4><code>Offer::Operation::Unreserve</code></h4>
-<p>A framework is able to unreserve resources through the resource offer cycle.
+<p>A framework can unreserve resources through the resource offer cycle.
In <a href="#offeroperationreserve">Offer::Operation::Reserve</a>, we reserved 8 CPUs
-and 4096 MB of RAM for our <code>role</code>. The master will continue to offer these
-resources to our <code>role</code>. Suppose we would like to unreserve these resources.
-First, we receive a resource offer (copy/pasted from above):</p>
+and 4096 MB of RAM on a particular slave for our <code>role</code>. The master will
+continue to only offer these resources to our <code>role</code>. Suppose we would like to
+unreserve these resources. First, we receive a resource offer (copy/pasted
+from above):</p>
<pre><code> {
"id": <offer_id>,
@@ -264,9 +280,9 @@ First, we receive a resource offer (copy
}
</code></pre>
-<p>We unreserve the 8 CPUs and 4096 MB of RAM by sending the following
+<p>We can unreserve the 8 CPUs and 4096 MB of RAM by sending the following
<code>Offer::Operation</code> message. <code>Offer::Operation::Unreserve</code> has a <code>resources</code> field
-which we specify with the resources to be unreserved.</p>
+which we can use to specify the resources to be unreserved.</p>
<pre><code> {
"type": Offer::Operation::UNRESERVE,
@@ -297,34 +313,44 @@ which we specify with the resources to b
<p>The unreserved resources may now be offered to other frameworks.</p>
-<h3><code>/reserve</code> (since 0.25.0)</h3>
+<h3>Operator HTTP Endpoints</h3>
-<p>Suppose we want to reserve 8 CPUs and 4096 MB of RAM for the <code>ads</code> role on
-a slave with id=<code><slave_id></code>. We send an HTTP POST request to the <code>/reserve</code>
-HTTP endpoint like so:</p>
+<p>As described above, dynamic reservations can be made by a framework scheduler,
+typically in response to a resource offer. However, dynamic reservations can
+also be created and deleted by sending HTTP requests to the <code>/reserve</code> and
+<code>/unreserve</code> endpoints, respectively. This capability is intended for use by
+operators and administrative tools.</p>
+
+<h4><code>/reserve</code> (since 0.25.0)</h4>
+
+<p>Suppose we want to reserve 8 CPUs and 4096 MB of RAM for the <code>ads</code> role on a
+slave with id=<code><slave_id></code> (note that it is up to the user to find the ID of the
+slave that hosts the desired resources; the request will fail if sufficient
+unreserved resources cannot be found on the slave). We send an HTTP POST request
+to the <code>/reserve</code> HTTP endpoint like so:</p>
<pre><code> $ curl -i \
-u <operator_principal>:<password> \
-d slaveId=<slave_id> \
- -d resources='[ \
- { \
- "name": "cpus", \
- "type": "SCALAR", \
- "scalar": { "value": 8 }, \
- "role": "ads", \
- "reservation": { \
- "principal": <operator_principal> \
- } \
- }, \
- { \
- "name": "mem", \
- "type": "SCALAR", \
- "scalar": { "value": 4096 }, \
- "role": "ads", \
- "reservation": { \
- "principal": <operator_principal> \
- } \
- } \
+ -d resources='[
+ {
+ "name": "cpus",
+ "type": "SCALAR",
+ "scalar": { "value": 8 },
+ "role": "ads",
+ "reservation": {
+ "principal": <operator_principal>
+ }
+ },
+ {
+ "name": "mem",
+ "type": "SCALAR",
+ "scalar": { "value": 4096 },
+ "role": "ads",
+ "reservation": {
+ "principal": <operator_principal>
+ }
+ }
]' \
-X POST http://<ip>:<port>/master/reserve
</code></pre>
@@ -332,14 +358,15 @@ HTTP endpoint like so:</p>
<p>The user receives one of the following HTTP responses:</p>
<ul>
-<li><code>200 OK</code>: Success</li>
-<li><code>400 BadRequest</code>: Invalid arguments (e.g. missing parameters).</li>
-<li><code>401 Unauthorized</code>: Unauthorized request.</li>
+<li><code>200 OK</code>: Success (the requested resources have been reserved).</li>
+<li><code>400 BadRequest</code>: Invalid arguments (e.g., missing parameters).</li>
+<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
+<li><code>403 Forbidden</code>: Unauthorized request.</li>
<li><code>409 Conflict</code>: Insufficient resources to satisfy the reserve operation.</li>
</ul>
-<h3><code>/unreserve</code> (since 0.25.0)</h3>
+<h4><code>/unreserve</code> (since 0.25.0)</h4>
<p>Suppose we want to unreserve the resources that we dynamically reserved above.
We can send an HTTP POST request to the <code>/unreserve</code> HTTP endpoint like so:</p>
@@ -347,25 +374,25 @@ We can send an HTTP POST request to the
<pre><code> $ curl -i \
-u <operator_principal>:<password> \
-d slaveId=<slave_id> \
- -d resources='[ \
- { \
- "name": "cpus", \
- "type": "SCALAR", \
- "scalar": { "value": 8 }, \
- "role": "ads", \
- "reservation": { \
- "principal": <operator_principal> \
- } \
- }, \
- { \
- "name": "mem", \
- "type": "SCALAR", \
- "scalar": { "value": 4096 }, \
- "role": "ads", \
- "reservation": { \
- "principal": <operator_principal> \
- } \
- } \
+ -d resources='[
+ {
+ "name": "cpus",
+ "type": "SCALAR",
+ "scalar": { "value": 8 },
+ "role": "ads",
+ "reservation": {
+ "principal": <operator_principal>
+ }
+ },
+ {
+ "name": "mem",
+ "type": "SCALAR",
+ "scalar": { "value": 4096 },
+ "role": "ads",
+ "reservation": {
+ "principal": <operator_principal>
+ }
+ }
]' \
-X POST http://<ip>:<port>/master/unreserve
</code></pre>
@@ -373,10 +400,11 @@ We can send an HTTP POST request to the
<p>The user receives one of the following HTTP responses:</p>
<ul>
-<li><code>200 OK</code>: Success</li>
-<li><code>400 BadRequest</code>: Invalid arguments (e.g. missing parameters).</li>
-<li><code>401 Unauthorized</code>: Unauthorized request.</li>
-<li><code>409 Conflict</code>: Insufficient resources to satisfy unreserve operation.</li>
+<li><code>200 OK</code>: Success (the requested resources have been unreserved).</li>
+<li><code>400 BadRequest</code>: Invalid arguments (e.g., missing parameters).</li>
+<li><code>401 Unauthorized</code>: Unauthenticated request.</li>
+<li><code>403 Forbidden</code>: Unauthorized request.</li>
+<li><code>409 Conflict</code>: Insufficient resources to satisfy the unreserve operation.</li>
</ul>