You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@aurora.apache.org by se...@apache.org on 2017/02/21 20:55:06 UTC
svn commit: r1783940 [6/20] - in /aurora/site: data/ publish/ publish/blog/
publish/blog/aurora-0-17-0-released/ publish/documentation/0.10.0/
publish/documentation/0.10.0/build-system/
publish/documentation/0.10.0/client-cluster-configuration/ publish...
Added: aurora/site/publish/documentation/0.17.0/features/custom-executors/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.17.0/features/custom-executors/index.html?rev=1783940&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.17.0/features/custom-executors/index.html (added)
+++ aurora/site/publish/documentation/0.17.0/features/custom-executors/index.html Tue Feb 21 20:54:58 2017
@@ -0,0 +1,373 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/custom-executors/'"
+ value="0.17.0">
+ <option value="0.17.0"
+ selected="selected">
+ 0.17.0
+ (latest)
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="custom-executors">Custom Executors</h1>
+
+<p>If the need arises to use a Mesos executor other than the Thermos executor, the scheduler can be
+configured to utilize a custom executor by specifying the <code>-custom_executor_config</code> flag.
+The flag must be set to the path of a valid executor configuration file.</p>
+
+<p>The configuration file must be a valid <strong>JSON array</strong> and contain, at minimum,
+one executor configuration including the name, command and resources fields and
+must be pointed to by the <code>-custom_executor_config</code> flag when the scheduler is
+started.</p>
+
+<h3 id="array-entry">Array Entry</h3>
+
+<table><thead>
+<tr>
+<th>Property</th>
+<th>Description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td>executor (required)</td>
+<td>Description of executor.</td>
+</tr>
+<tr>
+<td>task_prefix (required) )</td>
+<td>Prefix given to tasks launched with this executor’s configuration.</td>
+</tr>
+<tr>
+<td>volume_mounts (optional)</td>
+<td>Volumes to be mounted in container running executor.</td>
+</tr>
+</tbody></table>
+
+<h4 id="executor">executor</h4>
+
+<table><thead>
+<tr>
+<th>Property</th>
+<th>Description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td>name (required)</td>
+<td>Name of the executor.</td>
+</tr>
+<tr>
+<td>command (required)</td>
+<td>How to run the executor.</td>
+</tr>
+<tr>
+<td>resources (required)</td>
+<td>Overhead to use for each executor instance.</td>
+</tr>
+</tbody></table>
+
+<h4 id="command">command</h4>
+
+<table><thead>
+<tr>
+<th>Property</th>
+<th>Description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td>value (required)</td>
+<td>The command to execute.</td>
+</tr>
+<tr>
+<td>arguments (optional)</td>
+<td>A list of arguments to pass to the command.</td>
+</tr>
+<tr>
+<td>uris (optional)</td>
+<td>List of resources to download into the task sandbox.</td>
+</tr>
+<tr>
+<td>shell (optional)</td>
+<td>Run executor via shell.</td>
+</tr>
+</tbody></table>
+
+<p>A note on the command property (from <a href="https://github.com/apache/mesos/blob/master/include/mesos/mesos.proto">mesos.proto</a>):
+<code>
+1) If 'shell == true', the command will be launched via shell
+ (i.e., /bin/sh -c 'value'). The 'value' specified will be
+ treated as the shell command. The 'arguments' will be ignored.
+2) If 'shell == false', the command will be launched by passing
+ arguments to an executable. The 'value' specified will be
+ treated as the filename of the executable. The 'arguments'
+ will be treated as the arguments to the executable. This is
+ similar to how POSIX exec families launch processes (i.e.,
+ execlp(value, arguments(0), arguments(1), ...)).
+</code></p>
+
+<h5 id="uris-list">uris (list)</h5>
+
+<ul>
+<li>Follows the <a href="http://mesos.apache.org/documentation/latest/fetcher/">Mesos Fetcher schema</a></li>
+</ul>
+
+<table><thead>
+<tr>
+<th>Property</th>
+<th>Description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td>value (required)</td>
+<td>Path to the resource needed in the sandbox.</td>
+</tr>
+<tr>
+<td>executable (optional)</td>
+<td>Change resource to be executable via chmod.</td>
+</tr>
+<tr>
+<td>extract (optional)</td>
+<td>Extract files from packed or compressed archives into the sandbox.</td>
+</tr>
+<tr>
+<td>cache (optional)</td>
+<td>Use caching mechanism provided by Mesos for resources.</td>
+</tr>
+</tbody></table>
+
+<h4 id="resources-list">resources (list)</h4>
+
+<table><thead>
+<tr>
+<th>Property</th>
+<th>Description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td>name (required)</td>
+<td>Name of the resource: cpus or mem.</td>
+</tr>
+<tr>
+<td>type (required)</td>
+<td>Type of resource. Should always be SCALAR.</td>
+</tr>
+<tr>
+<td>scalar (required)</td>
+<td>Value in float for cpus or int for mem (in MBs)</td>
+</tr>
+</tbody></table>
+
+<h3 id="volume_mounts-list">volume_mounts (list)</h3>
+
+<table><thead>
+<tr>
+<th>Property</th>
+<th>Description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td>host_path (required)</td>
+<td>Host path to mount inside the container.</td>
+</tr>
+<tr>
+<td>container_path (required)</td>
+<td>Path inside the container where <code>host_path</code> will be mounted.</td>
+</tr>
+<tr>
+<td>mode (required)</td>
+<td>Mode in which to mount the volume, Read-Write (RW) or Read-Only (RO).</td>
+</tr>
+</tbody></table>
+
+<p>A sample configuration is as follows:
+“`
+[
+ {
+ "executor”: {
+ “name”: “myExecutor”,
+ “command”: {
+ “value”: “myExecutor.a”,
+ “shell”: “false”,
+ “arguments”: [
+ “localhost:2181”,
+ “-verbose”,
+ “-config myConfiguration.config”
+ ],
+ “uris”: [
+ {
+ “value”: “/dist/myExecutor.a”,
+ “executable”: true,
+ “extract”: false,
+ “cache”: true
+ },
+ {
+ “value”: “/home/user/myConfiguration.config”,
+ “executable”: false,
+ “extract”: false,
+ “cache”: false
+ }
+ ]
+ },
+ “resources”: [
+ {
+ “name”: “cpus”,
+ “type”: “SCALAR”,
+ “scalar”: {
+ “value”: 1.00
+ }
+ },
+ {
+ “name”: “mem”,
+ “type”: “SCALAR”,
+ “scalar”: {
+ “value”: 512
+ }
+ }
+ ]
+ },
+ “volume<em>mounts”: [
+ {
+ “mode”: “RO”,
+ “container</em>path”: “/path/on/container”,
+ “host<em>path”: “/path/to/host/directory”
+ },
+ {
+ “mode”: “RW”,
+ “container</em>path”: “/container”,
+ “host<em>path”: “/host”
+ }
+ ],
+ “task</em>prefix”: “my-executor-”
+ }
+]</p>
+<pre class="highlight plaintext"><code>
+It should be noted that if you do not use Thermos or a Thermos based executor, links in the scheduler's
+Web UI for tasks will not work (at least for the time being).
+Some information about launched tasks can still be accessed via the Mesos Web UI or via the Aurora Client.
+
+### Using a custom executor
+
+At this time, it is not currently possible create a job that runs on a custom executor using the default
+Aurora client. To allow the scheduler to pick the correct executor, the `JobConfiguration.TaskConfig.ExecutorConfig.name`
+field must be set to match the name used in the custom executor configuration blob. (e.g. to run a job using myExecutor,
+`JobConfiguration.TaskConfig.ExecutorConfig.name` must be set to `myExecutor`). While support for modifying
+this field in Pystachio created, the easiest way to launch jobs with custom executors is to use
+an existing custom Client such as [gorealis](https://github.com/rdelval/gorealis).
+</code></pre>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.17.0/features/job-updates/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.17.0/features/job-updates/index.html?rev=1783940&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.17.0/features/job-updates/index.html (added)
+++ aurora/site/publish/documentation/0.17.0/features/job-updates/index.html Tue Feb 21 20:54:58 2017
@@ -0,0 +1,235 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/job-updates/'"
+ value="0.17.0">
+ <option value="0.17.0"
+ selected="selected">
+ 0.17.0
+ (latest)
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="aurora-job-updates">Aurora Job Updates</h1>
+
+<p><code>Job</code> configurations can be updated at any point in their lifecycle.
+Usually updates are done incrementally using a process called a <em>rolling
+upgrade</em>, in which Tasks are upgraded in small groups, one group at a
+time. Updates are done using various Aurora Client commands.</p>
+
+<h2 id="rolling-job-updates">Rolling Job Updates</h2>
+
+<p>There are several sub-commands to manage job updates:</p>
+<pre class="highlight plaintext"><code>aurora update start <job key> <configuration file>
+aurora update info <job key>
+aurora update pause <job key>
+aurora update resume <job key>
+aurora update abort <job key>
+aurora update list <cluster>
+</code></pre>
+
+<p>When you <code>start</code> a job update, the command will return once it has sent the
+instructions to the scheduler. At that point, you may view detailed
+progress for the update with the <code>info</code> subcommand, in addition to viewing
+graphical progress in the web browser. You may also get a full listing of
+in-progress updates in a cluster with <code>list</code>.</p>
+
+<p>Once an update has been started, you can <code>pause</code> to keep the update but halt
+progress. This can be useful for doing things like debug a partially-updated
+job to determine whether you would like to proceed. You can <code>resume</code> to
+proceed.</p>
+
+<p>You may <code>abort</code> a job update regardless of the state it is in. This will
+instruct the scheduler to completely abandon the job update and leave the job
+in the current (possibly partially-updated) state.</p>
+
+<p>For a configuration update, the Aurora Scheduler calculates required changes
+by examining the current job config state and the new desired job config.
+It then starts a <em>rolling batched update process</em> by going through every batch
+and performing these operations:</p>
+
+<ul>
+<li>If an instance is present in the scheduler but isn’t in the new config,
+then that instance is killed.</li>
+<li>If an instance is not present in the scheduler but is present in
+the new config, then the instance is created.</li>
+<li>If an instance is present in both the scheduler and the new config, then
+the scheduler diffs both task configs. If it detects any changes, it
+performs an instance update by killing the old config instance and adds
+the new config instance.</li>
+</ul>
+
+<p>The Aurora Scheduler continues through the instance list until all tasks are
+updated and in <code>RUNNING</code>. If the scheduler determines the update is not going
+well (based on the criteria specified in the UpdateConfig), it cancels the update.</p>
+
+<p>Update cancellation runs a procedure similar to the described above
+update sequence, but in reverse order. New instance configs are swapped
+with old instance configs and batch updates proceed backwards
+from the point where the update failed. E.g.; (0,1,2) (3,4,5) (6,7,
+8-FAIL) results in a rollback in order (8,7,6) (5,4,3) (2,1,0).</p>
+
+<p>For details on how to control a job update, please see the
+<a href="../../reference/configuration/#updateconfig-objects">UpdateConfig</a> configuration object.</p>
+
+<h2 id="coordinated-job-updates">Coordinated Job Updates</h2>
+
+<p>Some Aurora services may benefit from having more control over updates by explicitly
+acknowledging (“heartbeating”) job update progress. This may be helpful for mission-critical
+service updates where explicit job health monitoring is vital during the entire job update
+lifecycle. Such job updates would rely on an external service (or a custom client) periodically
+pulsing an active coordinated job update via a
+<a href="https://github.com/apache/aurora/blob/rel/0.17.0/api/src/main/thrift/org/apache/aurora/gen/api.thrift">pulseJobUpdate RPC</a>.</p>
+
+<p>A coordinated update is defined by setting a positive
+<a href="../../reference/configuration/#updateconfig-objects">pulse<em>interval</em>secs</a> value in job configuration
+file. If no pulses are received within specified interval the update will be blocked. A blocked
+update is unable to continue rolling forward (or rolling back) but retains its active status.
+It may only be unblocked by a fresh <code>pulseJobUpdate</code> call.</p>
+
+<p>NOTE: A coordinated update starts in <code>ROLL_FORWARD_AWAITING_PULSE</code> state and will not make any
+progress until the first pulse arrives. However, a paused update (<code>ROLL_FORWARD_PAUSED</code> or
+<code>ROLL_BACK_PAUSED</code>) is still considered active and upon resuming will immediately make progress
+provided the pulse interval has not expired.</p>
+
+<h2 id="canary-deployments">Canary Deployments</h2>
+
+<p>Canary deployments are a pattern for rolling out updates to a subset of job instances,
+in order to test different code versions alongside the actual production job.
+It is a risk-mitigation strategy for job owners and commonly used in a form where
+job instance 0 runs with a different configuration than the instances 1-N.</p>
+
+<p>For example, consider a job with 4 instances that each
+request 1 core of cpu, 1 GB of RAM, and 1 GB of disk space as specified
+in the configuration file <code>hello_world.aurora</code>. If you want to
+update it so it requests 2 GB of RAM instead of 1. You can create a new
+configuration file to do that called <code>new_hello_world.aurora</code> and
+issue</p>
+<pre class="highlight plaintext"><code>aurora update start <job_key_value>/0-1 new_hello_world.aurora
+</code></pre>
+
+<p>This results in instances 0 and 1 having 1 cpu, 2 GB of RAM, and 1 GB of disk space,
+while instances 2 and 3 have 1 cpu, 1 GB of RAM, and 1 GB of disk space. If instance 3
+dies and restarts, it restarts with 1 cpu, 1 GB RAM, and 1 GB disk space.</p>
+
+<p>So that means there are two simultaneous task configurations for the same job
+at the same time, just valid for different ranges of instances. While this isn’t a recommended
+pattern, it is valid and supported by the Aurora scheduler.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.17.0/features/mesos-fetcher/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.17.0/features/mesos-fetcher/index.html?rev=1783940&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.17.0/features/mesos-fetcher/index.html (added)
+++ aurora/site/publish/documentation/0.17.0/features/mesos-fetcher/index.html Tue Feb 21 20:54:58 2017
@@ -0,0 +1,189 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/mesos-fetcher/'"
+ value="0.17.0">
+ <option value="0.17.0"
+ selected="selected">
+ 0.17.0
+ (latest)
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="mesos-fetcher">Mesos Fetcher</h1>
+
+<p>Mesos has support for downloading resources into the sandbox through the
+use of the <a href="http://mesos.apache.org/documentation/latest/fetcher/">Mesos Fetcher</a></p>
+
+<p>Aurora supports passing URIs to the Mesos Fetcher dynamically by including
+a list of URIs in job submissions.</p>
+
+<h2 id="how-to-use">How to use</h2>
+
+<p>The scheduler flag <code>-enable_mesos_fetcher</code> must be set to true.</p>
+
+<p>Currently only the scheduler side of this feature has been implemented
+so a modification to the existing client, or a custom Thrift client are required
+to make use of this feature.</p>
+
+<p>If using a custom Thrift client, the list of URIs must be included in TaskConfig
+as the <code>mesosFetcherUris</code> field.</p>
+
+<p>Each Mesos Fetcher URI has the following data members:</p>
+
+<table><thead>
+<tr>
+<th>Property</th>
+<th>Description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td>value (required)</td>
+<td>Path to the resource needed in the sandbox.</td>
+</tr>
+<tr>
+<td>extract (optional)</td>
+<td>Extract files from packed or compressed archives into the sandbox.</td>
+</tr>
+<tr>
+<td>cache (optional)</td>
+<td>Use caching mechanism provided by Mesos for resources.</td>
+</tr>
+</tbody></table>
+
+<p>Note that this structure is very similar to the one provided for downloading
+resources needed for a <a href="../../operations/configuration/">custom executor</a>.</p>
+
+<p>This is because both features use the Mesos fetcher to retrieve resources into
+the sandbox. However, one, the custom executor feature, has a static set of URIs
+set in the server side, and the other, the Mesos Fetcher feature, is a dynamic set
+of URIs set at the time of job submission.</p>
+
+<h2 id="security-implications">Security Implications</h2>
+
+<p>There are security implications that must be taken into account when enabling this feature.
+<strong>Enabling this feature may potentially enable any job submitting user to perform a privilege escalation.</strong></p>
+
+<p>Until a more through solution is created, one step that has been taken to mitigate this issue
+is to statically mark every user submitted URI as non-executable. This is in contrast to the set of URIs
+set in the custom executor feature which may mark any URI as executable.</p>
+
+<p>If the need arises to mark a downloaded URI as executable, please consider using the custom executor feature.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.17.0/features/multitenancy/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.17.0/features/multitenancy/index.html?rev=1783940&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.17.0/features/multitenancy/index.html (added)
+++ aurora/site/publish/documentation/0.17.0/features/multitenancy/index.html Tue Feb 21 20:54:58 2017
@@ -0,0 +1,210 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/multitenancy/'"
+ value="0.17.0">
+ <option value="0.17.0"
+ selected="selected">
+ 0.17.0
+ (latest)
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="multitenancy">Multitenancy</h1>
+
+<p>Aurora is a multi-tenant system that can run jobs of multiple clients/tenants.
+Going beyond the <a href="../resource-isolation/">resource isolation on an individual host</a>, it is
+crucial to prevent those jobs from stepping on each others toes.</p>
+
+<h2 id="job-namespaces">Job Namespaces</h2>
+
+<p>The namespace for jobs in Aurora follows a hierarchical structure. This is meant to make it easier
+to differentiate between different jobs. A job key consists of four parts. The four parts are
+<code><cluster>/<role>/<environment>/<jobname></code> in that order:</p>
+
+<ul>
+<li>Cluster refers to the name of a particular Aurora installation.</li>
+<li>Role names are user accounts.</li>
+<li>Environment names are namespaces.</li>
+<li>Jobname is the custom name of your job.</li>
+</ul>
+
+<p>Role names correspond to user accounts. They are used for
+<a href="../../operations/security/">authentication</a>, as the linux user used to run jobs, and for the
+assignment of <a href="#preemption">quota</a>. If you don’t know what accounts are available, contact your
+sysadmin.</p>
+
+<p>The environment component in the job key, serves as a namespace. The values for
+environment are validated in the client and the scheduler so as to allow any of <code>devel</code>, <code>test</code>,
+<code>production</code>, and any value matching the regular expression <code>staging[0-9]*</code>.</p>
+
+<p>None of the values imply any difference in the scheduling behavior. Conventionally, the
+“environment” is set so as to indicate a certain level of stability in the behavior of the job
+by ensuring that an appropriate level of testing has been performed on the application code. e.g.
+in the case of a typical Job, releases may progress through the following phases in order of
+increasing level of stability: <code>devel</code>, <code>test</code>, <code>staging</code>, <code>production</code>.</p>
+
+<h2 id="configuration-tiers">Configuration Tiers</h2>
+
+<p>Tier is a predefined bundle of task configuration options. Aurora schedules tasks and assigns them
+resources based on their tier assignment. The default scheduler tier configuration allows for
+3 tiers:</p>
+
+<ul>
+<li><code>revocable</code>: The <code>revocable</code> tier requires the task to run with <a href="../resource-isolation/#oversubscription">revocable</a>
+resources.</li>
+<li><code>preemptible</code>: Setting the task’s tier to <code>preemptible</code> allows for the possibility of that task
+being <a href="#preemption">preempted</a> by other tasks when cluster is running low on resources.</li>
+<li><code>preferred</code>: The <code>preferred</code> tier prevents the task from using <a href="../resource-isolation/#oversubscription">revocable</a>
+resources and from being <a href="#preemption">preempted</a>.</li>
+</ul>
+
+<p>Since it is possible that a cluster is configured with a custom tier configuration, users should
+consult their cluster administrator to be informed of the tiers supported by the cluster. Attempts
+to schedule jobs with an unsupported tier will be rejected by the scheduler.</p>
+
+<h2 id="preemption">Preemption</h2>
+
+<p>In order to guarantee that important production jobs are always running, Aurora supports
+preemption.</p>
+
+<p>Let’s consider we have a pending job that is candidate for scheduling but resource shortage pressure
+prevents this. Active tasks can become the victim of preemption, if:</p>
+
+<ul>
+<li>both candidate and victim are owned by the same role and the
+<a href="../../reference/configuration/#job-objects">priority</a> of a victim is lower than the
+<a href="../../reference/configuration/#job-objects">priority</a> of the candidate.</li>
+<li>OR a victim is a <code>preemptible</code> or <code>revocable</code> <a href="#configuration-tiers">tier</a> task and the candidate
+is a <code>preferred</code> <a href="#configuration-tiers">tier</a> task.</li>
+</ul>
+
+<p>In other words, tasks from <code>preferred</code> <a href="../../reference/configuration/#job-objects">tier</a> jobs may
+preempt tasks from any <code>preemptible</code> or <code>revocable</code> job. However, a <code>preferred</code> task may only be
+preempted by tasks from <code>preferred</code> jobs in the same role with higher <a href="../../reference/configuration/#job-objects">priority</a>.</p>
+
+<p>Aurora requires resource quotas for <a href="../../reference/configuration/#job-objects">production non-dedicated jobs</a>.
+Quota is enforced at the job role level and when set, defines a non-preemptible pool of compute resources within
+that role. All job types (service, adhoc or cron) require role resource quota unless a job has
+<a href="../constraints/#dedicated-attribute">dedicated constraint set</a>.</p>
+
+<p>To grant quota to a particular role in production, an operator can use the command
+<code>aurora_admin set_quota</code>.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.17.0/features/resource-isolation/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.17.0/features/resource-isolation/index.html?rev=1783940&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.17.0/features/resource-isolation/index.html (added)
+++ aurora/site/publish/documentation/0.17.0/features/resource-isolation/index.html Tue Feb 21 20:54:58 2017
@@ -0,0 +1,308 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/resource-isolation/'"
+ value="0.17.0">
+ <option value="0.17.0"
+ selected="selected">
+ 0.17.0
+ (latest)
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="resources-isolation-and-sizing">Resources Isolation and Sizing</h1>
+
+<p>This document assumes Aurora and Mesos have been configured
+using our <a href="../../operations/configuration/#resource-isolation">recommended resource isolation settings</a>.</p>
+
+<ul>
+<li><a href="#isolation">Isolation</a></li>
+<li><a href="#sizing">Sizing</a></li>
+<li><a href="#oversubscription">Oversubscription</a></li>
+</ul>
+
+<h2 id="isolation">Isolation</h2>
+
+<p>Aurora is a multi-tenant system; a single software instance runs on a
+server, serving multiple clients/tenants. To share resources among
+tenants, it leverages Mesos for isolation of:</p>
+
+<ul>
+<li>CPU</li>
+<li>GPU</li>
+<li>memory</li>
+<li>disk space</li>
+<li>ports</li>
+</ul>
+
+<p>CPU is a soft limit, and handled differently from memory and disk space.
+Too low a CPU value results in throttling your application and
+slowing it down. Memory and disk space are both hard limits; when your
+application goes over these values, it’s killed.</p>
+
+<h3 id="cpu-isolation">CPU Isolation</h3>
+
+<p>Mesos can be configured to use a quota based CPU scheduler (the <em>Completely</em>
+<em>Fair Scheduler</em>) to provide consistent and predictable performance.
+This is effectively a guarantee of resources – you receive at least what
+you requested, but also no more than you’ve requested.</p>
+
+<p>The scheduler gives applications a CPU quota for every 100 ms interval.
+When an application uses its quota for an interval, it is throttled for
+the rest of the 100 ms. Usage resets for each interval and unused
+quota does not carry over.</p>
+
+<p>For example, an application specifying 4.0 CPU has access to 400 ms of
+CPU time every 100 ms. This CPU quota can be used in different ways,
+depending on the application and available resources. Consider the
+scenarios shown in this diagram.</p>
+
+<p><img alt="CPU Availability" src="../../images/CPUavailability.png" /></p>
+
+<ul>
+<li><p><em>Scenario A</em>: the application can use up to 4 cores continuously for
+every 100 ms interval. It is never throttled and starts processing
+new requests immediately.</p></li>
+<li><p><em>Scenario B</em> : the application uses up to 8 cores (depending on
+availability) but is throttled after 50 ms. The CPU quota resets at the
+start of each new 100 ms interval.</p></li>
+<li><p><em>Scenario C</em> : is like Scenario A, but there is a garbage collection
+event in the second interval that consumes all CPU quota. The
+application throttles for the remaining 75 ms of that interval and
+cannot service requests until the next interval. In this example, the
+garbage collection finished in one interval but, depending on how much
+garbage needs collecting, it may take more than one interval and further
+delay service of requests.</p></li>
+</ul>
+
+<p><em>Technical Note</em>: Mesos considers logical cores, also known as
+hyperthreading or SMT cores, as the unit of CPU.</p>
+
+<h3 id="memory-isolation">Memory Isolation</h3>
+
+<p>Mesos uses dedicated memory allocation. Your application always has
+access to the amount of memory specified in your configuration. The
+application’s memory use is defined as the sum of the resident set size
+(RSS) of all processes in a shard. Each shard is considered
+independently.</p>
+
+<p>In other words, say you specified a memory size of 10GB. Each shard
+would receive 10GB of memory. If an individual shard’s memory demands
+exceed 10GB, that shard is killed, but the other shards continue
+working.</p>
+
+<p><em>Technical note</em>: Total memory size is not enforced at allocation time,
+so your application can request more than its allocation without getting
+an ENOMEM. However, it will be killed shortly after.</p>
+
+<h3 id="disk-space">Disk Space</h3>
+
+<p>Disk space used by your application is defined as the sum of the files’
+disk space in your application’s directory, including the <code>stdout</code> and
+<code>stderr</code> logged from your application. Each shard is considered
+independently. You should use off-node storage for your application’s
+data whenever possible.</p>
+
+<p>In other words, say you specified disk space size of 100MB. Each shard
+would receive 100MB of disk space. If an individual shard’s disk space
+demands exceed 100MB, that shard is killed, but the other shards
+continue working.</p>
+
+<p>After your application finishes running, its allocated disk space is
+reclaimed. Thus, your job’s final action should move any disk content
+that you want to keep, such as logs, to your home file system or other
+less transitory storage. Disk reclamation takes place an undefined
+period after the application finish time; until then, the disk contents
+are still available but you shouldn’t count on them being so.</p>
+
+<p><em>Technical note</em> : Disk space is not enforced at write so your
+application can write above its quota without getting an ENOSPC, but it
+will be killed shortly after. This is subject to change.</p>
+
+<h3 id="gpu-isolation">GPU Isolation</h3>
+
+<p>GPU isolation will be supported for Nvidia devices starting from Mesos 1.0.
+Access to the allocated units will be exclusive with no sharing between tasks
+allowed (e.g. no fractional GPU allocation). For more details, see the
+<a href="https://docs.google.com/document/d/10GJ1A80x4nIEo8kfdeo9B11PIbS1xJrrB4Z373Ifkpo/edit#heading=h.w84lz7p4eexl">Mesos design document</a>
+and the <a href="http://mesos.apache.org/documentation/latest/configuration/">Mesos agent configuration</a>.</p>
+
+<h3 id="other-resources">Other Resources</h3>
+
+<p>Other resources, such as network bandwidth, do not have any performance
+guarantees. For some resources, such as memory bandwidth, there are no
+practical sharing methods so some application combinations collocated on
+the same host may cause contention.</p>
+
+<h2 id="sizing">Sizing</h2>
+
+<h3 id="cpu-sizing">CPU Sizing</h3>
+
+<p>To correctly size Aurora-run Mesos tasks, specify a per-shard CPU value
+that lets the task run at its desired performance when at peak load
+distributed across all shards. Include reserve capacity of at least 50%,
+possibly more, depending on how critical your service is (or how
+confident you are about your original estimate : -)), ideally by
+increasing the number of shards to also improve resiliency. When running
+your application, observe its CPU stats over time. If consistently at or
+near your quota during peak load, you should consider increasing either
+per-shard CPU or the number of shards.</p>
+
+<h2 id="memory-sizing">Memory Sizing</h2>
+
+<p>Size for your application’s peak requirement. Observe the per-instance
+memory statistics over time, as memory requirements can vary over
+different periods. Remember that if your application exceeds its memory
+value, it will be killed, so you should also add a safety margin of
+around 10-20%. If you have the ability to do so, you may also want to
+put alerts on the per-instance memory.</p>
+
+<h2 id="disk-space-sizing">Disk Space Sizing</h2>
+
+<p>Size for your application’s peak requirement. Rotate and discard log
+files as needed to stay within your quota. When running a Java process,
+add the maximum size of the Java heap to your disk space requirement, in
+order to account for an out of memory error dumping the heap
+into the application’s sandbox space.</p>
+
+<h2 id="gpu-sizing">GPU Sizing</h2>
+
+<p>GPU is highly dependent on your application requirements and is only limited
+by the number of physical GPU units available on a target box.</p>
+
+<h2 id="oversubscription">Oversubscription</h2>
+
+<p>Mesos supports <a href="http://mesos.apache.org/documentation/latest/oversubscription/">oversubscription of machine resources</a>
+via the concept of revocable tasks. In contrast to non-revocable tasks, revocable tasks are best-effort.
+Mesos reserves the right to throttle or even kill them if they might affect existing high-priority
+user-facing services.</p>
+
+<p>As of today, the only revocable resource supported by Aurora are CPU and RAM resources. A job can
+opt-in to use those by specifying the <code>revocable</code> <a href="../../features/multitenancy/#configuration-tiers">Configuration Tier</a>.
+A revocable job will only be scheduled using revocable resources, even if there are plenty of
+non-revocable resources available.</p>
+
+<p>The Aurora scheduler must be <a href="../../operations/configuration/#resource-isolation">configured to receive revocable offers</a>
+from Mesos and accept revocable jobs. If not configured properly revocable tasks will never get
+assigned to hosts and will stay in <code>PENDING</code>.</p>
+
+<p>For details on how to mark a job as being revocable, see the
+<a href="../../reference/configuration/">Configuration Reference</a>.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.17.0/features/service-discovery/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.17.0/features/service-discovery/index.html?rev=1783940&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.17.0/features/service-discovery/index.html (added)
+++ aurora/site/publish/documentation/0.17.0/features/service-discovery/index.html Tue Feb 21 20:54:58 2017
@@ -0,0 +1,178 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/service-discovery/'"
+ value="0.17.0">
+ <option value="0.17.0"
+ selected="selected">
+ 0.17.0
+ (latest)
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="service-discovery">Service Discovery</h1>
+
+<p>It is possible for the Aurora executor to announce tasks into ServerSets for
+the purpose of service discovery. ServerSets use the Zookeeper <a href="http://zookeeper.apache.org/doc/trunk/recipes.html#sc_outOfTheBox">group membership pattern</a>
+of which there are several reference implementations:</p>
+
+<ul>
+<li><a href="https://github.com/apache/mesos/blob/master/src/zookeeper/group.cpp">C++</a></li>
+<li><a href="https://github.com/twitter/commons/blob/master/src/java/com/twitter/common/zookeeper/ServerSetImpl.java#L221">Java</a></li>
+<li><a href="https://github.com/twitter/commons/blob/master/src/python/twitter/common/zookeeper/serverset/serverset.py#L51">Python</a></li>
+</ul>
+
+<p>These can also be used natively in Finagle using the <a href="https://github.com/twitter/finagle/blob/master/finagle-serversets/src/main/scala/com/twitter/finagle/zookeeper/ZookeeperServerSetCluster.scala">ZookeeperServerSetCluster</a>.</p>
+
+<p>For more information about how to configure announcing, see the <a href="../../reference/configuration/">Configuration Reference</a>.</p>
+
+<h2 id="using-mesos-discoveryinfo">Using Mesos DiscoveryInfo</h2>
+
+<p>Experimental support for populating DiscoveryInfo in Mesos is introduced in Aurora. This can be used to build
+custom service discovery system not using zookeeper. Please see <code>Service Discovery</code> section in
+<a href="http://mesos.apache.org/documentation/latest/app-framework-development-guide/">Mesos Framework Development guide</a> for
+explanation of the protobuf message in Mesos.</p>
+
+<p>To use this feature, please enable <code>--populate_discovery_info</code> flag on scheduler. All jobs started by scheduler
+afterwards will have their portmap populated to Mesos and discoverable in <code>/state</code> endpoint in Mesos master and agent.</p>
+
+<h3 id="using-mesos-dns">Using Mesos DNS</h3>
+
+<p>An example is using <a href="https://github.com/mesosphere/mesos-dns">Mesos-DNS</a>, which is able to generate multiple DNS
+records. With current implementation, the example job with key <code>devcluster/vagrant/test/http-example</code> generates at
+least the following:</p>
+
+<ol>
+<li>An A record for <code>http_example.test.vagrant.aurora.mesos</code> (which only includes IP address);</li>
+<li>A <a href="https://en.wikipedia.org/wiki/SRV_record">SRV record</a> for
+<code>_http_example.test.vagrant._tcp.aurora.mesos</code>, which includes IP address and every port. This should only
+be used if the service has one port.</li>
+<li>A SRV record <code>_{port-name}._http_example.test.vagrant._tcp.aurora.mesos</code> for each port name
+defined. This should be used when the service has multiple ports.</li>
+</ol>
+
+<p>Things to note:</p>
+
+<ol>
+<li>The domain part (“.mesos” in above example) can be configured in <a href="http://mesosphere.github.io/mesos-dns/docs/configuration-parameters.html">Mesos DNS</a>;</li>
+<li>Right now, portmap and port aliases in announcer object are not reflected in DiscoveryInfo, therefore not visible in
+Mesos DNS records either. This is because they are only resolved in thermos executors.</li>
+</ol>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.17.0/features/services/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.17.0/features/services/index.html?rev=1783940&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.17.0/features/services/index.html (added)
+++ aurora/site/publish/documentation/0.17.0/features/services/index.html Tue Feb 21 20:54:58 2017
@@ -0,0 +1,236 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/services/'"
+ value="0.17.0">
+ <option value="0.17.0"
+ selected="selected">
+ 0.17.0
+ (latest)
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="long-running-services">Long-running Services</h1>
+
+<p>Jobs that are always restart on completion, whether successful or unsuccessful,
+are called services. This is useful for long-running processes
+such as webservices that should always be running, unless stopped explicitly.</p>
+
+<h2 id="service-specification">Service Specification</h2>
+
+<p>A job is identified as a service by the presence of the flag
+`<code>service=True</code> in the <a href="../../reference/configuration/#job-objects"><code>Job</code></a> object.
+The <code>Service</code> alias can be used as shorthand for <code>Job</code> with <code>service=True</code>.</p>
+
+<p>Example (available in the <a href="../../getting-started/vagrant/">Vagrant environment</a>):</p>
+<pre class="highlight plaintext"><code>$ cat /vagrant/examples/jobs/hello_world.aurora
+hello = Process(
+ name = 'hello',
+ cmdline = """
+ while true; do
+ echo hello world
+ sleep 10
+ done
+ """)
+
+task = SequentialTask(
+ processes = [hello],
+ resources = Resources(cpu = 1.0, ram = 128*MB, disk = 128*MB)
+)
+
+jobs = [
+ Service(
+ task = task,
+ cluster = 'devcluster',
+ role = 'www-data',
+ environment = 'prod',
+ name = 'hello'
+ )
+]
+</code></pre>
+
+<p>Jobs without the service bit set only restart up to <code>max_task_failures</code> times and only if they
+terminated unsuccessfully either due to human error or machine failure (see the
+<a href="../../reference/configuration/#job-objects"><code>Job</code></a> object for details).</p>
+
+<h2 id="ports">Ports</h2>
+
+<p>In order to be useful, most services have to bind to one or more ports. Aurora enables this
+usecase via the <a href="../../reference/configuration/#thermos-namespace"><code>thermos.ports</code> namespace</a> that
+allows to request arbitrarily named ports:</p>
+<pre class="highlight plaintext"><code>nginx = Process(
+ name = 'nginx',
+ cmdline = './run_nginx.sh -port {{thermos.ports[http]}}'
+)
+</code></pre>
+
+<p>When this process is included in a job, the job will be allocated a port, and the command line
+will be replaced with something like:</p>
+<pre class="highlight plaintext"><code>./run_nginx.sh -port 42816
+</code></pre>
+
+<p>Where 42816 happens to be the allocated port.</p>
+
+<p>For details on how to enable clients to discover this dynamically assigned port, see our
+<a href="../service-discovery/">Service Discovery</a> documentation.</p>
+
+<h2 id="health-checking">Health Checking</h2>
+
+<p>Typically, the Thermos executor monitors processes within a task only by liveness of the forked
+process. In addition to that, Aurora has support for rudimentary health checking: Either via HTTP
+via custom shell scripts.</p>
+
+<p>For example, simply by requesting a <code>health</code> port, a process can request to be health checked
+via repeated calls to the <code>/health</code> endpoint:</p>
+<pre class="highlight plaintext"><code>nginx = Process(
+ name = 'nginx',
+ cmdline = './run_nginx.sh -port {{thermos.ports[health]}}'
+)
+</code></pre>
+
+<p>Please see the
+<a href="../../reference/configuration/#healthcheckconfig-objects">configuration reference</a>
+for configuration options for this feature.</p>
+
+<p>Starting with the 0.17.0 release, job updates rely only on task health-checks by introducing
+a <code>min_consecutive_successes</code> parameter on the HealthCheckConfig object. This parameter represents
+the number of successful health checks needed before a task is moved into the <code>RUNNING</code> state. Tasks
+that do not have enough successful health checks within the first <code>n</code> attempts, are moved to the
+<code>FAILED</code> state, where <code>n = ceil(initial_interval_secs/interval_secs) + max_consecutive_failures +
+min_consecutive_successes</code>. In order to accommodate variability during task warm up, <code>initial_interval_secs</code>
+will act as a grace period. Any health-check failures during the first <code>m</code> attempts are ignored and
+do not count towards <code>max_consecutive_failures</code>, where <code>m = ceil(initial_interval_secs/interval_secs)</code>.</p>
+
+<p>As <a href="../job-updates/">job updates</a> are based only on health-checks, it is not necessary to set
+<code>watch_secs</code> to the worst-case update time, it can instead be set to 0. The scheduler considers a
+task that is in the <code>RUNNING</code> to be healthy and proceeds to updating the next batch of instances.
+For details on how to control health checks, please see the
+<a href="../../reference/configuration/#healthcheckconfig-objects">HealthCheckConfig</a> configuration object.
+Existing jobs that do not configure a health-check can fall-back to using <code>watch_secs</code> to
+monitor a task before considering it healthy.</p>
+
+<p>You can pause health checking by touching a file inside of your sandbox, named <code>.healthchecksnooze</code>.
+As long as that file is present, health checks will be disabled, enabling users to gather core
+dumps or other performance measurements without worrying about Aurora’s health check killing
+their process.</p>
+
+<p>WARNING: Remember to remove this when you are done, otherwise your instance will have permanently
+disabled health checks.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.17.0/features/sla-metrics/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.17.0/features/sla-metrics/index.html?rev=1783940&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.17.0/features/sla-metrics/index.html (added)
+++ aurora/site/publish/documentation/0.17.0/features/sla-metrics/index.html Tue Feb 21 20:54:58 2017
@@ -0,0 +1,368 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/sla-metrics/'"
+ value="0.17.0">
+ <option value="0.17.0"
+ selected="selected">
+ 0.17.0
+ (latest)
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="aurora-sla-measurement">Aurora SLA Measurement</h1>
+
+<ul>
+<li><a href="#overview">Overview</a></li>
+<li><a href="#metric-details">Metric Details</a>
+
+<ul>
+<li><a href="#platform-uptime">Platform Uptime</a></li>
+<li><a href="#job-uptime">Job Uptime</a></li>
+<li><a href="#median-time-to-assigned-(mtta)">Median Time To Assigned (MTTA)</a></li>
+<li><a href="#median-time-to-starting-(mtts)">Median Time To Starting (MTTS)</a></li>
+<li><a href="#median-time-to-running-(mttr)">Median Time To Running (MTTR)</a></li>
+</ul></li>
+<li><a href="#limitations">Limitations</a></li>
+</ul>
+
+<h2 id="overview">Overview</h2>
+
+<p>The primary goal of the feature is collection and monitoring of Aurora job SLA (Service Level
+Agreements) metrics that defining a contractual relationship between the Aurora/Mesos platform
+and hosted services.</p>
+
+<p>The Aurora SLA feature is by default only enabled for service (non-cron)
+production jobs (<code>"production=True"</code> in your <code>.aurora</code> config). It can be enabled for
+non-production services by an operator via the scheduler command line flag <code>-sla_non_prod_metrics</code>.</p>
+
+<p>Counters that track SLA measurements are computed periodically within the scheduler.
+The individual instance metrics are refreshed every minute (configurable via
+<code>sla_stat_refresh_interval</code>). The instance counters are subsequently aggregated by
+relevant grouping types before exporting to scheduler <code>/vars</code> endpoint (when using <code>vagrant</code>
+that would be <code>http://192.168.33.7:8081/vars</code>)</p>
+
+<h2 id="metric-details">Metric Details</h2>
+
+<h3 id="platform-uptime">Platform Uptime</h3>
+
+<p><em>Aggregate amount of time a job spends in a non-runnable state due to platform unavailability
+or scheduling delays. This metric tracks Aurora/Mesos uptime performance and reflects on any
+system-caused downtime events (tasks LOST or DRAINED). Any user-initiated task kills/restarts
+will not degrade this metric.</em></p>
+
+<p><strong>Collection scope:</strong></p>
+
+<ul>
+<li>Per job - <code>sla_<job_key>_platform_uptime_percent</code></li>
+<li>Per cluster - <code>sla_cluster_platform_uptime_percent</code></li>
+</ul>
+
+<p><strong>Units:</strong> percent</p>
+
+<p>A fault in the task environment may cause the Aurora/Mesos to have different views on the task state
+or lose track of the task existence. In such cases, the service task is marked as LOST and
+rescheduled by Aurora. For example, this may happen when the task stays in ASSIGNED or STARTING
+for too long or the Mesos agent becomes unhealthy (or disappears completely). The time between
+task entering LOST and its replacement reaching RUNNING state is counted towards platform downtime.</p>
+
+<p>Another example of a platform downtime event is the administrator-requested task rescheduling. This
+happens during planned Mesos agent maintenance when all agent tasks are marked as DRAINED and
+rescheduled elsewhere.</p>
+
+<p>To accurately calculate Platform Uptime, we must separate platform incurred downtime from user
+actions that put a service instance in a non-operational state. It is simpler to isolate
+user-incurred downtime and treat all other downtime as platform incurred.</p>
+
+<p>Currently, a user can cause a healthy service (task) downtime in only two ways: via <code>killTasks</code>
+or <code>restartShards</code> RPCs. For both, their affected tasks leave an audit state transition trail
+relevant to uptime calculations. By applying a special “SLA meaning” to exposed task state
+transition records, we can build a deterministic downtime trace for every given service instance.</p>
+
+<p>A task going through a state transition carries one of three possible SLA meanings
+(see <a href="https://github.com/apache/aurora/blob/rel/0.17.0/src/main/java/org/apache/aurora/scheduler/sla/SlaAlgorithm.java">SlaAlgorithm.java</a> for
+sla-to-task-state mapping):</p>
+
+<ul>
+<li><p>Task is UP: starts a period where the task is considered to be up and running from the Aurora
+platform standpoint.</p></li>
+<li><p>Task is DOWN: starts a period where the task cannot reach the UP state for some
+non-user-related reason. Counts towards instance downtime.</p></li>
+<li><p>Task is REMOVED from SLA: starts a period where the task is not expected to be UP due to
+user initiated action or failure. We ignore this period for the uptime calculation purposes.</p></li>
+</ul>
+
+<p>This metric is recalculated over the last sampling period (last minute) to account for
+any UP/DOWN/REMOVED events. It ignores any UP/DOWN events not immediately adjacent to the
+sampling interval as well as adjacent REMOVED events.</p>
+
+<h3 id="job-uptime">Job Uptime</h3>
+
+<p><em>Percentage of the job instances considered to be in RUNNING state for the specified duration
+relative to request time. This is a purely application side metric that is considering aggregate
+uptime of all RUNNING instances. Any user- or platform initiated restarts directly affect
+this metric.</em></p>
+
+<p><strong>Collection scope:</strong> We currently expose job uptime values at 5 pre-defined
+percentiles (50th,75th,90th,95th and 99th):</p>
+
+<ul>
+<li><code>sla_<job_key>_job_uptime_50_00_sec</code></li>
+<li><code>sla_<job_key>_job_uptime_75_00_sec</code></li>
+<li><code>sla_<job_key>_job_uptime_90_00_sec</code></li>
+<li><code>sla_<job_key>_job_uptime_95_00_sec</code></li>
+<li><code>sla_<job_key>_job_uptime_99_00_sec</code></li>
+</ul>
+
+<p><strong>Units:</strong> seconds
+You can also get customized real-time stats from aurora client. See <code>aurora sla -h</code> for
+more details.</p>
+
+<h3 id="median-time-to-assigned-mtta">Median Time To Assigned (MTTA)</h3>
+
+<p><em>Median time a job spends waiting for its tasks to be assigned to a host. This is a combined
+metric that helps track the dependency of scheduling performance on the requested resources
+(user scope) as well as the internal scheduler bin-packing algorithm efficiency (platform scope).</em></p>
+
+<p><strong>Collection scope:</strong></p>
+
+<ul>
+<li>Per job - <code>sla_<job_key>_mtta_ms</code></li>
+<li>Per cluster - <code>sla_cluster_mtta_ms</code></li>
+<li>Per instance size (small, medium, large, x-large, xx-large). Size are defined in:
+<a href="https://github.com/apache/aurora/blob/rel/0.17.0/src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java">ResourceBag.java</a>
+
+<ul>
+<li>By CPU:</li>
+<li><code>sla_cpu_small_mtta_ms</code></li>
+<li><code>sla_cpu_medium_mtta_ms</code></li>
+<li><code>sla_cpu_large_mtta_ms</code></li>
+<li><code>sla_cpu_xlarge_mtta_ms</code></li>
+<li><code>sla_cpu_xxlarge_mtta_ms</code></li>
+<li>By RAM:</li>
+<li><code>sla_ram_small_mtta_ms</code></li>
+<li><code>sla_ram_medium_mtta_ms</code></li>
+<li><code>sla_ram_large_mtta_ms</code></li>
+<li><code>sla_ram_xlarge_mtta_ms</code></li>
+<li><code>sla_ram_xxlarge_mtta_ms</code></li>
+<li>By DISK:</li>
+<li><code>sla_disk_small_mtta_ms</code></li>
+<li><code>sla_disk_medium_mtta_ms</code></li>
+<li><code>sla_disk_large_mtta_ms</code></li>
+<li><code>sla_disk_xlarge_mtta_ms</code></li>
+<li><code>sla_disk_xxlarge_mtta_ms</code></li>
+</ul></li>
+</ul>
+
+<p><strong>Units:</strong> milliseconds</p>
+
+<p>MTTA only considers instances that have already reached ASSIGNED state and ignores those
+that are still PENDING. This ensures straggler instances (e.g. with unreasonable resource
+constraints) do not affect metric curves.</p>
+
+<h3 id="median-time-to-starting-mtts">Median Time To Starting (MTTS)</h3>
+
+<p><em>Median time a job waits for its tasks to reach STARTING state. This is a comprehensive metric
+reflecting on the overall time it takes for the Aurora/Mesos to start initializing the sandbox
+for a task.</em></p>
+
+<p><strong>Collection scope:</strong></p>
+
+<ul>
+<li>Per job - <code>sla_<job_key>_mtts_ms</code></li>
+<li>Per cluster - <code>sla_cluster_mtts_ms</code></li>
+<li>Per instance size (small, medium, large, x-large, xx-large). Size are defined in:
+<a href="https://github.com/apache/aurora/blob/rel/0.17.0/src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java">ResourceBag.java</a>
+
+<ul>
+<li>By CPU:</li>
+<li><code>sla_cpu_small_mtts_ms</code></li>
+<li><code>sla_cpu_medium_mtts_ms</code></li>
+<li><code>sla_cpu_large_mtts_ms</code></li>
+<li><code>sla_cpu_xlarge_mtts_ms</code></li>
+<li><code>sla_cpu_xxlarge_mtts_ms</code></li>
+<li>By RAM:</li>
+<li><code>sla_ram_small_mtts_ms</code></li>
+<li><code>sla_ram_medium_mtts_ms</code></li>
+<li><code>sla_ram_large_mtts_ms</code></li>
+<li><code>sla_ram_xlarge_mtts_ms</code></li>
+<li><code>sla_ram_xxlarge_mtts_ms</code></li>
+<li>By DISK:</li>
+<li><code>sla_disk_small_mtts_ms</code></li>
+<li><code>sla_disk_medium_mtts_ms</code></li>
+<li><code>sla_disk_large_mtts_ms</code></li>
+<li><code>sla_disk_xlarge_mtts_ms</code></li>
+<li><code>sla_disk_xxlarge_mtts_ms</code></li>
+</ul></li>
+</ul>
+
+<p><strong>Units:</strong> milliseconds</p>
+
+<p>MTTS only considers instances in STARTING state. This ensures straggler instances (e.g. with
+unreasonable resource constraints) do not affect metric curves.</p>
+
+<h3 id="median-time-to-running-mttr">Median Time To Running (MTTR)</h3>
+
+<p><em>Median time a job waits for its tasks to reach RUNNING state. This is a comprehensive metric
+reflecting on the overall time it takes for the Aurora/Mesos to start executing user content.</em></p>
+
+<p><strong>Collection scope:</strong></p>
+
+<ul>
+<li>Per job - <code>sla_<job_key>_mttr_ms</code></li>
+<li>Per cluster - <code>sla_cluster_mttr_ms</code></li>
+<li>Per instance size (small, medium, large, x-large, xx-large). Size are defined in:
+<a href="https://github.com/apache/aurora/blob/rel/0.17.0/src/main/java/org/apache/aurora/scheduler/resources/ResourceBag.java">ResourceBag.java</a>
+
+<ul>
+<li>By CPU:</li>
+<li><code>sla_cpu_small_mttr_ms</code></li>
+<li><code>sla_cpu_medium_mttr_ms</code></li>
+<li><code>sla_cpu_large_mttr_ms</code></li>
+<li><code>sla_cpu_xlarge_mttr_ms</code></li>
+<li><code>sla_cpu_xxlarge_mttr_ms</code></li>
+<li>By RAM:</li>
+<li><code>sla_ram_small_mttr_ms</code></li>
+<li><code>sla_ram_medium_mttr_ms</code></li>
+<li><code>sla_ram_large_mttr_ms</code></li>
+<li><code>sla_ram_xlarge_mttr_ms</code></li>
+<li><code>sla_ram_xxlarge_mttr_ms</code></li>
+<li>By DISK:</li>
+<li><code>sla_disk_small_mttr_ms</code></li>
+<li><code>sla_disk_medium_mttr_ms</code></li>
+<li><code>sla_disk_large_mttr_ms</code></li>
+<li><code>sla_disk_xlarge_mttr_ms</code></li>
+<li><code>sla_disk_xxlarge_mttr_ms</code></li>
+</ul></li>
+</ul>
+
+<p><strong>Units:</strong> milliseconds</p>
+
+<p>MTTR only considers instances in RUNNING state. This ensures straggler instances (e.g. with
+unreasonable resource constraints) do not affect metric curves.</p>
+
+<h2 id="limitations">Limitations</h2>
+
+<ul>
+<li><p>The availability of Aurora SLA metrics is bound by the scheduler availability.</p></li>
+<li><p>All metrics are calculated at a pre-defined interval (currently set at 1 minute).
+Scheduler restarts may result in missed collections.</p></li>
+</ul>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>