You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@aurora.apache.org by jf...@apache.org on 2016/04/16 06:09:26 UTC
svn commit: r1739400 [2/7] - in /aurora/site/publish/documentation/0.13.0:
./ additional-resources/ additional-resources/presentations/
additional-resources/tools/ development/ development/client/
development/committers-guide/ development/db-migration/...
Added: aurora/site/publish/documentation/0.13.0/development/thermos/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.13.0/development/thermos/index.html?rev=1739400&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.13.0/development/thermos/index.html (added)
+++ aurora/site/publish/documentation/0.13.0/development/thermos/index.html Sat Apr 16 04:09:25 2016
@@ -0,0 +1,233 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/development/thermos/'"
+ value="0.13.0">
+ <option value="0.13.0"
+ selected="selected">
+ 0.13.0
+ (latest)
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<p>The Python components of Aurora are built using <a href="https://pantsbuild.github.io">Pants</a>.</p>
+
+<h1 id="python-build-conventions">Python Build Conventions</h1>
+
+<p>The Python code is laid out according to the following conventions:</p>
+
+<ol>
+<li><p>1 <code>BUILD</code> per 3rd level directory. For a list of current top-level packages run:</p>
+<pre class="highlight plaintext"><code>% find src/main/python -maxdepth 3 -mindepth 3 -type d |\
+while read dname; do echo $dname |\
+ sed 's@src/main/python/\(.*\)/\(.*\)/\(.*\).*@\1.\2.\3@'; done
+</code></pre></li>
+<li><p>Each <code>BUILD</code> file exports 1
+<a href="https://pantsbuild.github.io/build_dictionary.html#bdict_python_library"><code>python_library</code></a>
+that provides a
+<a href="https://pantsbuild.github.io/build_dictionary.html#setup_py"><code>setup_py</code></a>
+containing each
+<a href="https://pantsbuild.github.io/build_dictionary.html#python_binary"><code>python_binary</code></a>
+in the <code>BUILD</code> file, named the same as the directory it’s in so that it can be referenced
+without a ’:’ character. The <code>sources</code> field in the <code>python_library</code> will almost always be
+<code>rglobs('*.py')</code>.</p></li>
+<li><p>Other BUILD files may only depend on this single public <code>python_library</code>
+target. Any other target is considered a private implementation detail and
+should be prefixed with an <code>_</code>.</p></li>
+<li><p><code>python_binary</code> targets are always named the same as the exported console script.</p></li>
+<li><p><code>python_binary</code> targets must have identical <code>dependencies</code> to the <code>python_library</code> exported
+by the package and must use <code>entry_point</code>.</p>
+
+<p>The means a PEX file generated by pants will contain exactly the same files that will be
+available on the <code>PYTHONPATH</code> in the case of <code>pip install</code> of the corresponding library
+target. This will help our migration off of Pants in the future.</p></li>
+</ol>
+
+<h2 id="annotated-example-apache-thermos-runner">Annotated example - apache.thermos.runner</h2>
+<pre class="highlight plaintext"><code>% find src/main/python/apache/thermos/runner
+src/main/python/apache/thermos/runner
+src/main/python/apache/thermos/runner/__init__.py
+src/main/python/apache/thermos/runner/thermos_runner.py
+src/main/python/apache/thermos/runner/BUILD
+% cat src/main/python/apache/thermos/runner/BUILD
+# License boilerplate omitted
+import os
+
+
+# Private target so that a setup_py can exist without a circular dependency. Only targets within
+# this file should depend on this.
+python_library(
+ name = '_runner',
+ # The target covers every python file under this directory and subdirectories.
+ sources = rglobs('*.py'),
+ dependencies = [
+ '3rdparty/python:twitter.common.app',
+ '3rdparty/python:twitter.common.log',
+ # Source dependencies are always referenced without a ':'.
+ 'src/main/python/apache/thermos/common',
+ 'src/main/python/apache/thermos/config',
+ 'src/main/python/apache/thermos/core',
+ ],
+)
+
+# Binary target for thermos_runner.pex. Nothing should depend on this - it's only used as an
+# argument to ./pants binary.
+python_binary(
+ name = 'thermos_runner',
+ # Use entry_point, not source so the files used here are the same ones tests see.
+ entry_point = 'apache.thermos.bin.thermos_runner',
+ dependencies = [
+ # Notice that we depend only on the single private target from this BUILD file here.
+ ':_runner',
+ ],
+)
+
+# The public library that everyone importing the runner symbols uses.
+# The test targets and any other dependent source code should depend on this.
+python_library(
+ name = 'runner',
+ dependencies = [
+ # Again, notice that we depend only on the single private target from this BUILD file here.
+ ':_runner',
+ ],
+ # We always provide a setup_py. This will cause any dependee libraries to automatically
+ # reference this library in their requirements.txt rather than copy the source files into their
+ # sdist.
+ provides = setup_py(
+ # Conventionally named and versioned.
+ name = 'apache.thermos.runner',
+ version = open(os.path.join(get_buildroot(), '.auroraversion')).read().strip().upper(),
+ ).with_binaries({
+ # Every binary in this file should also be repeated here.
+ # Always use the dict-form of .with_binaries so that commands with dashes in their names are
+ # supported.
+ # The console script name is always the same as the PEX with .pex stripped.
+ 'thermos_runner': ':thermos_runner',
+ }),
+)
+</code></pre>
+
+<h1 id="thermos-test-resources">Thermos Test resources</h1>
+
+<p>The Aurora source repository and distributions contain several
+<a href="../../src/test/resources/org/apache/thermos/root/checkpoints">binary files</a> to
+qualify the backwards-compatibility of thermos with checkpoint data. Since
+thermos persists state to disk, to be read by the thermos observer), it is important that we have
+tests that prevent regressions affecting the ability to parse previously-written data.</p>
+
+<p>The files included represent persisted checkpoints that exercise different
+features of thermos. The existing files should not be modified unless
+we are accepting backwards incompatibility, such as with a major release.</p>
+
+<p>It is not practical to write source code to generate these files on the fly,
+as source would be vulnerable to drift (e.g. due to refactoring) in ways
+that would undermine the goal of ensuring backwards compatibility.</p>
+
+<p>The most common reason to add a new checkpoint file would be to provide
+coverage for new thermos features that alter the data format. This is
+accomplished by writing and running a
+<a href="../reference/configuration.md">job configuration</a> that exercises the feature, and
+copying the checkpoint file from the sandbox directory, by default this is
+<code>/var/run/thermos/checkpoints/<aurora task id></code>.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.13.0/development/thrift/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.13.0/development/thrift/index.html?rev=1739400&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.13.0/development/thrift/index.html (added)
+++ aurora/site/publish/documentation/0.13.0/development/thrift/index.html Sat Apr 16 04:09:25 2016
@@ -0,0 +1,168 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/development/thrift/'"
+ value="0.13.0">
+ <option value="0.13.0"
+ selected="selected">
+ 0.13.0
+ (latest)
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="thrift">Thrift</h1>
+
+<p>Aurora uses <a href="https://thrift.apache.org/">Apache Thrift</a> for representing structured data in
+client/server RPC protocol as well as for internal data storage. While Thrift is capable of
+correctly handling additions and renames of the existing members, field removals must be done
+carefully to ensure backwards compatibility and provide predictable deprecation cycle. This
+document describes general guidelines for making Thrift schema changes to the existing fields in
+<a href="../../api/src/main/thrift/org/apache/aurora/gen/api.thrift">api.thrift</a>.</p>
+
+<p>It is highly recommended to go through the
+<a href="http://diwakergupta.github.io/thrift-missing-guide/">Thrift: The Missing Guide</a> first to refresh on
+basic Thrift schema concepts.</p>
+
+<h2 id="checklist">Checklist</h2>
+
+<p>Every existing Thrift schema modification is unique in its requirements and must be analyzed
+carefully to identify its scope and expected consequences. The following checklist may help in that
+analysis:
+* Is this a new field/struct? If yes, go ahead
+* Is this a pure field/struct rename without any type/structure change? If yes, go ahead and rename
+* Anything else, read further to make sure your change is properly planned</p>
+
+<h2 id="deprecation-cycle">Deprecation cycle</h2>
+
+<p>Any time a breaking change (e.g.: field replacement or removal) is required, the following cycle
+must be followed:</p>
+
+<h3 id="vcurrent">vCurrent</h3>
+
+<p>Change is applied in a way that does not break scheduler/client with this version to
+communicate with scheduler/client from vCurrent-1.
+* Do not remove or rename the old field
+* Add a new field as an eventual replacement of the old one and implement a dual read/write
+anywhere the old field is used. If a thrift struct is mapped in the DB store make sure both columns
+are marked as <code>NOT NULL</code>
+* Check <a href="../../api/src/main/thrift/org/apache/aurora/gen/storage.thrift">storage.thrift</a> to see if
+the affected struct is stored in Aurora scheduler storage. If so, it’s almost certainly also
+necessary to perform a <a href="db-migration.md">DB migration</a>.
+* Add a deprecation jira ticket into the vCurrent+1 release candidate
+* Add a TODO for the deprecated field mentioning the jira ticket</p>
+
+<h3 id="vcurrent-1">vCurrent+1</h3>
+
+<p>Finalize the change by removing the deprecated fields from the Thrift schema.
+* Drop any dual read/write routines added in the previous version
+* Remove thrift backfilling in scheduler
+* Remove the deprecated Thrift field</p>
+
+<h2 id="testing">Testing</h2>
+
+<p>It’s always advisable to test your changes in the local vagrant environment to build more
+confidence that you change is backwards compatible. It’s easy to simulate different
+client/scheduler versions by playing with <code>aurorabuild</code> command. See <a href="../getting-started/vagrant.md">this document</a>
+for more.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.13.0/development/ui/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.13.0/development/ui/index.html?rev=1739400&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.13.0/development/ui/index.html (added)
+++ aurora/site/publish/documentation/0.13.0/development/ui/index.html Sat Apr 16 04:09:25 2016
@@ -0,0 +1,158 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/development/ui/'"
+ value="0.13.0">
+ <option value="0.13.0"
+ selected="selected">
+ 0.13.0
+ (latest)
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="developing-the-aurora-scheduler-ui">Developing the Aurora Scheduler UI</h1>
+
+<h2 id="installing-bower-optional">Installing bower (optional)</h2>
+
+<p>Third party JS libraries used in Aurora (located at 3rdparty/javascript/bower_components) are
+managed by bower, a JS dependency manager. Bower is only required if you plan to add, remove or
+update JS libraries. Bower can be installed using the following command:</p>
+<pre class="highlight plaintext"><code>npm install -g bower
+</code></pre>
+
+<p>Bower depends on node.js and npm. The easiest way to install node on a mac is via brew:</p>
+<pre class="highlight plaintext"><code>brew install node
+</code></pre>
+
+<p>For more node.js installation options refer to <a href="https://github.com/joyent/node/wiki/Installation">https://github.com/joyent/node/wiki/Installation</a>.</p>
+
+<p>More info on installing and using bower can be found at: <a href="http://bower.io/">http://bower.io/</a>. Once installed, you can
+use the following commands to view and modify the bower repo at
+3rdparty/javascript/bower_components</p>
+<pre class="highlight plaintext"><code>bower list
+bower install <library name>
+bower remove <library name>
+bower update <library name>
+bower help
+</code></pre>
+
+<h2 id="faster-iteration-in-vagrant">Faster Iteration in Vagrant</h2>
+
+<p>The scheduler serves UI assets from the classpath. For production deployments this means the assets
+are served from within a jar. However, for faster development iteration, the vagrant image is
+configured to add the <code>scheduler</code> subtree of <code>/vagrant/dist/resources/main</code> to the head of
+<code>CLASSPATH</code>. This path is configured as a shared filesystem to the path on the host system where
+your Aurora repository lives. This means that any updates under <code>dist/resources/main/scheduler</code> in
+your checkout will be reflected immediately in the UI served from within the vagrant image.</p>
+
+<p>The one caveat to this is that this path is under <code>dist</code> not <code>src</code>. This is because the assets must
+be processed by gradle before they can be served. So, unfortunately, you cannot just save your local
+changes and see them reflected in the UI, you must first run <code>./gradlew processResources</code>. This is
+less than ideal, but better than having to restart the scheduler after every change. Additionally,
+gradle makes this process somewhat easier with the use of the <code>--continuous</code> flag. If you run:
+<code>./gradlew processResources --continuous</code> gradle will monitor the filesystem for changes and run the
+task automatically as necessary. This doesn’t quite provide hot-reload capabilities, but it does
+allow for <5s from save to changes being visibile in the UI with no further action required on the
+part of the developer.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.13.0/features/constraints/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.13.0/features/constraints/index.html?rev=1739400&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.13.0/features/constraints/index.html (added)
+++ aurora/site/publish/documentation/0.13.0/features/constraints/index.html Sat Apr 16 04:09:25 2016
@@ -0,0 +1,228 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/constraints/'"
+ value="0.13.0">
+ <option value="0.13.0"
+ selected="selected">
+ 0.13.0
+ (latest)
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="scheduling-constraints">Scheduling Constraints</h1>
+
+<p>By default, Aurora will pick any random slave with sufficient resources
+in order to schedule a task. This scheduling choice can be further
+restricted with the help of constraints.</p>
+
+<h2 id="mesos-attributes">Mesos Attributes</h2>
+
+<p>Data centers are often organized with hierarchical failure domains. Common failure domains
+include hosts, racks, rows, and PDUs. If you have this information available, it is wise to tag
+the Mesos slave with them as
+<a href="https://mesos.apache.org/documentation/attributes-resources/">attributes</a>.</p>
+
+<p>The Mesos slave <code>--attributes</code> command line argument can be used to mark slaves with
+static key/value pairs, so called attributes (not to be confused with <code>--resources</code>, which are
+dynamic and accounted).</p>
+
+<p>For example, consider the host <code>cluster1-aaa-03-sr2</code> and its following attributes (given in
+key:value format): <code>host:cluster1-aaa-03-sr2</code> and <code>rack:aaa</code>.</p>
+
+<p>Aurora makes these attributes available for matching with scheduling constraints.</p>
+
+<h2 id="limit-constraints">Limit Constraints</h2>
+
+<p>Limit constraints allow to control machine diversity using constraints. The below
+constraint ensures that no more than two instances of your job may run on a single host.
+Think of this as a “group by” limit.</p>
+<pre class="highlight plaintext"><code>Service(
+ name = 'webservice',
+ role = 'www-data',
+ constraints = {
+ 'host': 'limit:2',
+ }
+ ...
+)
+</code></pre>
+
+<p>Likewise, you can use constraints to control rack diversity, e.g. at
+most one task per rack:</p>
+<pre class="highlight plaintext"><code>constraints = {
+ 'rack': 'limit:1',
+}
+</code></pre>
+
+<p>Use these constraints sparingly as they can dramatically reduce Tasks’ schedulability.
+Further details are available in the reference documentation on
+<a href="../reference/configuration.md#specifying-scheduling-constraints">Scheduling Constraints</a>.</p>
+
+<h2 id="value-constraints">Value Constraints</h2>
+
+<p>Value constraints can be used to express that a certain attribute with a certain value
+should be present on a Mesos slave. For example, the following job would only be
+scheduled on nodes that claim to have an <code>SSD</code> as their disk.</p>
+<pre class="highlight plaintext"><code>Service(
+ name = 'webservice',
+ role = 'www-data',
+ constraints = {
+ 'disk': 'SSD',
+ }
+ ...
+)
+</code></pre>
+
+<p>Further details are available in the reference documentation on
+<a href="../reference/configuration.md#specifying-scheduling-constraints">Scheduling Constraints</a>.</p>
+
+<h2 id="running-stateful-services">Running stateful services</h2>
+
+<p>Aurora is best suited to run stateless applications, but it also accommodates for stateful services
+like databases, or services that otherwise need to always run on the same machines.</p>
+
+<h3 id="dedicated-attribute">Dedicated attribute</h3>
+
+<p>Most of the Mesos attributes arbitrary and available for custom use. There is one exception,
+though: the <code>dedicated</code> attribute. Aurora treats this specially, and only allows matching jobs to
+run on these machines, and will only schedule matching jobs on these machines.</p>
+
+<h4 id="syntax">Syntax</h4>
+
+<p>The dedicated attribute has semantic meaning. The format is <code>$role(/.*)?</code>. When a job is created,
+the scheduler requires that the <code>$role</code> component matches the <code>role</code> field in the job
+configuration, and will reject the job creation otherwise. The remainder of the attribute is
+free-form. We’ve developed the idiom of formatting this attribute as <code>$role/$job</code>, but do not
+enforce this. For example: a job <code>devcluster/www-data/prod/hello</code> with a dedicated constraint set as
+<code>www-data/web.multi</code> will have its tasks scheduled only on Mesos slaves configured with:
+<code>--attributes=dedicated:www-data/web.multi</code>.</p>
+
+<p>A wildcard (<code>*</code>) may be used for the role portion of the dedicated attribute, which will allow any
+owner to elect for a job to run on the host(s). For example: tasks from both
+<code>devcluster/www-data/prod/hello</code> and <code>devcluster/vagrant/test/hello</code> with a dedicated constraint
+formatted as <code>*/web.multi</code> will be scheduled only on Mesos slaves configured with
+<code>--attributes=dedicated:*/web.multi</code>. This may be useful when assembling a virtual cluster of
+machines sharing the same set of traits or requirements.</p>
+
+<h5 id="example">Example</h5>
+
+<p>Consider the following slave command line:</p>
+<pre class="highlight plaintext"><code>mesos-slave --attributes="dedicated:db_team/redis" ...
+</code></pre>
+
+<p>And this job configuration:</p>
+<pre class="highlight plaintext"><code>Service(
+ name = 'redis',
+ role = 'db_team',
+ constraints = {
+ 'dedicated': 'db_team/redis'
+ }
+ ...
+)
+</code></pre>
+
+<p>The job configuration is indicating that it should only be scheduled on slaves with the attribute
+<code>dedicated:db_team/redis</code>. Additionally, Aurora will prevent any tasks that do <em>not</em> have that
+constraint from running on those slaves.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.13.0/features/containers/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.13.0/features/containers/index.html?rev=1739400&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.13.0/features/containers/index.html (added)
+++ aurora/site/publish/documentation/0.13.0/features/containers/index.html Sat Apr 16 04:09:25 2016
@@ -0,0 +1,152 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/containers/'"
+ value="0.13.0">
+ <option value="0.13.0"
+ selected="selected">
+ 0.13.0
+ (latest)
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="containers">Containers</h1>
+
+<h2 id="docker">Docker</h2>
+
+<p>Aurora has optional support for launching Docker containers, if correctly <a href="../operations/configuration.md#docker-containers">configured by an Operator</a>.</p>
+
+<p>Example (available in the <a href="../getting-started/vagrant.md">Vagrant environment</a>):</p>
+<pre class="highlight plaintext"><code>$ cat /vagrant/examples/jobs/docker/hello_docker.aurora
+hello_docker = Process(
+ name = 'hello',
+ cmdline = """
+ while true; do
+ echo hello world
+ sleep 10
+ done
+ """)
+
+hello_world_docker = Task(
+ name = 'hello docker',
+ processes = [hello_world_proc],
+ resources = Resources(cpu = 1, ram = 1*MB, disk=8*MB)
+)
+
+jobs = [
+ Service(
+ cluster = 'devcluster',
+ environment = 'devel',
+ role = 'docker-test',
+ name = 'hello_docker',
+ task = hello_world_docker,
+ container = Container(docker = Docker(image = 'python:2.7'))
+ )
+]
+</code></pre>
+
+<p>In order to correctly execute processes inside a job, the docker container must have Python 2.7
+installed. Further details of how to use Docker can be found in the
+<a href="../reference/configuration.md#docker-object">Reference Documentation</a>.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.13.0/features/cron-jobs/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.13.0/features/cron-jobs/index.html?rev=1739400&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.13.0/features/cron-jobs/index.html (added)
+++ aurora/site/publish/documentation/0.13.0/features/cron-jobs/index.html Sat Apr 16 04:09:25 2016
@@ -0,0 +1,253 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/cron-jobs/'"
+ value="0.13.0">
+ <option value="0.13.0"
+ selected="selected">
+ 0.13.0
+ (latest)
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="cron-jobs">Cron Jobs</h1>
+
+<p>Aurora supports execution of scheduled jobs on a Mesos cluster using cron-style syntax.</p>
+
+<ul>
+<li><a href="#overview">Overview</a></li>
+<li><a href="#collision-policies">Collision Policies</a></li>
+<li><a href="#failure-recovery">Failure recovery</a></li>
+<li><a href="#interacting-with-cron-jobs-via-the-aurora-cli">Interacting with cron jobs via the Aurora CLI</a>
+
+<ul>
+<li><a href="#cron-schedule">cron schedule</a></li>
+<li><a href="#cron-deschedule">cron deschedule</a></li>
+<li><a href="#cron-start">cron start</a></li>
+<li><a href="#job-killall-job-restart-job-kill">job killall, job restart, job kill</a></li>
+</ul></li>
+<li><a href="#technical-note-about-syntax">Technical Note About Syntax</a></li>
+<li><a href="#caveats">Caveats</a>
+
+<ul>
+<li><a href="#failovers">Failovers</a></li>
+<li><a href="#collision-policy-is-best-effort">Collision policy is best-effort</a></li>
+<li><a href="#timezone-configuration">Timezone Configuration</a></li>
+</ul></li>
+</ul>
+
+<h2 id="overview">Overview</h2>
+
+<p>A job is identified as a cron job by the presence of a
+<code>cron_schedule</code> attribute containing a cron-style schedule in the
+<a href="../reference/configuration.md#job-objects"><code>Job</code></a> object. Examples of cron schedules
+include “every 5 minutes” (<code>*/5 * * * *</code>), “Fridays at 17:00” (<code>* 17 * * FRI</code>), and
+“the 1st and 15th day of the month at 03:00” (<code>0 3 1,15 *</code>).</p>
+
+<p>Example (available in the <a href="../getting-started/vagrant.md">Vagrant environment</a>):</p>
+<pre class="highlight plaintext"><code>$ cat /vagrant/examples/jobs/cron_hello_world.aurora
+# A cron job that runs every 5 minutes.
+jobs = [
+ Job(
+ cluster = 'devcluster',
+ role = 'www-data',
+ environment = 'test',
+ name = 'cron_hello_world',
+ cron_schedule = '*/5 * * * *',
+ task = SimpleTask(
+ 'cron_hello_world',
+ 'echo "Hello world from cron, the time is now $(date --rfc-822)"'),
+ ),
+]
+</code></pre>
+
+<h2 id="collision-policies">Collision Policies</h2>
+
+<p>The <code>cron_collision_policy</code> field specifies the scheduler’s behavior when a new cron job is
+triggered while an older run hasn’t finished. The scheduler has two policies available:</p>
+
+<ul>
+<li><code>KILL_EXISTING</code>: The default policy - on a collision the old instances are killed and a instances with the current
+configuration are started.</li>
+<li><code>CANCEL_NEW</code>: On a collision the new run is cancelled.</li>
+</ul>
+
+<p>Note that the use of <code>CANCEL_NEW</code> is likely a code smell - interrupted cron jobs should be able
+to recover their progress on a subsequent invocation, otherwise they risk having their work queue
+grow faster than they can process it.</p>
+
+<h2 id="failure-recovery">Failure recovery</h2>
+
+<p>Unlike with services, which aurora will always re-execute regardless of exit status, instances of
+cron jobs retry according to the <code>max_task_failures</code> attribute of the
+<a href="../reference/configuration.md#task-object">Task</a> object. To get “run-until-success” semantics,
+set <code>max_task_failures</code> to <code>-1</code>.</p>
+
+<h2 id="interacting-with-cron-jobs-via-the-aurora-cli">Interacting with cron jobs via the Aurora CLI</h2>
+
+<p>Most interaction with cron jobs takes place using the <code>cron</code> subcommand. See <code>aurora cron -h</code>
+for up-to-date usage instructions.</p>
+
+<h3 id="cron-schedule">cron schedule</h3>
+
+<p>Schedules a new cron job on the Aurora cluster for later runs or replaces the existing cron template
+with a new one. Only future runs will be affected, any existing active tasks are left intact.</p>
+<pre class="highlight plaintext"><code>$ aurora cron schedule devcluster/www-data/test/cron_hello_world /vagrant/examples/jobs/cron_hello_world.aurora
+</code></pre>
+
+<h3 id="cron-deschedule">cron deschedule</h3>
+
+<p>Deschedules a cron job, preventing future runs but allowing current runs to complete.</p>
+<pre class="highlight plaintext"><code>$ aurora cron deschedule devcluster/www-data/test/cron_hello_world
+</code></pre>
+
+<h3 id="cron-start">cron start</h3>
+
+<p>Start a cron job immediately, outside of its normal cron schedule.</p>
+<pre class="highlight plaintext"><code>$ aurora cron start devcluster/www-data/test/cron_hello_world
+</code></pre>
+
+<h3 id="job-killall-job-restart-job-kill">job killall, job restart, job kill</h3>
+
+<p>Cron jobs create instances running on the cluster that you can interact with like normal Aurora
+tasks with <code>job kill</code> and <code>job restart</code>.</p>
+
+<h2 id="technical-note-about-syntax">Technical Note About Syntax</h2>
+
+<p><code>cron_schedule</code> uses a restricted subset of BSD crontab syntax. While the
+execution engine currently uses Quartz, the schedule parsing is custom, a subset of FreeBSD
+<a href="http://www.freebsd.org/cgi/man.cgi?crontab(5)">crontab(5)</a> syntax. See
+<a href="https://github.com/apache/aurora/blob/master/src/main/java/org/apache/aurora/scheduler/cron/CrontabEntry.java#L106-L124">the source</a>
+for details.</p>
+
+<h2 id="caveats">Caveats</h2>
+
+<h3 id="failovers">Failovers</h3>
+
+<p>No failover recovery. Aurora does not record the latest minute it fired
+triggers for across failovers. Therefore it’s possible to miss triggers
+on failover. Note that this behavior may change in the future.</p>
+
+<p>It’s necessary to sync time between schedulers with something like <code>ntpd</code>.
+Clock skew could cause double or missed triggers in the case of a failover.</p>
+
+<h3 id="collision-policy-is-best-effort">Collision policy is best-effort</h3>
+
+<p>Aurora aims to always have <em>at least one copy</em> of a given instance running at a time - it’s
+an AP system, meaning it chooses Availability and Partition Tolerance at the expense of
+Consistency.</p>
+
+<p>If your collision policy was <code>CANCEL_NEW</code> and a task has terminated but
+Aurora has not noticed this Aurora will go ahead and create your new
+task.</p>
+
+<p>If your collision policy was <code>KILL_EXISTING</code> and a task was marked <code>LOST</code>
+but not yet GCed Aurora will go ahead and create your new task without
+attempting to kill the old one (outside the GC interval).</p>
+
+<h3 id="timezone-configuration">Timezone Configuration</h3>
+
+<p>Cron timezone is configured indepdendently of JVM timezone with the <code>-cron_timezone</code> flag and
+defaults to UTC.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.13.0/features/job-updates/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.13.0/features/job-updates/index.html?rev=1739400&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.13.0/features/job-updates/index.html (added)
+++ aurora/site/publish/documentation/0.13.0/features/job-updates/index.html Sat Apr 16 04:09:25 2016
@@ -0,0 +1,220 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/job-updates/'"
+ value="0.13.0">
+ <option value="0.13.0"
+ selected="selected">
+ 0.13.0
+ (latest)
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="aurora-job-updates">Aurora Job Updates</h1>
+
+<p><code>Job</code> configurations can be updated at any point in their lifecycle.
+Usually updates are done incrementally using a process called a <em>rolling
+upgrade</em>, in which Tasks are upgraded in small groups, one group at a
+time. Updates are done using various Aurora Client commands.</p>
+
+<h2 id="rolling-job-updates">Rolling Job Updates</h2>
+
+<p>There are several sub-commands to manage job updates:</p>
+<pre class="highlight plaintext"><code>aurora update start <job key> <configuration file>
+aurora update info <job key>
+aurora update pause <job key>
+aurora update resume <job key>
+aurora update abort <job key>
+aurora update list <cluster>
+</code></pre>
+
+<p>When you <code>start</code> a job update, the command will return once it has sent the
+instructions to the scheduler. At that point, you may view detailed
+progress for the update with the <code>info</code> subcommand, in addition to viewing
+graphical progress in the web browser. You may also get a full listing of
+in-progress updates in a cluster with <code>list</code>.</p>
+
+<p>Once an update has been started, you can <code>pause</code> to keep the update but halt
+progress. This can be useful for doing things like debug a partially-updated
+job to determine whether you would like to proceed. You can <code>resume</code> to
+proceed.</p>
+
+<p>You may <code>abort</code> a job update regardless of the state it is in. This will
+instruct the scheduler to completely abandon the job update and leave the job
+in the current (possibly partially-updated) state.</p>
+
+<p>For a configuration update, the Aurora Client calculates required changes
+by examining the current job config state and the new desired job config.
+It then starts a <em>rolling batched update process</em> by going through every batch
+and performing these operations:</p>
+
+<ul>
+<li>If an instance is present in the scheduler but isn’t in the new config,
+then that instance is killed.</li>
+<li>If an instance is not present in the scheduler but is present in
+the new config, then the instance is created.</li>
+<li>If an instance is present in both the scheduler and the new config, then
+the client diffs both task configs. If it detects any changes, it
+performs an instance update by killing the old config instance and adds
+the new config instance.</li>
+</ul>
+
+<p>The Aurora client continues through the instance list until all tasks are
+updated, in <code>RUNNING,</code> and healthy for a configurable amount of time.
+If the client determines the update is not going well (a percentage of health
+checks have failed), it cancels the update.</p>
+
+<p>Update cancellation runs a procedure similar to the described above
+update sequence, but in reverse order. New instance configs are swapped
+with old instance configs and batch updates proceed backwards
+from the point where the update failed. E.g.; (0,1,2) (3,4,5) (6,7,
+8-FAIL) results in a rollback in order (8,7,6) (5,4,3) (2,1,0).</p>
+
+<p>For details how to control a job update, please see the
+<a href="../reference/configuration.md#updateconfig-objects">UpdateConfig</a> configuration object.</p>
+
+<h2 id="coordinated-job-updates">Coordinated Job Updates</h2>
+
+<p>Some Aurora services may benefit from having more control over updates by explicitly
+acknowledging (“heartbeating”) job update progress. This may be helpful for mission-critical
+service updates where explicit job health monitoring is vital during the entire job update
+lifecycle. Such job updates would rely on an external service (or a custom client) periodically
+pulsing an active coordinated job update via a
+<a href="../../api/src/main/thrift/org/apache/aurora/gen/api.thrift">pulseJobUpdate RPC</a>.</p>
+
+<p>A coordinated update is defined by setting a positive
+<a href="../reference/configuration.md#updateconfig-objects">pulse<em>interval</em>secs</a> value in job configuration
+file. If no pulses are received within specified interval the update will be blocked. A blocked
+update is unable to continue rolling forward (or rolling back) but retains its active status.
+It may only be unblocked by a fresh <code>pulseJobUpdate</code> call.</p>
+
+<p>NOTE: A coordinated update starts in <code>ROLL_FORWARD_AWAITING_PULSE</code> state and will not make any
+progress until the first pulse arrives. However, a paused update (<code>ROLL_FORWARD_PAUSED</code> or
+<code>ROLL_BACK_PAUSED</code>) is still considered active and upon resuming will immediately make progress
+provided the pulse interval has not expired.</p>
+
+<h2 id="canary-deployments">Canary Deployments</h2>
+
+<p>Canary deployments are a pattern for rolling out updates to a subset of job instances,
+in order to test different code versions alongside the actual production job.
+It is a risk-mitigation strategy for job owners and commonly used in a form where
+job instance 0 runs with a different configuration than the instances 1-N.</p>
+
+<p>For example, consider a job with 4 instances that each
+request 1 core of cpu, 1 GB of RAM, and 1 GB of disk space as specified
+in the configuration file <code>hello_world.aurora</code>. If you want to
+update it so it requests 2 GB of RAM instead of 1. You can create a new
+configuration file to do that called <code>new_hello_world.aurora</code> and
+issue</p>
+<pre class="highlight plaintext"><code>aurora update start <job_key_value>/0-1 new_hello_world.aurora
+</code></pre>
+
+<p>This results in instances 0 and 1 having 1 cpu, 2 GB of RAM, and 1 GB of disk space,
+while instances 2 and 3 have 1 cpu, 1 GB of RAM, and 1 GB of disk space. If instance 3
+dies and restarts, it restarts with 1 cpu, 1 GB RAM, and 1 GB disk space.</p>
+
+<p>So that means there are two simultaneous task configurations for the same job
+at the same time, just valid for different ranges of instances. While this isn’t a recommended
+pattern, it is valid and supported by the Aurora scheduler.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.13.0/features/multitenancy/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.13.0/features/multitenancy/index.html?rev=1739400&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.13.0/features/multitenancy/index.html (added)
+++ aurora/site/publish/documentation/0.13.0/features/multitenancy/index.html Sat Apr 16 04:09:25 2016
@@ -0,0 +1,175 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/multitenancy/'"
+ value="0.13.0">
+ <option value="0.13.0"
+ selected="selected">
+ 0.13.0
+ (latest)
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="multitenancy">Multitenancy</h1>
+
+<p>Aurora is a multi-tenant system that can run jobs of multiple clients/tenants.
+Going beyond the <a href="resource-isolation.md">resource isolation on an individual host</a>, it is
+crucial to prevent those jobs from stepping on each others toes.</p>
+
+<h2 id="job-namespaces">Job Namespaces</h2>
+
+<p>The namespace for jobs in Aurora follows a hierarchical structure. This is meant to make it easier
+to differentiate between different jobs. A job key consists of four parts. The four parts are
+<code><cluster>/<role>/<environment>/<jobname></code> in that order:</p>
+
+<ul>
+<li>Cluster refers to the name of a particular Aurora installation.</li>
+<li>Role names are user accounts.</li>
+<li>Environment names are namespaces.</li>
+<li>Jobname is the custom name of your job.</li>
+</ul>
+
+<p>Role names correspond to user accounts. They are used for
+<a href="../operations/security.md">authentication</a>, as the linux user used to run jobs, and for the
+assignment of <a href="#preemption">quota</a>. If you don’t know what accounts are available, contact your
+sysadmin.</p>
+
+<p>The environment component in the job key, serves as a namespace. The values for
+environment are validated in the client and the scheduler so as to allow any of <code>devel</code>, <code>test</code>,
+<code>production</code>, and any value matching the regular expression <code>staging[0-9]*</code>.</p>
+
+<p>None of the values imply any difference in the scheduling behavior. Conventionally, the
+“environment” is set so as to indicate a certain level of stability in the behavior of the job
+by ensuring that an appropriate level of testing has been performed on the application code. e.g.
+in the case of a typical Job, releases may progress through the following phases in order of
+increasing level of stability: <code>devel</code>, <code>test</code>, <code>staging</code>, <code>production</code>.</p>
+
+<h2 id="preemption">Preemption</h2>
+
+<p>In order to guarantee that important production jobs are always running, Aurora supports
+preemption.</p>
+
+<p>Let’s consider we have a pending job that is candidate for scheduling but resource shortage pressure
+prevents this. Active tasks can become the victim of preemption, if:</p>
+
+<ul>
+<li>both candidate and victim are owned by the same role and the
+<a href="../reference/configuration.md#job-objects">priority</a> of a victim is lower than the
+<a href="../reference/configuration.md#job-objects">priority</a> of the candidate.</li>
+<li>OR a victim is non-<a href="../reference/configuration.md#job-objects">production</a> and the candidate is
+<a href="../reference/configuration.md#job-objects">production</a>.</li>
+</ul>
+
+<p>In other words, tasks from <a href="../reference/configuration.md#job-objects">production</a> jobs may preempt
+tasks from any non-production job. However, a production task may only be preempted by tasks from
+production jobs in the same role with higher <a href="../reference/configuration.md#job-objects">priority</a>.</p>
+
+<p>Aurora requires resource quotas for <a href="../reference/configuration.md#job-objects">production non-dedicated jobs</a>.
+Quota is enforced at the job role level and when set, defines a non-preemptible pool of compute resources within
+that role. All job types (service, adhoc or cron) require role resource quota unless a job has
+<a href="constraints.md#dedicated-attribute">dedicated constraint set</a>.</p>
+
+<p>To grant quota to a particular role in production, an operator can use the command
+<code>aurora_admin set_quota</code>.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.13.0/features/resource-isolation/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.13.0/features/resource-isolation/index.html?rev=1739400&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.13.0/features/resource-isolation/index.html (added)
+++ aurora/site/publish/documentation/0.13.0/features/resource-isolation/index.html Sat Apr 16 04:09:25 2016
@@ -0,0 +1,277 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/resource-isolation/'"
+ value="0.13.0">
+ <option value="0.13.0"
+ selected="selected">
+ 0.13.0
+ (latest)
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="resources-isolation-and-sizing">Resources Isolation and Sizing</h1>
+
+<ul>
+<li><a href="#isolation">Isolation</a></li>
+<li><a href="#sizing">Sizing</a></li>
+<li><a href="#oversubscription">Oversubscription</a></li>
+</ul>
+
+<h2 id="isolation">Isolation</h2>
+
+<p>Aurora is a multi-tenant system; a single software instance runs on a
+server, serving multiple clients/tenants. To share resources among
+tenants, it implements isolation of:</p>
+
+<ul>
+<li>CPU</li>
+<li>memory</li>
+<li>disk space</li>
+</ul>
+
+<p>CPU is a soft limit, and handled differently from memory and disk space.
+Too low a CPU value results in throttling your application and
+slowing it down. Memory and disk space are both hard limits; when your
+application goes over these values, it’s killed.</p>
+
+<h3 id="cpu-isolation">CPU Isolation</h3>
+
+<p>Mesos uses a quota based CPU scheduler (the <em>Completely Fair Scheduler</em>)
+to provide consistent and predictable performance. This is effectively
+a guarantee of resources – you receive at least what you requested, but
+also no more than you’ve requested.</p>
+
+<p>The scheduler gives applications a CPU quota for every 100 ms interval.
+When an application uses its quota for an interval, it is throttled for
+the rest of the 100 ms. Usage resets for each interval and unused
+quota does not carry over.</p>
+
+<p>For example, an application specifying 4.0 CPU has access to 400 ms of
+CPU time every 100 ms. This CPU quota can be used in different ways,
+depending on the application and available resources. Consider the
+scenarios shown in this diagram.</p>
+
+<p><img alt="CPU Availability" src="../../images/CPUavailability.png" /></p>
+
+<ul>
+<li><p><em>Scenario A</em>: the application can use up to 4 cores continuously for
+every 100 ms interval. It is never throttled and starts processing
+new requests immediately.</p></li>
+<li><p><em>Scenario B</em> : the application uses up to 8 cores (depending on
+availability) but is throttled after 50 ms. The CPU quota resets at the
+start of each new 100 ms interval.</p></li>
+<li><p><em>Scenario C</em> : is like Scenario A, but there is a garbage collection
+event in the second interval that consumes all CPU quota. The
+application throttles for the remaining 75 ms of that interval and
+cannot service requests until the next interval. In this example, the
+garbage collection finished in one interval but, depending on how much
+garbage needs collecting, it may take more than one interval and further
+delay service of requests.</p></li>
+</ul>
+
+<p><em>Technical Note</em>: Mesos considers logical cores, also known as
+hyperthreading or SMT cores, as the unit of CPU.</p>
+
+<h3 id="memory-isolation">Memory Isolation</h3>
+
+<p>Mesos uses dedicated memory allocation. Your application always has
+access to the amount of memory specified in your configuration. The
+application’s memory use is defined as the sum of the resident set size
+(RSS) of all processes in a shard. Each shard is considered
+independently.</p>
+
+<p>In other words, say you specified a memory size of 10GB. Each shard
+would receive 10GB of memory. If an individual shard’s memory demands
+exceed 10GB, that shard is killed, but the other shards continue
+working.</p>
+
+<p><em>Technical note</em>: Total memory size is not enforced at allocation time,
+so your application can request more than its allocation without getting
+an ENOMEM. However, it will be killed shortly after.</p>
+
+<h3 id="disk-space">Disk Space</h3>
+
+<p>Disk space used by your application is defined as the sum of the files’
+disk space in your application’s directory, including the <code>stdout</code> and
+<code>stderr</code> logged from your application. Each shard is considered
+independently. You should use off-node storage for your application’s
+data whenever possible.</p>
+
+<p>In other words, say you specified disk space size of 100MB. Each shard
+would receive 100MB of disk space. If an individual shard’s disk space
+demands exceed 100MB, that shard is killed, but the other shards
+continue working.</p>
+
+<p>After your application finishes running, its allocated disk space is
+reclaimed. Thus, your job’s final action should move any disk content
+that you want to keep, such as logs, to your home file system or other
+less transitory storage. Disk reclamation takes place an undefined
+period after the application finish time; until then, the disk contents
+are still available but you shouldn’t count on them being so.</p>
+
+<p><em>Technical note</em> : Disk space is not enforced at write so your
+application can write above its quota without getting an ENOSPC, but it
+will be killed shortly after. This is subject to change.</p>
+
+<h3 id="other-resources">Other Resources</h3>
+
+<p>Other resources, such as network bandwidth, do not have any performance
+guarantees. For some resources, such as memory bandwidth, there are no
+practical sharing methods so some application combinations collocated on
+the same host may cause contention.</p>
+
+<h2 id="sizing">Sizing</h2>
+
+<h3 id="cpu-sizing">CPU Sizing</h3>
+
+<p>To correctly size Aurora-run Mesos tasks, specify a per-shard CPU value
+that lets the task run at its desired performance when at peak load
+distributed across all shards. Include reserve capacity of at least 50%,
+possibly more, depending on how critical your service is (or how
+confident you are about your original estimate : -)), ideally by
+increasing the number of shards to also improve resiliency. When running
+your application, observe its CPU stats over time. If consistently at or
+near your quota during peak load, you should consider increasing either
+per-shard CPU or the number of shards.</p>
+
+<h2 id="memory-sizing">Memory Sizing</h2>
+
+<p>Size for your application’s peak requirement. Observe the per-instance
+memory statistics over time, as memory requirements can vary over
+different periods. Remember that if your application exceeds its memory
+value, it will be killed, so you should also add a safety margin of
+around 10-20%. If you have the ability to do so, you may also want to
+put alerts on the per-instance memory.</p>
+
+<h2 id="disk-space-sizing">Disk Space Sizing</h2>
+
+<p>Size for your application’s peak requirement. Rotate and discard log
+files as needed to stay within your quota. When running a Java process,
+add the maximum size of the Java heap to your disk space requirement, in
+order to account for an out of memory error dumping the heap
+into the application’s sandbox space.</p>
+
+<h2 id="oversubscription">Oversubscription</h2>
+
+<p><strong>WARNING</strong>: This feature is currently in alpha status. Do not use it in production clusters!</p>
+
+<p>Mesos <a href="http://mesos.apache.org/documentation/latest/oversubscription/">supports a concept of revocable tasks</a>
+by oversubscribing machine resources by the amount deemed safe to not affect the existing
+non-revocable tasks. Aurora now supports revocable jobs via a <code>tier</code> setting set to <code>revocable</code>
+value.</p>
+
+<p>The Aurora scheduler must be configured to receive revocable offers from Mesos and accept revocable
+jobs. If not configured properly revocable tasks will never get assigned to hosts and will stay in
+<code>PENDING</code>. Set these scheduler flag to allow receiving revocable Mesos offers:</p>
+<pre class="highlight plaintext"><code>-receive_revocable_resources=true
+</code></pre>
+
+<p>Specify a tier configuration file path (unless you want to use the <a href="../../src/main/resources/org/apache/aurora/scheduler/tiers.json">default</a>):</p>
+<pre class="highlight plaintext"><code>-tier_config=path/to/tiers/config.json
+</code></pre>
+
+<p>See the <a href="../references/configuration.md">Configuration Reference</a> for details on how to mark a job
+as being revocable.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>