You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@aurora.apache.org by re...@apache.org on 2018/09/11 05:28:12 UTC
svn commit: r1840515 [2/15] - in /aurora/site:
publish/blog/aurora-0-21-0-released/ publish/documentation/0.21.0/
publish/documentation/0.21.0/additional-resources/
publish/documentation/0.21.0/additional-resources/presentations/
publish/documentation/...
Added: aurora/site/publish/documentation/0.21.0/development/db-migration/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.21.0/development/db-migration/index.html?rev=1840515&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.21.0/development/db-migration/index.html (added)
+++ aurora/site/publish/documentation/0.21.0/development/db-migration/index.html Tue Sep 11 05:28:10 2018
@@ -0,0 +1,187 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/development/db-migration/'"
+ value="0.21.0">
+ <option value="0.21.0"
+ selected="selected">
+ 0.21.0
+ (latest)
+ </option>
+ <option value="0.20.0"
+ >
+ 0.20.0
+ </option>
+ <option value="0.19.1"
+ >
+ 0.19.1
+ </option>
+ <option value="0.19.0"
+ >
+ 0.19.0
+ </option>
+ <option value="0.18.1"
+ >
+ 0.18.1
+ </option>
+ <option value="0.18.0"
+ >
+ 0.18.0
+ </option>
+ <option value="0.17.0"
+ >
+ 0.17.0
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="db-migrations">DB Migrations</h1>
+
+<p>Changes to the DB schema should be made in the form of migrations. This ensures that all changes
+are applied correctly after a DB dump from a previous version is restored.</p>
+
+<p>DB migrations are managed through a system built on top of
+<a href="http://www.mybatis.org/migrations/">MyBatis Migrations</a>. The migrations are run automatically when
+a snapshot is restored, no manual interaction is required by cluster operators.</p>
+
+<h2 id="upgrades">Upgrades</h2>
+
+<p>When adding or altering tables or changing data, in addition to making to change in
+<a href="../../src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql">schema.sql</a>, a new
+migration class should be created under the org.apache.aurora.scheduler.storage.db.migration
+package. The class should implement the <a href="https://github.com/mybatis/migrations/blob/master/src/main/java/org/apache/ibatis/migration/MigrationScript.java">MigrationScript</a>
+interface (see <a href="https://github.com/apache/aurora/blob/rel/0.21.0/src/test/java/org/apache/aurora/scheduler/storage/db/testmigration/V001_TestMigration.java">V001_TestMigration</a>
+as an example). The upgrade and downgrade scripts are defined in this class. When restoring a
+snapshot the list of migrations on the classpath is compared to the list of applied changes in the
+DB. Any changes that have not yet been applied are executed and their downgrade script is stored
+alongside the changelog entry in the database to faciliate downgrades in the event of a rollback.</p>
+
+<h2 id="downgrades">Downgrades</h2>
+
+<p>If, while running migrations, a rollback is detected, i.e. a change exists in the DB changelog that
+does not exist on the classpath, the downgrade script associated with each affected change is
+applied.</p>
+
+<h2 id="baselines">Baselines</h2>
+
+<p>After enough time has passed (at least 1 official release), it should be safe to baseline migrations
+if desired. This can be accomplished by ensuring the changes from migrations have been applied to
+<a href="../../src/main/resources/org/apache/aurora/scheduler/storage/db/schema.sql">schema.sql</a> and then
+removing the corresponding migration classes and adding a migration to remove the changelog entries.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.21.0/development/design-documents/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.21.0/development/design-documents/index.html?rev=1840515&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.21.0/development/design-documents/index.html (added)
+++ aurora/site/publish/documentation/0.21.0/development/design-documents/index.html Tue Sep 11 05:28:10 2018
@@ -0,0 +1,178 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/development/design-documents/'"
+ value="0.21.0">
+ <option value="0.21.0"
+ selected="selected">
+ 0.21.0
+ (latest)
+ </option>
+ <option value="0.20.0"
+ >
+ 0.20.0
+ </option>
+ <option value="0.19.1"
+ >
+ 0.19.1
+ </option>
+ <option value="0.19.0"
+ >
+ 0.19.0
+ </option>
+ <option value="0.18.1"
+ >
+ 0.18.1
+ </option>
+ <option value="0.18.0"
+ >
+ 0.18.0
+ </option>
+ <option value="0.17.0"
+ >
+ 0.17.0
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="design-documents">Design Documents</h1>
+
+<p>Since its inception as an Apache project, larger feature additions to the
+Aurora code base are discussed in form of design documents. Design documents
+are living documents until a consensus has been reached to implement a feature
+in the proposed form.</p>
+
+<p>Current and past documents:</p>
+
+<ul>
+<li><a href="../design/command-hooks/">Command Hooks for the Aurora Client</a></li>
+<li><a href="https://docs.google.com/document/d/19gV8Po6DIHO14tOC7Qouk8RnboY8UCfRTninwn_5-7c/edit">Dynamic Reservations</a></li>
+<li><a href="https://docs.google.com/document/d/1J9SIswRMpVKQpnlvJAMAJtKfPP7ZARFknuyXl-2aZ-M/edit">GPU Resources in Aurora</a></li>
+<li><a href="https://docs.google.com/document/d/1KOO0LC046k75TqQqJ4c0FQcVGbxvrn71E10wAjMorVY/edit">Health Checks for Updates</a></li>
+<li><a href="https://docs.google.com/document/d/1Fc_YhhV7fc4D9Xv6gJzpfooxbK4YWZcvzw6Bd3qVTL8/edit">JobUpdateDiff thrift API</a></li>
+<li><a href="https://docs.google.com/document/d/11_lAsYIRlD5ETRzF2eSd3oa8LXAHYFD8rSetspYXaf4/edit">REST API RFC</a></li>
+<li><a href="https://docs.google.com/document/d/1r1WCHgmPJp5wbrqSZLsgtxPNj3sULfHrSFmxp2GyPTo/edit">Revocable Mesos offers in Aurora</a></li>
+<li><a href="https://docs.google.com/document/d/111T09NBF2zjjl7HE95xglsDpRdKoZqhCRM5hHmOfTLA/edit?usp=sharing">Supporting the Mesos Universal Containerizer</a></li>
+<li><a href="https://docs.google.com/document/d/1erszT-HsWf1zCIfhbqHlsotHxWUvDyI2xUwNQQQxLgs/edit?usp=sharing">Tier Management In Apache Aurora</a></li>
+<li><a href="https://docs.google.com/document/d/12hr6GnUZU3mc7xsWRzMi3nQILGB-3vyUxvbG-6YmvdE/edit">Ubiquitous Jobs</a></li>
+<li><a href="https://docs.google.com/document/d/1fVHLt9AF-YbOCVCDMQmi5DATVusn-tqY8DldKbjVEm0/edit">Pluggable Scheduling</a></li>
+</ul>
+
+<p>Design documents can be found in the Aurora issue tracker via the query <a href="https://issues.apache.org/jira/browse/AURORA-1528?jql=project%20%3D%20AURORA%20AND%20text%20~%20%22docs.google.com%22%20ORDER%20BY%20created"><code>project = AURORA AND text ~ "docs.google.com" ORDER BY created</code></a>.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.21.0/development/design/command-hooks/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.21.0/development/design/command-hooks/index.html?rev=1840515&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.21.0/development/design/command-hooks/index.html (added)
+++ aurora/site/publish/documentation/0.21.0/development/design/command-hooks/index.html Tue Sep 11 05:28:10 2018
@@ -0,0 +1,257 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/development/design/command-hooks/'"
+ value="0.21.0">
+ <option value="0.21.0"
+ selected="selected">
+ 0.21.0
+ (latest)
+ </option>
+ <option value="0.20.0"
+ >
+ 0.20.0
+ </option>
+ <option value="0.19.1"
+ >
+ 0.19.1
+ </option>
+ <option value="0.19.0"
+ >
+ 0.19.0
+ </option>
+ <option value="0.18.1"
+ >
+ 0.18.1
+ </option>
+ <option value="0.18.0"
+ >
+ 0.18.0
+ </option>
+ <option value="0.17.0"
+ >
+ 0.17.0
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="command-hooks-for-the-aurora-client">Command Hooks for the Aurora Client</h1>
+
+<h2 id="introduction-motivation">Introduction/Motivation</h2>
+
+<p>We’ve got hooks in the client that surround API calls. These are
+pretty awkward, because they don’t correlate with user actions. For
+example, suppose we wanted a policy that said users weren’t allowed to
+kill all instances of a production job at once.</p>
+
+<p>Right now, all that we could hook would be the “killJob” api call. But
+kill (at least in newer versions of the client) normally runs in
+batches. If a user called killall, what we would see on the API level
+is a series of “killJob” calls, each of which specified a batch of
+instances. We woudn’t be able to distinguish between really killing
+all instances of a job (which is forbidden under this policy), and
+carefully killing in batches (which is permitted.) In each case, the
+hook would just see a series of API calls, and couldn’t find out what
+the actual command being executed was!</p>
+
+<p>For most policy enforcement, what we really want to be able to do is
+look at and vet the commands that a user is performing, not the API
+calls that the client uses to implement those commands.</p>
+
+<p>So I propose that we add a new kind of hooks, which surround noun/verb
+commands. A hook will register itself to handle a collection of (noun,
+verb) pairs. Whenever any of those noun/verb commands are invoked, the
+hooks methods will be called around the execution of the verb. A
+pre-hook will have the ability to reject a command, preventing the
+verb from being executed.</p>
+
+<h2 id="registering-hooks">Registering Hooks</h2>
+
+<p>These hooks will be registered via configuration plugins. A configuration plugin
+can register hooks using an API. Hooks registered this way are, effectively,
+hardwired into the client executable.</p>
+
+<p>The order of execution of hooks is unspecified: they may be called in
+any order. There is no way to guarantee that one hook will execute
+before some other hook.</p>
+
+<h3 id="global-hooks">Global Hooks</h3>
+
+<p>Commands registered by the python call are called <em>global</em> hooks,
+because they will run for all configurations, whether or not they
+specify any hooks in the configuration file.</p>
+
+<p>In the implementation, hooks are registered in the module
+<code>apache.aurora.client.cli.command_hooks</code>, using the class
+<code>GlobalCommandHookRegistry</code>. A global hook can be registered by calling
+<code>GlobalCommandHookRegistry.register_command_hook</code> in a configuration plugin.</p>
+
+<h3 id="the-api">The API</h3>
+<pre class="highlight objective_c"><code><span style="background-color: #f8f8f8">class</span> <span style="background-color: #f8f8f8">CommandHook</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">object</span><span style="background-color: #f8f8f8">)</span>
+ <span style="color: #000000;font-weight: bold">@property</span>
+ <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">name</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span>
+ <span style="color: #d14">"""Returns a name for the hook."</span>
+
+ <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">get_nouns</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span>
+ <span style="color: #d14">"""Return the nouns that have verbs that should invoke this hook."""</span>
+
+ <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">get_verbs</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">noun</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span>
+ <span style="color: #d14">"""Return the verbs for a particular noun that should invoke his hook."""</span>
+
+ <span style="color: #a61717;background-color: #e3d2d2">@abstractmethod</span>
+ <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">pre_command</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">noun</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">verb</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">context</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">commandline</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span>
+ <span style="color: #d14">"""Execute a hook before invoking a verb.</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> * noun: the noun being invoked.</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> * verb: the verb being invoked.</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> * context: the context object that will be used to invoke the verb.</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> The options object will be initialized before calling the hook</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> * commandline: the original argv collection used to invoke the client.</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> Returns: True if the command should be allowed to proceed; False if the command</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> should be rejected.</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> """</span>
+
+ <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">post_command</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">noun</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">verb</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">context</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">commandline</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">result</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span>
+ <span style="color: #d14">"""Execute a hook after invoking a verb.</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> * noun: the noun being invoked.</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> * verb: the verb being invoked.</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> * context: the context object that will be used to invoke the verb.</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> The options object will be initialized before calling the hook</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> * commandline: the original argv collection used to invoke the client.</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> * result: the result code returned by the verb.</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> Returns: nothing</span><span style="color: #a61717;background-color: #e3d2d2">
+</span><span style="color: #d14"> """</span>
+
+<span style="background-color: #f8f8f8">class</span> <span style="background-color: #f8f8f8">GlobalCommandHookRegistry</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">object</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span>
+ <span style="color: #a61717;background-color: #e3d2d2">@classmethod</span>
+ <span style="background-color: #f8f8f8">def</span> <span style="background-color: #f8f8f8">register_command_hook</span><span style="background-color: #f8f8f8">(</span><span style="background-color: #f8f8f8">self</span><span style="background-color: #f8f8f8">,</span> <span style="background-color: #f8f8f8">hook</span><span style="background-color: #f8f8f8">)</span><span style="color: #000000;font-weight: bold">:</span>
+ <span style="background-color: #f8f8f8">pass</span>
+</code></pre>
+
+<h3 id="skipping-hooks">Skipping Hooks</h3>
+
+<p>To skip a hook, a user uses a command-line option, <code>--skip-hooks</code>. The option can either
+specify specific hooks to skip, or “all”:</p>
+
+<ul>
+<li><code>aurora --skip-hooks=all job create east/bozo/devel/myjob</code> will create a job
+without running any hooks.</li>
+<li><code>aurora --skip-hooks=test,iq create east/bozo/devel/myjob</code> will create a job,
+and will skip only the hooks named “test” and “iq”.</li>
+</ul>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.21.0/development/scheduler/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.21.0/development/scheduler/index.html?rev=1840515&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.21.0/development/scheduler/index.html (added)
+++ aurora/site/publish/documentation/0.21.0/development/scheduler/index.html Tue Sep 11 05:28:10 2018
@@ -0,0 +1,262 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/development/scheduler/'"
+ value="0.21.0">
+ <option value="0.21.0"
+ selected="selected">
+ 0.21.0
+ (latest)
+ </option>
+ <option value="0.20.0"
+ >
+ 0.20.0
+ </option>
+ <option value="0.19.1"
+ >
+ 0.19.1
+ </option>
+ <option value="0.19.0"
+ >
+ 0.19.0
+ </option>
+ <option value="0.18.1"
+ >
+ 0.18.1
+ </option>
+ <option value="0.18.0"
+ >
+ 0.18.0
+ </option>
+ <option value="0.17.0"
+ >
+ 0.17.0
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="developing-the-aurora-scheduler">Developing the Aurora Scheduler</h1>
+
+<p>The Aurora scheduler is written in Java code and built with <a href="http://gradle.org">Gradle</a>.</p>
+
+<h1 id="prerequisite">Prerequisite</h1>
+
+<p>When using Apache Aurora checked out from the source repository or the binary
+distribution, the Gradle wrapper and JavaScript dependencies are provided.
+However, you need to manually install them when using the source release
+downloads:</p>
+
+<ol>
+<li>Install Gradle following the instructions on the <a href="http://gradle.org">Gradle web site</a></li>
+<li><p>From the root directory of the Apache Aurora project generate the Gradle
+wrapper by running:</p>
+
+<p>gradle wrapper</p></li>
+</ol>
+
+<h1 id="getting-started">Getting Started</h1>
+
+<p>You will need Java 8 installed and on your <code>PATH</code> or unzipped somewhere with <code>JAVA_HOME</code> set. Then</p>
+<pre class="highlight plaintext"><code>./gradlew tasks
+</code></pre>
+
+<p>will bootstrap the build system and show available tasks. This can take a while the first time you
+run it but subsequent runs will be much faster due to cached artifacts.</p>
+
+<h2 id="running-the-tests">Running the Tests</h2>
+
+<p>Aurora has a comprehensive unit test suite. To run the tests use</p>
+<pre class="highlight plaintext"><code>./gradlew build
+</code></pre>
+
+<p>Gradle will only re-run tests when dependencies of them have changed. To force a re-run of all
+tests use</p>
+<pre class="highlight plaintext"><code>./gradlew clean build
+</code></pre>
+
+<h2 id="running-the-build-with-code-quality-checks">Running the build with code quality checks</h2>
+
+<p>To speed up development iteration, the plain gradle commands will not run static analysis tools.
+However, you should run these before posting a review diff, and <strong>always</strong> run this before pushing a
+commit to origin/master.</p>
+<pre class="highlight plaintext"><code>./gradlew build -Pq
+</code></pre>
+
+<h2 id="running-integration-tests">Running integration tests</h2>
+
+<p>To run the same tests that are run in the Apache Aurora continuous integration
+environment:</p>
+<pre class="highlight plaintext"><code>./build-support/jenkins/build.sh
+</code></pre>
+
+<p>In addition, there is an end-to-end test that runs a suite of aurora commands
+using a virtual cluster:</p>
+<pre class="highlight plaintext"><code>./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh
+</code></pre>
+
+<h2 id="creating-a-bundle-for-deployment">Creating a bundle for deployment</h2>
+
+<p>Gradle can create a zip file containing Aurora, all of its dependencies, and a launch script with</p>
+<pre class="highlight plaintext"><code>./gradlew distZip
+</code></pre>
+
+<p>or a tar file containing the same files with</p>
+<pre class="highlight plaintext"><code>./gradlew distTar
+</code></pre>
+
+<p>The output file will be written to <code>dist/distributions/aurora-scheduler.zip</code> or
+<code>dist/distributions/aurora-scheduler.tar</code>.</p>
+
+<h1 id="developing-aurora-java-code">Developing Aurora Java code</h1>
+
+<h2 id="setting-up-an-ide">Setting up an IDE</h2>
+
+<p>Gradle can generate project files for your IDE. To generate an IntelliJ IDEA project run</p>
+<pre class="highlight plaintext"><code>./gradlew idea
+</code></pre>
+
+<p>and import the generated <code>aurora.ipr</code> file.</p>
+
+<h2 id="adding-or-upgrading-a-dependency">Adding or Upgrading a Dependency</h2>
+
+<p>New dependencies can be added from Maven central by adding a <code>compile</code> dependency to <code>build.gradle</code>.
+For example, to add a dependency on <code>com.example</code>’s <code>example-lib</code> 1.0 add this block:</p>
+<pre class="highlight plaintext"><code>compile 'com.example:example-lib:1.0'
+</code></pre>
+
+<p>NOTE: Anyone thinking about adding a new dependency should first familiarize themselves with the
+Apache Foundation’s third-party licensing
+<a href="http://www.apache.org/legal/resolved.html#category-x">policy</a>.</p>
+
+<h1 id="developing-the-aurora-build-system">Developing the Aurora Build System</h1>
+
+<h2 id="bootstrapping-gradle">Bootstrapping Gradle</h2>
+
+<p>The following files were autogenerated by <code>gradle wrapper</code> using gradle’s
+<a href="http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.wrapper.Wrapper.html">Wrapper</a> plugin and
+should not be modified directly:</p>
+<pre class="highlight plaintext"><code>./gradlew
+./gradlew.bat
+./gradle/wrapper/gradle-wrapper.jar
+./gradle/wrapper/gradle-wrapper.properties
+</code></pre>
+
+<p>To upgrade Gradle unpack the new version somewhere, run <code>/path/to/new/gradle wrapper</code> in the
+repository root and commit the changed files.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.21.0/development/thermos/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.21.0/development/thermos/index.html?rev=1840515&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.21.0/development/thermos/index.html (added)
+++ aurora/site/publish/documentation/0.21.0/development/thermos/index.html Tue Sep 11 05:28:10 2018
@@ -0,0 +1,273 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/development/thermos/'"
+ value="0.21.0">
+ <option value="0.21.0"
+ selected="selected">
+ 0.21.0
+ (latest)
+ </option>
+ <option value="0.20.0"
+ >
+ 0.20.0
+ </option>
+ <option value="0.19.1"
+ >
+ 0.19.1
+ </option>
+ <option value="0.19.0"
+ >
+ 0.19.0
+ </option>
+ <option value="0.18.1"
+ >
+ 0.18.1
+ </option>
+ <option value="0.18.0"
+ >
+ 0.18.0
+ </option>
+ <option value="0.17.0"
+ >
+ 0.17.0
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<p>The Python components of Aurora are built using <a href="https://pantsbuild.github.io">Pants</a>.</p>
+
+<h1 id="python-build-conventions">Python Build Conventions</h1>
+
+<p>The Python code is laid out according to the following conventions:</p>
+
+<ol>
+<li><p>1 <code>BUILD</code> per 3rd level directory. For a list of current top-level packages run:</p>
+<pre class="highlight plaintext"><code>% find src/main/python -maxdepth 3 -mindepth 3 -type d |\
+while read dname; do echo $dname |\
+ sed 's@src/main/python/\(.*\)/\(.*\)/\(.*\).*@\1.\2.\3@'; done
+</code></pre></li>
+<li><p>Each <code>BUILD</code> file exports 1
+<a href="https://pantsbuild.github.io/build_dictionary.html#bdict_python_library"><code>python_library</code></a>
+that provides a
+<a href="https://pantsbuild.github.io/build_dictionary.html#setup_py"><code>setup_py</code></a>
+containing each
+<a href="https://pantsbuild.github.io/build_dictionary.html#python_binary"><code>python_binary</code></a>
+in the <code>BUILD</code> file, named the same as the directory it’s in so that it can be referenced
+without a ’:’ character. The <code>sources</code> field in the <code>python_library</code> will almost always be
+<code>rglobs('*.py')</code>.</p></li>
+<li><p>Other BUILD files may only depend on this single public <code>python_library</code>
+target. Any other target is considered a private implementation detail and
+should be prefixed with an <code>_</code>.</p></li>
+<li><p><code>python_binary</code> targets are always named the same as the exported console script.</p></li>
+<li><p><code>python_binary</code> targets must have identical <code>dependencies</code> to the <code>python_library</code> exported
+by the package and must use <code>entry_point</code>.</p>
+
+<p>The means a PEX file generated by pants will contain exactly the same files that will be
+available on the <code>PYTHONPATH</code> in the case of <code>pip install</code> of the corresponding library
+target. This will help our migration off of Pants in the future.</p></li>
+</ol>
+
+<h2 id="annotated-example-apache-thermos-runner">Annotated example - apache.thermos.runner</h2>
+<pre class="highlight plaintext"><code>% find src/main/python/apache/thermos/runner
+src/main/python/apache/thermos/runner
+src/main/python/apache/thermos/runner/__init__.py
+src/main/python/apache/thermos/runner/thermos_runner.py
+src/main/python/apache/thermos/runner/BUILD
+% cat src/main/python/apache/thermos/runner/BUILD
+# License boilerplate omitted
+import os
+
+
+# Private target so that a setup_py can exist without a circular dependency. Only targets within
+# this file should depend on this.
+python_library(
+ name = '_runner',
+ # The target covers every python file under this directory and subdirectories.
+ sources = rglobs('*.py'),
+ dependencies = [
+ '3rdparty/python:twitter.common.app',
+ '3rdparty/python:twitter.common.log',
+ # Source dependencies are always referenced without a ':'.
+ 'src/main/python/apache/thermos/common',
+ 'src/main/python/apache/thermos/config',
+ 'src/main/python/apache/thermos/core',
+ ],
+)
+
+# Binary target for thermos_runner.pex. Nothing should depend on this - it's only used as an
+# argument to ./pants binary.
+python_binary(
+ name = 'thermos_runner',
+ # Use entry_point, not source so the files used here are the same ones tests see.
+ entry_point = 'apache.thermos.bin.thermos_runner',
+ dependencies = [
+ # Notice that we depend only on the single private target from this BUILD file here.
+ ':_runner',
+ ],
+)
+
+# The public library that everyone importing the runner symbols uses.
+# The test targets and any other dependent source code should depend on this.
+python_library(
+ name = 'runner',
+ dependencies = [
+ # Again, notice that we depend only on the single private target from this BUILD file here.
+ ':_runner',
+ ],
+ # We always provide a setup_py. This will cause any dependee libraries to automatically
+ # reference this library in their requirements.txt rather than copy the source files into their
+ # sdist.
+ provides = setup_py(
+ # Conventionally named and versioned.
+ name = 'apache.thermos.runner',
+ version = open(os.path.join(get_buildroot(), '.auroraversion')).read().strip().upper(),
+ ).with_binaries({
+ # Every binary in this file should also be repeated here.
+ # Always use the dict-form of .with_binaries so that commands with dashes in their names are
+ # supported.
+ # The console script name is always the same as the PEX with .pex stripped.
+ 'thermos_runner': ':thermos_runner',
+ }),
+)
+</code></pre>
+
+<h1 id="thermos-test-resources">Thermos Test resources</h1>
+
+<p>The Aurora source repository and distributions contain several
+<a href="../../src/test/resources/org/apache/thermos/root/checkpoints">binary files</a> to
+qualify the backwards-compatibility of thermos with checkpoint data. Since
+thermos persists state to disk, to be read by the thermos observer), it is important that we have
+tests that prevent regressions affecting the ability to parse previously-written data.</p>
+
+<p>The files included represent persisted checkpoints that exercise different
+features of thermos. The existing files should not be modified unless
+we are accepting backwards incompatibility, such as with a major release.</p>
+
+<p>It is not practical to write source code to generate these files on the fly,
+as source would be vulnerable to drift (e.g. due to refactoring) in ways
+that would undermine the goal of ensuring backwards compatibility.</p>
+
+<p>The most common reason to add a new checkpoint file would be to provide
+coverage for new thermos features that alter the data format. This is
+accomplished by writing and running a
+<a href="../../reference/configuration/">job configuration</a> that exercises the feature, and
+copying the checkpoint file from the sandbox directory, by default this is
+<code>/var/run/thermos/checkpoints/<aurora task id></code>.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.21.0/development/thrift/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.21.0/development/thrift/index.html?rev=1840515&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.21.0/development/thrift/index.html (added)
+++ aurora/site/publish/documentation/0.21.0/development/thrift/index.html Tue Sep 11 05:28:10 2018
@@ -0,0 +1,208 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/development/thrift/'"
+ value="0.21.0">
+ <option value="0.21.0"
+ selected="selected">
+ 0.21.0
+ (latest)
+ </option>
+ <option value="0.20.0"
+ >
+ 0.20.0
+ </option>
+ <option value="0.19.1"
+ >
+ 0.19.1
+ </option>
+ <option value="0.19.0"
+ >
+ 0.19.0
+ </option>
+ <option value="0.18.1"
+ >
+ 0.18.1
+ </option>
+ <option value="0.18.0"
+ >
+ 0.18.0
+ </option>
+ <option value="0.17.0"
+ >
+ 0.17.0
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="thrift">Thrift</h1>
+
+<p>Aurora uses <a href="https://thrift.apache.org/">Apache Thrift</a> for representing structured data in
+client/server RPC protocol as well as for internal data storage. While Thrift is capable of
+correctly handling additions and renames of the existing members, field removals must be done
+carefully to ensure backwards compatibility and provide predictable deprecation cycle. This
+document describes general guidelines for making Thrift schema changes to the existing fields in
+<a href="https://github.com/apache/aurora/blob/rel/0.21.0/api/src/main/thrift/org/apache/aurora/gen/api.thrift">api.thrift</a>.</p>
+
+<p>It is highly recommended to go through the
+<a href="http://diwakergupta.github.io/thrift-missing-guide/">Thrift: The Missing Guide</a> first to refresh on
+basic Thrift schema concepts.</p>
+
+<h2 id="checklist">Checklist</h2>
+
+<p>Every existing Thrift schema modification is unique in its requirements and must be analyzed
+carefully to identify its scope and expected consequences. The following checklist may help in that
+analysis:
+* Is this a new field/struct? If yes, go ahead
+* Is this a pure field/struct rename without any type/structure change? If yes, go ahead and rename
+* Anything else, read further to make sure your change is properly planned</p>
+
+<h2 id="deprecation-cycle">Deprecation cycle</h2>
+
+<p>Any time a breaking change (e.g.: field replacement or removal) is required, the following cycle
+must be followed:</p>
+
+<h3 id="vcurrent">vCurrent</h3>
+
+<p>Change is applied in a way that does not break scheduler/client with this version to
+communicate with scheduler/client from vCurrent-1.
+* Do not remove or rename the old field
+* Add a new field as an eventual replacement of the old one and implement a dual read/write
+anywhere the old field is used. If a thrift struct is mapped in the DB store make sure both columns
+are marked as <code>NOT NULL</code>
+* Check <a href="https://github.com/apache/aurora/blob/rel/0.21.0/api/src/main/thrift/org/apache/aurora/gen/storage.thrift">storage.thrift</a> to see if
+the affected struct is stored in Aurora scheduler storage. If so, it’s almost certainly also
+necessary to perform a <a href="../db-migration/">DB migration</a>.
+* Add a deprecation jira ticket into the vCurrent+1 release candidate
+* Add a TODO for the deprecated field mentioning the jira ticket</p>
+
+<h3 id="vcurrent-1">vCurrent+1</h3>
+
+<p>Finalize the change by removing the deprecated fields from the Thrift schema.
+* Drop any dual read/write routines added in the previous version
+* Remove thrift backfilling in scheduler
+* Remove the deprecated Thrift field</p>
+
+<h2 id="testing">Testing</h2>
+
+<p>It’s always advisable to test your changes in the local vagrant environment to build more
+confidence that you change is backwards compatible. It’s easy to simulate different
+client/scheduler versions by playing with <code>aurorabuild</code> command. See <a href="../../getting-started/vagrant/">this document</a>
+for more.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.21.0/development/ui/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.21.0/development/ui/index.html?rev=1840515&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.21.0/development/ui/index.html (added)
+++ aurora/site/publish/documentation/0.21.0/development/ui/index.html Tue Sep 11 05:28:10 2018
@@ -0,0 +1,198 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/development/ui/'"
+ value="0.21.0">
+ <option value="0.21.0"
+ selected="selected">
+ 0.21.0
+ (latest)
+ </option>
+ <option value="0.20.0"
+ >
+ 0.20.0
+ </option>
+ <option value="0.19.1"
+ >
+ 0.19.1
+ </option>
+ <option value="0.19.0"
+ >
+ 0.19.0
+ </option>
+ <option value="0.18.1"
+ >
+ 0.18.1
+ </option>
+ <option value="0.18.0"
+ >
+ 0.18.0
+ </option>
+ <option value="0.17.0"
+ >
+ 0.17.0
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="developing-the-aurora-scheduler-ui">Developing the Aurora Scheduler UI</h1>
+
+<h2 id="installing-bower-optional">Installing bower (optional)</h2>
+
+<p>Third party JS libraries used in Aurora (located at 3rdparty/javascript/bower_components) are
+managed by bower, a JS dependency manager. Bower is only required if you plan to add, remove or
+update JS libraries. Bower can be installed using the following command:</p>
+<pre class="highlight plaintext"><code>npm install -g bower
+</code></pre>
+
+<p>Bower depends on node.js and npm. The easiest way to install node on a mac is via brew:</p>
+<pre class="highlight plaintext"><code>brew install node
+</code></pre>
+
+<p>For more node.js installation options refer to <a href="https://github.com/joyent/node/wiki/Installation">https://github.com/joyent/node/wiki/Installation</a>.</p>
+
+<p>More info on installing and using bower can be found at: <a href="http://bower.io/">http://bower.io/</a>. Once installed, you can
+use the following commands to view and modify the bower repo at
+3rdparty/javascript/bower_components</p>
+<pre class="highlight plaintext"><code>bower list
+bower install <library name>
+bower remove <library name>
+bower update <library name>
+bower help
+</code></pre>
+
+<h2 id="faster-iteration-in-vagrant">Faster Iteration in Vagrant</h2>
+
+<p>The scheduler serves UI assets from the classpath. For production deployments this means the assets
+are served from within a jar. However, for faster development iteration, the vagrant image is
+configured to add the <code>scheduler</code> subtree of <code>/vagrant/dist/resources/main</code> to the head of
+<code>CLASSPATH</code>. This path is configured as a shared filesystem to the path on the host system where
+your Aurora repository lives. This means that any updates under <code>dist/resources/main/scheduler</code> in
+your checkout will be reflected immediately in the UI served from within the vagrant image.</p>
+
+<p>The one caveat to this is that this path is under <code>dist</code> not <code>src</code>. This is because the assets must
+be processed by gradle before they can be served. So, unfortunately, you cannot just save your local
+changes and see them reflected in the UI, you must first run <code>./gradlew processResources</code>. This is
+less than ideal, but better than having to restart the scheduler after every change. Additionally,
+gradle makes this process somewhat easier with the use of the <code>--continuous</code> flag. If you run:
+<code>./gradlew processResources --continuous</code> gradle will monitor the filesystem for changes and run the
+task automatically as necessary. This doesn’t quite provide hot-reload capabilities, but it does
+allow for <5s from save to changes being visibile in the UI with no further action required on the
+part of the developer.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.21.0/features/constraints/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.21.0/features/constraints/index.html?rev=1840515&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.21.0/features/constraints/index.html (added)
+++ aurora/site/publish/documentation/0.21.0/features/constraints/index.html Tue Sep 11 05:28:10 2018
@@ -0,0 +1,268 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/constraints/'"
+ value="0.21.0">
+ <option value="0.21.0"
+ selected="selected">
+ 0.21.0
+ (latest)
+ </option>
+ <option value="0.20.0"
+ >
+ 0.20.0
+ </option>
+ <option value="0.19.1"
+ >
+ 0.19.1
+ </option>
+ <option value="0.19.0"
+ >
+ 0.19.0
+ </option>
+ <option value="0.18.1"
+ >
+ 0.18.1
+ </option>
+ <option value="0.18.0"
+ >
+ 0.18.0
+ </option>
+ <option value="0.17.0"
+ >
+ 0.17.0
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="scheduling-constraints">Scheduling Constraints</h1>
+
+<p>By default, Aurora will pick any random agent with sufficient resources
+in order to schedule a task. This scheduling choice can be further
+restricted with the help of constraints.</p>
+
+<h2 id="mesos-attributes">Mesos Attributes</h2>
+
+<p>Data centers are often organized with hierarchical failure domains. Common failure domains
+include hosts, racks, rows, and PDUs. If you have this information available, it is wise to tag
+the Mesos agent with them as
+<a href="https://mesos.apache.org/documentation/attributes-resources/">attributes</a>.</p>
+
+<p>The Mesos agent <code>--attributes</code> command line argument can be used to mark agents with
+static key/value pairs, so called attributes (not to be confused with <code>--resources</code>, which are
+dynamic and accounted).</p>
+
+<p>For example, consider the host <code>cluster1-aaa-03-sr2</code> and its following attributes (given in
+key:value format): <code>host:cluster1-aaa-03-sr2</code> and <code>rack:aaa</code>.</p>
+
+<p>Aurora makes these attributes available for matching with scheduling constraints.</p>
+
+<h2 id="limit-constraints">Limit Constraints</h2>
+
+<p>Limit constraints allow to control machine diversity using constraints. The below
+constraint ensures that no more than two instances of your job may run on a single host.
+Think of this as a “group by” limit.</p>
+<pre class="highlight plaintext"><code>Service(
+ name = 'webservice',
+ role = 'www-data',
+ constraints = {
+ 'host': 'limit:2',
+ }
+ ...
+)
+</code></pre>
+
+<p>Likewise, you can use constraints to control rack diversity, e.g. at
+most one task per rack:</p>
+<pre class="highlight plaintext"><code>constraints = {
+ 'rack': 'limit:1',
+}
+</code></pre>
+
+<p>Use these constraints sparingly as they can dramatically reduce Tasks’ schedulability.
+Further details are available in the reference documentation on
+<a href="../../reference/configuration/#specifying-scheduling-constraints">Scheduling Constraints</a>.</p>
+
+<h2 id="value-constraints">Value Constraints</h2>
+
+<p>Value constraints can be used to express that a certain attribute with a certain value
+should be present on a Mesos agent. For example, the following job would only be
+scheduled on nodes that claim to have an <code>SSD</code> as their disk.</p>
+<pre class="highlight plaintext"><code>Service(
+ name = 'webservice',
+ role = 'www-data',
+ constraints = {
+ 'disk': 'SSD',
+ }
+ ...
+)
+</code></pre>
+
+<p>Further details are available in the reference documentation on
+<a href="../../reference/configuration/#specifying-scheduling-constraints">Scheduling Constraints</a>.</p>
+
+<h2 id="running-stateful-services">Running stateful services</h2>
+
+<p>Aurora is best suited to run stateless applications, but it also accommodates for stateful services
+like databases, or services that otherwise need to always run on the same machines.</p>
+
+<h3 id="dedicated-attribute">Dedicated attribute</h3>
+
+<p>Most of the Mesos attributes arbitrary and available for custom use. There is one exception,
+though: the <code>dedicated</code> attribute. Aurora treats this specially, and only allows matching jobs to
+run on these machines, and will only schedule matching jobs on these machines.</p>
+
+<h4 id="syntax">Syntax</h4>
+
+<p>The dedicated attribute has semantic meaning. The format is <code>$role(/.*)?</code>. When a job is created,
+the scheduler requires that the <code>$role</code> component matches the <code>role</code> field in the job
+configuration, and will reject the job creation otherwise. The remainder of the attribute is
+free-form. We’ve developed the idiom of formatting this attribute as <code>$role/$job</code>, but do not
+enforce this. For example: a job <code>devcluster/www-data/prod/hello</code> with a dedicated constraint set as
+<code>www-data/web.multi</code> will have its tasks scheduled only on Mesos agents configured with:
+<code>--attributes=dedicated:www-data/web.multi</code>.</p>
+
+<p>A wildcard (<code>*</code>) may be used for the role portion of the dedicated attribute, which will allow any
+owner to elect for a job to run on the host(s). For example: tasks from both
+<code>devcluster/www-data/prod/hello</code> and <code>devcluster/vagrant/test/hello</code> with a dedicated constraint
+formatted as <code>*/web.multi</code> will be scheduled only on Mesos agents configured with
+<code>--attributes=dedicated:*/web.multi</code>. This may be useful when assembling a virtual cluster of
+machines sharing the same set of traits or requirements.</p>
+
+<h5 id="example">Example</h5>
+
+<p>Consider the following agent command line:</p>
+<pre class="highlight plaintext"><code>mesos-slave --attributes="dedicated:db_team/redis" ...
+</code></pre>
+
+<p>And this job configuration:</p>
+<pre class="highlight plaintext"><code>Service(
+ name = 'redis',
+ role = 'db_team',
+ constraints = {
+ 'dedicated': 'db_team/redis'
+ }
+ ...
+)
+</code></pre>
+
+<p>The job configuration is indicating that it should only be scheduled on agents with the attribute
+<code>dedicated:db_team/redis</code>. Additionally, Aurora will prevent any tasks that do <em>not</em> have that
+constraint from running on those agents.</p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>
Added: aurora/site/publish/documentation/0.21.0/features/containers/index.html
URL: http://svn.apache.org/viewvc/aurora/site/publish/documentation/0.21.0/features/containers/index.html?rev=1840515&view=auto
==============================================================================
--- aurora/site/publish/documentation/0.21.0/features/containers/index.html (added)
+++ aurora/site/publish/documentation/0.21.0/features/containers/index.html Tue Sep 11 05:28:10 2018
@@ -0,0 +1,280 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_dkbkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+ <div class="container-fluid">
+ <div class="container content">
+ <div class="col-md-12 documentation">
+<h5 class="page-header text-uppercase">Documentation
+<select onChange="window.location.href='/documentation/' + this.value + '/features/containers/'"
+ value="0.21.0">
+ <option value="0.21.0"
+ selected="selected">
+ 0.21.0
+ (latest)
+ </option>
+ <option value="0.20.0"
+ >
+ 0.20.0
+ </option>
+ <option value="0.19.1"
+ >
+ 0.19.1
+ </option>
+ <option value="0.19.0"
+ >
+ 0.19.0
+ </option>
+ <option value="0.18.1"
+ >
+ 0.18.1
+ </option>
+ <option value="0.18.0"
+ >
+ 0.18.0
+ </option>
+ <option value="0.17.0"
+ >
+ 0.17.0
+ </option>
+ <option value="0.16.0"
+ >
+ 0.16.0
+ </option>
+ <option value="0.15.0"
+ >
+ 0.15.0
+ </option>
+ <option value="0.14.0"
+ >
+ 0.14.0
+ </option>
+ <option value="0.13.0"
+ >
+ 0.13.0
+ </option>
+ <option value="0.12.0"
+ >
+ 0.12.0
+ </option>
+ <option value="0.11.0"
+ >
+ 0.11.0
+ </option>
+ <option value="0.10.0"
+ >
+ 0.10.0
+ </option>
+ <option value="0.9.0"
+ >
+ 0.9.0
+ </option>
+ <option value="0.8.0"
+ >
+ 0.8.0
+ </option>
+ <option value="0.7.0-incubating"
+ >
+ 0.7.0-incubating
+ </option>
+ <option value="0.6.0-incubating"
+ >
+ 0.6.0-incubating
+ </option>
+ <option value="0.5.0-incubating"
+ >
+ 0.5.0-incubating
+ </option>
+</select>
+</h5>
+<h1 id="containers">Containers</h1>
+
+<p>Aurora supports several containerizers, notably the Mesos containerizer and the Docker
+containerizer. The Mesos containerizer uses native OS features directly to provide isolation between
+containers, while the Docker containerizer delegates container management to the Docker engine.</p>
+
+<p>The support for launching container images via both containerizers has to be
+<a href="../../operations/configuration/#containers">enabled by a cluster operator</a>.</p>
+
+<h2 id="mesos-containerizer">Mesos Containerizer</h2>
+
+<p>The Mesos containerizer is the native Mesos containerizer solution. It allows tasks to be
+run with an array of <a href="../resource-isolation/">pluggable isolators</a> and can launch tasks using
+<a href="https://github.com/docker/docker/blob/master/image/spec/v1.md">Docker</a> images,
+<a href="https://github.com/appc/spec/blob/master/SPEC.md">AppC</a> images, or directly on the agent host
+filesystem.</p>
+
+<p>The following example (available in our <a href="../../getting-started/vagrant/">Vagrant environment</a>)
+launches a hello world example within a <code>debian/jessie</code> Docker image:</p>
+<pre class="highlight plaintext"><code>$ cat /vagrant/examples/jobs/hello_docker_image.aurora
+hello_loop = Process(
+ name = 'hello',
+ cmdline = """
+ while true; do
+ echo hello world
+ sleep 10
+ done
+ """)
+
+task = Task(
+ processes = [hello_loop],
+ resources = Resources(cpu=1, ram=1*MB, disk=8*MB)
+)
+
+jobs = [
+ Service(
+ cluster = 'devcluster',
+ environment = 'devel',
+ role = 'www-data',
+ name = 'hello_docker_image',
+ task = task,
+ container = Mesos(image=DockerImage(name='debian', tag='jessie'))
+ )
+]
+</code></pre>
+
+<p>Docker and Appc images are designated using an appropriate <code>image</code> property of the <code>Mesos</code>
+configuration object. If either <code>container</code> or <code>image</code> is left unspecified, the host filesystem
+will be used. Further details of how to specify images can be found in the
+<a href="../../reference/configuration/#mesos-object">Reference Documentation</a>.</p>
+
+<p>By default, Aurora launches processes as the Linux user named like the used role (e.g. <code>www-data</code>
+in the example above). This user has to exist on the host filesystem. If it does not exist within
+the container image, it will be created automatically. Otherwise, this user and its primary group
+has to exist in the image with matching uid/gid.</p>
+
+<p>For more information on the Mesos containerizer filesystem, namespace, and isolator features, visit
+<a href="http://mesos.apache.org/documentation/latest/mesos-containerizer/">Mesos Containerizer</a> and
+<a href="http://mesos.apache.org/documentation/latest/container-image/">Mesos Container Images</a>.</p>
+
+<h2 id="docker-containerizer">Docker Containerizer</h2>
+
+<p>The Docker containerizer launches container images using the Docker engine. It may often provide
+more advanced features than the native Mesos containerizer, but has to be installed separately to
+Mesos on each agent host.</p>
+
+<p>Starting with the 0.17.0 release, <code>image</code> can be specified with a <code>{{docker.image[name][tag]}}</code> binder so that
+the tag can be resolved to a concrete image digest. This ensures that the job always uses the same image
+across restarts, even if the version identified by the tag has been updated, guaranteeing that only job
+updates can mutate configuration.</p>
+
+<p>Example (available in the <a href="../../getting-started/vagrant/">Vagrant environment</a>):</p>
+<pre class="highlight plaintext"><code>$ cat /vagrant/examples/jobs/hello_docker_engine.aurora
+hello_loop = Process(
+ name = 'hello',
+ cmdline = """
+ while true; do
+ echo hello world
+ sleep 10
+ done
+ """)
+
+task = Task(
+ processes = [hello_loop],
+ resources = Resources(cpu=1, ram=1*MB, disk=8*MB)
+)
+
+jobs = [
+ Service(
+ cluster = 'devcluster',
+ environment = 'devel',
+ role = 'www-data',
+ name = 'hello_docker',
+ task = task,
+ container = Docker(image = 'python:2.7')
+ ), Service(
+ cluster = 'devcluster',
+ environment = 'devel',
+ role = 'www-data',
+ name = 'hello_docker_engine_binding',
+ task = task,
+ container = Docker(image = '{{docker.image[library/python][2.7]}}')
+ )
+]
+</code></pre>
+
+<p>Note, this feature requires a v2 Docker registry. If using a private Docker registry its url
+must be specified in the <code>clusters.json</code> configuration file under the key <code>docker_registry</code>.
+If not specified <code>docker_registry</code> defaults to <code>https://registry-1.docker.io</code> (Docker Hub).</p>
+
+<p>Example:
+ # clusters.json
+ [{
+ “name”: “devcluster”,
+ …
+ “docker_registry”: “<a href="https://registry.example.com">https://registry.example.com</a>”
+ }]</p>
+
+<p>Details of how to use Docker via the Docker engine can be found in the
+<a href="../../reference/configuration/#docker-object">Reference Documentation</a>. Please note that in order to
+correctly execute processes inside a job, the Docker container must have Python 2.7 and potentitally
+further Mesos dependencies installed. This limitation does not hold for Docker containers used via
+the Mesos containerizer.</p>
+
+<p>For more information on launching Docker containers through the Docker containerizer, visit
+<a href="http://mesos.apache.org/documentation/latest/docker-containerizer/">Docker Containerizer</a></p>
+
+</div>
+
+ </div>
+ </div>
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">© 2014-2017 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+
+ </body>
+</html>