You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@aurora.apache.org by jf...@apache.org on 2014/12/17 19:30:38 UTC
[13/18] incubator-aurora-website git commit: Initial move of website
over to git
http://git-wip-us.apache.org/repos/asf/incubator-aurora-website/blob/c43a3a2d/publish/documentation/latest/clientv2/index.html
----------------------------------------------------------------------
diff --git a/publish/documentation/latest/clientv2/index.html b/publish/documentation/latest/clientv2/index.html
new file mode 100644
index 0000000..8609e8d
--- /dev/null
+++ b/publish/documentation/latest/clientv2/index.html
@@ -0,0 +1,491 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_white_bkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+ <div class="container-fluid">
+ <div class="container content">
+ <h1 id="aurora-client-v2">Aurora Client v2</h1>
+
+<h2 id="overview">Overview</h2>
+
+<p>Our goal is to replace the current Aurora command-line client. The
+current client suffers from an early monolithic structure, and a long
+development history of rapid unplanned evolution.</p>
+
+<p>In addition to its internal problems, the current Aurora client is
+confusing for users. There are several different kinds of objects
+manipulated by the Aurora command line, and the difference between
+them is often not clear. (What’s the difference between a job and a
+configuration?) For each type of object, there are different commands,
+and it’s hard to remember which command should be used for which kind
+of object.</p>
+
+<p>Instead of continuing to let the Aurora client develop and evolve
+randomly, it’s time to take a principled look at the Aurora command
+line, and figure out how to make our command line processing make
+sense. At the same time, the code needs to be cleaned up, and divided
+into small comprehensible units based on a plugin architecture.</p>
+
+<p>Doing this now will give us a more intuitive, consistent, and easy to
+use client, as well as a sound platform for future development.</p>
+
+<h2 id="goals">Goals</h2>
+
+<ul>
+<li>A command line tool for interacting with Aurora that is easy for
+users to understand.</li>
+<li>A noun/verb command model.</li>
+<li>A modular source-code architecture.</li>
+<li>Non-disruptive transition for users.</li>
+</ul>
+
+<h2 id="non-goals">Non-Goals</h2>
+
+<ul>
+<li>The most important non-goal is that we’re not trying to redesign the
+Aurora scheduler, the Aurora executor, or any of the peripheral tools
+that the Aurora command line interacts with; we only want to create a
+better command line client.</li>
+<li>We do not want to change thermos, mesos, hadoop, etc.</li>
+<li>We do not want to create new objects that users will work with to
+interact with Mesos or Aurora.</li>
+<li>We do not want to change Aurora job configuration files or file formats.</li>
+<li>We do not want to change the Aurora API.</li>
+<li>We don’t want to boil the ocean: there are many things that we could
+include in the scope of this project, but we don’t want to be
+distracted by re-implementing all of twitter.commons in order to
+create a perfect Aurora client.</li>
+</ul>
+
+<h2 id="background">Background</h2>
+
+<p>Aurora is a system that’s used to run and manage services and
+service-like jobs running in a datacenter. Aurora takes care of
+allocating resources in order to schedule and run jobs without
+requiring teams to manage dedicated hardware. The heart of Aurora is
+called the scheduler, and is responsible for finding and assigning
+resources to tasks.</p>
+
+<p>The Aurora scheduler provides a thrift API. The scheduler API is
+low-level and difficult to interact with. Users do not interact
+directly with the Aurora API; instead, they use a command-line tool,
+which provides a collection of easy-to-use commands. This command-line
+tool, in turn, talks to the scheduler API to launch and manage jobs in
+datacenter clusters. The command-line tool is called the Aurora
+client.</p>
+
+<p>The current implementation of the Aurora client is haphazard,
+and really needs to be cleaned up:</p>
+
+<ul>
+<li>The code is monolithic and hard to maintain. It’s implemented using
+<code>twitter.common.app</code>, which assumes that all of the command code lives
+in a single source file. To work around this, and allow some
+subdivision, it uses a hack of <code>twitter.common.app</code> to force
+registration of commands from multiple modules. It’s hard to
+understand, and hard to modify.</li>
+<li>The current code is very difficult to test. Because of the way it’s
+built, there is no consistent way of passing key application data
+around. As a result, each unit test of client operations needs a
+difficult-to-assemble custom setup of mock objects.</li>
+<li>The current code handles errors poorly, and it is difficult to
+fix. Many common errors produce unacceptable results. For example,
+issuing an unknown command generates an error message “main takes 0
+parameters but received 1”; passing an invalid parameter to other
+commands frequently produces a stack trace.</li>
+<li>The current command line is confusing for users. There are several
+different kinds of objects manipulated by the Aurora command line,
+and the difference between them is often not entirely clear. (What’s
+the difference between a job and a configuration?)
+For each type of object, there are different
+commands, and it’s frequently not clear just which command should be
+used for which object.</li>
+</ul>
+
+<p>Instead of continuing to let it develop and evolve randomly, it’s time
+to take a principled look at the Aurora command line, and figure out
+how to make command line processing make sense. At the same time, the
+code needs to be cleaned up, and divided into small comprehensible
+units based on a plugin architecture.</p>
+
+<h2 id="requirements">Requirements</h2>
+
+<p>Aurora is aimed at engineers who run jobs and services in a
+datacenter. As a result, the requirements for the aurora client are
+all engineering focused:</p>
+
+<ul>
+<li><strong>Consistency</strong>: commands should follow a consistent structure, so that
+users can apply knowledge and intuition gained from working with
+some aurora commands to new commands. This means that when commands
+can re-use the same options, they should; that objects should be
+referred to by consistent syntax throughout the tool.</li>
+<li><strong>Helpfulness</strong>: commands should be structured so that the system can
+generate helpful error messages. If a user just runs “aurora”, they
+should get a basic usage message. If they try to run an invalid
+command, they should get a message that the command is invalid, not
+a stack dump or “command main() takes 0 parameters but received
+2”. Commands should not generate extraneous output that obscures the
+key facts that the user needs to know, and the default behavior of
+commands should not generate outputs that will be routinely ignored
+by users.</li>
+<li><strong>Extensibility</strong>: it should be easy to plug in new commands,
+including custom commands, to adapt the Aurora client to new
+environments.</li>
+<li><strong>Script-friendly command output</strong>: every command should at least include
+an option that generates output that’s script-friendly. Scripts should be
+able to work with command-output without needing to do screen scraping.</li>
+<li><strong>Scalability</strong>: the tools should be usable for any foreseeable size
+of Aurora datacenters and machine clusters.</li>
+</ul>
+
+<h2 id="design-overview">Design Overview</h2>
+
+<p>The Aurora client will be reimplemented using a noun-verb model,
+similar to the cmdlet model used by Monad/Windows Powershell. Users
+will work by providing a noun for the type of object being operated
+on, and a verb for the specific operation being performed on the
+object, followed by parameters. For example, to create a job, the user
+would execute: “<code>aurora job create smfd/mchucarroll/devel/jobname
+job.aurora</code>”. The noun is <code>job</code> and the verb is <code>create</code>.</p>
+
+<p>The client will be implemented following that noun-verb
+convention. Each noun will be a separate component, which can be
+registered into the command-line framework. Each verb will be
+implemented by a class that registers with the appropriate noun. Nouns
+and verbs will each provide methods that add their command line
+options and parameters to the options parser, using the Python
+argparse library.</p>
+
+<h2 id="detailed-design">Detailed Design</h2>
+
+<h3 id="interface">Interface</h3>
+
+<p>In this section, we’ll walk through the types of objects that the
+client can manipulate, and the operations that need to be provided for
+each object. These form the primary interface that engineers will use
+to interact with Aurora.</p>
+
+<p>In the command-line, each of the object types will have an Aurora
+subcommand. The commands to manipulate the object type will follow the
+type. For example, here are several commands in the old syntax
+contrasted against the new noun/verb syntax.</p>
+
+<ul>
+<li>Get quota for a role:
+
+<ul>
+<li>Noun/Verb syntax: <code>aurora quota get west/www-data</code></li>
+<li>Old syntax: <code>aurora get_quota --cluster=smf1 www-data</code></li>
+</ul></li>
+<li>Create job:
+
+<ul>
+<li>Noun/Verb syntax: <code>aurora job create west/www-data/test/job job.aurora</code></li>
+<li>Old syntax: <code>aurora create west/www-data/test/job job.aurora</code></li>
+</ul></li>
+<li>Schedule a job to run at a specific interval:
+
+<ul>
+<li>Noun/verb: <code>aurora cron schedule east/www-data/test/job job.aurora</code></li>
+<li>Old: <code>aurora create east/www-data/test/job job.aurora</code></li>
+</ul></li>
+</ul>
+
+<p>As you can see in these examples, the new syntax is more consistent:
+you always specify the cluster where a command executes as part of an
+identifier, where in the old syntax, it was sometimes part of the
+jobkey and sometimes specified with a “–cluster” option.</p>
+
+<p>The new syntax is also more clear and explicit: even without knowing
+much about Aurora, it’s clear what objects each command is acting on,
+where in the old syntax, commands like “create” are unclear.</p>
+
+<h3 id="the-job-noun">The Job Noun</h3>
+
+<p>A job is a configured program ready to run in Aurora. A job is,
+conceptually, a task factory: when a job is submitted to the Aurora
+scheduler, it creates a collection of tasks. The job contains a
+complete description of everything it needs to create a collection of
+tasks. (Note that this subsumes “service” commands. A service is just
+a task whose configuration sets the is_service flag, so we don’t have
+separate commands for working with services.) Jobs are specified using
+<code>cluster/role/env/name</code> jobkey syntax.</p>
+
+<ul>
+<li><code>aurora job create *jobkey* *config*</code>: submits a job to a cluster, launching the task(s) specified by the job config.</li>
+<li><code>aurora job status *jobkey*</code>: query job status. Prints information about the job,
+whether it’s running, etc., to standard out. If jobkey includes
+globs, it should list all jobs that match the glob</li>
+<li><code>aurora job kill *jobkey*/*instanceids*</code>: kill/stop some of a jobs instances. This stops a job' tasks; if the job
+has service tasks, they’ll be disabled, so that they won’t restart.</li>
+<li><code>aurora job killall *jobkey*</code>: kill all of the instances of a job. This
+is distinct from the <em>kill</em> command as a safety measure: omitting the
+instances from a kill command shouldn’t result in destroying the entire job.</li>
+<li><code>aurora job restart *jobkey*</code>: conceptually, this will kill a job, and then
+launch it again. If the job does not exist, then fail with an error
+message. In fact, the underlying implementation does the
+kill/relaunch on a rolling basis - so it’s not an immediate kill of
+all shards/instances, followed by a delay as all instances relaunch,
+but rather a controlled gradual process.</li>
+<li><code>aurora job list *jobkey*</code>: list all jobs that match the jobkey spec that are
+registered with the scheduler. This will include both jobs that are
+currently running, and jobs that are scheduled to run at a later
+time. The job key can be partial: if it specifies cluster, all jobs
+on the cluster will be listed; cluster/role, all jobs running on the cluster under the role will be listed, etc.</li>
+</ul>
+
+<h2 id="the-schedule-noun-(cron)">The Schedule Noun (Cron)</h2>
+
+<p>Note (3/21/2014): The “cron” noun is <em>not</em> implemented yet.</p>
+
+<p>Cron is a scheduler adjunct that periodically runs a job on a
+schedule. The cron commands all manipulate cron schedule entries. The
+schedules are specified as a part of the job configuration.</p>
+
+<ul>
+<li><code>aurora cron schedule jobkey config</code>: schedule a job to run by cron. If a cron job already exists
+replaces its template with a new one.</li>
+<li><code>aurora cron deschedule jobkey</code>: removes a jobs entry from the cron schedule.</li>
+<li><code>aurora cron status jobkey</code>: query for a scheduled job’s status.</li>
+</ul>
+
+<h2 id="the-quota-noun">The Quota Noun</h2>
+
+<p>A quota is a data object maintained by the scheduler that specifies the maximum
+resources that may be consumed by jobs owned by a particular role. In the future,
+we may add new quota types. At some point, we’ll also probably add an administrators
+command to set quotas.</p>
+
+<ul>
+<li><code>aurora quota get *cluster/role*</code></li>
+</ul>
+
+<h2 id="implementation">Implementation</h2>
+
+<p>The current command line is monolithic. Every command on an Aurora
+object is a top-level command in the Aurora client. In the
+restructured command line, each of the primary object types
+manipulated by Aurora should have its own sub-command.</p>
+
+<ul>
+<li>Advantages of this approach:
+
+<ul>
+<li>Easier to detangle the command-line processing. The top-level
+command-processing will be a small set of subcommand
+processors. Option processing for each subcommand can be offloaded
+to a separate module.</li>
+<li>The aurora top-level help command will be much more
+comprehensible. Instead of giving a huge list of every possible
+command, it will present the list of top-level object types, and
+then users can request help on the commands for a specific type
+of object.</li>
+<li>The sub-commands can be separated into distinct command-line
+tools when appropriate.</li>
+</ul></li>
+</ul>
+
+<h3 id="command-structure-and-options-processing">Command Structure and Options Processing</h3>
+
+<p>The implementation will follow closely on Pants goals. Pants goals use
+a static registration system to add new subcommands. In pants, each
+goal command is an implementation of a command interface, and provides
+implementations of methods to register options and parameters, and to
+actually execute the command. In this design, commands are modular and
+easy to implement, debug, and combine in different ways.</p>
+
+<p>For the Aurora client, we plan to use a two-level variation of the
+basic concept from pants. At the top-level we will have nouns. A noun
+will define some common command-line parameters required by all of its
+verbs, and will provide a registration hook for attaching verbs. Nouns
+will be implemented as a subclass of a basic Noun type.</p>
+
+<p>Each verb will, similarly, be implemented as a subclass of Verb. Verbs
+will be able to specify command-line options and parameters.</p>
+
+<p>Both <code>Noun</code> and <code>Verb</code> will be subclasses of a common base-class <code>AuroraCommand</code>:</p>
+<pre class="highlight text">class AuroraCommand(object):
+ def get_options(self):
+ """Gets the set of command-line options objects for this command.
+ The result is a list of CommandOption objects.
+ """
+ pass
+
+ @property
+ def help(self):
+ """Returns the help message for this command"""
+
+ @property
+ def usage(self):
+ """Returns a short usage description of the command"""
+
+ @property
+ def name(self):
+ """Returns the command name"""
+</pre>
+<p>A command-line tool will be implemented as an instance of a <code>CommandLine</code>:</p>
+<pre class="highlight text">class CommandLine(object):
+ """The top-level object implementing a command-line application."""
+
+ @property
+ def name(self):
+ """Returns the name of this command-line tool"""
+
+ def print_out(self, str):
+ print(str)
+
+ def print_err(self, str):
+ print(str, file=sys.stderr)
+
+ def register_noun(self, noun):
+ """Adds a noun to the application"""
+
+ def register_plugin(self, plugin):
+ """Adds a configuration plugin to the system"""
+</pre>
+<p>Nouns are registered into a command-line using the <code>register_noun</code>
+method. They are weakly coupled to the application, making it easy to
+use a single noun in several different command-line tools. Nouns allow
+the registration of verbs using the <code>register_verb</code> method.</p>
+
+<p>When commands execute, they’re given an instance of a <em>context object</em>.
+The context object must be an instance of a subclass of <code>AuroraCommandContext</code>.
+Options, parameters, and IO are all accessed using the context object. The context
+is created dynamically by the noun object owning the verb being executed. Developers
+are strongly encouraged to implement custom contexts for their nouns, and move functionality
+shared by the noun’s verbs into the context object. The context interface is:</p>
+<pre class="highlight text">class Context(object):
+ class Error(Exception): pass
+
+ class ArgumentException(Error): pass
+
+ class CommandError(Error):
+
+ @classmethod
+ def exit(cls, code, msg):
+ """Exit the application with an error message"""
+ raise cls.CommandError(code, msg)
+
+ def print_out(self, msg, indent=0):
+ """Prints a message to standard out, with an indent"""
+
+ def print_err(self, msg, indent=0):
+ """Prints a message to standard err, with an indent"""
+</pre>
+<p>In addition to nouns and verbs, there’s one more kind of registerable
+component, called a <em>configuration plugin</em>. These objects add a set of
+command-line options that can be passed to <em>all</em> of the commands
+implemented in the tool. Before the command is executed, the
+configuration plugin will be invoked, and will process its
+command-line arguments. This is useful for general configuration
+changes, like establish a secure tunnel to talk to machines in a
+datacenter. (A useful way to think of a plugin is as something like an
+aspect that can be woven in to aurora to provide environment-specific
+configuration.) A configuration plugin is implemented as an instance
+of class <code>ConfigurationPlugin</code>, and registered with the
+<code>register_plugin</code> method of the <code>CommandLine</code> object. The interface of
+a plugin is:</p>
+<pre class="highlight text">class ConfigurationPlugin(object):
+ """A component that can be plugged in to a command-line."""
+
+ @abstractmethod
+ def get_options(self):
+ """Return the set of options processed by this plugin"""
+
+ @abstractmethod
+ def execute(self, context):
+ """Run the context/command line initialization code for this plugin."""
+</pre>
+<h3 id="command-execution">Command Execution</h3>
+
+<p>The options process and command execution is built as a facade over Python’s
+standard argparse. All of the actual argument processing is done by the
+argparse library.</p>
+
+<p>Once the options are processed, the framework will start to execute the command. Command execution consists of:</p>
+
+<h1 id="create-a-context-object.-the-framework-will-use-the-argparse-options-to-identify">Create a context object. The framework will use the argparse options to identify</h1>
+
+<p>which noun is being invoked, and will call that noun’s <code>create_context</code> method.
+ The argparse options object will be stored in the context.</p>
+
+<h1 id="execute-any-configuration-plugins.-before-any-command-is-invoked,-the-framework">Execute any configuration plugins. Before any command is invoked, the framework</h1>
+
+<p>will first iterate over all of the registered configuration plugins. For each
+ plugin, it will invoke the <code>execute</code> method.</p>
+
+<h1 id="the-noun-will-use-the-context-to-find-out-what-verb-is-being-invoked,-and-it-will">The noun will use the context to find out what verb is being invoked, and it will</h1>
+
+<p>then call that verb’s <code>execute</code> method.</p>
+
+<h1 id="the-command-will-exit.-its-return-code-will-be-whatever-was-returned-by-the-verb's">The command will exit. Its return code will be whatever was returned by the verb’s</h1>
+
+<p><code>execute</code> method.</p>
+
+<p>Commands are expected to return a code from a list of standard exit codes,
+which can be found in <code>src/main/python/apache/aurora/client/cli/__init__.py</code>.</p>
+
+ </div>
+ </div>
+
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Apache Aurora is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.</p>
+ <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+ </body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-aurora-website/blob/c43a3a2d/publish/documentation/latest/committers/index.html
----------------------------------------------------------------------
diff --git a/publish/documentation/latest/committers/index.html b/publish/documentation/latest/committers/index.html
new file mode 100644
index 0000000..bda3db4
--- /dev/null
+++ b/publish/documentation/latest/committers/index.html
@@ -0,0 +1,141 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_white_bkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+ <div class="container-fluid">
+ <div class="container content">
+ <h2 id="setting-up-your-email-account">Setting up your email account</h2>
+
+<p>Once your Apache ID has been set up you can configure your account and add ssh keys and setup an
+email forwarding address at</p>
+
+<p><a href="http://id.apache.org">http://id.apache.org</a></p>
+
+<p>Additional instructions for setting up your new committer email can be found at</p>
+
+<p><a href="http://www.apache.org/dev/user-email.html">http://www.apache.org/dev/user-email.html</a></p>
+
+<p>The recommended setup is to configure all services (mailing lists, JIRA, ReviewBoard) to send
+emails to your @apache.org email address.</p>
+
+<h2 id="creating-a-gpg-key-for-releases">Creating a gpg key for releases</h2>
+
+<p>In order to create a release candidate you will need a gpg key published to an external key server
+and that key will need to be added to our KEYS file as well.</p>
+
+<ol>
+<li><p>Create a key:</p>
+<pre class="highlight text"> gpg --gen-key
+</pre></li>
+<li><p>Add your gpg key to the Apache Aurora KEYS file:</p>
+<pre class="highlight text"> git clone https://git-wip-us.apache.org/repos/asf/incubator-aurora.git
+ (gpg --list-sigs <KEY ID> && gpg --armor --export <KEY ID>) >> KEYS
+ git add KEYS && git commit -m "Adding gpg key for <APACHE ID>"
+ ./rbt post -o -g
+</pre></li>
+<li><p>Publish the key to an external key server:</p>
+<pre class="highlight text"> gpg --keyserver pgp.mit.edu --send-keys <KEY ID>
+</pre></li>
+<li><p>Update the changes to the KEYS file to the Apache Aurora svn dist locations listed below:</p>
+<pre class="highlight text"> https://dist.apache.org/repos/dist/dev/incubator/aurora/KEYS
+ https://dist.apache.org/repos/dist/release/incubator/aurora/KEYS
+</pre></li>
+<li><p>Add your key to git config for use with the release scripts:</p>
+<pre class="highlight text"> git config --global user.signingkey <KEY ID>
+</pre></li>
+</ol>
+
+<h2 id="creating-a-release">Creating a release</h2>
+
+<p>The following will guide you through the steps to create a release candidate, vote, and finally an
+official Apache Aurora release. Before starting your gpg key should be in the KEYS file and you
+must have access to commit to the dist.a.o repositories.</p>
+
+<ol>
+<li><p>Ensure that all issues resolved for this release candidate are tagged with the correct Fix
+Version in Jira, the changelog script will use this to generate the CHANGELOG in step #2.</p></li>
+<li><p>Create a release candidate. This will automatically update the CHANGELOG and commit it, create a
+branch and update the current version within the trunk. To create a minor version update and publish
+it run</p>
+<pre class="highlight text"> ./build-support/release/release-candidate -l m -p
+</pre></li>
+<li><p>Update, if necessary, the draft email created from the <code>release-candidate</code> script in step #2 and
+send the [VOTE] email to the dev@ and private@ mailing lists. You can verify the release signature
+and checksums by running</p>
+<pre class="highlight text"> ./build-support/release/verify-release-candidate
+</pre></li>
+<li><p>Wait for the vote to complete. If the vote fails address any issues and go back to step #1 and
+run again, this time you will use the -r flag to increment the release candidate version. This will
+automatically clean up the release candidate rc0 branch and source distribution.</p>
+<pre class="highlight text"> ./build-support/release/release-candidate -l m -r 1 -p
+</pre></li>
+<li><p>Once the vote has successfully passed create the release</p>
+<pre class="highlight text"> ./build-support/release/release
+</pre></li>
+<li><p>Update the draft email created fom the <code>release</code> script in step #5 to include the Apache ID’s for
+all binding votes and send the [RESULT][VOTE] email to the dev@ and private@ mailing lists.</p></li>
+</ol>
+
+ </div>
+ </div>
+
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Apache Aurora is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.</p>
+ <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+ </body>
+</html>
\ No newline at end of file
http://git-wip-us.apache.org/repos/asf/incubator-aurora-website/blob/c43a3a2d/publish/documentation/latest/configuration-reference/index.html
----------------------------------------------------------------------
diff --git a/publish/documentation/latest/configuration-reference/index.html b/publish/documentation/latest/configuration-reference/index.html
new file mode 100644
index 0000000..d0e4473
--- /dev/null
+++ b/publish/documentation/latest/configuration-reference/index.html
@@ -0,0 +1,879 @@
+<!DOCTYPE html>
+<html lang="en">
+ <head>
+ <meta charset="utf-8">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <title>Apache Aurora</title>
+ <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.1/css/bootstrap.min.css">
+ <link href="/assets/css/main.css" rel="stylesheet">
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+
+ <div class="container-fluid section-header">
+ <div class="container">
+ <div class="nav nav-bar">
+ <a href="/"><img src="/assets/img/aurora_logo_white_bkg.svg" width="300" alt="Transparent Apache Aurora logo with dark background"/></a>
+ <ul class="nav navbar-nav navbar-right">
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/community/">Community</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/blog/">Blog</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+ <div class="container-fluid">
+ <div class="container content">
+ <h1 id="aurora-+-thermos-configuration-reference">Aurora + Thermos Configuration Reference</h1>
+
+<ul>
+<li><a href="#aurora--thermos-configuration-reference">Aurora + Thermos Configuration Reference</a></li>
+<li><a href="#introduction">Introduction</a></li>
+<li><a href="#process-schema">Process Schema</a>
+
+<ul>
+<li><a href="#process-objects">Process Objects</a>
+
+<ul>
+<li><a href="#name">name</a></li>
+<li><a href="#cmdline">cmdline</a></li>
+<li><a href="#max_failures">max_failures</a></li>
+<li><a href="#daemon">daemon</a></li>
+<li><a href="#ephemeral">ephemeral</a></li>
+<li><a href="#min_duration">min_duration</a></li>
+<li><a href="#final">final</a></li>
+</ul></li>
+</ul></li>
+<li><a href="#task-schema">Task Schema</a>
+
+<ul>
+<li><a href="#task-object">Task Object</a>
+
+<ul>
+<li><a href="#name-1">name</a></li>
+<li><a href="#processes">processes</a></li>
+<li><a href="#constraints">constraints</a></li>
+<li><a href="#resources">resources</a></li>
+<li><a href="#max_failures-1">max_failures</a></li>
+<li><a href="#max_concurrency">max_concurrency</a></li>
+<li><a href="#finalization_wait">finalization_wait</a></li>
+</ul></li>
+<li><a href="#constraint-object">Constraint Object</a></li>
+<li><a href="#resource-object">Resource Object</a></li>
+</ul></li>
+<li><a href="#job-schema">Job Schema</a>
+
+<ul>
+<li><a href="#job-objects">Job Objects</a></li>
+<li><a href="#services">Services</a></li>
+<li><a href="#updateconfig-objects">UpdateConfig Objects</a></li>
+<li><a href="#healthcheckconfig-objects">HealthCheckConfig Objects</a></li>
+<li><a href="#announcer-objects">Announcer Objects</a></li>
+</ul></li>
+<li><a href="#specifying-scheduling-constraints">Specifying Scheduling Constraints</a></li>
+<li><a href="#template-namespaces">Template Namespaces</a>
+
+<ul>
+<li><a href="#mesos-namespace">mesos Namespace</a></li>
+<li><a href="#thermos-namespace">thermos Namespace</a></li>
+</ul></li>
+<li><a href="#basic-examples">Basic Examples</a>
+
+<ul>
+<li><a href="#hello_worldaurora">hello_world.aurora</a></li>
+<li><a href="#environment-tailoring">Environment Tailoring</a>
+
+<ul>
+<li><a href="#hello_world_productionizedaurora">hello<em>world</em>productionized.aurora</a></li>
+</ul></li>
+</ul></li>
+</ul>
+
+<h1 id="introduction">Introduction</h1>
+
+<p>Don’t know where to start? The Aurora configuration schema is very
+powerful, and configurations can become quite complex for advanced use
+cases.</p>
+
+<p>For examples of simple configurations to get something up and running
+quickly, check out the <a href="/documentation/latest/tutorial/">Tutorial</a>. When you feel comfortable with the basics, move
+on to the <a href="/documentation/latest/configuration-tutorial/">Configuration Tutorial</a> for more in-depth coverage of
+configuration design.</p>
+
+<p>For additional basic configuration examples, see <a href="#BasicExamples">the end of this document</a>.</p>
+
+<h1 id="process-schema">Process Schema</h1>
+
+<p>Process objects consist of required <code>name</code> and <code>cmdline</code> attributes. You can customize Process
+behavior with its optional attributes. Remember, Processes are handled by Thermos.</p>
+
+<h3 id="process-objects">Process Objects</h3>
+
+<table><thead>
+<tr>
+<th><strong>Attribute Name</strong></th>
+<th style="text-align: center"><strong>Type</strong></th>
+<th><strong>Description</strong></th>
+</tr>
+</thead><tbody>
+<tr>
+<td><strong>name</strong></td>
+<td style="text-align: center">String</td>
+<td>Process name (Required)</td>
+</tr>
+<tr>
+<td><strong>cmdline</strong></td>
+<td style="text-align: center">String</td>
+<td>Command line (Required)</td>
+</tr>
+<tr>
+<td><strong>max_failures</strong></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum process failures (Default: 1)</td>
+</tr>
+<tr>
+<td><strong>daemon</strong></td>
+<td style="text-align: center">Boolean</td>
+<td>When True, this is a daemon process. (Default: False)</td>
+</tr>
+<tr>
+<td><strong>ephemeral</strong></td>
+<td style="text-align: center">Boolean</td>
+<td>When True, this is an ephemeral process. (Default: False)</td>
+</tr>
+<tr>
+<td><strong>min_duration</strong></td>
+<td style="text-align: center">Integer</td>
+<td>Minimum duration between process restarts in seconds. (Default: 15)</td>
+</tr>
+<tr>
+<td><strong>final</strong></td>
+<td style="text-align: center">Boolean</td>
+<td>When True, this process is a finalizing one that should run last. (Default: False)</td>
+</tr>
+</tbody></table>
+
+<h4 id="name">name</h4>
+
+<p>The name is any valid UNIX filename string (specifically no
+slashes, NULLs or leading periods). Within a Task object, each Process name
+must be unique.</p>
+
+<h4 id="cmdline">cmdline</h4>
+
+<p>The command line run by the process. The command line is invoked in a bash
+subshell, so can involve fully-blown bash scripts. However, nothing is
+supplied for command-line arguments so <code>$*</code> is unspecified.</p>
+
+<h4 id="max_failures">max_failures</h4>
+
+<p>The maximum number of failures (non-zero exit statuses) this process can
+have before being marked permanently failed and not retried. If a
+process permanently fails, Thermos looks at the failure limit of the task
+containing the process (usually 1) to determine if the task has
+failed as well.</p>
+
+<p>Setting <code>max_failures</code> to 0 makes the process retry
+indefinitely until it achieves a successful (zero) exit status.
+It retries at most once every <code>min_duration</code> seconds to prevent
+an effective denial of service attack on the coordinating Thermos scheduler.</p>
+
+<h4 id="daemon">daemon</h4>
+
+<p>By default, Thermos processes are non-daemon. If <code>daemon</code> is set to True, a
+successful (zero) exit status does not prevent future process runs.
+Instead, the process reinvokes after <code>min_duration</code> seconds.
+However, the maximum failure limit still applies. A combination of
+<code>daemon=True</code> and <code>max_failures=0</code> causes a process to retry
+indefinitely regardless of exit status. This should be avoided
+for very short-lived processes because of the accumulation of
+checkpointed state for each process run. When running in Mesos
+specifically, <code>max_failures</code> is capped at 100.</p>
+
+<h4 id="ephemeral">ephemeral</h4>
+
+<p>By default, Thermos processes are non-ephemeral. If <code>ephemeral</code> is set to
+True, the process' status is not used to determine if its containing task
+has completed. For example, consider a task with a non-ephemeral
+webserver process and an ephemeral logsaver process
+that periodically checkpoints its log files to a centralized data store.
+The task is considered finished once the webserver process has
+completed, regardless of the logsaver’s current status.</p>
+
+<h4 id="min_duration">min_duration</h4>
+
+<p>Processes may succeed or fail multiple times during a single task’s
+duration. Each of these is called a <em>process run</em>. <code>min_duration</code> is
+the minimum number of seconds the scheduler waits before running the
+same process.</p>
+
+<h4 id="final">final</h4>
+
+<p>Processes can be grouped into two classes: ordinary processes and
+finalizing processes. By default, Thermos processes are ordinary. They
+run as long as the task is considered healthy (i.e., no failure
+limits have been reached.) But once all regular Thermos processes
+finish or the task reaches a certain failure threshold, it
+moves into a “finalization” stage and runs all finalizing
+processes. These are typically processes necessary for cleaning up the
+task, such as log checkpointers, or perhaps e-mail notifications that
+the task completed.</p>
+
+<p>Finalizing processes may not depend upon ordinary processes or
+vice-versa, however finalizing processes may depend upon other
+finalizing processes and otherwise run as a typical process
+schedule.</p>
+
+<h1 id="task-schema">Task Schema</h1>
+
+<p>Tasks fundamentally consist of a <code>name</code> and a list of Process objects stored as the
+value of the <code>processes</code> attribute. Processes can be further constrained with
+<code>constraints</code>. By default, <code>name</code>’s value inherits from the first Process in the
+<code>processes</code> list, so for simple <code>Task</code> objects with one Process, <code>name</code>
+can be omitted. In Mesos, <code>resources</code> is also required.</p>
+
+<h3 id="task-object">Task Object</h3>
+
+<table><thead>
+<tr>
+<th><strong>param</strong></th>
+<th style="text-align: center"><strong>type</strong></th>
+<th><strong>description</strong></th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>name</code></td>
+<td style="text-align: center">String</td>
+<td>Process name (Required) (Default: <code>processes0.name</code>)</td>
+</tr>
+<tr>
+<td><code>processes</code></td>
+<td style="text-align: center">List of <code>Process</code> objects</td>
+<td>List of <code>Process</code> objects bound to this task. (Required)</td>
+</tr>
+<tr>
+<td><code>constraints</code></td>
+<td style="text-align: center">List of <code>Constraint</code> objects</td>
+<td>List of <code>Constraint</code> objects constraining processes.</td>
+</tr>
+<tr>
+<td><code>resources</code></td>
+<td style="text-align: center"><code>Resource</code> object</td>
+<td>Resource footprint. (Required)</td>
+</tr>
+<tr>
+<td><code>max_failures</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum process failures before being considered failed (Default: 1)</td>
+</tr>
+<tr>
+<td><code>max_concurrency</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of concurrent processes (Default: 0, unlimited concurrency.)</td>
+</tr>
+<tr>
+<td><code>finalization_wait</code></td>
+<td style="text-align: center">Integer</td>
+<td>Amount of time allocated for finalizing processes, in seconds. (Default: 30)</td>
+</tr>
+</tbody></table>
+
+<h4 id="name">name</h4>
+
+<p><code>name</code> is a string denoting the name of this task. It defaults to the name of the first Process in
+the list of Processes associated with the <code>processes</code> attribute.</p>
+
+<h4 id="processes">processes</h4>
+
+<p><code>processes</code> is an unordered list of <code>Process</code> objects. To constrain the order
+in which they run, use <code>constraints</code>.</p>
+
+<h5 id="constraints">constraints</h5>
+
+<p>A list of <code>Constraint</code> objects. Currently it supports only one type,
+the <code>order</code> constraint. <code>order</code> is a list of process names
+that should run in the order given. For example,</p>
+<pre class="highlight text"> process = Process(cmdline = "echo hello {{name}}")
+ task = Task(name = "echoes",
+ processes = [process(name = "jim"), process(name = "bob")],
+ constraints = [Constraint(order = ["jim", "bob"]))
+</pre>
+<p>Constraints can be supplied ad-hoc and in duplicate. Not all
+Processes need be constrained, however Tasks with cycles are
+rejected by the Thermos scheduler.</p>
+
+<p>Use the <code>order</code> function as shorthand to generate <code>Constraint</code> lists.
+The following:</p>
+<pre class="highlight text"> order(process1, process2)
+</pre>
+<p>is shorthand for</p>
+<pre class="highlight text"> [Constraint(order = [process1.name(), process2.name()])]
+</pre>
+<h4 id="resources">resources</h4>
+
+<p>Takes a <code>Resource</code> object, which specifies the amounts of CPU, memory, and disk space resources
+to allocate to the Task.</p>
+
+<h4 id="max_failures">max_failures</h4>
+
+<p><code>max_failures</code> is the number of times processes that are part of this
+Task can fail before the entire Task is marked for failure.</p>
+
+<p>For example:</p>
+<pre class="highlight text"> template = Process(max_failures=10)
+ task = Task(
+ name = "fail",
+ processes = [
+ template(name = "failing", cmdline = "exit 1"),
+ template(name = "succeeding", cmdline = "exit 0")
+ ],
+ max_failures=2)
+</pre>
+<p>The <code>failing</code> Process could fail 10 times before being marked as
+permanently failed, and the <code>succeeding</code> Process would succeed on the
+first run. The task would succeed despite only allowing for two failed
+processes. To be more specific, there would be 10 failed process runs
+yet 1 failed process.</p>
+
+<h4 id="max_concurrency">max_concurrency</h4>
+
+<p>For Tasks with a number of expensive but otherwise independent
+processes, you may want to limit the amount of concurrency
+the Thermos scheduler provides rather than artificially constraining
+it via <code>order</code> constraints. For example, a test framework may
+generate a task with 100 test run processes, but wants to run it on
+a machine with only 4 cores. You can limit the amount of parallelism to
+4 by setting <code>max_concurrency=4</code> in your task configuration.</p>
+
+<p>For example, the following task spawns 180 Processes (“mappers”)
+to compute individual elements of a 180 degree sine table, all dependent
+upon one final Process (“reducer”) to tabulate the results:</p>
+<pre class="highlight text">def make_mapper(id):
+ return Process(
+ name = "mapper%03d" % id,
+ cmdline = "echo 'scale=50;s(%d\*4\*a(1)/180)' | bc -l >
+ temp.sine_table.%03d" % (id, id))
+
+def make_reducer():
+ return Process(name = "reducer", cmdline = "cat temp.\* | nl \> sine\_table.txt
+ && rm -f temp.\*")
+
+processes = map(make_mapper, range(180))
+
+task = Task(
+ name = "mapreduce",
+ processes = processes + [make\_reducer()],
+ constraints = [Constraint(order = [mapper.name(), 'reducer']) for mapper
+ in processes],
+ max_concurrency = 8)
+</pre>
+<h4 id="finalization_wait">finalization_wait</h4>
+
+<p>Tasks have three active stages: <code>ACTIVE</code>, <code>CLEANING</code>, and <code>FINALIZING</code>. The
+<code>ACTIVE</code> stage is when ordinary processes run. This stage lasts as
+long as Processes are running and the Task is healthy. The moment either
+all Processes have finished successfully or the Task has reached a
+maximum Process failure limit, it goes into <code>CLEANING</code> stage and send
+SIGTERMs to all currently running Processes and their process trees.
+Once all Processes have terminated, the Task goes into <code>FINALIZING</code> stage
+and invokes the schedule of all Processes with the “final” attribute set to True.</p>
+
+<p>This whole process from the end of <code>ACTIVE</code> stage to the end of <code>FINALIZING</code>
+must happen within <code>finalization_wait</code> seconds. If it does not
+finish during that time, all remaining Processes are sent SIGKILLs
+(or if they depend upon uncompleted Processes, are
+never invoked.)</p>
+
+<p>Client applications with higher priority may force a shorter
+finalization wait (e.g. through parameters to <code>thermos kill</code>), so this
+is mostly a best-effort signal.</p>
+
+<h3 id="constraint-object">Constraint Object</h3>
+
+<p>Current constraint objects only support a single ordering constraint, <code>order</code>,
+which specifies its processes run sequentially in the order given. By
+default, all processes run in parallel when bound to a <code>Task</code> without
+ordering constraints.</p>
+
+<table><thead>
+<tr>
+<th>param</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td>order</td>
+<td style="text-align: center">List of String</td>
+<td>List of processes by name (String) that should be run serially.</td>
+</tr>
+</tbody></table>
+
+<h3 id="resource-object">Resource Object</h3>
+
+<p>Specifies the amount of CPU, Ram, and disk resources the task needs. See the
+<a href="/documentation/latest/resource-isolation/">Resource Isolation document</a> for suggested values and to understand how
+resources are allocated.</p>
+
+<table><thead>
+<tr>
+<th>param</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>cpu</code></td>
+<td style="text-align: center">Float</td>
+<td>Fractional number of cores required by the task.</td>
+</tr>
+<tr>
+<td><code>ram</code></td>
+<td style="text-align: center">Integer</td>
+<td>Bytes of RAM required by the task.</td>
+</tr>
+<tr>
+<td><code>disk</code></td>
+<td style="text-align: center">Integer</td>
+<td>Bytes of disk required by the task.</td>
+</tr>
+</tbody></table>
+
+<h1 id="job-schema">Job Schema</h1>
+
+<h3 id="job-objects">Job Objects</h3>
+
+<table><thead>
+<tr>
+<th>name</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>task</code></td>
+<td style="text-align: center">Task</td>
+<td>The Task object to bind to this job. Required.</td>
+</tr>
+<tr>
+<td><code>name</code></td>
+<td style="text-align: center">String</td>
+<td>Job name. (Default: inherited from the task attribute’s name)</td>
+</tr>
+<tr>
+<td><code>role</code></td>
+<td style="text-align: center">String</td>
+<td>Job role account. Required.</td>
+</tr>
+<tr>
+<td><code>cluster</code></td>
+<td style="text-align: center">String</td>
+<td>Cluster in which this job is scheduled. Required.</td>
+</tr>
+<tr>
+<td><code>environment</code></td>
+<td style="text-align: center">String</td>
+<td>Job environment, default <code>devel</code>. Must be one of <code>prod</code>, <code>devel</code>, <code>test</code> or <code>staging<number></code>.</td>
+</tr>
+<tr>
+<td><code>contact</code></td>
+<td style="text-align: center">String</td>
+<td>Best email address to reach the owner of the job. For production jobs, this is usually a team mailing list.</td>
+</tr>
+<tr>
+<td><code>instances</code></td>
+<td style="text-align: center">Integer</td>
+<td>Number of instances (sometimes referred to as replicas or shards) of the task to create. (Default: 1)</td>
+</tr>
+<tr>
+<td><code>cron_schedule</code></td>
+<td style="text-align: center">String</td>
+<td>Cron schedule in cron format. May only be used with non-service jobs. See <a href="/documentation/latest/cron-jobs/">Cron Jobs</a> for more information. Default: None (not a cron job.)</td>
+</tr>
+<tr>
+<td><code>cron_collision_policy</code></td>
+<td style="text-align: center">String</td>
+<td>Policy to use when a cron job is triggered while a previous run is still active. KILL<em>EXISTING Kill the previous run, and schedule the new run CANCEL</em>NEW Let the previous run continue, and cancel the new run. (Default: KILL_EXISTING)</td>
+</tr>
+<tr>
+<td><code>update_config</code></td>
+<td style="text-align: center"><code>update_config</code> object</td>
+<td>Parameters for controlling the rate and policy of rolling updates.</td>
+</tr>
+<tr>
+<td><code>constraints</code></td>
+<td style="text-align: center">dict</td>
+<td>Scheduling constraints for the tasks. See the section on the <a href="#Specifying-Scheduling-Constraints">constraint specification language</a></td>
+</tr>
+<tr>
+<td><code>service</code></td>
+<td style="text-align: center">Boolean</td>
+<td>If True, restart tasks regardless of success or failure. (Default: False)</td>
+</tr>
+<tr>
+<td><code>max_task_failures</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of failures after which the task is considered to have failed (Default: 1) Set to -1 to allow for infinite failures</td>
+</tr>
+<tr>
+<td><code>priority</code></td>
+<td style="text-align: center">Integer</td>
+<td>Preemption priority to give the task (Default 0). Tasks with higher priorities may preempt tasks at lower priorities.</td>
+</tr>
+<tr>
+<td><code>production</code></td>
+<td style="text-align: center">Boolean</td>
+<td>Whether or not this is a production task backed by quota (Default: False). Production jobs may preempt any non-production job, and may only be preempted by production jobs in the same role and of higher priority. To run jobs at this level, the job role must have the appropriate quota.</td>
+</tr>
+<tr>
+<td><code>health_check_config</code></td>
+<td style="text-align: center"><code>heath_check_config</code> object</td>
+<td>Parameters for controlling a task’s health checks via HTTP. Only used if a health port was assigned with a command line wildcard.</td>
+</tr>
+</tbody></table>
+
+<h3 id="services">Services</h3>
+
+<p>Jobs with the <code>service</code> flag set to True are called Services. The <code>Service</code>
+alias can be used as shorthand for <code>Job</code> with <code>service=True</code>.
+Services are differentiated from non-service Jobs in that tasks
+always restart on completion, whether successful or unsuccessful.
+Jobs without the service bit set only restart up to
+<code>max_task_failures</code> times and only if they terminated unsuccessfully
+either due to human error or machine failure.</p>
+
+<h3 id="updateconfig-objects">UpdateConfig Objects</h3>
+
+<p>Parameters for controlling the rate and policy of rolling updates.</p>
+
+<table><thead>
+<tr>
+<th>object</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>batch_size</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of shards to be updated in one iteration (Default: 1)</td>
+</tr>
+<tr>
+<td><code>restart_threshold</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of seconds before a shard must move into the <code>RUNNING</code> state before considered a failure (Default: 60)</td>
+</tr>
+<tr>
+<td><code>watch_secs</code></td>
+<td style="text-align: center">Integer</td>
+<td>Minimum number of seconds a shard must remain in <code>RUNNING</code> state before considered a success (Default: 45)</td>
+</tr>
+<tr>
+<td><code>max_per_shard_failures</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of restarts per shard during update. Increments total failure count when this limit is exceeded. (Default: 0)</td>
+</tr>
+<tr>
+<td><code>max_total_failures</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of shard failures to be tolerated in total during an update. Cannot be greater than or equal to the total number of tasks in a job. (Default: 0)</td>
+</tr>
+</tbody></table>
+
+<h3 id="healthcheckconfig-objects">HealthCheckConfig Objects</h3>
+
+<p>Parameters for controlling a task’s health checks via HTTP.</p>
+
+<table><thead>
+<tr>
+<th>object</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>initial_interval_secs</code></td>
+<td style="text-align: center">Integer</td>
+<td>Initial delay for performing an HTTP health check. (Default: 15)</td>
+</tr>
+<tr>
+<td><code>interval_secs</code></td>
+<td style="text-align: center">Integer</td>
+<td>Interval on which to check the task’s health via HTTP. (Default: 10)</td>
+</tr>
+<tr>
+<td><code>timeout_secs</code></td>
+<td style="text-align: center">Integer</td>
+<td>HTTP request timeout. (Default: 1)</td>
+</tr>
+<tr>
+<td><code>max_consecutive_failures</code></td>
+<td style="text-align: center">Integer</td>
+<td>Maximum number of consecutive failures that tolerated before considering a task unhealthy (Default: 0)</td>
+</tr>
+</tbody></table>
+
+<h3 id="announcer-objects">Announcer Objects</h3>
+
+<p>If the <code>announce</code> field in the Job configuration is set, each task will be
+registered in the ServerSet <code>/aurora/role/environment/jobname</code> in the
+zookeeper ensemble configured by the executor. If no Announcer object is specified,
+no announcement will take place. For more information about ServerSets, see the <a href="/documentation/latest/user-guide/">User Guide</a>.</p>
+
+<table><thead>
+<tr>
+<th>object</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>primary_port</code></td>
+<td style="text-align: center">String</td>
+<td>Which named port to register as the primary endpoint in the ServerSet (Default: <code>http</code>)</td>
+</tr>
+<tr>
+<td><code>portmap</code></td>
+<td style="text-align: center">dict</td>
+<td>A mapping of additional endpoints to announced in the ServerSet (Default: <code>{ 'aurora': '{{primary_port}}' }</code>)</td>
+</tr>
+</tbody></table>
+
+<h3 id="port-aliasing-with-the-announcer-portmap">Port aliasing with the Announcer <code>portmap</code></h3>
+
+<p>The primary endpoint registered in the ServerSet is the one allocated to the port
+specified by the <code>primary_port</code> in the <code>Announcer</code> object, by default
+the <code>http</code> port. This port can be referenced from anywhere within a configuration
+as <code>{{thermos.ports[http]}}</code>.</p>
+
+<p>Without the port map, each named port would be allocated a unique port number.
+The <code>portmap</code> allows two different named ports to be aliased together. The default
+<code>portmap</code> aliases the <code>aurora</code> port (i.e. <code>{{thermos.ports[aurora]}}</code>) to
+the <code>http</code> port. Even though the two ports can be referenced independently,
+only one port is allocated by Mesos. Any port referenced in a <code>Process</code> object
+but which is not in the portmap will be allocated dynamically by Mesos and announced as well.</p>
+
+<p>It is possible to use the portmap to alias names to static port numbers, e.g.
+<code>{'http': 80, 'https': 443, 'aurora': 'http'}</code>. In this case, referencing
+<code>{{thermos.ports[aurora]}}</code> would look up <code>{{thermos.ports[http]}}</code> then
+find a static port 80. No port would be requested of or allocated by Mesos.</p>
+
+<p>Static ports should be used cautiously as Aurora does nothing to prevent two
+tasks with the same static port allocations from being co-scheduled.
+External constraints such as slave attributes should be used to enforce such
+guarantees should they be needed.</p>
+
+<h1 id="specifying-scheduling-constraints">Specifying Scheduling Constraints</h1>
+
+<p>Most users will not need to specify constraints explicitly, as the
+scheduler automatically inserts reasonable defaults that attempt to
+ensure reliability without impacting schedulability. For example, the
+scheduler inserts a <code>host: limit:1</code> constraint, ensuring
+that your shards run on different physical machines. Please do not
+set this field unless you are sure of what you are doing.</p>
+
+<p>In the <code>Job</code> object there is a map <code>constraints</code> from String to String
+allowing the user to tailor the schedulability of tasks within the job.</p>
+
+<p>Each slave in the cluster is assigned a set of string-valued
+key/value pairs called attributes. For example, consider the host
+<code>cluster1-aaa-03-sr2</code> and its following attributes (given in key:value
+format): <code>host:cluster1-aaa-03-sr2</code> and <code>rack:aaa</code>.</p>
+
+<p>The constraint map’s key value is the attribute name in which we
+constrain Tasks within our Job. The value is how we constrain them.
+There are two types of constraints: <em>limit constraints</em> and <em>value
+constraints</em>.</p>
+
+<table><thead>
+<tr>
+<th>constraint</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td>Limit</td>
+<td>A string that specifies a limit for a constraint. Starts with <code>'limit:</code> followed by an Integer and closing single quote, such as <code>'limit:1'</code>.</td>
+</tr>
+<tr>
+<td>Value</td>
+<td>A string that specifies a value for a constraint. To include a list of values, separate the values using commas. To negate the values of a constraint, start with a <code>!</code> <code>.</code></td>
+</tr>
+</tbody></table>
+
+<p>You can also control machine diversity using constraints. The below
+constraint ensures that no more than two instances of your job may run
+on a single host. Think of this as a “group by” limit.</p>
+<pre class="highlight text">constraints = {
+ 'host': 'limit:2',
+}
+</pre>
+<p>Likewise, you can use constraints to control rack diversity, e.g. at
+most one task per rack:</p>
+<pre class="highlight text">constraints = {
+ 'rack': 'limit:1',
+}
+</pre>
+<p>Use these constraints sparingly as they can dramatically reduce Tasks' schedulability.</p>
+
+<h1 id="template-namespaces">Template Namespaces</h1>
+
+<p>Currently, a few Pystachio namespaces have special semantics. Using them
+in your configuration allow you to tailor application behavior
+through environment introspection or interact in special ways with the
+Aurora client or Aurora-provided services.</p>
+
+<h3 id="mesos-namespace">mesos Namespace</h3>
+
+<p>The <code>mesos</code> namespace contains the <code>instance</code> variable that can be used
+to distinguish between Task replicas.</p>
+
+<table><thead>
+<tr>
+<th>variable name</th>
+<th style="text-align: center">type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td><code>instance</code></td>
+<td style="text-align: center">Integer</td>
+<td>The instance number of the created task. A job with 5 replicas has instance numbers 0, 1, 2, 3, and 4.</td>
+</tr>
+</tbody></table>
+
+<h3 id="thermos-namespace">thermos Namespace</h3>
+
+<p>The <code>thermos</code> namespace contains variables that work directly on the
+Thermos platform in addition to Aurora. This namespace is fully
+compatible with Tasks invoked via the <code>thermos</code> CLI.</p>
+
+<table><thead>
+<tr>
+<th style="text-align: center">variable</th>
+<th>type</th>
+<th>description</th>
+</tr>
+</thead><tbody>
+<tr>
+<td style="text-align: center"><code>ports</code></td>
+<td>map of string to Integer</td>
+<td>A map of names to port numbers</td>
+</tr>
+<tr>
+<td style="text-align: center"><code>task_id</code></td>
+<td>string</td>
+<td>The task ID assigned to this task.</td>
+</tr>
+</tbody></table>
+
+<p>The <code>thermos.ports</code> namespace is automatically populated by Aurora when
+invoking tasks on Mesos. When running the <code>thermos</code> command directly,
+these ports must be explicitly mapped with the <code>-P</code> option.</p>
+
+<p>For example, if ‘{{<code>thermos.ports[http]</code>}}’ is specified in a <code>Process</code>
+configuration, it is automatically extracted and auto-populated by
+Aurora, but must be specified with, for example, <code>thermos -P http:12345</code>
+to map <code>http</code> to port 12345 when running via the CLI.</p>
+
+<h1 id="basic-examples">Basic Examples</h1>
+
+<p>These are provided to give a basic understanding of simple Aurora jobs.</p>
+
+<h3 id="hello_world.aurora">hello_world.aurora</h3>
+
+<p>Put the following in a file named <code>hello_world.aurora</code>, substituting your own values
+for values such as <code>cluster</code>s.</p>
+<pre class="highlight text">import os
+hello_world_process = Process(name = 'hello_world', cmdline = 'echo hello world')
+
+hello_world_task = Task(
+ resources = Resources(cpu = 0.1, ram = 16 * MB, disk = 16 * MB),
+ processes = [hello_world_process])
+
+hello_world_job = Job(
+ cluster = 'cluster1',
+ role = os.getenv('USER'),
+ task = hello_world_task)
+
+jobs = [hello_world_job]
+</pre>
+<p>Then issue the following commands to create and kill the job, using your own values for the job key.</p>
+<pre class="highlight text">aurora create cluster1/$USER/test/hello_world hello_world.aurora
+
+aurora kill cluster1/$USER/test/hello_world
+</pre>
+<h3 id="environment-tailoring">Environment Tailoring</h3>
+
+<h4 id="helloworldproductionized.aurora">hello<em>world</em>productionized.aurora</h4>
+
+<p>Put the following in a file named <code>hello_world_productionized.aurora</code>, substituting your own values
+for values such as <code>cluster</code>s.</p>
+<pre class="highlight text">include('hello_world.aurora')
+
+production_resources = Resources(cpu = 1.0, ram = 512 * MB, disk = 2 * GB)
+staging_resources = Resources(cpu = 0.1, ram = 32 * MB, disk = 512 * MB)
+hello_world_template = hello_world(
+ name = "hello_world-{{cluster}}"
+ task = hello_world(resources=production_resources))
+
+jobs = [
+ # production jobs
+ hello_world_template(cluster = 'cluster1', instances = 25),
+ hello_world_template(cluster = 'cluster2', instances = 15),
+
+ # staging jobs
+ hello_world_template(
+ cluster = 'local',
+ instances = 1,
+ task = hello_world(resources=staging_resources)),
+]
+</pre>
+<p>Then issue the following commands to create and kill the job, using your own values for the job key</p>
+<pre class="highlight text">aurora create cluster1/$USER/test/hello_world-cluster1 hello_world_productionized.aurora
+
+aurora kill cluster1/$USER/test/hello_world-cluster1
+</pre>
+ </div>
+ </div>
+
+ <div class="container-fluid section-footer buffer">
+ <div class="container">
+ <div class="row">
+ <div class="col-md-2 col-md-offset-1"><h3>Quick Links</h3>
+ <ul>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/AURORA">Issue Tracking</a></li>
+ <li><a href="/documentation/latest/contributing/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="col-md-2"><h3>The ASF</h3>
+ <ul>
+ <li><a href="http://www.apache.org/licenses/">License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ <div class="col-md-6">
+ <p class="disclaimer">Apache Aurora is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.</p>
+ <p class="disclaimer">Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. The <a href="https://www.flickr.com/photos/trondk/12706051375/">Aurora Borealis IX photo</a> displayed on the homepage is available under a <a href="https://creativecommons.org/licenses/by-nc-nd/2.0/">Creative Commons BY-NC-ND 2.0 license</a>. Apache, Apache Aurora, and the Apache feather logo are trademarks of The Apache Software Foundation.</p>
+ </div>
+ </div>
+ </div>
+ </body>
+</html>
\ No newline at end of file