You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@aurora.apache.org by dl...@apache.org on 2014/03/25 07:10:07 UTC
svn commit: r1581246 [3/6] - in /incubator/aurora/site: ./ publish/
publish/community/ publish/developers/ publish/docs/gettingstarted/
publish/docs/howtocontribute/ publish/documentation/
publish/documentation/latest/ publish/documentation/latest/clie...
Added: incubator/aurora/site/publish/documentation/latest/hooks/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/hooks/index.html?rev=1581246&view=auto
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/hooks/index.html (added)
+++ incubator/aurora/site/publish/documentation/latest/hooks/index.html Tue Mar 25 06:10:05 2014
@@ -0,0 +1,366 @@
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title>Apache Aurora</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+ <meta name="description" content="">
+ <meta name="author" content="">
+
+ <link href="/assets/css/bootstrap.css" rel="stylesheet">
+ <link href="/assets/css/bootstrap-responsive.min.css" rel="stylesheet">
+ <link href="/assets/css/main.css" rel="stylesheet">
+
+ <!-- JS -->
+ <script type="text/javascript" src="/assets/js/jquery-1.10.1.min.js"></script>
+ <script type="text/javascript" src="/assets/js/bootstrap-dropdown.js"></script>
+
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="navbar navbar-static-top">
+ <div class="navbar-inner">
+ <div class="container">
+ <a href="/" class="logo"><img src="/assets/img/aurora_logo.png" alt="Apache Aurora logo" /></a>
+ <ul class="nav">
+ <li><a href="/docs/gettingstarted/">Getting Started</a></li>
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/downloads/">Download</a></li>
+ <li><a href="/community">Community</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+<div class="container">
+<!-- magical breadcrumbs -->
+<ul class="breadcrumb">
+ <li>
+ <div class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a>
+ <ul class="dropdown-menu" role="menu">
+ <li><a href="http://www.apache.org">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/">Apache License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ </li>
+ <li><span class="divider">•</span></li>
+ <li><a href="http://incubator.apache.org">Apache Incubator</a></li>
+ <li><span class="divider">•</span></li>
+ <li><a href="http://aurora.incubator.apache.org">Apache Aurora</a></li>
+</ul>
+<!-- /breadcrumb -->
+
+ <div class="container">
+ <h1 id="hooks-for-aurora-client-api">Hooks for Aurora Client API</h1>
+
+<ul>
+<li><a href="#introduction">Introduction</a></li>
+<li><a href="#hook-types">Hook Types</a></li>
+<li><a href="#execution-order">Execution Order</a></li>
+<li><a href="#hookable-methods">Hookable Methods</a></li>
+<li><a href="#activating-and-using-hooks">Activating and Using Hooks</a></li>
+<li><a href="#aurora-config-file-settings">.aurora Config File Settings</a></li>
+<li><a href="#command-line">Command Line</a></li>
+<li><a href="#hooks-protocol">Hooks Protocol</a>
+
+<ul>
+<li><a href="#pre_-methods">pre_ Methods</a></li>
+<li><a href="#err_-methods">err_ Methods</a></li>
+<li><a href="#post_-methods">post_ Methods</a></li>
+</ul></li>
+<li><a href="#generic-hooks">Generic Hooks</a></li>
+<li><a href="#hooks-process-checklist">Hooks Process Checklist</a></li>
+</ul>
+
+<h2 id="introduction">Introduction</h2>
+
+<p>You can execute hook methods around Aurora API Client methods when they are called by the Aurora Command Line commands.</p>
+
+<p>Explaining how hooks work is a bit tricky because of some indirection about what they apply to. Basically, a hook is code that executes when a particular Aurora Client API method runs, letting you extend the method’s actions. The hook executes on the client side, specifically on the machine executing Aurora commands.</p>
+
+<p>The catch is that hooks are associated with Aurora Client API methods, which users don’t directly call. Instead, users call Aurora Command Line commands, which call Client API methods during their execution. Since which hooks run depend on which Client API methods get called, you will need to know which Command Line commands call which API methods. Later on, there is a table showing the various associations.</p>
+
+<p><strong>Terminology Note</strong>: From now on, “method(s)” refer to Client API methods, and “command(s)” refer to Command Line commands.</p>
+
+<h2 id="hook-types">Hook Types</h2>
+
+<p>Hooks have three basic types, differing by when they run with respect to their associated method.</p>
+
+<p><code>pre_<method_name></code>: When its associated method is called, the <code>pre_</code> hook executes first, then the called method. If the <code>pre_</code> hook fails, the method never runs. Later code that expected the method to succeed may be affected by this, and result in terminating the Aurora client.</p>
+
+<p>Note that a <code>pre_</code> hook can error-trap internally so it does not
+return <code>False</code>. Designers/contributors of new <code>pre_</code> hooks should
+consider whether or not to error-trap them. You can error trap at the
+highest level very generally and always pass the <code>pre_</code> hook by
+returning <code>True</code>. For example:</p>
+<pre class="highlight python"><span class="k">def</span> <span class="nf">pre_create</span><span class="p">(</span><span class="o">...</span><span class="p">):</span>
+ <span class="n">do_something</span><span class="p">()</span> <span class="c"># if do_something fails with an exception, the create_job is not attempted!</span>
+ <span class="k">return</span> <span class="bp">True</span>
+
+<span class="c"># However...</span>
+<span class="k">def</span> <span class="nf">pre_create</span><span class="p">(</span><span class="o">...</span><span class="p">):</span>
+ <span class="k">try</span><span class="p">:</span>
+ <span class="n">do_something</span><span class="p">()</span> <span class="c"># may cause exception</span>
+ <span class="k">except</span> <span class="n">Exception</span><span class="p">:</span> <span class="c"># generic error trap will catch it</span>
+ <span class="k">pass</span> <span class="c"># and ignore the exception</span>
+ <span class="k">return</span> <span class="bp">True</span> <span class="c"># create_job will run in any case!</span>
+</pre>
+<p><code>post_<method_name></code>: A <code>post_</code> hook executes after its associated method successfully finishes running. If it fails, the already executed method is unaffected. A <code>post_</code> hook’s error is trapped, and any later operations are unaffected.</p>
+
+<p><code>err_<method_name></code>: Executes only when its associated method returns a status other than OK or throws an exception. If an <code>err_</code> hook fails, the already executed method is unaffected. An <code>err_</code> hook’s error is trapped, and any later operations are unaffected.</p>
+
+<h2 id="execution-order">Execution Order</h2>
+
+<p>A command with <code>pre_</code>, <code>post_</code>, and <code>err_</code> hooks defined and activated for its called method executes in the following order when the method successfully executes:</p>
+
+<ol>
+<li>Command called</li>
+<li>Command code executes</li>
+<li>Method Called</li>
+<li><code>pre_</code> method hook runs</li>
+<li>Method runs and successfully finishes</li>
+<li><code>post_</code> method hook runs</li>
+<li>Command code executes</li>
+<li>Command execution ends</li>
+</ol>
+
+<p>The following is what happens when, for the same command and hooks, the method associated with the command suffers an error and does not successfully finish executing:</p>
+
+<ol>
+<li>Command called</li>
+<li>Command code executes</li>
+<li>Method Called</li>
+<li><code>pre_</code> method hook runs</li>
+<li>Method runs and fails</li>
+<li><code>err_</code> method hook runs</li>
+<li>Command Code executes (if <code>err_</code> method does not end the command execution)</li>
+<li>Command execution ends</li>
+</ol>
+
+<p>Note that the <code>post_</code> and <code>err_</code> hooks for the same method can never both run for a single execution of that method.</p>
+
+<h2 id="hookable-methods">Hookable Methods</h2>
+
+<p>You can associate <code>pre_</code>, <code>post_</code>, and <code>err_</code> hooks with the following methods. Since you do not directly interact with the methods, but rather the Aurora Command Line commands that call them, for each method we also list the command(s) that can call the method. Note that a different method or methods may be called by a command depending on how the command’s other code executes. Similarly, multiple commands can call the same method. We also list the methods' argument signatures, which are used by their associated hooks. <a name="Chart"></a></p>
+
+<table border="1" cellpadding="0" cellspacing="0">
+ <tbody>
+ <tr>
+ <th>Aurora Client API Method</td>
+ <th>Client API Method Argument Signature</td>
+ <th>Aurora Command Line Command</td>
+ </tr>
+ <tr>
+ <td><code>cancel_update</code></td>
+ <td><code>self</code>, <code>job_key</code></td>
+ <td><code>cancel_update</code></td>
+ </tr>
+ <tr>
+ <td><code>create_job</code></td>
+ <td><code>self</code>, <code>config</code></td>
+ <td><code>create</code>, <code>runtask</code></td>
+ </tr>
+ <tr>
+ <td><code>restart</code></td>
+ <td><code>self</code>, <code>job_key</code>, <code>shards</code>, <code>update_config</code>, <code>health_check_interval_seconds</code></td>
+ <td><code>restart</code></td>
+ </tr>
+ <tr>
+ <td><code>update_job</code></td>
+ <td><code>self</code>, <code>config</code>, <code>health_check_interval_seconds=3</code>, <code>shards=None</code></td>
+ <td><code>update</code></td>
+ </tr>
+ <tr>
+ <td><code>kill_job</code></td>
+ <td><code>self</code>, <code>job_key</code>, <code>shards=None</code></td>
+ <td><code>kill</code></td>
+ </tr>
+ </table>
+
+<p>Some specific examples:</p>
+
+<ul>
+<li><p><code>pre_create_job</code> executes when a <code>create_job</code> method is called, and before the <code>create_job</code> method itself executes.</p></li>
+<li><p><code>post_cancel_update</code> executes after a <code>cancel_update</code> method has successfully finished running.</p></li>
+<li><p><code>err_kill_job</code> executes when the <code>kill_job</code> method is called, but doesn’t successfully finish running.</p></li>
+</ul>
+
+<h2 id="activating-and-using-hooks">Activating and Using Hooks</h2>
+
+<p>By default, hooks are inactive. If you do not want to use hooks, you do not need to make any changes to your code. If you do want to use hooks, you will need to alter your <code>.aurora</code> config file to activate them both for the configuration as a whole as well as for individual <code>Job</code>s. And, of course, you will need to define in your config file what happens when a particular hook executes.</p>
+
+<h2 id=".aurora-config-file-settings">.aurora Config File Settings</h2>
+
+<p>You can define a top-level <code>hooks</code> variable in any <code>.aurora</code> config file. <code>hooks</code> is a list of all objects that define hooks used by <code>Job</code>s defined in that config file. If you do not want to define any hooks for a configuration, <code>hooks</code> is optional.</p>
+<pre class="highlight text">hooks = [Object_with_defined_hooks1, Object_with_defined_hooks2]
+</pre>
+<p>Be careful when assembling a config file using <code>include</code> on multiple smaller config files. If there are multiple files that assign a value to <code>hooks</code>, only the last assignment made will stick. For example, if <code>x.aurora</code> has <code>hooks = [a, b, c]</code> and <code>y.aurora</code> has <code>hooks = [d, e, f]</code> and <code>z.aurora</code> has, in this order, <code>include x.aurora</code> and <code>include y.aurora</code>, the <code>hooks</code> value will be <code>[d, e, f]</code>.</p>
+
+<p>Also, for any <code>Job</code> that you want to use hooks with, its <code>Job</code> definition in the <code>.aurora</code> config file must set an <code>enable_hooks</code> flag to <code>True</code> (it defaults to <code>False</code>). By default, hooks are disabled and you must enable them for <code>Job</code>s of your choice.</p>
+
+<p>To summarize, to use hooks for a particular job, you must both activate hooks for your config file as a whole, and for that job. Activating hooks only for individual jobs won’t work, nor will only activating hooks for your config file as a whole. You must also specify the hooks' defining object in the <code>hooks</code> variable.</p>
+
+<p>Recall that <code>.aurora</code> config files are written in Pystachio. So the following turns on hooks for production jobs at cluster1 and cluster2, but leaves them off for similar jobs with a defined user role. Of course, you also need to list the objects that define the hooks in your config file’s <code>hooks</code> variable.</p>
+<pre class="highlight python"><span class="n">jobs</span> <span class="o">=</span> <span class="p">[</span>
+ <span class="n">Job</span><span class="p">(</span><span class="n">enable_hooks</span> <span class="o">=</span> <span class="bp">True</span><span class="p">,</span> <span class="n">cluster</span> <span class="o">=</span> <span class="n">c</span><span class="p">,</span> <span class="n">env</span> <span class="o">=</span> <span class="s">'prod'</span><span class="p">)</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="p">(</span><span class="s">'cluster1'</span><span class="p">,</span> <span class="s">'cluster2'</span><span class="p">)</span>
+ <span class="p">]</span>
+<span class="n">jobs</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span>
+ <span class="n">Job</span><span class="p">(</span><span class="n">cluster</span> <span class="o">=</span> <span class="n">c</span><span class="p">,</span> <span class="n">env</span> <span class="o">=</span> <span class="s">'prod'</span><span class="p">,</span> <span class="n">role</span> <span class="o">=</span> <span class="n">getpass</span><span class="o">.</span><span class="n">getuser</span><span class="p">())</span> <span class="k">for</span> <span class="n">c</span> <span class="ow">in</span> <span class="p">(</span><span class="s">'cluster1'</span><span class="p">,</span> <span class="s">'cluster2'</span><span class="p">))</span>
+ <span class="c"># Hooks disabled for these jobs</span>
+</pre>
+<h2 id="command-line">Command Line</h2>
+
+<p>All Aurora Command Line commands now accept an <code>.aurora</code> config file as an optional parameter (some, of course, accept it as a required parameter). Whenever a command has a <code>.aurora</code> file parameter, any hooks specified and activated in the <code>.aurora</code> file can be used. For example:</p>
+<pre class="highlight text">aurora restart cluster1/role/env/app myapp.aurora
+</pre>
+<p>The command activates any hooks specified and activated in <code>myapp.aurora</code>. For the <code>restart</code> command, that is the only thing the <code>myapp.aurora</code> parameter does. So, if the command was the following, since there is no <code>.aurora</code> config file to specify any hooks, no hooks on the <code>restart</code> command can run.</p>
+<pre class="highlight text">aurora restart cluster1/role/env/app
+</pre>
+<h2 id="hooks-protocol">Hooks Protocol</h2>
+
+<p>Any object defined in the <code>.aurora</code> config file can define hook methods. You should define your hook methods within a class, and then use the class name as a value in the <code>hooks</code> list in your config file.</p>
+
+<p>Note that you can define other methods in the class that its hook methods can call; all the logic of a hook does not have to be in its definition.</p>
+
+<p>The following example defines a class containing a <code>pre_kill_job</code> hook definition that calls another method defined in the class.</p>
+<pre class="highlight python"><span class="c"># Defines a method pre_kill_job</span>
+<span class="k">class</span> <span class="nc">KillConfirmer</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
+ <span class="k">def</span> <span class="nf">confirm</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">msg</span><span class="p">):</span>
+ <span class="k">return</span> <span class="nb">raw_input</span><span class="p">(</span><span class="n">msg</span><span class="p">)</span><span class="o">.</span><span class="n">lower</span><span class="p">()</span> <span class="o">==</span> <span class="s">'yes'</span>
+
+ <span class="k">def</span> <span class="nf">pre_kill_job</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">job_key</span><span class="p">,</span> <span class="n">shards</span><span class="o">=</span><span class="bp">None</span><span class="p">):</span>
+ <span class="n">shards</span> <span class="o">=</span> <span class="p">(</span><span class="s">'shards </span><span class="si">%</span><span class="s">s'</span> <span class="o">%</span> <span class="n">shards</span><span class="p">)</span> <span class="k">if</span> <span class="n">shards</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">None</span> <span class="k">else</span> <span class="s">'all shards'</span>
+ <span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">confirm</span><span class="p">(</span><span class="s">'Are you sure you want to kill </span><span class="si">%</span><span class="s">s (</span><span class="si">%</span><span class="s">s)? (yes/no): '</span>
+ <span class="o">%</span> <span class="p">(</span><span class="n">job_key</span><span class="p">,</span> <span class="n">shards</span><span class="p">))</span>
+</pre>
+<h3 id="pre_-methods">pre_ Methods</h3>
+
+<p><code>pre_</code> methods have the signature:</p>
+<pre class="highlight text">pre_<API method name>(self, <associated method's signature>)
+</pre>
+<p><code>pre_</code> methods have the same signature as their associated method, with the addition of <code>self</code> as the first parameter. See the <a href="#Chart">chart</a> above for the mapping of parameters to methods. When writing <code>pre_</code> methods, you can use the <code>*</code> and <code>**</code> syntax to designate that all unspecified parameters are passed in a list to the <code>*</code>ed variable and all named parameters with values are passed as name/value pairs to the <code>**</code>ed variable.</p>
+
+<p>If this method returns False, the API command call aborts.</p>
+
+<h3 id="err_-methods">err_ Methods</h3>
+
+<p><code>err_</code> methods have the signature:</p>
+<pre class="highlight text">err_<API method name>(self, exc, <associated method's signature>)
+</pre>
+<p><code>err_</code> methods have the same signature as their associated method, with the addition of a first parameter <code>self</code> and a second parameter <code>exc</code>. <code>exc</code> is either a result with responseCode other than <code>ResponseCode.OK</code> or an <code>Exception</code>. See the <a href="#Chart">chart</a> above for the mapping of parameters to methods. When writing <code>err</code>_ methods, you can use the <code>*</code> and <code>**</code> syntax to designate that all unspecified parameters are passed in a list to the <code>*</code>ed variable and all named parameters with values are passed as name/value pairs to the <code>**</code>ed variable.</p>
+
+<p><code>err_</code> method return codes are ignored.</p>
+
+<h3 id="post_-methods">post_ Methods</h3>
+
+<p><code>post_</code> methods have the signature:</p>
+<pre class="highlight text">post_<API method name>(self, result, <associated method signature>)
+</pre>
+<p><code>post_</code> method parameters are <code>self</code>, then <code>result</code>, followed by the same parameter signature as their associated method. <code>result</code> is the result of the associated method call. See the <a href="#chart">chart</a> above for the mapping of parameters to methods. When writing <code>post_</code> methods, you can use the <code>*</code> and <code>**</code> syntax to designate that all unspecified arguments are passed in a list to the <code>*</code>ed parameter and all unspecified named arguments with values are passed as name/value pairs to the <code>**</code>ed parameter.</p>
+
+<p><code>post_</code> method return codes are ignored.</p>
+
+<h2 id="generic-hooks">Generic Hooks</h2>
+
+<p>There are five Aurora API Methods which any of the three hook types can attach to. Thus, there are 15 possible hook/method combinations for a single <code>.aurora</code> config file. Say that you define <code>pre_</code> and <code>post_</code> hooks for the <code>restart</code> method. That leaves 13 undefined hook/method combinations; <code>err_restart</code> and the 3 <code>pre_</code>, <code>post_</code>, and <code>err_</code> hooks for each of the other 4 hookable methods. You can define what happens when any of these otherwise undefined 13 hooks execute via a generic hook, whose signature is:</p>
+<pre class="highlight python"><span class="n">generic_hook</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">hook_config</span><span class="p">,</span> <span class="n">event</span><span class="p">,</span> <span class="n">method_name</span><span class="p">,</span> <span class="n">result_or_err</span><span class="p">,</span> <span class="n">args</span><span class="o">*</span><span class="p">,</span> <span class="n">kw</span><span class="o">**</span><span class="p">)</span>
+</pre>
+<p>where:</p>
+
+<ul>
+<li><p><code>hook_config</code> is a named tuple of <code>config</code> (the Pystashio <code>config</code> object) and <code>job_key</code>.</p></li>
+<li><p><code>event</code> is one of <code>pre</code>, <code>err</code>, or <code>post</code>, indicating which type of hook the genetic hook is standing in for. For example, assume no specific hooks were defined for the <code>restart</code> API command. If <code>generic_hook</code> is defined and activated, and <code>restart</code> is called, <code>generic_hook</code> will effectively run as <code>pre_restart</code>, <code>post_restart</code>, and <code>err_restart</code>. You can use a selection statement on this value so that <code>generic_hook</code> will act differently based on whether it is standing in for a <code>pre_</code>, <code>post_</code>, or <code>err_</code> hook.</p></li>
+<li><p><code>method_name</code> is the Client API method name whose execution is causing this execution of the <code>generic_hook</code>.</p></li>
+<li><p><code>args*</code>, <code>kw**</code> are the API method arguments and keyword arguments respectively.</p></li>
+<li><p><code>result_or_err</code> is a tri-state parameter taking one of these three values:</p>
+
+<ol>
+<li>None for <code>pre_</code>hooks</li>
+<li><code>result</code> for <code>post_</code> nooks</li>
+<li><code>exc</code> for <code>err_</code> hooks</li>
+</ol></li>
+</ul>
+
+<p>Example:</p>
+<pre class="highlight python"><span class="c"># Overrides the standard do-nothing generic_hook by adding a log writing operation.</span>
+<span class="kn">from</span> <span class="nn">twitter.common</span> <span class="kn">import</span> <span class="n">log</span>
+ <span class="k">class</span> <span class="nc">Logger</span><span class="p">(</span><span class="nb">object</span><span class="p">):</span>
+ <span class="s">'''Adds to the log every time a hookable API method is called'''</span>
+ <span class="k">def</span> <span class="nf">generic_hook</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">hook_config</span><span class="p">,</span> <span class="n">event</span><span class="p">,</span> <span class="n">method_name</span><span class="p">,</span> <span class="n">result_or_err</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">,</span> <span class="o">**</span><span class="n">kw</span><span class="p">)</span>
+ <span class="n">log</span><span class="o">.</span><span class="n">info</span><span class="p">(</span><span class="s">'</span><span class="si">%</span><span class="s">s: </span><span class="si">%</span><span class="s">s_</span><span class="si">%</span><span class="s">s of </span><span class="si">%</span><span class="s">s'</span>
+ <span class="o">%</span> <span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">__class__</span><span class="o">.</span><span class="n">__name__</span><span class="p">,</span> <span class="n">event</span><span class="p">,</span> <span class="n">method_name</span><span class="p">,</span> <span class="n">hook_config</span><span class="o">.</span><span class="n">job_key</span><span class="p">))</span>
+</pre>
+<h2 id="hooks-process-checklist">Hooks Process Checklist</h2>
+
+<ol>
+<li>In your <code>.aurora</code> config file, add a <code>hooks</code> variable. Note that you may want to define a <code>.aurora</code> file only for hook definitions and then include this file in multiple other config files that you want to use the same hooks.</li>
+</ol>
+<pre class="highlight python"><span class="n">hooks</span> <span class="o">=</span> <span class="p">[]</span>
+</pre>
+<ol>
+<li>In the <code>hooks</code> variable, list all objects that define hooks used by <code>Job</code>s defined in this config:</li>
+</ol>
+<pre class="highlight python"><span class="n">hooks</span> <span class="o">=</span> <span class="p">[</span><span class="n">Object_hook_definer1</span><span class="p">,</span> <span class="n">Object_hook_definer2</span><span class="p">]</span>
+</pre>
+<ol>
+<li><p>For each job that uses hooks in this config file, add <code>enable_hooks = True</code> to the <code>Job</code> definition. Note that this is necessary even if you only want to use the generic hook.</p></li>
+<li><p>Write your <code>pre_</code>, <code>post_</code>, and <code>err_</code> hook definitions as part of an object definition in your <code>.aurora</code> config file.</p></li>
+<li><p>If desired, write your <code>generic_hook</code> definition as part of an object definition in your <code>.aurora</code> config file. Remember, the object must be listed as a member of <code>hooks</code>.</p></li>
+<li><p>If your Aurora command line command does not otherwise take an <code>.aurora</code> config file argument, add the appropriate <code>.aurora</code> file as an argument in order to define and activate the configuration’s hooks.</p></li>
+</ol>
+
+ </div>
+ <div class="container">
+ <hr>
+ <footer class="footer">
+ <div class="row-fluid">
+ <div class="span2 text-left">
+ <h3>Links</h3>
+ <ul class="unstyled">
+ <li><a href="/docs/gettingstarted/">Getting Started</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/developers/">Developers</a></li>
+ </ul>
+ </div>
+ <div class="span3 text-left">
+ <h3>Community</h3>
+ <ul class="unstyled">
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/aurora">Issue Tracking</a></li>
+ <li><a href="/docs/howtocontribute/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="span7 text-left">
+ <h3>Apache Software Foundation</h3>
+
+ <div class="span8">
+ Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. Apache, Apache Thrift, and the Apache feather logo are trademarks of The Apache Software Foundation. Currently part of the <a href="http://incubator.apache.org">Apache Incubator</a>.
+ </div>
+ <div class=" pull-right">
+ <a href="http://incubator.apache.org" class="logo"><img src="/assets/img/apache_incubator_logo.png" alt="Apache Incubator" class="pull-right"/></a>
+ </div>
+ </div>
+
+ </div>
+
+ </footer>
+</div>
+
+ </body>
+</html>
+
Added: incubator/aurora/site/publish/documentation/latest/images/CPUavailability.png
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/images/CPUavailability.png?rev=1581246&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/aurora/site/publish/documentation/latest/images/CPUavailability.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/aurora/site/publish/documentation/latest/images/HelloWorldJob.png
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/images/HelloWorldJob.png?rev=1581246&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/aurora/site/publish/documentation/latest/images/HelloWorldJob.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/aurora/site/publish/documentation/latest/images/RoleJobs.png
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/images/RoleJobs.png?rev=1581246&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/aurora/site/publish/documentation/latest/images/RoleJobs.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/aurora/site/publish/documentation/latest/images/ScheduledJobs.png
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/images/ScheduledJobs.png?rev=1581246&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/aurora/site/publish/documentation/latest/images/ScheduledJobs.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/aurora/site/publish/documentation/latest/images/TaskBreakdown.png
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/images/TaskBreakdown.png?rev=1581246&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/aurora/site/publish/documentation/latest/images/TaskBreakdown.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/aurora/site/publish/documentation/latest/images/aurora_hierarchy.png
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/images/aurora_hierarchy.png?rev=1581246&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/aurora/site/publish/documentation/latest/images/aurora_hierarchy.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/aurora/site/publish/documentation/latest/images/killedtask.png
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/images/killedtask.png?rev=1581246&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/aurora/site/publish/documentation/latest/images/killedtask.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/aurora/site/publish/documentation/latest/images/lifeofatask.png
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/images/lifeofatask.png?rev=1581246&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/aurora/site/publish/documentation/latest/images/lifeofatask.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/aurora/site/publish/documentation/latest/images/runningtask.png
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/images/runningtask.png?rev=1581246&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/aurora/site/publish/documentation/latest/images/runningtask.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/aurora/site/publish/documentation/latest/images/stderr.png
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/images/stderr.png?rev=1581246&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/aurora/site/publish/documentation/latest/images/stderr.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/aurora/site/publish/documentation/latest/images/stdout.png
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/images/stdout.png?rev=1581246&view=auto
==============================================================================
Binary file - no diff available.
Propchange: incubator/aurora/site/publish/documentation/latest/images/stdout.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
Added: incubator/aurora/site/publish/documentation/latest/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/index.html?rev=1581246&view=auto
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/index.html (added)
+++ incubator/aurora/site/publish/documentation/latest/index.html Tue Mar 25 06:10:05 2014
@@ -0,0 +1,131 @@
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title>Apache Aurora</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+ <meta name="description" content="">
+ <meta name="author" content="">
+
+ <link href="/assets/css/bootstrap.css" rel="stylesheet">
+ <link href="/assets/css/bootstrap-responsive.min.css" rel="stylesheet">
+ <link href="/assets/css/main.css" rel="stylesheet">
+
+ <!-- JS -->
+ <script type="text/javascript" src="/assets/js/jquery-1.10.1.min.js"></script>
+ <script type="text/javascript" src="/assets/js/bootstrap-dropdown.js"></script>
+
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="navbar navbar-static-top">
+ <div class="navbar-inner">
+ <div class="container">
+ <a href="/" class="logo"><img src="/assets/img/aurora_logo.png" alt="Apache Aurora logo" /></a>
+ <ul class="nav">
+ <li><a href="/docs/gettingstarted/">Getting Started</a></li>
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/downloads/">Download</a></li>
+ <li><a href="/community">Community</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+<div class="container">
+<!-- magical breadcrumbs -->
+<ul class="breadcrumb">
+ <li>
+ <div class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a>
+ <ul class="dropdown-menu" role="menu">
+ <li><a href="http://www.apache.org">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/">Apache License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ </li>
+ <li><span class="divider">•</span></li>
+ <li><a href="http://incubator.apache.org">Apache Incubator</a></li>
+ <li><span class="divider">•</span></li>
+ <li><a href="http://aurora.incubator.apache.org">Apache Aurora</a></li>
+</ul>
+<!-- /breadcrumb -->
+
+ <div class="container">
+ <h1 id="overview">Overview</h1>
+
+<p><em>Aurora</em> is a service scheduler that schedules jobs onto <em>Mesos</em>, which runs tasks at a specified cluster. Typical services consist of up to hundreds of task replicas.</p>
+
+<p>Aurora provides a <em>Job</em> abstraction consisting of a <em>Task</em> template and instructions for creating near-identical replicas of that Task (modulo things like “instance id” or specific port numbers which may differ from machine to machine).</p>
+
+<p><em>Terminology Note</em>: <em>Replicas</em> are also referred to as <em>shards</em> and <em>instances</em>. While there is a general desire to move to using “instances”, “shard” is still found in commands and help strings.</p>
+
+<p>Typically a Task is a single <em>Process</em> corresponding to a single command line, such as <code>python2.6 my_script.py</code>. However, sometimes you must colocate separate Processes together within a single Task, which runs within a single container and <code>chroot</code>, often referred to as a “sandbox”. For example, if you run multiple cooperating agents together such as <code>logrotate</code>, <code>installer</code>, and master or slave processes. <em>Thermos</em> provides a Process abstraction under the Mesos Tasks.</p>
+
+<p>To use and get up to speed on Aurora, you should look the docs in this directory in this order:</p>
+
+<ol>
+<li>How to <a href="/documentation/latest/deploying-aurora-scheduler/">deploy Aurora</a> or, how to <a href="/documentation/latest/vagrant/">install Aurora on virtual machines on your private machine</a> (the Tutorial uses the virtual machine approach).</li>
+<li>As a user, get started quickly with a <a href="/documentation/latest/tutorial/">Tutorial</a>.</li>
+<li>For an overview of Aurora’s process flow under the hood, see the <a href="/documentation/latest/userguide/">User Guide</a>.</li>
+<li>To learn how to write a configuration file, look at our <a href="/documentation/latest/configurationtutorial/">Configuration Tutorial</a>. From there, look at the <a href="/documentation/latest/configurationreference/">Aurora + Thermos Reference</a>.</li>
+<li>Then read up on the <a href="/documentation/latest/clientcommands/">Aurora Command Line Client</a>.</li>
+<li>Find out general information and useful tips about how Aurora does <a href="/documentation/latest/resourceisolation/">Resource Isolation</a>.</li>
+</ol>
+
+<p>To contact the Aurora Developer List, email <a href="mailto:dev@aurora.incubator.apache.org">dev@aurora.incubator.apache.org</a>. You may want to read the list <a href="http://mail-archives.apache.org/mod_mbox/incubator-aurora-dev/">archives</a>. You can also use the IRC channel <code>#aurora</code> on <code>irc.freenode.net</code></p>
+
+ </div>
+ <div class="container">
+ <hr>
+ <footer class="footer">
+ <div class="row-fluid">
+ <div class="span2 text-left">
+ <h3>Links</h3>
+ <ul class="unstyled">
+ <li><a href="/docs/gettingstarted/">Getting Started</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/developers/">Developers</a></li>
+ </ul>
+ </div>
+ <div class="span3 text-left">
+ <h3>Community</h3>
+ <ul class="unstyled">
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/aurora">Issue Tracking</a></li>
+ <li><a href="/docs/howtocontribute/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="span7 text-left">
+ <h3>Apache Software Foundation</h3>
+
+ <div class="span8">
+ Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. Apache, Apache Thrift, and the Apache feather logo are trademarks of The Apache Software Foundation. Currently part of the <a href="http://incubator.apache.org">Apache Incubator</a>.
+ </div>
+ <div class=" pull-right">
+ <a href="http://incubator.apache.org" class="logo"><img src="/assets/img/apache_incubator_logo.png" alt="Apache Incubator" class="pull-right"/></a>
+ </div>
+ </div>
+
+ </div>
+
+ </footer>
+</div>
+
+ </body>
+</html>
+
Added: incubator/aurora/site/publish/documentation/latest/resourceisolation/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/resourceisolation/index.html?rev=1581246&view=auto
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/resourceisolation/index.html (added)
+++ incubator/aurora/site/publish/documentation/latest/resourceisolation/index.html Tue Mar 25 06:10:05 2014
@@ -0,0 +1,259 @@
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title>Apache Aurora</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+ <meta name="description" content="">
+ <meta name="author" content="">
+
+ <link href="/assets/css/bootstrap.css" rel="stylesheet">
+ <link href="/assets/css/bootstrap-responsive.min.css" rel="stylesheet">
+ <link href="/assets/css/main.css" rel="stylesheet">
+
+ <!-- JS -->
+ <script type="text/javascript" src="/assets/js/jquery-1.10.1.min.js"></script>
+ <script type="text/javascript" src="/assets/js/bootstrap-dropdown.js"></script>
+
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="navbar navbar-static-top">
+ <div class="navbar-inner">
+ <div class="container">
+ <a href="/" class="logo"><img src="/assets/img/aurora_logo.png" alt="Apache Aurora logo" /></a>
+ <ul class="nav">
+ <li><a href="/docs/gettingstarted/">Getting Started</a></li>
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/downloads/">Download</a></li>
+ <li><a href="/community">Community</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+<div class="container">
+<!-- magical breadcrumbs -->
+<ul class="breadcrumb">
+ <li>
+ <div class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a>
+ <ul class="dropdown-menu" role="menu">
+ <li><a href="http://www.apache.org">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/">Apache License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ </li>
+ <li><span class="divider">•</span></li>
+ <li><a href="http://incubator.apache.org">Apache Incubator</a></li>
+ <li><span class="divider">•</span></li>
+ <li><a href="http://aurora.incubator.apache.org">Apache Aurora</a></li>
+</ul>
+<!-- /breadcrumb -->
+
+ <div class="container">
+ <h1 id="resource-isolation-and-sizing">Resource Isolation and Sizing</h1>
+
+<p><strong>NOTE</strong>: Resource Isolation and Sizing is very much a work in progress.
+Both user-facing aspects and how it works under the hood are subject to
+change.</p>
+
+<ul>
+<li><a href="#introduction">Introduction</a></li>
+<li><a href="#cpu-isolation">CPU Isolation</a></li>
+<li><a href="#cpu-sizing">CPU Sizing</a></li>
+<li><a href="#memory-isolation">Memory Isolation</a></li>
+<li><a href="#memory-sizing">Memory Sizing</a></li>
+<li><a href="#disk-space">Disk Space</a></li>
+<li><a href="#disk-space-sizing">Disk Space Sizing</a></li>
+<li><a href="#other-resources">Other Resources</a></li>
+</ul>
+
+<h2 id="introduction">Introduction</h2>
+
+<p>Aurora is a multi-tenant system; a single software instance runs on a
+server, serving multiple clients/tenants. To share resources among
+tenants, it implements isolation of:</p>
+
+<ul>
+<li>CPU</li>
+<li>memory</li>
+<li>disk space</li>
+</ul>
+
+<p>CPU is a soft limit, and handled differently from memory and disk space.
+Too low a CPU value results in throttling your application and
+slowing it down. Memory and disk space are both hard limits; when your
+application goes over these values, it’s killed.</p>
+
+<p>Let’s look at each resource type in more detail:</p>
+
+<h2 id="cpu-isolation">CPU Isolation</h2>
+
+<p>Mesos uses a quota based CPU scheduler (the <em>Completely Fair Scheduler</em>)
+to provide consistent and predictable performance. This is effectively
+a guarantee of resources – you receive at least what you requested, but
+also no more than you’ve requested.</p>
+
+<p>The scheduler gives applications a CPU quota for every 100 ms interval.
+When an application uses its quota for an interval, it is throttled for
+the rest of the 100 ms. Usage resets for each interval and unused
+quota does not carry over.</p>
+
+<p>For example, an application specifying 4.0 CPU has access to 400 ms of
+CPU time every 100 ms. This CPU quota can be used in different ways,
+depending on the application and available resources. Consider the
+scenarios shown in this diagram.</p>
+
+<p><img alt="CPU Availability" src="../images/CPUavailability.png" /></p>
+
+<ul>
+<li><p><em>Scenario A</em>: the application can use up to 4 cores continuously for
+every 100 ms interval. It is never throttled and starts processing
+new requests immediately.</p></li>
+<li><p><em>Scenario B</em> : the application uses up to 8 cores (depending on
+availability) but is throttled after 50 ms. The CPU quota resets at the
+start of each new 100 ms interval.</p></li>
+<li><p><em>Scenario C</em> : is like Scenario A, but there is a garbage collection
+event in the second interval that consumes all CPU quota. The
+application throttles for the remaining 75 ms of that interval and
+cannot service requests until the next interval. In this example, the
+garbage collection finished in one interval but, depending on how much
+garbage needs collecting, it may take more than one interval and further
+delay service of requests.</p></li>
+</ul>
+
+<p><em>Technical Note</em>: Mesos considers logical cores, also known as
+hyperthreading or SMT cores, as the unit of CPU.</p>
+
+<h2 id="cpu-sizing">CPU Sizing</h2>
+
+<p>To correctly size Aurora-run Mesos tasks, specify a per-shard CPU value
+that lets the task run at its desired performance when at peak load
+distributed across all shards. Include reserve capacity of at least 50%,
+possibly more, depending on how critical your service is (or how
+confident you are about your original estimate : -)), ideally by
+increasing the number of shards to also improve resiliency. When running
+your application, observe its CPU stats over time. If consistently at or
+near your quota during peak load, you should consider increasing either
+per-shard CPU or the number of shards.</p>
+
+<h2 id="memory-isolation">Memory Isolation</h2>
+
+<p>Mesos uses dedicated memory allocation. Your application always has
+access to the amount of memory specified in your configuration. The
+application’s memory use is defined as the sum of the resident set size
+(RSS) of all processes in a shard. Each shard is considered
+independently.</p>
+
+<p>In other words, say you specified a memory size of 10GB. Each shard
+would receive 10GB of memory. If an individual shard’s memory demands
+exceed 10GB, that shard is killed, but the other shards continue
+working.</p>
+
+<p><em>Technical note</em>: Total memory size is not enforced at allocation time,
+so your application can request more than its allocation without getting
+an ENOMEM. However, it will be killed shortly after.</p>
+
+<h2 id="memory-sizing">Memory Sizing</h2>
+
+<p>Size for your application’s peak requirement. Observe the per-instance
+memory statistics over time, as memory requirements can vary over
+different periods. Remember that if your application exceeds its memory
+value, it will be killed, so you should also add a safety margin of
+around 10-20%. If you have the ability to do so, you may also want to
+put alerts on the per-instance memory.</p>
+
+<h2 id="disk-space">Disk Space</h2>
+
+<p>Disk space used by your application is defined as the sum of the files'
+disk space in your application’s directory, including the <code>stdout</code> and
+<code>stderr</code> logged from your application. Each shard is considered
+independently. You should use off-node storage for your application’s
+data whenever possible.</p>
+
+<p>In other words, say you specified disk space size of 100MB. Each shard
+would receive 100MB of disk space. If an individual shard’s disk space
+demands exceed 100MB, that shard is killed, but the other shards
+continue working.</p>
+
+<p>After your application finishes running, its allocated disk space is
+reclaimed. Thus, your job’s final action should move any disk content
+that you want to keep, such as logs, to your home file system or other
+less transitory storage. Disk reclamation takes place an undefined
+period after the application finish time; until then, the disk contents
+are still available but you shouldn’t count on them being so.</p>
+
+<p><em>Technical note</em> : Disk space is not enforced at write so your
+application can write above its quota without getting an ENOSPC, but it
+will be killed shortly after. This is subject to change.</p>
+
+<h2 id="disk-space-sizing">Disk Space Sizing</h2>
+
+<p>Size for your application’s peak requirement. Rotate and discard log
+files as needed to stay within your quota. When running a Java process,
+add the maximum size of the Java heap to your disk space requirement, in
+order to account for an out of memory error dumping the heap
+into the application’s sandbox space.</p>
+
+<h2 id="other-resources">Other Resources</h2>
+
+<p>Other resources, such as network bandwidth, do not have any performance
+guarantees. For some resources, such as memory bandwidth, there are no
+practical sharing methods so some application combinations collocated on
+the same host may cause contention.</p>
+
+ </div>
+ <div class="container">
+ <hr>
+ <footer class="footer">
+ <div class="row-fluid">
+ <div class="span2 text-left">
+ <h3>Links</h3>
+ <ul class="unstyled">
+ <li><a href="/docs/gettingstarted/">Getting Started</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/developers/">Developers</a></li>
+ </ul>
+ </div>
+ <div class="span3 text-left">
+ <h3>Community</h3>
+ <ul class="unstyled">
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/aurora">Issue Tracking</a></li>
+ <li><a href="/docs/howtocontribute/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="span7 text-left">
+ <h3>Apache Software Foundation</h3>
+
+ <div class="span8">
+ Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. Apache, Apache Thrift, and the Apache feather logo are trademarks of The Apache Software Foundation. Currently part of the <a href="http://incubator.apache.org">Apache Incubator</a>.
+ </div>
+ <div class=" pull-right">
+ <a href="http://incubator.apache.org" class="logo"><img src="/assets/img/apache_incubator_logo.png" alt="Apache Incubator" class="pull-right"/></a>
+ </div>
+ </div>
+
+ </div>
+
+ </footer>
+</div>
+
+ </body>
+</html>
+
Added: incubator/aurora/site/publish/documentation/latest/tutorial/index.html
URL: http://svn.apache.org/viewvc/incubator/aurora/site/publish/documentation/latest/tutorial/index.html?rev=1581246&view=auto
==============================================================================
--- incubator/aurora/site/publish/documentation/latest/tutorial/index.html (added)
+++ incubator/aurora/site/publish/documentation/latest/tutorial/index.html Tue Mar 25 06:10:05 2014
@@ -0,0 +1,360 @@
+<html>
+ <head>
+ <meta charset="utf-8">
+ <title>Apache Aurora</title>
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+ <meta name="description" content="">
+ <meta name="author" content="">
+
+ <link href="/assets/css/bootstrap.css" rel="stylesheet">
+ <link href="/assets/css/bootstrap-responsive.min.css" rel="stylesheet">
+ <link href="/assets/css/main.css" rel="stylesheet">
+
+ <!-- JS -->
+ <script type="text/javascript" src="/assets/js/jquery-1.10.1.min.js"></script>
+ <script type="text/javascript" src="/assets/js/bootstrap-dropdown.js"></script>
+
+ <!-- Analytics -->
+ <script type="text/javascript">
+ var _gaq = _gaq || [];
+ _gaq.push(['_setAccount', 'UA-45879646-1']);
+ _gaq.push(['_setDomainName', 'apache.org']);
+ _gaq.push(['_trackPageview']);
+
+ (function() {
+ var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
+ ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
+ var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
+ })();
+ </script>
+ </head>
+ <body>
+ <div class="navbar navbar-static-top">
+ <div class="navbar-inner">
+ <div class="container">
+ <a href="/" class="logo"><img src="/assets/img/aurora_logo.png" alt="Apache Aurora logo" /></a>
+ <ul class="nav">
+ <li><a href="/docs/gettingstarted/">Getting Started</a></li>
+ <li><a href="/documentation/latest/">Documentation</a></li>
+ <li><a href="/downloads/">Download</a></li>
+ <li><a href="/community">Community</a></li>
+ </ul>
+ </div>
+ </div>
+</div>
+
+<div class="container">
+<!-- magical breadcrumbs -->
+<ul class="breadcrumb">
+ <li>
+ <div class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">Apache Software Foundation <b class="caret"></b></a>
+ <ul class="dropdown-menu" role="menu">
+ <li><a href="http://www.apache.org">Apache Homepage</a></li>
+ <li><a href="http://www.apache.org/licenses/">Apache License</a></li>
+ <li><a href="http://www.apache.org/foundation/sponsorship.html">Sponsorship</a></li>
+ <li><a href="http://www.apache.org/foundation/thanks.html">Thanks</a></li>
+ <li><a href="http://www.apache.org/security/">Security</a></li>
+ </ul>
+ </div>
+ </li>
+ <li><span class="divider">•</span></li>
+ <li><a href="http://incubator.apache.org">Apache Incubator</a></li>
+ <li><span class="divider">•</span></li>
+ <li><a href="http://aurora.incubator.apache.org">Apache Aurora</a></li>
+</ul>
+<!-- /breadcrumb -->
+
+ <div class="container">
+ <h2 id="aurora-tutorial">Aurora Tutorial</h2>
+
+<p>Before reading this document, you should read over the (short) <a href="/documentation/latest/README/">README</a>
+for the Aurora docs.</p>
+
+<ul>
+<li><a href="#introduction">Introduction</a></li>
+<li><a href="#setup-install-aurora">Setup: Install Aurora</a></li>
+<li><a href="#the-script">The Script</a></li>
+<li><a href="#aurora-configuration">Aurora Configuration</a></li>
+<li><a href="#whats-going-on-in-that-configuration-file">What’s Going On In That Configuration File?</a></li>
+<li><a href="#creating-the-job">Creating the Job</a></li>
+<li><a href="#watching-the-job-run">Watching the Job Run</a></li>
+<li><a href="#cleanup">Cleanup</a></li>
+<li><a href="#next-steps">Next Steps</a></li>
+</ul>
+
+<h2 id="introduction">Introduction</h2>
+
+<p>This tutorial shows how to use the Aurora scheduler to run (and
+“<code>printf-debug</code>”) a hello world program on Mesos. The operational
+hierarchy is:</p>
+
+<ul>
+<li>Aurora manages and schedules jobs for Mesos to run.</li>
+<li>Mesos manages the individual tasks that make up a job.</li>
+<li>Thermos manages the individual processes that make up a task.</li>
+</ul>
+
+<p>This is the recommended first Aurora users document to read to start
+getting up to speed on the system.</p>
+
+<p>To get help, email questions to the Aurora Developer List,
+<a href="mailto:dev@aurora.incubator.apache.org">dev@aurora.incubator.apache.org</a></p>
+
+<h2 id="setup:-install-aurora">Setup: Install Aurora</h2>
+
+<p>You use the Aurora client and web UI to interact with Aurora jobs. To
+install it locally, see <a href="/documentation/latest/vagrant/">vagrant.md</a>. The remainder of this
+Tutorial assumes you are running Aurora using Vagrant.</p>
+
+<h2 id="the-script">The Script</h2>
+
+<p>Our “hello world” application is a simple Python script that loops
+forever, displaying the time every few seconds. Copy the code below and
+put it in a file named <code>hello_world.py</code> in the root of your Aurora repository clone (Note:
+this directory is the same as <code>/vagrant</code> inside the Vagrant VMs).</p>
+
+<p>The script has an intentional bug, which we will explain later on.</p>
+<pre class="highlight python"><span class="kn">import</span> <span class="nn">sys</span>
+<span class="kn">import</span> <span class="nn">time</span>
+
+<span class="k">def</span> <span class="nf">main</span><span class="p">(</span><span class="n">argv</span><span class="p">):</span>
+ <span class="n">SLEEP_DELAY</span> <span class="o">=</span> <span class="mi">10</span>
+ <span class="c"># Python ninjas - ignore this blatant bug.</span>
+ <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="n">xrang</span><span class="p">(</span><span class="mi">100</span><span class="p">):</span>
+ <span class="k">print</span><span class="p">(</span><span class="s">"Hello world! The time is now: </span><span class="si">%</span><span class="s">s. Sleeping for </span><span class="si">%</span><span class="s">d secs"</span> <span class="o">%</span> <span class="p">(</span>
+ <span class="n">time</span><span class="o">.</span><span class="n">asctime</span><span class="p">(),</span> <span class="n">SLEEP_DELAY</span><span class="p">))</span>
+ <span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="o">.</span><span class="n">flush</span><span class="p">()</span>
+ <span class="n">time</span><span class="o">.</span><span class="n">sleep</span><span class="p">(</span><span class="n">SLEEP_DELAY</span><span class="p">)</span>
+
+<span class="k">if</span> <span class="n">__name__</span> <span class="o">==</span> <span class="s">"__main__"</span><span class="p">:</span>
+ <span class="n">main</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">)</span>
+</pre>
+<h2 id="aurora-configuration">Aurora Configuration</h2>
+
+<p>Once we have our script/program, we need to create a <em>configuration
+file</em> that tells Aurora how to manage and launch our Job. Save the below
+code in the file <code>hello_world.aurora</code> in the same directory as your
+<code>hello_world.py</code> file. (all Aurora configuration files end with <code>.aurora</code> and
+are written in a Python variant).</p>
+<pre class="highlight python"><span class="kn">import</span> <span class="nn">os</span>
+
+<span class="c"># copy hello_world.py into the local sandbox</span>
+<span class="n">install</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span>
+ <span class="n">name</span> <span class="o">=</span> <span class="s">'fetch_package'</span><span class="p">,</span>
+ <span class="n">cmdline</span> <span class="o">=</span> <span class="s">'cp /vagrant/hello_world.py . && chmod +x hello_world.py'</span><span class="p">)</span>
+
+<span class="c"># run the script</span>
+<span class="n">hello_world</span> <span class="o">=</span> <span class="n">Process</span><span class="p">(</span>
+ <span class="n">name</span> <span class="o">=</span> <span class="s">'hello_world'</span><span class="p">,</span>
+ <span class="n">cmdline</span> <span class="o">=</span> <span class="s">'python hello_world.py'</span><span class="p">)</span>
+
+<span class="c"># describe the task</span>
+<span class="n">hello_world_task</span> <span class="o">=</span> <span class="n">SequentialTask</span><span class="p">(</span>
+ <span class="n">processes</span> <span class="o">=</span> <span class="p">[</span><span class="n">install</span><span class="p">,</span> <span class="n">hello_world</span><span class="p">],</span>
+ <span class="n">resources</span> <span class="o">=</span> <span class="n">Resources</span><span class="p">(</span><span class="n">cpu</span> <span class="o">=</span> <span class="mi">1</span><span class="p">,</span> <span class="n">ram</span> <span class="o">=</span> <span class="mi">1</span><span class="o">*</span><span class="n">MB</span><span class="p">,</span> <span class="n">disk</span><span class="o">=</span><span class="mi">8</span><span class="o">*</span><span class="n">MB</span><span class="p">))</span>
+
+<span class="n">jobs</span> <span class="o">=</span> <span class="p">[</span>
+ <span class="n">Job</span><span class="p">(</span><span class="n">name</span> <span class="o">=</span> <span class="s">'hello_world'</span><span class="p">,</span> <span class="n">cluster</span> <span class="o">=</span> <span class="s">'example'</span><span class="p">,</span> <span class="n">role</span> <span class="o">=</span> <span class="s">'www-data'</span><span class="p">,</span>
+ <span class="n">environment</span> <span class="o">=</span> <span class="s">'devel'</span><span class="p">,</span> <span class="n">task</span> <span class="o">=</span> <span class="n">hello_world_task</span><span class="p">)</span>
+<span class="p">]</span>
+</pre>
+<p>For more about Aurora configuration files, see the <a href="/documentation/latest/configurationtutorial/">Configuration
+Tutorial</a> and the <a href="/documentation/latest/configurationreference/">Aurora + Thermos
+Reference</a> (preferably after finishing this
+tutorial).</p>
+
+<h2 id="what's-going-on-in-that-configuration-file?">What’s Going On In That Configuration File?</h2>
+
+<p>More than you might think.</p>
+
+<ol>
+<li><p>From a “big picture” viewpoint, it first defines two
+Processes. Then it defines a Task that runs the two Processes in the
+order specified in the Task definition, as well as specifying what
+computational and memory resources are available for them. Finally,
+it defines a Job that will schedule the Task on available and suitable
+machines. This Job is the sole member of a list of Jobs; you can
+specify more than one Job in a config file.</p></li>
+<li><p>At the Process level, it specifies how to get your code into the
+local sandbox in which it will run. It then specifies how the code is
+actually run once the second Process starts.</p></li>
+</ol>
+
+<h2 id="creating-the-job">Creating the Job</h2>
+
+<p>We’re ready to launch our job! To do so, we use the Aurora Client to
+issue a Job creation request to the Aurora scheduler.</p>
+
+<p>Many Aurora Client commands take a <em>job key</em> argument, which uniquely
+identifies a Job. A job key consists of four parts, each separated by a
+“/”. The four parts are <code><cluster>/<role>/<environment>/<jobname></code>
+in that order. When comparing two job keys, if any of the
+four parts is different from its counterpart in the other key, then the
+two job keys identify two separate jobs. If all four values are
+identical, the job keys identify the same job.</p>
+
+<p><code>/etc/aurora/clusters.json</code> within the Aurora scheduler has the available
+cluster names. For Vagrant, from the top-level of your Aurora repository clone,
+do:</p>
+<pre class="highlight text">$ vagrant ssh aurora-scheduler
+</pre>
+<p>Followed by:</p>
+<pre class="highlight text">vagrant@precise64:~$ cat /etc/aurora/clusters.json
+</pre>
+<p>You’ll see something like:</p>
+<pre class="highlight javascript"><span class="p">[{</span>
+ <span class="s2">"name"</span><span class="err">:</span> <span class="s2">"example"</span><span class="p">,</span>
+ <span class="s2">"zk"</span><span class="err">:</span> <span class="s2">"192.168.33.2"</span><span class="p">,</span>
+ <span class="s2">"scheduler_zk_path"</span><span class="err">:</span> <span class="s2">"/aurora/scheduler"</span><span class="p">,</span>
+ <span class="s2">"auth_mechanism"</span><span class="err">:</span> <span class="s2">"UNAUTHENTICATED"</span>
+<span class="p">}]</span>
+</pre>
+<p>Use a <code>name</code> value for your job key’s cluster value.</p>
+
+<p>Role names are user accounts existing on the slave machines. If you don’t know what accounts
+are available, contact your sysadmin.</p>
+
+<p>Environment names are namespaces; you can count on <code>prod</code>, <code>devel</code> and <code>test</code> existing.</p>
+
+<p>The Aurora Client command that actually runs our Job is <code>aurora create</code>. It creates a Job as
+specified by its job key and configuration file arguments and runs it.</p>
+<pre class="highlight text">aurora create <cluster>/<role>/<environment>/<jobname> <config_file>
+</pre>
+<p>Or for our example:</p>
+<pre class="highlight text">aurora create example/www-data/devel/hello_world /vagrant/hello_world.aurora
+</pre>
+<p>Note: Remember, the job key’s <code><jobname></code> value is the name of the Job, not the name
+of its code file.</p>
+
+<p>This returns:</p>
+<pre class="highlight text">$ vagrant ssh aurora-scheduler
+Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-23-generic x86_64)
+
+ * Documentation: https://help.ubuntu.com/
+Welcome to your Vagrant-built virtual machine.
+Last login: Fri Jan 3 02:18:55 2014 from 10.0.2.2
+vagrant@precise64:~$ aurora create example/www-data/devel/hello_world \
+ /vagrant/hello_world.aurora
+ INFO] Creating job hello_world
+ INFO] Response from scheduler: OK (message: 1 new tasks pending for job
+ www-data/devel/hello_world)
+ INFO] Job url: http://precise64:8081/scheduler/www-data/devel/hello_world
+</pre>
+<h2 id="watching-the-job-run">Watching the Job Run</h2>
+
+<p>Now that our job is running, let’s see what it’s doing. Access the
+scheduler web interface at <code>http://$scheduler_hostname:$scheduler_port/scheduler</code>
+Or when using <code>vagrant</code>, <code>http://192.168.33.6:8081/scheduler</code>
+First we see what Jobs are scheduled:</p>
+
+<p><img alt="Scheduled Jobs" src="../images/ScheduledJobs.png" /></p>
+
+<p>Click on your user name, which in this case was <code>www-data</code>, and we see the Jobs associated
+with that role:</p>
+
+<p><img alt="Role Jobs" src="../images/RoleJobs.png" /></p>
+
+<p>Uh oh, that <code>Unstable</code> next to our <code>hello_world</code> Job doesn’t look good. Click the
+<code>hello_world</code> Job, and you’ll see:</p>
+
+<p><img alt="hello_world Job" src="../images/HelloWorldJob.png" /></p>
+
+<p>Oops, looks like our first job didn’t quite work! The task failed, so we have
+to figure out what went wrong.</p>
+
+<p>Access the page for our Task by clicking on its Host.</p>
+
+<p><img alt="Task page" src="../images/TaskBreakdown.png" /></p>
+
+<p>Once there, we see that the
+<code>hello_world</code> process failed. The Task page captures the standard error and
+standard output streams and makes them available. Clicking through
+to <code>stderr</code> on the failed <code>hello_world</code> process, we see what happened.</p>
+
+<p><img alt="stderr page" src="../images/stderr.png" /></p>
+
+<p>It looks like we made a typo in our Python script. We wanted <code>xrange</code>,
+not <code>xrang</code>. Edit the <code>hello_world.py</code> script, save as <code>hello_world_v2.py</code> and change your
+<code>hello_world.aurora</code> config file to use <code>hello_world_v2.py</code> instead of <code>hello_world.py</code>.</p>
+
+<p>Now that we’ve updated our configuration, let’s restart the job:</p>
+<pre class="highlight text">aurora update example/www-data/devel/hello_world /vagrant/hello_world.aurora
+</pre>
+<p>This time, the task comes up, we inspect the page, and see that the
+<code>hello_world</code> process is running.</p>
+
+<p><img alt="Running Task page" src="../images/runningtask.png" /></p>
+
+<p>We then inspect the output by clicking on <code>stdout</code> and see our process'
+output:</p>
+
+<p><img alt="stdout page" src="../images/stdout.png" /></p>
+
+<h2 id="cleanup">Cleanup</h2>
+
+<p>Now that we’re done, we kill the job using the Aurora client:</p>
+<pre class="highlight text">vagrant@precise64:~$ aurora kill example/www-data/devel/hello_world
+ INFO] Killing tasks for job: example/www-data/devel/hello_world
+ INFO] Response from scheduler: OK (message: Tasks killed.)
+ INFO] Job url: http://precise64:8081/scheduler/www-data/devel/hello_world
+vagrant@precise64:~$
+</pre>
+<p>The Task scheduler page now shows the <code>hello_world</code> process as <code>KILLED</code>.</p>
+
+<p><img alt="Killed Task page" src="../images/killedtask.png" /></p>
+
+<h2 id="next-steps">Next Steps</h2>
+
+<p>Now that you’ve finished this Tutorial, you should read or do the following:</p>
+
+<ul>
+<li><a href="/documentation/latest/configurationtutorial/">The Aurora Configuration Tutorial</a>, which provides more examples
+and best practices for writing Aurora configurations. You should also look at
+the <a href="/documentation/latest/configurationreference/">Aurora + Thermos Configuration Reference</a>.</li>
+<li>The <a href="/documentation/latest/userguide/">Aurora User Guide</a> provides an overview of how Aurora, Mesos, and
+Thermos work “under the hood”.</li>
+<li>Explore the Aurora Client - use the <code>aurora help</code> subcommand, and read the
+<a href="/documentation/latest/clientcommands/">Aurora Client Commands</a> document.</li>
+</ul>
+
+ </div>
+ <div class="container">
+ <hr>
+ <footer class="footer">
+ <div class="row-fluid">
+ <div class="span2 text-left">
+ <h3>Links</h3>
+ <ul class="unstyled">
+ <li><a href="/docs/gettingstarted/">Getting Started</a></li>
+ <li><a href="/downloads/">Downloads</a></li>
+ <li><a href="/developers/">Developers</a></li>
+ </ul>
+ </div>
+ <div class="span3 text-left">
+ <h3>Community</h3>
+ <ul class="unstyled">
+ <li><a href="/community/">Mailing Lists</a></li>
+ <li><a href="http://issues.apache.org/jira/browse/aurora">Issue Tracking</a></li>
+ <li><a href="/docs/howtocontribute/">How To Contribute</a></li>
+ </ul>
+ </div>
+ <div class="span7 text-left">
+ <h3>Apache Software Foundation</h3>
+
+ <div class="span8">
+ Copyright 2014 <a href="http://www.apache.org/">Apache Software Foundation</a>. Licensed under the <a href="http://www.apache.org/licenses/">Apache License v2.0</a>. Apache, Apache Thrift, and the Apache feather logo are trademarks of The Apache Software Foundation. Currently part of the <a href="http://incubator.apache.org">Apache Incubator</a>.
+ </div>
+ <div class=" pull-right">
+ <a href="http://incubator.apache.org" class="logo"><img src="/assets/img/apache_incubator_logo.png" alt="Apache Incubator" class="pull-right"/></a>
+ </div>
+ </div>
+
+ </div>
+
+ </footer>
+</div>
+
+ </body>
+</html>
+