You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by bu...@apache.org on 2015/11/02 18:24:48 UTC
svn commit: r971096 [4/8] - in /websites/staging/slider/trunk/content: ./
design/ design/registry/ design/specification/ developing/ docs/ docs/api/
docs/configuration/ docs/configuration/revision-1/ docs/slider_specs/
downloads/ release_notes/
Modified: websites/staging/slider/trunk/content/developing/submitting_patches.html
==============================================================================
--- websites/staging/slider/trunk/content/developing/submitting_patches.html (original)
+++ websites/staging/slider/trunk/content/developing/submitting_patches.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,8 +193,8 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="title">title</h1>
-<h2 id="recommended-patch-development-process">Recommended Patch development process</h2>
+<h1 id="title">title<a class="headerlink" href="#title" title="Permanent link">¶</a></h1>
+<h2 id="recommended-patch-development-process">Recommended Patch development process<a class="headerlink" href="#recommended-patch-development-process" title="Permanent link">¶</a></h2>
<ol>
<li>Git clone the repository, checking out <code>develop</code></li>
<li>Make sure it builds and tests, following the instructions.</li>
@@ -192,7 +203,7 @@ Latest release: <strong>0.80.0-incubatin
<li>Check in your code as you go along, including the relevant JIRA value in the commit message, e.g. "SLIDER-149 patch pom"</li>
<li>Rebase before the final patch submission, and ideally regularly</li>
</ol>
-<h3 id="rebasing">Rebasing</h3>
+<h3 id="rebasing">Rebasing<a class="headerlink" href="#rebasing" title="Permanent link">¶</a></h3>
<p>Patches will be applied to the head of the develop branch, so need to be submitted as diffs against that. This is best done during development by rebasing your commit history <em>provided you are not collaborating with others on that branch</em></p>
<ol>
<li>Tag the current version of the branch, something like <code>tag_<JIRA-NAME>_<DATE>_rebasing</code>. Example <code>tag_SLIDER-149_2014-08-14_rebasing</code>. This ensures the pre-rebase commit tree is still retained somewhere.</li>
@@ -200,7 +211,7 @@ Latest release: <strong>0.80.0-incubatin
<li>check out your feature branch, then rebase it with <code>git rebase develop</code>.</li>
<li>Fix any conflicts.</li>
</ol>
-<h2 id="generating-patches">Generating Patches</h2>
+<h2 id="generating-patches">Generating Patches<a class="headerlink" href="#generating-patches" title="Permanent link">¶</a></h2>
<p>There's two ways you can create patches. </p>
<ol>
<li>
@@ -216,7 +227,7 @@ Latest release: <strong>0.80.0-incubatin
<li>Every time you create a new patch, update the patch revision number.</li>
<li>Consider tagging your branch whenever you create a patch, including the patch filename in the tag, e.g. <code>tag_patch_SLIDER-149-002</code>. This makes the work history easier to trace.</li>
</ol>
-<h3 id="simple-git-diff">Simple git diff</h3>
+<h3 id="simple-git-diff">Simple git diff<a class="headerlink" href="#simple-git-diff" title="Permanent link">¶</a></h3>
<p>Create a diff between the <code>develop</code> branch and your latest commit</p>
<div class="codehilite"><pre><span class="n">git</span> <span class="n">diff</span> <span class="n">develop</span><span class="p">...</span><span class="n">HEAD</span> <span class="o">></span> <span class="o"><</span><span class="n">PATCH_FILENAME</span><span class="o">></span>
</pre></div>
@@ -227,7 +238,7 @@ Latest release: <strong>0.80.0-incubatin
</pre></div>
-<h3 id="formatted-patches">formatted patches</h3>
+<h3 id="formatted-patches">formatted patches<a class="headerlink" href="#formatted-patches" title="Permanent link">¶</a></h3>
<p>A git formatted patch includes all the commits applied during the development process. This must be called against a branch which has either been rebased, or had its patch history squashed to a single patch. The process of doing that is <a href="http://stackoverflow.com/questions/616556/how-do-you-squash-commits-into-one-patch-with-git-format-patch">documented on stackoverflow</a></p>
<div class="codehilite"><pre><span class="n">git</span> <span class="n">format</span><span class="o">-</span><span class="n">patch</span> <span class="o">--</span><span class="n">stdout</span> <span class="n">develop</span> <span class="o">></span> <span class="o"><</span><span class="n">PATCH_FILENAME</span><span class="o">></span>
</pre></div>
@@ -238,11 +249,11 @@ Latest release: <strong>0.80.0-incubatin
</pre></div>
-<h3 id="submitting-patches">Submitting Patches</h3>
+<h3 id="submitting-patches">Submitting Patches<a class="headerlink" href="#submitting-patches" title="Permanent link">¶</a></h3>
<p>Add the patch to the JIRA to which it is related.</p>
-<h2 id="applying-patches">Applying patches</h2>
+<h2 id="applying-patches">Applying patches<a class="headerlink" href="#applying-patches" title="Permanent link">¶</a></h2>
<p>Patches MUST be applied to a new feature branch created off the latest commit of <code>develop</code>; if a long-lived branch is being developed, it MUST be rebased first.</p>
-<h3 id="simple-patches">simple patches</h3>
+<h3 id="simple-patches">simple patches<a class="headerlink" href="#simple-patches" title="Permanent link">¶</a></h3>
<p>Before applying a patch, check it actually works:</p>
<div class="codehilite"><pre><span class="n">git</span> <span class="n">apply</span> <span class="o">-</span><span class="n">p0</span> <span class="o">--</span><span class="n">check</span> <span class="o">--</span><span class="n">verbose</span> <span class="o">--</span><span class="n">whitespace</span><span class="p">=</span><span class="nb">fix</span> <span class="o"><</span><span class="n">PATH</span><span class="o">-</span><span class="n">TO</span><span class="o">-</span><span class="n">PATCH</span><span class="o">></span>
</pre></div>
@@ -254,7 +265,7 @@ Latest release: <strong>0.80.0-incubatin
<p>Then: clean build, test. </p>
-<h3 id="formatted-patches_1">Formatted patches</h3>
+<h3 id="formatted-patches_1">Formatted patches<a class="headerlink" href="#formatted-patches_1" title="Permanent link">¶</a></h3>
<p>Formatted patches, with the commit history, are applied with <code>git am</code></p>
<div class="codehilite"><pre><span class="n">git</span> <span class="n">am</span> <span class="o">--</span><span class="n">whitespace</span><span class="p">=</span><span class="nb">fix</span> <span class="o"><</span> <span class="o"><</span><span class="n">PATH</span><span class="o">-</span><span class="n">TO</span><span class="o">-</span><span class="n">PATCH</span><span class="o">></span>
</pre></div>
@@ -265,7 +276,7 @@ Latest release: <strong>0.80.0-incubatin
</pre></div>
-<h2 id="reviewing-patches">Reviewing patches</h2>
+<h2 id="reviewing-patches">Reviewing patches<a class="headerlink" href="#reviewing-patches" title="Permanent link">¶</a></h2>
<p>After being applied, the feature branch must be committed. This ensures that there is a history of the submitted patch. The commit message must include the patch filename.</p>
<p>As well as reviewing the code and its style, all tests that it patch must be run before attempting to merge the code back into the develop branch.</p>
<p>Ideally, the full test run, minicluster and functional should be performed. A less-reliable shortcut is to run all the tests that are changed in the patch, and all tests which are known to depend on the expected feature. As an example, if a client command is changed, search for uses of that command and execute those tests.</p>
Modified: websites/staging/slider/trunk/content/developing/testing.html
==============================================================================
--- websites/staging/slider/trunk/content/developing/testing.html (original)
+++ websites/staging/slider/trunk/content/developing/testing.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,7 +196,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="testing-apache-slider">Testing Apache Slider</h1>
+<h1 id="testing-apache-slider">Testing Apache Slider<a class="headerlink" href="#testing-apache-slider" title="Permanent link">¶</a></h1>
<div class="codehilite"><pre> <span class="n">The</span> <span class="n">key</span> <span class="n">words</span> "<span class="n">MUST</span>"<span class="p">,</span> "<span class="n">MUST</span> <span class="n">NOT</span>"<span class="p">,</span> "<span class="n">REQUIRED</span>"<span class="p">,</span> "<span class="n">SHALL</span>"<span class="p">,</span> "<span class="n">SHALL</span>
<span class="n">NOT</span>"<span class="p">,</span> "<span class="n">SHOULD</span>"<span class="p">,</span> "<span class="n">SHOULD</span> <span class="n">NOT</span>"<span class="p">,</span> "<span class="n">RECOMMENDED</span>"<span class="p">,</span> "<span class="n">MAY</span>"<span class="p">,</span> <span class="n">and</span>
"<span class="n">OPTIONAL</span>" <span class="n">in</span> <span class="n">this</span> <span class="n">document</span> <span class="n">are</span> <span class="n">to</span> <span class="n">be</span> <span class="n">interpreted</span> <span class="n">as</span> <span class="n">described</span> <span class="n">in</span>
@@ -193,7 +204,7 @@ Latest release: <strong>0.80.0-incubatin
</pre></div>
-<h2 id="standalone-tests">Standalone Tests</h2>
+<h2 id="standalone-tests">Standalone Tests<a class="headerlink" href="#standalone-tests" title="Permanent link">¶</a></h2>
<p>Slider core contains a suite of tests that are designed to run on the local machine,
using Hadoop's <code>MiniDFSCluster</code> and <code>MiniYARNCluster</code> classes to create small,
one-node test clusters. All the YARN/HDFS code runs in the JUnit process; the
Modified: websites/staging/slider/trunk/content/developing/windows.html
==============================================================================
--- websites/staging/slider/trunk/content/developing/windows.html (original)
+++ websites/staging/slider/trunk/content/developing/windows.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,37 +193,37 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="windows-development-and-testing">Windows Development and Testing</h1>
+<h1 id="windows-development-and-testing">Windows Development and Testing<a class="headerlink" href="#windows-development-and-testing" title="Permanent link">¶</a></h1>
<p>This documents what you need to build and test on Windows</p>
<ol>
<li>Windows server 2012</li>
<li>Create an admin account for yourself</li>
<li>On a virtual machine: take snapshots as you go along</li>
</ol>
-<h1 id="core-installs">Core installs</h1>
+<h1 id="core-installs">Core installs<a class="headerlink" href="#core-installs" title="Permanent link">¶</a></h1>
<ul>
<li>Install <a href="http://slproweb.com/products/Win32OpenSSL.html">openssl 1.0.1i 64-bit</a>
into a dir with no spaces, <code>C:\bin\openssl</code>. Add its bin dir to your path.
(this will also need the Visual C++ 2008 Redistributables (DLL libraries)
installed.</li>
</ul>
-<h2 id="virtual-box">Virtual Box</h2>
+<h2 id="virtual-box">Virtual Box<a class="headerlink" href="#virtual-box" title="Permanent link">¶</a></h2>
<ol>
<li>Ask for lots of RAM, CPU. You can vary CPU and limits as you go, and
increase them for test runs/full screen work.</li>
<li>2D acceleration of video</li>
<li>Critical: Enable PAE under "processor". this avoids random hangs.</li>
</ol>
-<h3 id="os-features">OS features</h3>
+<h3 id="os-features">OS features<a class="headerlink" href="#os-features" title="Permanent link">¶</a></h3>
<p>In the server management app, go to "manage features", where you can add</p>
<ul>
<li>Telnet server: lets you telnet in.</li>
<li>Maybe: desktop experience</li>
</ul>
-<h3 id="os-settings">OS settings</h3>
+<h3 id="os-settings">OS settings<a class="headerlink" href="#os-settings" title="Permanent link">¶</a></h3>
<p>Firewall setup is a mystery. You need to convince the OS that your Ether network
is "private", then edit the firewall to open it up.</p>
-<h2 id="dev-tools">Dev tools</h2>
+<h2 id="dev-tools">Dev tools<a class="headerlink" href="#dev-tools" title="Permanent link">¶</a></h2>
<ol>
<li>Bitvise SSH server; add your private key to the account you create.
(even with this installed, you will still be mostly working from the windows
@@ -230,7 +241,7 @@ view all web pages in IE to verify they
<li><a href="http://technet.microsoft.com/en-us/sysinternals/bb545021.aspx">Sysinternals</a>.
Their <code>tcpview</code> tool is ideal for seeing what ports Hadoop services are listening on.</li>
</ol>
-<h3 id="knowledge">Knowledge</h3>
+<h3 id="knowledge">Knowledge<a class="headerlink" href="#knowledge" title="Permanent link">¶</a></h3>
<ol>
<li>Knowledge of the <a href="http://technet.microsoft.com/en-us/library/bb490954.aspx">CMD.EXE command line</a></li>
<li>
@@ -246,7 +257,7 @@ Their <code>tcpview</code> tool is ideal
</ol>
</li>
</ol>
-<h2 id="windows">Windows</h2>
+<h2 id="windows">Windows<a class="headerlink" href="#windows" title="Permanent link">¶</a></h2>
<ol>
<li>Follow the Hadoop <code>BUILDING.TXT</code> instructions. </li>
<li>install cygwin64 with it's <code>sh</code> command.</li>
@@ -265,11 +276,11 @@ not any cygwin one.</li>
<em> <code>awk</code>
</em> <code>grep</code>
* <code>xargs</code></p>
-<h2 id="avoid">Avoid</h2>
+<h2 id="avoid">Avoid<a class="headerlink" href="#avoid" title="Permanent link">¶</a></h2>
<ul>
<li>having cygwin <code>cmake</code> on your path; it must be the native cmake</li>
</ul>
-<h3 id="env-variables">env variables</h3>
+<h3 id="env-variables">env variables<a class="headerlink" href="#env-variables" title="Permanent link">¶</a></h3>
<div class="codehilite"><pre><span class="n">CYGWIN</span><span class="p">=</span><span class="n">nodosfilewarning</span>
<span class="n">PATH</span> <span class="p">=</span> <span class="n">cmake</span> <span class="n">bin</span><span class="p">,</span> <span class="n">cygwin</span> <span class="n">bin</span><span class="p">,</span> <span class="n">python</span> <span class="n">bin</span><span class="p">,</span> <span class="n">hadoop_home</span> <span class="n">bin</span> <span class="p">(</span><span class="n">see</span> <span class="n">below</span><span class="p">)</span>
</pre></div>
@@ -286,7 +297,7 @@ not any cygwin one.</li>
</pre></div>
-<h3 id="commands-to-test-cli-setup">Commands to test CLI setup</h3>
+<h3 id="commands-to-test-cli-setup">Commands to test CLI setup<a class="headerlink" href="#commands-to-test-cli-setup" title="Permanent link">¶</a></h3>
<div class="codehilite"><pre><span class="n">cmake</span> <span class="o">--</span><span class="n">version</span>
<span class="n">openssl</span> <span class="n">version</span>
<span class="n">python</span> <span class="o">--</span><span class="n">version</span>
@@ -308,14 +319,14 @@ not any cygwin one.</li>
</pre></div>
-<h3 id="sliders-test-for-windows-support">Slider's test for windows support</h3>
+<h3 id="sliders-test-for-windows-support">Slider's test for windows support<a class="headerlink" href="#sliders-test-for-windows-support" title="Permanent link">¶</a></h3>
<p>You can test for windows support with the slider-core test</p>
<div class="codehilite"><pre><span class="n">mvn</span> <span class="n">test</span> <span class="o">-</span><span class="n">Dtest</span><span class="p">=</span><span class="n">TestWindowsSupport</span>
</pre></div>
<p>This looks for the arguments and other valid settings</p>
-<h2 id="building-hadoop">Building Hadoop</h2>
+<h2 id="building-hadoop">Building Hadoop<a class="headerlink" href="#building-hadoop" title="Permanent link">¶</a></h2>
<ol>
<li>Find the "Windows SDK 7.1 command prompt" using the search field in the
windows 8 screen</li>
Modified: websites/staging/slider/trunk/content/docs/api/index.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/api/index.html (original)
+++ websites/staging/slider/trunk/content/docs/api/index.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,7 +193,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="slider-rest-apis">Slider REST APIs</h1>
+<h1 id="slider-rest-apis">Slider REST APIs<a class="headerlink" href="#slider-rest-apis" title="Permanent link">¶</a></h1>
<ul>
<li><a href="slider_REST_v1.html">REST API v1</a>: Implemented in Slider 0.60</li>
<li><a href="slider_REST_api_v2.html">REST API v2</a>: V2 API</li>
Modified: websites/staging/slider/trunk/content/docs/api/slider_REST_api_v2.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/api/slider_REST_api_v2.html (original)
+++ websites/staging/slider/trunk/content/docs/api/slider_REST_api_v2.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,7 +193,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="slider-am-rest-api-v2">Slider AM REST API: v2</h1>
+<h1 id="slider-am-rest-api-v2">Slider AM REST API: v2<a class="headerlink" href="#slider-am-rest-api-v2" title="Permanent link">¶</a></h1>
<p>This is a successor to the <a href="slider_REST_v1.html">Slider v1 REST API</a></p>
<p>This document represents the third iteration of designing a REST API to be implemented by the Slider Application Master.</p>
<div class="codehilite"><pre> <span class="n">The</span> <span class="n">key</span> <span class="n">words</span> "<span class="n">MUST</span>"<span class="p">,</span> "<span class="n">MUST</span> <span class="n">NOT</span>"<span class="p">,</span> "<span class="n">REQUIRED</span>"<span class="p">,</span> "<span class="n">SHALL</span>"<span class="p">,</span> "<span class="n">SHALL</span>
@@ -192,11 +203,11 @@ Latest release: <strong>0.80.0-incubatin
</pre></div>
-<h2 id="history">History</h2>
+<h2 id="history">History<a class="headerlink" href="#history" title="Permanent link">¶</a></h2>
<ol>
<li>Created: January 2014</li>
</ol>
-<h1 id="introduction-and-background">Introduction and Background</h1>
+<h1 id="introduction-and-background">Introduction and Background<a class="headerlink" href="#introduction-and-background" title="Permanent link">¶</a></h1>
<p>Slider 0.60 uses Hadoop IPC for communications between the Slider client and the per-instance application master,
with a READ-only JSON view of the cluster, as documented in the <a href="slider_REST_v1.html">Slider v1 REST API</a></p>
<p>Were Slider to support a read/write REST API, it would be possible to:</p>
@@ -211,7 +222,7 @@ with a READ-only JSON view of the cluste
<p>Offer alternative methods of constructing an application</p>
</li>
</ol>
-<h2 id="slider-configuration-model-and-rest">Slider Configuration Model and REST</h2>
+<h2 id="slider-configuration-model-and-rest">Slider Configuration Model and REST<a class="headerlink" href="#slider-configuration-model-and-rest" title="Permanent link">¶</a></h2>
<p>Slider's declarative view of the application to deploy fits in well with a REST world view: one does not make calls to operations such as "increase region server count by two", instead the JSON specification of YARN resources is altered such that the region server count is implemented, then the new JSON document submitted. currently this is done via IPC.</p>
<p>Where Slider does not integrate well with REST is</p>
<ol>
@@ -228,7 +239,7 @@ with a READ-only JSON view of the cluste
<p>No support of configuration update of a running application. The content may be read, but a write is not supported (or if it is, they are not addressed until the application is next restarted).</p>
</li>
</ol>
-<h2 id="existing-ipc-api">Existing IPC API</h2>
+<h2 id="existing-ipc-api">Existing IPC API<a class="headerlink" href="#existing-ipc-api" title="Permanent link">¶</a></h2>
<p>The slider IPC layer uses protobuf-formatted payloads, with the Hadoop IPC layer handling security: authorization, authentication and encryption.</p>
<div class="codehilite"><pre><span class="n">service</span> <span class="n">SliderClusterProtocolPB</span> <span class="p">{</span>
@@ -271,10 +282,10 @@ with a READ-only JSON view of the cluste
</li>
</ol>
<p>Note that for a "version robust" marshalling format, Protobuf is (a) hard to work with in Java and (b) very hard to examine at the payload layer in Java (e.g. to enumerate elements which were not known at compile time) and (c) due to Google's lack of backwards compatibility in libraries and generated code, very brittle in the Java source. </p>
-<h1 id="use-cases-of-an-am-rest-api">Use Cases of an AM REST API</h1>
+<h1 id="use-cases-of-an-am-rest-api">Use Cases of an AM REST API<a class="headerlink" href="#use-cases-of-an-am-rest-api" title="Permanent link">¶</a></h1>
<p>Here are the possible different cases of a Slider REST API. </p>
<p>Each one has different requirements âso the priority of supporting different use cases will scope and direct effort</p>
-<h2 id="command-line-tooling">Command Line tooling</h2>
+<h2 id="command-line-tooling">Command Line tooling<a class="headerlink" href="#command-line-tooling" title="Permanent link">¶</a></h2>
<p>Direct communication with Slider via <code>curl</code>, <code>wget</code>, Python, and other lightweight tooling, rather than exclusively via the Java Slider JAR</p>
<ul>
<li>
@@ -295,7 +306,7 @@ with a READ-only JSON view of the cluste
<p>Scripts may need to have the ability to block on an operation until the application reaches a desired state (e.g containers match requested count)</p>
</li>
</ul>
-<h2 id="web-ui">Web UI</h2>
+<h2 id="web-ui">Web UI<a class="headerlink" href="#web-ui" title="Permanent link">¶</a></h2>
<p>The (currently minimal) Slider Web UI could forward operations the REST API via HTML/HTML5 forms.</p>
<p>The state of the application could be presented better than it is today.</p>
<p>It would also be possible to build a more complex web application that that offered by slider today.</p>
@@ -305,7 +316,7 @@ with a READ-only JSON view of the cluste
<ul>
<li>Support for HTML form submission required.</li>
</ul>
-<h2 id="management-tooling">Management tooling</h2>
+<h2 id="management-tooling">Management tooling<a class="headerlink" href="#management-tooling" title="Permanent link">¶</a></h2>
<p>The example of this is Apache Ambari, but it is not restricted to this program; Ambari is merely a representative example of "<em>a web application launching and controlling an application via slider, on behalf of users"</em>.</p>
<p>We know today that such applications built in Java do not need a REST API; the slider client itself can be used for this. </p>
<p>What a REST API could do is</p>
@@ -342,7 +353,7 @@ with a READ-only JSON view of the cluste
<p>May want more slider application metrics.</p>
</li>
</ul>
-<h2 id="long-haul-client">Long-haul Client</h2>
+<h2 id="long-haul-client">Long-haul Client<a class="headerlink" href="#long-haul-client" title="Permanent link">¶</a></h2>
<p>A long-haul clients is probably the most complex client application. It can be probably be done within the slider client codebase, so allowing remote application creation and manipulation.</p>
<p>Remote cloud deployments are a key target here âso we cannot expect the cluster's HDFS storage to be persistent over time. Instead we must keep persistent data (packages, JSON configurations) in the platform's persistent store (amazon S3, Azure AVS, etc). YARN node managers do already "localize" resources served up this way; persisting application state may be more complex if the consistency model of the object store does not match that of HDFS.</p>
<p><strong>Requirements</strong></p>
@@ -381,7 +392,7 @@ with a READ-only JSON view of the cluste
<p>Persistent application instance configuration to be in the persistent data store rather than transient HDFS storage.</p>
</li>
</ul>
-<h2 id="functional-testing">Functional Testing</h2>
+<h2 id="functional-testing">Functional Testing<a class="headerlink" href="#functional-testing" title="Permanent link">¶</a></h2>
<p>Slider uses the slider-client as an in-VM library during its minicluster unit tests.</p>
<p>For functional testing, it uses the slider CLI as an external application. This guarantees full testing of the CLI, including the shell/python scripts themselves. It was this testing which picked up some problems with the python script on windows, and a later regression related to accumulo. It also forces us to ensure that the return codes of operations differentiate between different failure causes, rather than providing a simple "-1" error indicating that an operation failed. Our <a href="/docs/exitcodes.html">exit codes</a> are now something which may be used for support and debugging.</p>
<p>A REST API could also be used for testing, though not, for the reasons above, by slider itself, except in the special case of functional tests of the REST API itself.</p>
@@ -413,7 +424,7 @@ with a READ-only JSON view of the cluste
<p>API calls to probe for state (ideally blocking)</p>
</li>
</ul>
-<h2 id="deployed-application">Deployed Application</h2>
+<h2 id="deployed-application">Deployed Application<a class="headerlink" href="#deployed-application" title="Permanent link">¶</a></h2>
<p>This is a use case which came from the Storm team: give the application the ability to talk to Slider and so query and manipulate its own deployment.</p>
<p>This allows the application to expand and contract itself based on perceived need, and to explicitly release specific components which are no longer required. It can also expose the YARN cluster details to the application, so allowing the deployed application to build a model of the YARN cluster without talking directly to it.</p>
<p>In this design, the Slider AM's REST API is no longer for clients of the application, or even management tools âit becomes the API by which deployed applications integrate with YARN. To use the current fashionable terminology, it becomes a "microservice" rather than a library.</p>
@@ -444,8 +455,8 @@ with a READ-only JSON view of the cluste
<p>Ability to query recent event history.</p>
</li>
</ul>
-<h1 id="api-principles-high-rest-with-asynchronous-state-changes">API Principles: High Rest with Asynchronous state changes</h1>
-<h2 id="uris-for-overall-and-detailed-access">URIs for overall and detailed access</h2>
+<h1 id="api-principles-high-rest-with-asynchronous-state-changes">API Principles: High Rest with Asynchronous state changes<a class="headerlink" href="#api-principles-high-rest-with-asynchronous-state-changes" title="Permanent link">¶</a></h1>
+<h2 id="uris-for-overall-and-detailed-access">URIs for overall and detailed access<a class="headerlink" href="#uris-for-overall-and-detailed-access" title="Permanent link">¶</a></h2>
<p>Resources SHOULD use URIs over ? parameters or arguments within the body. </p>
<ul>
<li>
@@ -466,10 +477,10 @@ with a READ-only JSON view of the cluste
</ol>
</li>
</ul>
-<h3 id="use-and-generate-standard-http-headers-when-possible">Use and generate standard HTTP Headers when possible</h3>
+<h3 id="use-and-generate-standard-http-headers-when-possible">Use and generate standard HTTP Headers when possible<a class="headerlink" href="#use-and-generate-standard-http-headers-when-possible" title="Permanent link">¶</a></h3>
<p>If there is a standard HTTP header for an option (such as a <code>range:</code> header), it MUST be used. This boosts compatibility with browsers and existing applications.</p>
<p>The services MUST return information that defines cache duration of retrieved data, possibly 0 seconds. Proxy caching MUST be disabled. (this comes for free with the <code>NoCacheFilter</code> âtests are needed to verify the filter is adding the values)</p>
-<h2 id="get-for-queries-and-only-queries">GET for queries âand only queries</h2>
+<h2 id="get-for-queries-and-only-queries">GET for queries âand only queries<a class="headerlink" href="#get-for-queries-and-only-queries" title="Permanent link">¶</a></h2>
<ol>
<li>
<p>All side-effect free queries MUST be implemented via GET operations.</p>
@@ -479,23 +490,23 @@ with a READ-only JSON view of the cluste
</li>
</ol>
<p>Rule #1 is for a coherent REST API. Rule #2 is mandated in the HTTP specification, and assumed to hold by those browsers which perform pre-emptive fetching.</p>
-<h3 id="put-for-overwrites-to-existing-resources-or-explicit-creation-of-new-ones">PUT for overwrites to existing resources, or explicit creation of new ones.</h3>
+<h3 id="put-for-overwrites-to-existing-resources-or-explicit-creation-of-new-ones">PUT for overwrites to existing resources, or explicit creation of new ones.<a class="headerlink" href="#put-for-overwrites-to-existing-resources-or-explicit-creation-of-new-ones" title="Permanent link">¶</a></h3>
<p>If a URL references a valid resource, and an update to it makes sense (e.g. overwriting an existing resource topology with a new declaration), then the PUT verb SHOULD be preferred to POST.</p>
<p>It MAY also be used for resource creation operations âbut only if the result of the PUT is a new resource at the final URL specified.</p>
-<h2 id="put-operations-must-be-idempotent">PUT operations MUST be idempotent</h2>
+<h2 id="put-operations-must-be-idempotent">PUT operations MUST be idempotent<a class="headerlink" href="#put-operations-must-be-idempotent" title="Permanent link">¶</a></h2>
<p>If a PUT operation is repeated, the final state of the model MUST be the same.</p>
<p>Processing of the initial PUT may result in external/visible actions. These actions SHOULD NOT be repeated when the second PUT is received. As an example, a PUT, twice, of a new resources.json model should eventually result in the final resource counts matching the desired state, without more container creation and deletion than if a single PUT had occurred. </p>
<p>Operations which aren't idempotent MUST NOT be implemented as PUT; instead they </p>
-<h2 id="delete-for-resource-deletion-operations">DELETE for resource deletion operations</h2>
+<h2 id="delete-for-resource-deletion-operations">DELETE for resource deletion operations<a class="headerlink" href="#delete-for-resource-deletion-operations" title="Permanent link">¶</a></h2>
<p>If resources are to be deleted, then DELETE is the operation of choice.</p>
-<h3 id="post-operations-for-system-state-changes-that-do-not-match-a-resource-model">POST operations for system state changes that do not match a resource model.</h3>
+<h3 id="post-operations-for-system-state-changes-that-do-not-match-a-resource-model">POST operations for system state changes that do not match a resource model.<a class="headerlink" href="#post-operations-for-system-state-changes-that-do-not-match-a-resource-model" title="Permanent link">¶</a></h3>
<p>Operations which do not map directly the resource model SHOULD be implemented as POST operations. </p>
-<h3 id="post-operations-may-be-non-idempotent">POST operations MAY be non-idempotent</h3>
+<h3 id="post-operations-may-be-non-idempotent">POST operations MAY be non-idempotent<a class="headerlink" href="#post-operations-may-be-non-idempotent" title="Permanent link">¶</a></h3>
<p>There is no requirement for POST operations to be idempotent.</p>
-<h2 id="payloads-should-be-json-payloads">Payloads SHOULD be JSON payloads</h2>
+<h2 id="payloads-should-be-json-payloads">Payloads SHOULD be JSON payloads<a class="headerlink" href="#payloads-should-be-json-payloads" title="Permanent link">¶</a></h2>
<p>The bodies of operations SHOULD be JSON.</p>
<p>POST operations MAY ALSO support application/x-www-form-urlencoded, so as to handle data directly from an HTML form.</p>
-<h2 id="errors-must-be-meaningful">Errors MUST be meaningful</h2>
+<h2 id="errors-must-be-meaningful">Errors MUST be meaningful<a class="headerlink" href="#errors-must-be-meaningful" title="Permanent link">¶</a></h2>
<p>If an operation fails enough information should be provided to aid diagnosis of the problem.</p>
<ol>
<li>
@@ -509,17 +520,17 @@ with a READ-only JSON view of the cluste
</li>
</ol>
<p>Jersey is going to interfere here with its own exception logic; methods must catch all exceptions and convert them to <code>WebAppException</code> instances to avoid them being mishandled.</p>
-<h2 id="the-api-must-be-secure">The API MUST be Secure</h2>
+<h2 id="the-api-must-be-secure">The API MUST be Secure<a class="headerlink" href="#the-api-must-be-secure" title="Permanent link">¶</a></h2>
<p>The REST API must be secure. In the context of a YARN application, this means all communications in a secure cluster must be via the Kerberos/SPNEGO-authenticated ResourceManager proxy.</p>
<p><strong>Development time exception</strong> : disable the proxy on the /ws/ path of the web application, so that the full set of HTTP verbs can be used, without depending on Hadoop-2.7 proxy improvements.</p>
-<h2 id="minimal">Minimal</h2>
+<h2 id="minimal">Minimal<a class="headerlink" href="#minimal" title="Permanent link">¶</a></h2>
<p>Features implemented via HDFS and YARN SHOULD NOT be re-implemented in the Slider AM REST API. </p>
-<h2 id="asynchronous-actions-and-state-changes">Asynchronous Actions and state-changes</h2>
+<h2 id="asynchronous-actions-and-state-changes">Asynchronous Actions and state-changes<a class="headerlink" href="#asynchronous-actions-and-state-changes" title="Permanent link">¶</a></h2>
<p>All state changes are asynchronous, serialized and queued within the AM.</p>
<p>This is what happens today; there is some optimisation for handling multiple cluster-size changing events in the queue such that a "review and request containers" operation is postponed until all pending size-changing events (flex, container-loss, ...) have been processed. </p>
<p>This means that while REST operations (and YARN events) are queued in the order of receipt, some operations âsuch as a flex operationâ may not have any work performed while later events arrive in the queue.</p>
<p>The response code to an asynchronous operation SHOULD be 201, ACCEPTED</p>
-<h1 id="yarn-dependencies">YARN dependencies</h1>
+<h1 id="yarn-dependencies">YARN dependencies<a class="headerlink" href="#yarn-dependencies" title="Permanent link">¶</a></h1>
<p>What do we need from YARN?</p>
<ol>
<li>
@@ -541,7 +552,7 @@ with a READ-only JSON view of the cluste
<p>For a pure-REST client, a RESTy registry API in both YARN <em>and KNOX.</em></p>
</li>
</ol>
-<h1 id="resource-structure">Resource Structure</h1>
+<h1 id="resource-structure">Resource Structure<a class="headerlink" href="#resource-structure" title="Permanent link">¶</a></h1>
<p><img alt="Slider REST model" src="slider_REST_model_v2.jpg" title="Slider REST Resource model" /></p>
<p>Core concepts:</p>
<ol>
@@ -552,12 +563,12 @@ with a READ-only JSON view of the cluste
<p>The live view of what is going on in the application under <code>/application/model</code>.</p>
</li>
</ol>
-<h2 id="application">/application</h2>
-<h3 id="all-application-resources">All Application resources</h3>
+<h2 id="application">/application<a class="headerlink" href="#application" title="Permanent link">¶</a></h2>
+<h3 id="all-application-resources">All Application resources<a class="headerlink" href="#all-application-resources" title="Permanent link">¶</a></h3>
<p>All entries will be under the service path <code>/application</code>, which itself is under the <code>/ws/v1/</code> path of the Slider web interface.</p>
-<h2 id="applicationmodel">/application/model/ :</h2>
-<h3 id="get-and-for-some-urls-put-view-of-the-specification">GET/ and, for some URLs, PUT view of the specification</h3>
-<h3 id="applicationmodeldesired">/application/model/desired/</h3>
+<h2 id="applicationmodel">/application/model/ :<a class="headerlink" href="#applicationmodel" title="Permanent link">¶</a></h2>
+<h3 id="get-and-for-some-urls-put-view-of-the-specification">GET/ and, for some URLs, PUT view of the specification<a class="headerlink" href="#get-and-for-some-urls-put-view-of-the-specification" title="Permanent link">¶</a></h3>
+<h3 id="applicationmodeldesired">/application/model/desired/<a class="headerlink" href="#applicationmodeldesired" title="Permanent link">¶</a></h3>
<p>This is where the specification of the application: resources and configuration, can be read and written. </p>
<ol>
<li>
@@ -567,12 +578,12 @@ with a READ-only JSON view of the cluste
<p>Write accesses to <code>configuration</code> will only take effect on a cluster upgrade or restart</p>
</li>
</ol>
-<h3 id="applicationmodelresolved">/application/model/resolved/</h3>
+<h3 id="applicationmodelresolved">/application/model/resolved/<a class="headerlink" href="#applicationmodelresolved" title="Permanent link">¶</a></h3>
<p>The resolved specification, the one where we implement the inheritance, and, when we eventually do x-refs, all non-LAZY references. This lets the caller see the final configuration model.</p>
-<h3 id="applicationmodelinternal">/application/model/internal/</h3>
+<h3 id="applicationmodelinternal">/application/model/internal/<a class="headerlink" href="#applicationmodelinternal" title="Permanent link">¶</a></h3>
<p>Read-only view of <code>internal.json</code>. Exported for diagnostics and completeness.</p>
-<h2 id="applicationlive">/application/live/ :</h2>
-<h3 id="get-and-delete-view-of-the-live-application">GET and DELETE view of the live application</h3>
+<h2 id="applicationlive">/application/live/ :<a class="headerlink" href="#applicationlive" title="Permanent link">¶</a></h2>
+<h3 id="get-and-delete-view-of-the-live-application">GET and DELETE view of the live application<a class="headerlink" href="#get-and-delete-view-of-the-live-application" title="Permanent link">¶</a></h3>
<p>This provides different views of the system, something which we can delve into</p>
<ol>
<li>
@@ -601,10 +612,10 @@ DELETE node_id will decommission all con
<p>"system" state: AM state, outstanding requests, upgrade in progress</p>
</li>
</ol>
-<h2 id="applicationactions">/application/actions</h2>
-<h3 id="post-state-changing-operations">POST state changing operations</h3>
+<h2 id="applicationactions">/application/actions<a class="headerlink" href="#applicationactions" title="Permanent link">¶</a></h2>
+<h3 id="post-state-changing-operations">POST state changing operations<a class="headerlink" href="#post-state-changing-operations" title="Permanent link">¶</a></h3>
<p>These are for operations which are hard to represent in a simple REST view within the AM itself.</p>
-<h1 id="proposed-state-query-operations">Proposed State Query Operations</h1>
+<h1 id="proposed-state-query-operations">Proposed State Query Operations<a class="headerlink" href="#proposed-state-query-operations" title="Permanent link">¶</a></h1>
<p>All of these are GET operations on data that is not directly mutable</p>
<table>
<tr>
@@ -679,7 +690,7 @@ DELETE node_id will decommission all con
<p>All live values will be cached and refreshed regularly; the caching ensures that a heavy load of GET operations does not overload the application master.
It does mean that there may be a delay of under a second
before an updated value is visible.</p>
-<h1 id="actions">Actions</h1>
+<h1 id="actions">Actions<a class="headerlink" href="#actions" title="Permanent link">¶</a></h1>
<p>Actions are POST operations.</p>
<table>
<tr>
@@ -713,8 +724,8 @@ before an updated value is visible.</p>
<p>It MAY be possible to overwrite an existing operation with a new one, though that will depend on the active operation. Specifically, "upgrade" would only support STOP; "stop" would only support "stop". the empty operation, "" will support anything</p>
</li>
</ol>
-<h1 id="non-normative-example-data-structures">Non-normative Example Data structures</h1>
-<h2 id="applicationliveresources">application/live/resources</h2>
+<h1 id="non-normative-example-data-structures">Non-normative Example Data structures<a class="headerlink" href="#non-normative-example-data-structures" title="Permanent link">¶</a></h1>
+<h2 id="applicationliveresources">application/live/resources<a class="headerlink" href="#applicationliveresources" title="Permanent link">¶</a></h2>
<p>The contents of application/live/resources on an application which only has an application master deployed. The entries in italic are the statistics related to the live state; the remainder the original values.</p>
<div class="codehilite"><pre><span class="p">{</span>
"<span class="n">schema</span>" <span class="p">:</span> "<span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">example</span><span class="p">.</span><span class="n">org</span><span class="o">/</span><span class="n">specification</span><span class="o">/</span><span class="n">v2</span><span class="p">.</span>0<span class="p">.</span>0"<span class="p">,</span>
@@ -741,7 +752,7 @@ before an updated value is visible.</p>
</pre></div>
-<h2 id="liveliveness"><code>live/liveness</code></h2>
+<h2 id="liveliveness"><code>live/liveness</code><a class="headerlink" href="#liveliveness" title="Permanent link">¶</a></h2>
<p>The liveness URL returns a JSON structure on the liveness of the
application as perceived by Slider itself.</p>
<p>See <code>org.apache.slider.api.types.ApplicationLivenessInformation</code></p>
@@ -757,7 +768,7 @@ application as perceived by Slider itsel
percentage of desired containers of each component type have been allocated
âor even how many were actually running.</p>
<p>Any new liveness probes will supplement rather than replace the current values.</p>
-<h2 id="to-be-defined-livestatisticts">To be defined: <code>live/statisticts</code></h2>
+<h2 id="to-be-defined-livestatisticts">To be defined: <code>live/statisticts</code><a class="headerlink" href="#to-be-defined-livestatisticts" title="Permanent link">¶</a></h2>
<p>The statistics will cover the collected statistics on a component type, as well
as aggregate statistics on an application instance.</p>
<p>When an Application Master is restarted, all statistics will be lost.</p>
Modified: websites/staging/slider/trunk/content/docs/api/slider_REST_v1.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/api/slider_REST_v1.html (original)
+++ websites/staging/slider/trunk/content/docs/api/slider_REST_v1.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,8 +193,8 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="slider-rest-apis-v1">Slider REST APIs v1</h1>
-<h2 id="december-2014">December 2014</h2>
+<h1 id="slider-rest-apis-v1">Slider REST APIs v1<a class="headerlink" href="#slider-rest-apis-v1" title="Permanent link">¶</a></h1>
+<h2 id="december-2014">December 2014<a class="headerlink" href="#december-2014" title="Permanent link">¶</a></h2>
<p>This document covers the external REST APIs of Slider</p>
<div class="codehilite"><pre> <span class="n">The</span> <span class="n">key</span> <span class="n">words</span> "<span class="n">MUST</span>"<span class="p">,</span> "<span class="n">MUST</span> <span class="n">NOT</span>"<span class="p">,</span> "<span class="n">REQUIRED</span>"<span class="p">,</span> "<span class="n">SHALL</span>"<span class="p">,</span> "<span class="n">SHALL</span>
<span class="n">NOT</span>"<span class="p">,</span> "<span class="n">SHOULD</span>"<span class="p">,</span> "<span class="n">SHOULD</span> <span class="n">NOT</span>"<span class="p">,</span> "<span class="n">RECOMMENDED</span>"<span class="p">,</span> "<span class="n">MAY</span>"<span class="p">,</span> <span class="n">and</span>
@@ -192,11 +203,11 @@ Latest release: <strong>0.80.0-incubatin
</pre></div>
-<h1 id="introduction-and-background">Introduction and Background</h1>
+<h1 id="introduction-and-background">Introduction and Background<a class="headerlink" href="#introduction-and-background" title="Permanent link">¶</a></h1>
<p>Slider 0.60 Uses Hadoop IPC for communications between the Slider client and the per-instance application master. This uses protobuf-formatted payloads, with the Hadoop IPC layer handling security: authorization, authentication and encryption.</p>
<p>It provides read-only views of generated content and state through a set of web service URLs. All these services are implemented by the Jersey JAX-RS engine, operating in an embedded Jetty Web engine, with the YARN <code>AmWebFilter</code> class redirecting all requests not coming from the RM Proxy IP address to that RM Proxy via a 302 (redirect-as-GET)</p>
<p>Excluding the internal-use-only the AM<->Agent channel operating on a private HTTP port, there are a number of current services</p>
-<h1 id="read-only-access-to-the-yarn-registry-wsv1sliderregistry">Read only access to the YARN registry: /ws/v1/slider/registry</h1>
+<h1 id="read-only-access-to-the-yarn-registry-wsv1sliderregistry">Read only access to the YARN registry: /ws/v1/slider/registry<a class="headerlink" href="#read-only-access-to-the-yarn-registry-wsv1sliderregistry" title="Permanent link">¶</a></h1>
<p>This was done primarily to get a browseable view of what the YARN-913 registry was publishing in the absence of any web view built in to that registry. The unit tests against this service actually discovered some JSON marshalling problems with the registry design and Jersey integration. </p>
<p>The root path <code>/v1/slider/registry</code> lists the root path of the registry; all entries below it follow the path in the registry. </p>
<div class="codehilite"><pre><span class="o">/</span><span class="n">ws</span><span class="o">/</span><span class="n">v1</span><span class="o">/</span><span class="n">slider</span><span class="o">/</span><span class="n">registry</span><span class="p">:</span> <span class="p">{</span>"<span class="n">nodes</span>"<span class="p">:[</span>"<span class="n">services</span>"<span class="p">,</span>"<span class="n">users</span>"<span class="p">]}</span>
@@ -272,7 +283,7 @@ Latest release: <strong>0.80.0-incubatin
<p>The slider code makes no direct use of this registry service except for a pair of tests. The Agents MUST use the core (ZK-based) registry service to locate a restarted AM. The slider client operations to enumerate and retrieve configuration files could use it, but they currently also use the YARN registry directly.</p>
<p>As and when the YARN registry itself implements a REST view (possibly an R/W one), then this view will become obsolete âand possibly disabled.</p>
-<h1 id="generated-configurations-wsv1sliderpublisher">Generated configurations: <code>/ws/v1/slider/publisher</code></h1>
+<h1 id="generated-configurations-wsv1sliderpublisher">Generated configurations: <code>/ws/v1/slider/publisher</code><a class="headerlink" href="#generated-configurations-wsv1sliderpublisher" title="Permanent link">¶</a></h1>
<p>This service contains a one or more "published configuration sets"</p>
<div class="codehilite"><pre><span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">devix</span><span class="p">.</span><span class="n">cotham</span><span class="p">.</span><span class="n">uk</span><span class="p">:</span>8088<span class="o">/</span><span class="n">proxy</span><span class="o">/</span><span class="n">application_1418130391694_0002</span><span class="o">/</span><span class="n">ws</span><span class="o">/</span><span class="n">v1</span><span class="o">/</span><span class="n">slider</span><span class="o">/</span><span class="n">publisher</span>
@@ -328,7 +339,7 @@ Latest release: <strong>0.80.0-incubatin
<p>This configuration publishing service is listed in the YARN Registry, and could be implemented by other applications to publish their configurations âincluding the standard YARN services.</p>
-<h1 id="application-state-in-slider-v1-format">Application state in slider v1 format</h1>
+<h1 id="application-state-in-slider-v1-format">Application state in slider v1 format<a class="headerlink" href="#application-state-in-slider-v1-format" title="Permanent link">¶</a></h1>
<p>The original "v1" formatted slider specification/state is provided, wrapped in HTML, at:</p>
<div class="codehilite"><pre><span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">devix</span><span class="p">.</span><span class="n">cotham</span><span class="p">.</span><span class="n">uk</span><span class="p">:</span>8088<span class="o">/</span><span class="n">proxy</span><span class="o">/</span><span class="n">application_1418130391694_0002</span><span class="o">/</span><span class="n">slideram</span><span class="o">/</span><span class="n">spec</span>
</pre></div>
@@ -336,7 +347,7 @@ Latest release: <strong>0.80.0-incubatin
<p>This combines the declaration of what is desired with information about what has actually been provided</p>
<p>(this shows the merits of a single file model which can describe both the desired state and the actual state; the same architecture of <a href="http://www.hpl.hp.com/techreports/2012/HPL-2012-158R1.pdf">the HPL Cells cloud infrastructure</a>)</p>
-<h1 id="slider-070-codahale-metrics-servlets">Slider 0.70: codahale metrics servlets</h1>
+<h1 id="slider-070-codahale-metrics-servlets">Slider 0.70: codahale metrics servlets<a class="headerlink" href="#slider-070-codahale-metrics-servlets" title="Permanent link">¶</a></h1>
<p>Slider 0.70 is adding metrics via the Codahale metrics library; as well as (optionally) publishing to Ganglia, the metrics service export metrics,health and liveness information, as well as a thread dump</p>
<table>
<tr>
Modified: websites/staging/slider/trunk/content/docs/client-configuration.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/client-configuration.html (original)
+++ websites/staging/slider/trunk/content/docs/client-configuration.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,9 +196,9 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="apache-slider-client-configuration">Apache Slider Client Configuration</h1>
+<h1 id="apache-slider-client-configuration">Apache Slider Client Configuration<a class="headerlink" href="#apache-slider-client-configuration" title="Permanent link">¶</a></h1>
<p>This document covers how the client application is itself configured.</p>
-<h2 id="summary">Summary</h2>
+<h2 id="summary">Summary<a class="headerlink" href="#summary" title="Permanent link">¶</a></h2>
<p>The client application can be configured</p>
<ol>
<li>On the command line, which can set client options and JVM system properties.</li>
@@ -201,7 +212,7 @@ directory.</li>
</ol>
<p>The options defined in a Slider cluster configuration are only used by the client
when creating a cluster -not for the actual client itself.</p>
-<h2 id="introduction">Introduction</h2>
+<h2 id="introduction">Introduction<a class="headerlink" href="#introduction" title="Permanent link">¶</a></h2>
<p>The Slider client needs to be configured to talk to a Hadoop filesystem and a
YARN resource manager ("the RM"). In a secure cluster it needs to be told the Kerberos
identity, the <em>principal</em> of both the HDFS namenode and the YARN RM -and it may
@@ -213,7 +224,7 @@ work on client machines that may not hav
<p>If these files are present, and the <code>HADOOP_CONF_DIR</code> environment variable is
set to the directory that contains them âslider will pick up the settings in
these files. If not, all settings must be placed in the file <code>slider-client.xml</code></p>
-<h2 id="setting-slider-jvm-options">Setting Slider JVM options</h2>
+<h2 id="setting-slider-jvm-options">Setting Slider JVM options<a class="headerlink" href="#setting-slider-jvm-options" title="Permanent link">¶</a></h2>
<p>Core JVM options can be set in the environment variable <code>SLIDER_JVM_OPTS</code>;
if unset the <code>bin/slider</code> script will use the default values that were
current when that version of Slider was released. These values may change
@@ -244,7 +255,7 @@ duplicate declarations. In such a case t
<p>For any system property that the user expects to have to issue on every command
-including any kerberos-related properties, adding them to the JVM options
environment variable guarantees that they are always set.</p>
-<h2 id="setting-slider-client-options-on-the-command-line-with-the-d-parameter">Setting Slider client options on the command line with the <code>-D</code> parameter</h2>
+<h2 id="setting-slider-client-options-on-the-command-line-with-the-d-parameter">Setting Slider client options on the command line with the <code>-D</code> parameter<a class="headerlink" href="#setting-slider-client-options-on-the-command-line-with-the-d-parameter" title="Permanent link">¶</a></h2>
<p>The slider client is configured via Hadoop-style configuration options.
To be precise, all standard Hadoop-common, hadoop-hdfs client and hadoop-yar
client-side options control how Slider communicates with the Hadoop YARN cluster.</p>
@@ -268,7 +279,7 @@ parameter followed by the appropriate <c
<p>If duplicate declarations are made the order of assignment is undefined.</p>
-<h1 id="setting-common-options-through-specific-command-line-arguments">Setting common options through specific command-line arguments</h1>
+<h1 id="setting-common-options-through-specific-command-line-arguments">Setting common options through specific command-line arguments<a class="headerlink" href="#setting-common-options-through-specific-command-line-arguments" title="Permanent link">¶</a></h1>
<p>Some Hadoop and Slider options are so common that they have specific
shortcut commands to aid their use</p>
<p><code>-m</code>, <code>--manager</code> : sets the YARN resource manager. Equivalent to setting the
@@ -277,7 +288,7 @@ shortcut commands to aid their use</p>
<code>fs.defaultFS</code> option</p>
<p>If these shortcuts are used and the options are also defined via <code>-D</code>
declarations, the order of assignment is undefined.</p>
-<h1 id="defining-hadoop-and-slider-options-in-the-slider-clientxml-file">Defining Hadoop and Slider Options in the <code>slider-client.xml</code> file.</h1>
+<h1 id="defining-hadoop-and-slider-options-in-the-slider-clientxml-file">Defining Hadoop and Slider Options in the <code>slider-client.xml</code> file.<a class="headerlink" href="#defining-hadoop-and-slider-options-in-the-slider-clientxml-file" title="Permanent link">¶</a></h1>
<p>In the Slider installation, alongside the <code>bin/slider</code> script is
a configuration directory <code>conf</code>. This contains the files:</p>
<ol>
@@ -312,11 +323,11 @@ to declare either on the command line.</
<p>If an option is defined in the <code>slider-client.xml</code> file and on the command line
-be it by a <code>-D key=value</code> declaration or a <code>--manager</code> or <code>--filesystem</code>
definition. (this holds even if the value is declared with <code><final>true</final></code>).</p>
-<h2 id="selecting-an-alternate-slider-configuration-directory">Selecting an alternate Slider configuration directory</h2>
+<h2 id="selecting-an-alternate-slider-configuration-directory">Selecting an alternate Slider configuration directory<a class="headerlink" href="#selecting-an-alternate-slider-configuration-directory" title="Permanent link">¶</a></h2>
<p>The environment variable <code>SLIDER_CONF_DIR</code> can be used to declare an alternate
configuration directory. If set, the directory it identifies will be used
as the source of the <code>log4j.properties</code> and <code>slider-client.xml</code> files.</p>
-<h2 id="slider-client-configuration-options">Slider Client Configuration options</h2>
+<h2 id="slider-client-configuration-options">Slider Client Configuration options<a class="headerlink" href="#slider-client-configuration-options" title="Permanent link">¶</a></h2>
<p>As well as standard YARN and Hadoop configuration options, Slider supports
a limited number of slider-specific configuration parameters.</p>
<div class="codehilite"><pre><span class="nt"><property></span>
@@ -348,7 +359,7 @@ a limited number of slider-specific conf
</pre></div>
-<h3 id="slideryarnrestartlimit-set-limit-on-application-master-restarts"><code>slider.yarn.restart.limit</code> - set limit on Application Master Restarts</h3>
+<h3 id="slideryarnrestartlimit-set-limit-on-application-master-restarts"><code>slider.yarn.restart.limit</code> - set limit on Application Master Restarts<a class="headerlink" href="#slideryarnrestartlimit-set-limit-on-application-master-restarts" title="Permanent link">¶</a></h3>
<p>This limits how many times YARN should start a failed application master.</p>
<p>A short restart limit is useful when initially creating a cluster, as it
ensures that YARN does not repeatedly try to restart a failing application.</p>
@@ -361,12 +372,12 @@ permitted.</p>
places an upper limit on the number of retries that any application can request.
If the application fails after less restarts than requested, check this cluster
setting.</p>
-<h3 id="slideryarnqueue-the-name-of-the-yarn-queue-for-the-cluster"><code>slider.yarn.queue</code> - the name of the YARN queue for the cluster.</h3>
+<h3 id="slideryarnqueue-the-name-of-the-yarn-queue-for-the-cluster"><code>slider.yarn.queue</code> - the name of the YARN queue for the cluster.<a class="headerlink" href="#slideryarnqueue-the-name-of-the-yarn-queue-for-the-cluster" title="Permanent link">¶</a></h3>
<p>This identifies the queue submit the application creation request to, which can
define the priority, resource limits and other values of an application. All
containers created in the Slider cluster will share this same queue.</p>
<p>Default value: <code>default</code>.</p>
-<h3 id="slideryarnqueuepriority-the-name-of-the-yarn-queue-for-the-cluster"><code>slider.yarn.queue.priority</code> - the name of the YARN queue for the cluster.</h3>
+<h3 id="slideryarnqueuepriority-the-name-of-the-yarn-queue-for-the-cluster"><code>slider.yarn.queue.priority</code> - the name of the YARN queue for the cluster.<a class="headerlink" href="#slideryarnqueuepriority-the-name-of-the-yarn-queue-for-the-cluster" title="Permanent link">¶</a></h3>
<p>This identifies the priority within the queue. The lower the value, the higher the
priority</p>
<p>Default value: <code>1</code>.</p>
@@ -374,7 +385,7 @@ priority</p>
</pre></div>
-<h4 id="sliderclusterdirectorypermissions"><code>slider.cluster.directory.permissions</code></h4>
+<h4 id="sliderclusterdirectorypermissions"><code>slider.cluster.directory.permissions</code><a class="headerlink" href="#sliderclusterdirectorypermissions" title="Permanent link">¶</a></h4>
<p>An octal-format (<code>chmod</code>-style) permissions mask for the directory
that contains the cluster specification <code>${user.home}/.slider/clusters/${clustername}</code></p>
<div class="codehilite"><pre><span class="nt"><property></span>
@@ -384,7 +395,7 @@ that contains the cluster specification
</pre></div>
-<h4 id="sliderdatadirectorypermissions"><code>slider.data.directory.permissions</code></h4>
+<h4 id="sliderdatadirectorypermissions"><code>slider.data.directory.permissions</code><a class="headerlink" href="#sliderdatadirectorypermissions" title="Permanent link">¶</a></h4>
<p>An octal-format (<code>chmod</code>-style) permissions mask for the directory
that contains the application data <code>${user.home}/.slider/clusters/${clustername}/database</code></p>
<div class="codehilite"><pre><span class="nt"><property></span>
@@ -394,7 +405,7 @@ that contains the application data <code
</pre></div>
-<h2 id="yarn-registry-binding">YARN Registry binding</h2>
+<h2 id="yarn-registry-binding">YARN Registry binding<a class="headerlink" href="#yarn-registry-binding" title="Permanent link">¶</a></h2>
<p>Slider uses the (evolving) YARN service registry to register application
instances, to locate them, and as a means to publish the configuration information
needed to connect to the slider-deployed applications.</p>
@@ -403,7 +414,7 @@ for a cluster, for use by all applicatio
Resource Manager is set to automatically create the paths in the registry for
users âa required feature in secure clustersâ then it must share the same settings.</p>
<p>Consult the YARN registry documentation for more information.</p>
-<h3 id="hadoopregistryzkquorum-the-zookeeper-quorum"><code>hadoop.registry.zk.quorum</code> - the zookeeper quorum.</h3>
+<h3 id="hadoopregistryzkquorum-the-zookeeper-quorum"><code>hadoop.registry.zk.quorum</code> - the zookeeper quorum.<a class="headerlink" href="#hadoopregistryzkquorum-the-zookeeper-quorum" title="Permanent link">¶</a></h3>
<p>This defines the zookeeper quorum for this YARN cluster. </p>
<p>It is used to locate the service registry, enable running instances to publish
information about their application, and for clients to query this. </p>
@@ -418,7 +429,7 @@ defined will be copied into the instance
</pre></div>
-<h3 id="hadoopregistryzkroot-the-zookeeper-path-for-the-service-registry"><code>hadoop.registry.zk.root</code> - the zookeeper path for the service registry</h3>
+<h3 id="hadoopregistryzkroot-the-zookeeper-path-for-the-service-registry"><code>hadoop.registry.zk.root</code> - the zookeeper path for the service registry<a class="headerlink" href="#hadoopregistryzkroot-the-zookeeper-path-for-the-service-registry" title="Permanent link">¶</a></h3>
<p>This declares the the zookeeper path for the service registry. </p>
<div class="codehilite"><pre><span class="nt"><property></span>
<span class="nt"><name></span>hadoop.registry.zk.root<span class="nt"></name></span>
@@ -427,10 +438,10 @@ defined will be copied into the instance
</pre></div>
-<h2 id="debugging-configuration-issues">Debugging configuration issues</h2>
+<h2 id="debugging-configuration-issues">Debugging configuration issues<a class="headerlink" href="#debugging-configuration-issues" title="Permanent link">¶</a></h2>
<p>If the slider packages are set to log at debug level in the log4j configuration
file, details on properties will be part of the copious output.</p>
-<h2 id="how-client-options-are-passed-down-to-created-clusters">How client options are passed down to created clusters.</h2>
+<h2 id="how-client-options-are-passed-down-to-created-clusters">How client options are passed down to created clusters.<a class="headerlink" href="#how-client-options-are-passed-down-to-created-clusters" title="Permanent link">¶</a></h2>
<p>Most cient configuration options are
not passed down to the XML site specification of the created cluster.</p>
<p>Exceptions:</p>
Modified: websites/staging/slider/trunk/content/docs/configuration/appconfig.html.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/configuration/appconfig.html.html (original)
+++ websites/staging/slider/trunk/content/docs/configuration/appconfig.html.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,7 +193,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="title">title</h1>
+<h1 id="title">title<a class="headerlink" href="#title" title="Permanent link">¶</a></h1>
</div>
<div id="footer">
Modified: websites/staging/slider/trunk/content/docs/configuration/core.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/configuration/core.html (original)
+++ websites/staging/slider/trunk/content/docs/configuration/core.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,8 +196,8 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="apache-slider-core-configuration-specification-version-20">Apache Slider Core Configuration Specification, version 2.0</h1>
-<h2 id="terminology">Terminology</h2>
+<h1 id="apache-slider-core-configuration-specification-version-20">Apache Slider Core Configuration Specification, version 2.0<a class="headerlink" href="#apache-slider-core-configuration-specification-version-20" title="Permanent link">¶</a></h1>
+<h2 id="terminology">Terminology<a class="headerlink" href="#terminology" title="Permanent link">¶</a></h2>
<p><em>Application</em> A single application, such as an HBase cluster. An application
is distribed across the YARN cluster.</p>
<p><em>Component</em> A single executable part of the larger application. An application
@@ -205,7 +216,7 @@ size of the application in terms of its
and what their resource requirements are. </p>
<p><em><code>internal.json</code></em>: A file which contains Slider's internal configuration
parameters.</p>
-<h2 id="structure">Structure</h2>
+<h2 id="structure">Structure<a class="headerlink" href="#structure" title="Permanent link">¶</a></h2>
<p>Configurations are stored in well-formed JSON files.
1. Text MUST be saved in the UTF-8 format.
1. Duplicate entries MUST NOT occur in any section.
@@ -241,7 +252,7 @@ author, or any other information that is
</pre></div>
-<h2 id="property-inheritance-model-and-resolution">Property inheritance model and <em>resolution</em></h2>
+<h2 id="property-inheritance-model-and-resolution">Property inheritance model and <em>resolution</em><a class="headerlink" href="#property-inheritance-model-and-resolution" title="Permanent link">¶</a></h2>
<p>There is a simple global to component inheritance model.</p>
<ol>
<li>Properties defined in <code>/global</code> define parameters across the entire application.</li>
@@ -258,7 +269,7 @@ MAY be added in future.</li>
<li>There is NO sharing of information from the different <code>.json</code> files in a
an application configuration.</li>
</ol>
-<h3 id="example">Example</h3>
+<h3 id="example">Example<a class="headerlink" href="#example" title="Permanent link">¶</a></h3>
<p>Here is an example configuration</p>
<div class="codehilite"><pre><span class="p">{</span>
"<span class="n">schema</span>"<span class="p">:</span> "<span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">example</span><span class="p">.</span><span class="n">org</span><span class="o">/</span><span class="n">specification</span><span class="o">/</span><span class="n">v2</span><span class="p">.</span>0<span class="p">.</span>0"<span class="p">,</span>
@@ -294,7 +305,7 @@ an application configuration.</li>
<p>These are the values visible to any part of the application which is
not itself one of the components. </p>
<p>There are three components defined, <code>simple</code>, <code>master</code> and <code>worker</code>.</p>
-<h4 id="component-simple">component <code>simple</code>:</h4>
+<h4 id="component-simple">component <code>simple</code>:<a class="headerlink" href="#component-simple" title="Permanent link">¶</a></h4>
<div class="codehilite"><pre><span class="n">g1</span><span class="p">=</span>"<span class="n">a</span>"
<span class="n">g2</span><span class="p">=</span>"<span class="n">b</span>"
</pre></div>
@@ -302,7 +313,7 @@ not itself one of the components. </p>
<p>No settings have been defined specifically for the component; the global
settings are applied.</p>
-<h4 id="component-master">component <code>master</code>:</h4>
+<h4 id="component-master">component <code>master</code>:<a class="headerlink" href="#component-master" title="Permanent link">¶</a></h4>
<div class="codehilite"><pre><span class="n">name</span><span class="p">=</span>"<span class="n">m</span>"<span class="p">,</span>
<span class="n">g1</span><span class="p">=</span>"<span class="n">overridden</span>"
<span class="n">g2</span><span class="p">=</span>"<span class="n">b</span>"
@@ -312,7 +323,7 @@ settings are applied.</p>
<p>A new attribute, <code>name</code>, has been defined with the value <code>"m"</code>, and the
global property <code>g1</code> has been overridden with the new value, <code>"overridden"</code>.
The global property <code>g2</code> is passed down unchanged.</p>
-<h4 id="component-worker">component <code>worker</code>:</h4>
+<h4 id="component-worker">component <code>worker</code>:<a class="headerlink" href="#component-worker" title="Permanent link">¶</a></h4>
<div class="codehilite"><pre><span class="n">name</span><span class="p">=</span>"<span class="n">w</span>"<span class="p">,</span>
<span class="n">g1</span><span class="p">=</span>"<span class="n">overridden</span><span class="o">-</span><span class="n">by</span><span class="o">-</span><span class="n">worker</span>"
<span class="n">g2</span><span class="p">=</span>"<span class="n">b</span>"
@@ -366,12 +377,12 @@ in the example, without any propagation
to individual components. </p>
<p>Note that a resolved specification can still have the resolution operation applied
to it -it just does not have any effect.</p>
-<h2 id="metadata">Metadata</h2>
+<h2 id="metadata">Metadata<a class="headerlink" href="#metadata" title="Permanent link">¶</a></h2>
<p>The metadata section can contain arbitrary string values for use in diagnostics
and by other applications.</p>
<p>To avoid conflict with other applications, please use a unique name in strings,
such as java-style package names.</p>
-<h1 id="resource-requirements-resourcesjson">Resource Requirements: <code>resources.json</code></h1>
+<h1 id="resource-requirements-resourcesjson">Resource Requirements: <code>resources.json</code><a class="headerlink" href="#resource-requirements-resourcesjson" title="Permanent link">¶</a></h1>
<p>This file declares the resource requirements for YARN for the components
of an application.</p>
<p><code>instances</code>: the number of instances of a role desired.
@@ -434,7 +445,7 @@ of an application.</p>
<p>This declares this deployment of the application to consist of one instance of
the master component, using 1 vcore and 1024MB of RAM, and five worker components
each using one vcore and 512 MB of RAM.</p>
-<h2 id="deployment-specification-app_configurationjson">Deployment specification: <code>app_configuration.json</code></h2>
+<h2 id="deployment-specification-app_configurationjson">Deployment specification: <code>app_configuration.json</code><a class="headerlink" href="#deployment-specification-app_configurationjson" title="Permanent link">¶</a></h2>
<p>This defines parameters that are to be used when creating the instance of the
application, and instances of the individual components.</p>
<div class="codehilite"><pre><span class="p">{</span>
@@ -522,10 +533,10 @@ does not prevent any component from over
to create local configurations incompatible with the global state. (i.e.,
there is no way to declare an attribute as final). It is the responsibility
of the author of the configuration file (and their tools) to detect such issues.</p>
-<h3 id="key-application-configuration-items">Key Application Configuration Items</h3>
+<h3 id="key-application-configuration-items">Key Application Configuration Items<a class="headerlink" href="#key-application-configuration-items" title="Permanent link">¶</a></h3>
<p>The following sections provides details about certain application configuration
properties that can be utilized to tailor the deployment of a given application:</p>
-<h4 id="controlling-assigned-port-ranges">Controlling assigned port ranges</h4>
+<h4 id="controlling-assigned-port-ranges">Controlling assigned port ranges<a class="headerlink" href="#controlling-assigned-port-ranges" title="Permanent link">¶</a></h4>
<p>For certain deployments, the ports available for communication with clients
(Web UI ports, RPC ports, etc) are restricted to a specific set (e.g when using a firewall).
In those situations you can designate the set of allowed ports with the
@@ -537,7 +548,7 @@ In those situations you can designate th
<p>The AM exposed ports (Web UI, RPC), as well as the ports allocated to launched
application containers, will be limited to the ranges specified by the property value.</p>
-<h4 id="delaying-container-launch">Delaying container launch</h4>
+<h4 id="delaying-container-launch">Delaying container launch<a class="headerlink" href="#delaying-container-launch" title="Permanent link">¶</a></h4>
<p>In situations where container restarts may need to be delayed to allow for
platform resources to be released (e.g. a port assigned to a previous container
may be slow to release), a delay can be designated by setting the <code>container.launch.delay.sec</code> property.</p>
@@ -548,7 +559,7 @@ may be slow to release), a delay can be
</pre></div>
-<h4 id="specifying-the-python-executable-path">Specifying the Python Executable Path</h4>
+<h4 id="specifying-the-python-executable-path">Specifying the Python Executable Path<a class="headerlink" href="#specifying-the-python-executable-path" title="Permanent link">¶</a></h4>
<p>Slider containers use python for component scripts in the containers.
When deploying applications on certain variations of linux or other operating systems (e.g. Centos 5),
the version of python on the system PATH may be incompatible with the component script
Modified: websites/staging/slider/trunk/content/docs/configuration/index.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/configuration/index.html (original)
+++ websites/staging/slider/trunk/content/docs/configuration/index.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,7 +196,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="apache-slider-specification-of-an-application-instance-revision-20">Apache Slider: Specification of an application instance, revision 2.0</h1>
+<h1 id="apache-slider-specification-of-an-application-instance-revision-20">Apache Slider: Specification of an application instance, revision 2.0<a class="headerlink" href="#apache-slider-specification-of-an-application-instance-revision-20" title="Permanent link">¶</a></h1>
<p>The specification of an application comprises</p>
<ol>
<li>The persistent description of an application's configuration</li>