You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by bu...@apache.org on 2015/11/02 18:24:48 UTC
svn commit: r971096 [5/8] - in /websites/staging/slider/trunk/content: ./
design/ design/registry/ design/specification/ developing/ docs/ docs/api/
docs/configuration/ docs/configuration/revision-1/ docs/slider_specs/
downloads/ release_notes/
Modified: websites/staging/slider/trunk/content/docs/configuration/internal.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/configuration/internal.html (original)
+++ websites/staging/slider/trunk/content/docs/configuration/internal.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,8 +193,8 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="internaljson-sliders-internal-configuration-document">internal.json: slider's internal configuration document</h1>
-<h2 id="internal-information-internaljson">Internal information, <code>internal.json</code></h2>
+<h1 id="internaljson-sliders-internal-configuration-document">internal.json: slider's internal configuration document<a class="headerlink" href="#internaljson-sliders-internal-configuration-document" title="Permanent link">¶</a></h1>
+<h2 id="internal-information-internaljson">Internal information, <code>internal.json</code><a class="headerlink" href="#internal-information-internaljson" title="Permanent link">¶</a></h2>
<p>This contains internal data related to the deployment âit is not
intended for normal use.</p>
<p>There MAY be a component, <code>diagnostics</code>. If defined, its content contains
@@ -213,7 +224,7 @@ of diagnostics reports)</p>
</pre></div>
-<h2 id="chaos-monkey">Chaos Monkey</h2>
+<h2 id="chaos-monkey">Chaos Monkey<a class="headerlink" href="#chaos-monkey" title="Permanent link">¶</a></h2>
<p>The Slider application has a built in "Chaos Monkey", which is configured in the <code>internal.json</code>
file.:</p>
<p>Consult <a href="../developing/chaosmonkey.html">"configuring the Slider Chaos Monkey"</a> for details.</p>
Modified: websites/staging/slider/trunk/content/docs/configuration/resources.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/configuration/resources.html (original)
+++ websites/staging/slider/trunk/content/docs/configuration/resources.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
Modified: websites/staging/slider/trunk/content/docs/configuration/revision-1/index.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/configuration/revision-1/index.html (original)
+++ websites/staging/slider/trunk/content/docs/configuration/revision-1/index.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,7 +196,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="apache-slider-specification-of-an-application-instance-revision-1">Apache Slider: Specification of an application instance, revision 1</h1>
+<h1 id="apache-slider-specification-of-an-application-instance-revision-1">Apache Slider: Specification of an application instance, revision 1<a class="headerlink" href="#apache-slider-specification-of-an-application-instance-revision-1" title="Permanent link">¶</a></h1>
<p>This is the original specification of an application instance, including discussion
on a proposed rework.</p>
<ol>
Modified: websites/staging/slider/trunk/content/docs/configuration/revision-1/redesign.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/configuration/revision-1/redesign.html (original)
+++ websites/staging/slider/trunk/content/docs/configuration/revision-1/redesign.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,13 +196,13 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="apache-slider-cluster-specification">Apache Slider Cluster Specification</h1>
-<h3 id="notation">Notation:</h3>
+<h1 id="apache-slider-cluster-specification">Apache Slider Cluster Specification<a class="headerlink" href="#apache-slider-cluster-specification" title="Permanent link">¶</a></h1>
+<h3 id="notation">Notation:<a class="headerlink" href="#notation" title="Permanent link">¶</a></h3>
<p>In this document, a full path to a value is represented as a path
<code>options/zookeeper.port</code> ; an assigment as <code>options/zookeeper.port=2181</code>.</p>
<p>A wildcard indicates all entries matching a path: <code>options/zookeeper.*</code>
or <code>/roles/*/yarn.memory</code></p>
-<h2 id="history">History</h2>
+<h2 id="history">History<a class="headerlink" href="#history" title="Permanent link">¶</a></h2>
<p>The Slider cluster specification was implicitly defined in the file
<code>org.apache.slider.api.ClusterDescription</code>. It had a number of roles</p>
<ol>
@@ -216,8 +227,8 @@ more attributes into specific sections <
<li><code>info</code>: read-only information about the cluster.</li>
<li><code>statistics</code>: Numeric statistics about the cluster</li>
</ul>
-<h1 id="sections">Sections</h1>
-<h2 id="root">Root</h2>
+<h1 id="sections">Sections<a class="headerlink" href="#sections" title="Permanent link">¶</a></h1>
+<h2 id="root">Root<a class="headerlink" href="#root" title="Permanent link">¶</a></h2>
<p>Contains various string and integer values</p>
<div class="codehilite"><pre>"<span class="n">version</span>"<span class="p">:</span> "1<span class="p">.</span>0"<span class="p">,</span>
"<span class="n">name</span>"<span class="p">:</span> "<span class="n">test_cluster_lifecycle</span>"<span class="p">,</span>
@@ -270,10 +281,10 @@ into a section <code>/slider-internal</c
1. The cluster state is moved from an enum to a simple
boolean, <code>valid</code>, set to true when the cluster JSON
has been fully constructed.</p>
-<h2 id="info"><code>/info</code></h2>
+<h2 id="info"><code>/info</code><a class="headerlink" href="#info" title="Permanent link">¶</a></h2>
<p>Read-only list of information about the application. Generally this is
intended to be used for debugging and testing.</p>
-<h3 id="persisted-values-static-information-about-the-file-history">Persisted values: static information about the file history</h3>
+<h3 id="persisted-values-static-information-about-the-file-history">Persisted values: static information about the file history<a class="headerlink" href="#persisted-values-static-information-about-the-file-history" title="Permanent link">¶</a></h3>
<div class="codehilite"><pre>"<span class="n">info</span>" <span class="p">:</span> <span class="p">{</span>
"<span class="n">create</span><span class="p">.</span><span class="n">hadoop</span><span class="p">.</span><span class="n">deployed</span><span class="p">.</span><span class="n">info</span>" <span class="p">:</span> "<span class="p">(</span><span class="n">detached</span> <span class="n">from</span> <span class="n">release</span><span class="o">-</span>2<span class="p">.</span>3<span class="p">.</span>0<span class="p">)</span> <span class="p">@</span><span class="n">dfe46336fbc6a044bc124392ec06b85</span>"<span class="p">,</span>
"<span class="n">create</span><span class="p">.</span><span class="n">application</span><span class="p">.</span><span class="n">build</span><span class="p">.</span><span class="n">info</span>" <span class="p">:</span> "<span class="n">Slider</span> <span class="n">Core</span><span class="o">-</span>0<span class="p">.</span>13<span class="p">.</span>0<span class="o">-</span><span class="n">SNAPSHOT</span> <span class="n">Built</span> <span class="n">against</span> <span class="n">commit</span># 1<span class="n">a94ee4aa1</span> <span class="n">on</span> <span class="n">Java</span> 1<span class="p">.</span>7<span class="p">.</span>0<span class="n">_45</span> <span class="n">by</span> <span class="n">stevel</span>"<span class="p">,</span>
@@ -284,7 +295,7 @@ intended to be used for debugging and te
<p><em>Proposed</em>: move persisted info K-V pairs to a section <code>/diagnostics</code>.</p>
-<h3 id="dynamic-values">Dynamic values:</h3>
+<h3 id="dynamic-values">Dynamic values:<a class="headerlink" href="#dynamic-values" title="Permanent link">¶</a></h3>
<p>whether the AM supports service restart without killing all the containers hosting
the role instances:</p>
<div class="codehilite"><pre>"<span class="n">slider</span><span class="p">.</span><span class="n">am</span><span class="p">.</span><span class="n">restart</span><span class="p">.</span><span class="n">supported</span>" <span class="p">:</span> "<span class="n">false</span>"<span class="p">,</span>
@@ -441,7 +452,7 @@ and to move the roles' statistics under
<p>This approach allows extra statistics sections to be added (perhaps
by providers), without any changes to the toplevel section.</p>
-<h2 id="options">Options</h2>
+<h2 id="options">Options<a class="headerlink" href="#options" title="Permanent link">¶</a></h2>
<p>A list of options used by Slider and its providers to build up the AM
and the configurations of the deployed service components</p>
<div class="codehilite"><pre>"<span class="n">options</span>"<span class="p">:</span> <span class="p">{</span>
@@ -466,18 +477,18 @@ and the configurations of the deployed s
for internal use -and are cluster specific. If/when the ability to use
an existing json file as a template for a new cluster is added, having these
options in the configuration will create problems</p>
-<h1 id="proposed-changes">Proposed Changes</h1>
-<h2 id="move-slider-internal-state-to-slider-internal">Move Slider internal state to <code>/slider-internal</code></h2>
+<h1 id="proposed-changes">Proposed Changes<a class="headerlink" href="#proposed-changes" title="Permanent link">¶</a></h1>
+<h2 id="move-slider-internal-state-to-slider-internal">Move Slider internal state to <code>/slider-internal</code><a class="headerlink" href="#move-slider-internal-state-to-slider-internal" title="Permanent link">¶</a></h2>
<p>Move all slider "private" data to an internal section,<code>/slider-internal</code>
including those in the toplevel directory and in <code>/options</code></p>
-<h2 id="allow-options-and-roles-options-entries-to-take-the-value-null">Allow <code>/options</code> and <code>roles/*/</code> options entries to take the value "null".</h2>
+<h2 id="allow-options-and-roles-options-entries-to-take-the-value-null">Allow <code>/options</code> and <code>roles/*/</code> options entries to take the value "null".<a class="headerlink" href="#allow-options-and-roles-options-entries-to-take-the-value-null" title="Permanent link">¶</a></h2>
<p>This would be a definition that the value must be defined before the cluster
can start. Provider templates could declare this.</p>
-<h2 id="make-client-configuration-retrieval-hierarchical-and-maybe-move-out-of-the">Make client configuration retrieval hierarchical -and maybe move out of the</h2>
+<h2 id="make-client-configuration-retrieval-hierarchical-and-maybe-move-out-of-the">Make client configuration retrieval hierarchical -and maybe move out of the<a class="headerlink" href="#make-client-configuration-retrieval-hierarchical-and-maybe-move-out-of-the" title="Permanent link">¶</a></h2>
<p>status</p>
<p>The current design assumes that it is a -site.xml file being served up. This
does not work for alternate file formats generated by the Provider.</p>
-<h2 id="role-options">Role Options</h2>
+<h2 id="role-options">Role Options<a class="headerlink" href="#role-options" title="Permanent link">¶</a></h2>
<p>The <code>/roles/$ROLENAME/</code> clauses each provide options for a
specific role.</p>
<p>This includes
@@ -531,20 +542,20 @@ about the cluster.</p>
</pre></div>
-<h3 id="proposed">Proposed:</h3>
+<h3 id="proposed">Proposed:<a class="headerlink" href="#proposed" title="Permanent link">¶</a></h3>
<ol>
<li>move all dynamic role status to its own clauses.</li>
<li>use a simple inheritance model from <code>/options</code></li>
<li>don't allow role entries to alter the cluster state. </li>
</ol>
-<h3 id="proposed-clientproperties-continues-return-key-val-pairs">Proposed: <code>/clientProperties</code> continues return Key-val pairs</h3>
+<h3 id="proposed-clientproperties-continues-return-key-val-pairs">Proposed: <code>/clientProperties</code> continues return Key-val pairs<a class="headerlink" href="#proposed-clientproperties-continues-return-key-val-pairs" title="Permanent link">¶</a></h3>
<p>The <code>/clientProperties</code> section will remain, with key-val pairs of type
string, the expectation being this is where providers can insert specific
single attributes for client applications.</p>
<p>These values can be converted to application-specific files on the client,
in code -as done today in the Slider CLI-, or via template expansion (beyond
the scope of this document.</p>
-<h3 id="proposed-alongside-clientproperties-comes-clientfiles">Proposed: alongside <code>/clientProperties</code> comes <code>/clientfiles</code></h3>
+<h3 id="proposed-alongside-clientproperties-comes-clientfiles">Proposed: alongside <code>/clientProperties</code> comes <code>/clientfiles</code><a class="headerlink" href="#proposed-alongside-clientproperties-comes-clientfiles" title="Permanent link">¶</a></h3>
<p>This section will list all files that an application instance can generate
for clients, along with with a description.</p>
<div class="codehilite"><pre>"<span class="o">/</span><span class="n">clientfiles</span><span class="o">/</span><span class="n">hbase</span><span class="o">-</span><span class="n">site</span><span class="p">.</span><span class="n">xml</span>"<span class="p">:</span> "<span class="n">site</span> <span class="n">information</span> <span class="k">for</span> <span class="n">HBase</span>"
@@ -582,14 +593,14 @@ is restricted to those files which the a
generating. The advantage of the server-side option is that for those files
about which the provider is aware of, they will be visible through the
REST and Web UIs, so trivially retrieved.</p>
-<h3 id="stop-intermixing-role-specification-with-role-current-state">Stop intermixing role specification with role current state</h3>
+<h3 id="stop-intermixing-role-specification-with-role-current-state">Stop intermixing role specification with role current state<a class="headerlink" href="#stop-intermixing-role-specification-with-role-current-state" title="Permanent link">¶</a></h3>
<p>Create a new section, <code>rolestatus</code>, which lists the current status
of the roles: how many are running vs requested, how many are being
released.</p>
<p>There's some overlap here with the <code>/statistics</code> field, so we should
either merge them or clearly separate the two. Only the <code>role.failed</code>
properties match entries in the statistics -perhaps they should be cut.</p>
-<h4 id="provider-specific-status">provider-specific status</h4>
+<h4 id="provider-specific-status">provider-specific status<a class="headerlink" href="#provider-specific-status" title="Permanent link">¶</a></h4>
<p>Allow providers to publish information to the status, in their
own section.</p>
<p>There already is support for providers updating the cluster status
Modified: websites/staging/slider/trunk/content/docs/configuration/revision-1/specification.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/configuration/revision-1/specification.html (original)
+++ websites/staging/slider/trunk/content/docs/configuration/revision-1/specification.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,21 +196,21 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="apache-slider-specification-of-the-cluster-description">Apache Slider Specification of the "Cluster Description"</h1>
+<h1 id="apache-slider-specification-of-the-cluster-description">Apache Slider Specification of the "Cluster Description"<a class="headerlink" href="#apache-slider-specification-of-the-cluster-description" title="Permanent link">¶</a></h1>
<ul>
<li>This is partially obsolete. Slider still returns the Slider Cluster Description
as changing it will break most of the unit tests -once these are updated
this document will be completely obsolete and replaced with a new one.</li>
</ul>
-<h3 id="notation">Notation:</h3>
+<h3 id="notation">Notation:<a class="headerlink" href="#notation" title="Permanent link">¶</a></h3>
<p>In this document, a full path to a value is represented as a path
<code>options/zookeeper.port</code> ; an assigment as <code>options/zookeeper.port=2181</code>.</p>
<p>A wildcard indicates all entries matching a path: <code>options/zookeeper.*</code>
or <code>/roles/*/yarn.memory</code></p>
-<h2 id="core-concepts">Core Concepts</h2>
+<h2 id="core-concepts">Core Concepts<a class="headerlink" href="#core-concepts" title="Permanent link">¶</a></h2>
<p>The specificaton of an application instance is defined in an application instance
directory, <code>${user.home}/.slidera/clusters/${clustername}/cluster.json</code>)</p>
-<h2 id="sections-for-specifying-and-describing-cluster-state">Sections for specifying and describing cluster state</h2>
+<h2 id="sections-for-specifying-and-describing-cluster-state">Sections for specifying and describing cluster state<a class="headerlink" href="#sections-for-specifying-and-describing-cluster-state" title="Permanent link">¶</a></h2>
<p>The cluster desciption is hierarchal, with standardized sections.</p>
<p>Different sections have one of three roles.</p>
<ol>
@@ -228,7 +239,7 @@ aspects of the Slider application master
may be confusing, but it is designed to unify the logic needed to parse
and process cluster descriptions. There is only one JSON file to parse
-merely different sections of relevance at different times.</p>
-<h2 id="role-by-role-subsections">Role-by-role subsections</h2>
+<h2 id="role-by-role-subsections">Role-by-role subsections<a class="headerlink" href="#role-by-role-subsections" title="Permanent link">¶</a></h2>
<p>A slider-deployed application consists of the single Slider application master,
and one or more roles -specific components in the actual application.</p>
<p>The <code>/roles</code> section contains a listing for each role,
@@ -237,7 +248,7 @@ possibly along with some details definin
<p>The <code>/statistics/roles/</code> section returns statistics on each role,
while <code>/instances</code> has a per-role entry listing the YARN
containers hosting instances. </p>
-<h2 id="cluster-information-for-applications">Cluster information for applications</h2>
+<h2 id="cluster-information-for-applications">Cluster information for applications<a class="headerlink" href="#cluster-information-for-applications" title="Permanent link">¶</a></h2>
<p>The AM/application provider may generate information for use by client applications.</p>
<p>There are three ways to provide this</p>
<ol>
@@ -246,8 +257,8 @@ by client applications -usually to gener
<li>A listing of files that may be provided directly to a client. The API to provide these files is not covered by this document.</li>
<li>A provider-specific section in which arbitrary values and structures may be defined. This allows greater flexibility in the information that a provider can publish -though it does imply custom code to process this data on the client.</li>
</ol>
-<h1 id="persistent-specification-sections">Persistent Specification Sections</h1>
-<h2 id="root">"/" : root</h2>
+<h1 id="persistent-specification-sections">Persistent Specification Sections<a class="headerlink" href="#persistent-specification-sections" title="Permanent link">¶</a></h1>
+<h2 id="root">"/" : root<a class="headerlink" href="#root" title="Permanent link">¶</a></h2>
<p>The root contains a limited number of key-value pairs, </p>
<ul>
<li>
@@ -273,10 +284,10 @@ property lookup to find the implementati
Flag to indicate whether or not a specification is considered valid.
If false, the rest of the document is in an unknown state.</li>
</ul>
-<h2 id="slider-internal-internal-confiugration"><code>/slider-internal</code>: internal confiugration</h2>
+<h2 id="slider-internal-internal-confiugration"><code>/slider-internal</code>: internal confiugration<a class="headerlink" href="#slider-internal-internal-confiugration" title="Permanent link">¶</a></h2>
<p>Stores internal configuration options. These parameters
are not defined in this document.</p>
-<h2 id="diagnostics-diagnostics-sections"><code>/diagnostics</code>: diagnostics sections</h2>
+<h2 id="diagnostics-diagnostics-sections"><code>/diagnostics</code>: diagnostics sections<a class="headerlink" href="#diagnostics-diagnostics-sections" title="Permanent link">¶</a></h2>
<p>Persisted list of information about Slider. </p>
<p>Static information about the file history</p>
<div class="codehilite"><pre>"<span class="n">diagnostics</span>" <span class="p">:</span> <span class="p">{</span>
@@ -294,7 +305,7 @@ are not defined in this document.</p>
than diagnostics to an application; the values and their meaning
are not defined. All applications MUST be able to process
an empty or absent <code>/diagnostics</code> section.</p>
-<h2 id="options-cluster-options">Options: cluster options</h2>
+<h2 id="options-cluster-options">Options: cluster options<a class="headerlink" href="#options-cluster-options" title="Permanent link">¶</a></h2>
<p>A persisted list of options used by Slider and its providers to build up the AM
and the configurations of the deployed service components</p>
<div class="codehilite"><pre>"<span class="n">options</span>"<span class="p">:</span> <span class="p">{</span>
@@ -317,9 +328,9 @@ and the configurations of the deployed s
<p>Many of the properties are automatically set by Slider when a cluster is constructed.
They may be edited afterwards.</p>
-<h3 id="standard-option-types">Standard Option types</h3>
+<h3 id="standard-option-types">Standard Option types<a class="headerlink" href="#standard-option-types" title="Permanent link">¶</a></h3>
<p>All option values MUST be strings.</p>
-<h4 id="slider"><code>slider.</code></h4>
+<h4 id="slider"><code>slider.</code><a class="headerlink" href="#slider" title="Permanent link">¶</a></h4>
<p>All options that begin with <code>slider.</code> are intended for use by slider and
providers to configure the Slider application master itself, and the
application. For example, <code>slider.container.failure.threshold</code> defines
@@ -328,7 +339,7 @@ is considered to have failed. As another
such as <code>zookeeper.hosts</code> are read by the HBase and Ambari providers, and
used to modify the applications' site configurations with application-specific
properties.</p>
-<h4 id="site"><code>site.</code></h4>
+<h4 id="site"><code>site.</code><a class="headerlink" href="#site" title="Permanent link">¶</a></h4>
<p>These are properties that are expected to be propagated to an application's
<code>site</code> configuration -if such a configuration is created. For HBase, the
site file is <code>hbase-site.xml</code>; for Accumulo it is <code>accumulo-site.xml</code></p>
@@ -340,7 +351,7 @@ ignore the settings.</li>
<li>Providers MAY validate site settings to recognise invalid values. This
aids identifying and diagnosing startup problems.</li>
</ol>
-<h4 id="env"><code>env.</code></h4>
+<h4 id="env"><code>env.</code><a class="headerlink" href="#env" title="Permanent link">¶</a></h4>
<p>These are options to configure environment variables in the roles. When
a container is started, all <code>env.</code> options have the prefix removed, and
are then set as environment variables in the target context.</p>
@@ -349,7 +360,7 @@ are then set as environment variables in
option/role merge process.</li>
<li>Application providers SHOULD follow the same process.</li>
</ol>
-<h2 id="roles-role-declarations">'/roles': role declarations</h2>
+<h2 id="roles-role-declarations">'/roles': role declarations<a class="headerlink" href="#roles-role-declarations" title="Permanent link">¶</a></h2>
<p>The <code>/roles/$ROLENAME/</code> clauses each provide options for a
specific role.</p>
<p>This includes
@@ -384,7 +395,7 @@ fix the heap size of a component.</p>
<p>Providers may support a fixed number of roles -or they may support a dynamic
number of roles defined at run-time, potentially from other data sources.</p>
-<h2 id="how-options-and-role-options-are-merged">How <code>/options</code> and role options are merged.</h2>
+<h2 id="how-options-and-role-options-are-merged">How <code>/options</code> and role options are merged.<a class="headerlink" href="#how-options-and-role-options-are-merged" title="Permanent link">¶</a></h2>
<p>The options declared for a specific role are merged with the cluster-wide options
to define the final options for a role. This is implemented in a simple
override model: role-specific options can override any site-wide options.</p>
@@ -401,7 +412,7 @@ from the <code>/options</code> section.<
are specific to the invidual role being created. </p>
<p>As such, overwriting a <code>site.</code> option may have no effect -or it it may
change the value of a site configuration document <em>in that specific role instance</em>.</p>
-<h3 id="standard-role-options">Standard role options</h3>
+<h3 id="standard-role-options">Standard role options<a class="headerlink" href="#standard-role-options" title="Permanent link">¶</a></h3>
<ul>
<li><code>role.instances</code> : number; required.
The number of instances of that role desired in the application.</li>
@@ -424,16 +435,16 @@ change the value of a site configuration
String environment variables to use when setting up the container</p>
</li>
</ul>
-<h3 id="provider-specific-role-options">Provider-specific role options</h3>
+<h3 id="provider-specific-role-options">Provider-specific role options<a class="headerlink" href="#provider-specific-role-options" title="Permanent link">¶</a></h3>
<ul>
<li><code>jvm.heapsize</code> -the amount of memory for a provider to allocate for
a processes JVM. Example "512M". This option MAY be implemented by a provider.</li>
</ul>
-<h1 id="dynamic-information-sections">Dynamic Information Sections</h1>
+<h1 id="dynamic-information-sections">Dynamic Information Sections<a class="headerlink" href="#dynamic-information-sections" title="Permanent link">¶</a></h1>
<p>These are the parts of the document that provide dynamic run-time
information about an application. They are provided by the
Slider Application Master when a request for the cluster status is issued.</p>
-<h2 id="info"><code>/info</code></h2>
+<h2 id="info"><code>/info</code><a class="headerlink" href="#info" title="Permanent link">¶</a></h2>
<p>Dynamic set of string key-value pairs containing
information about the running application -as provided by th </p>
<p>The values in this section are not normatively defined. </p>
@@ -492,9 +503,9 @@ for debugging.</p>
</pre></div>
-<h2 id="status-detailed-dynamic-state"><code>/status</code>: detailed dynamic state</h2>
+<h2 id="status-detailed-dynamic-state"><code>/status</code>: detailed dynamic state<a class="headerlink" href="#status-detailed-dynamic-state" title="Permanent link">¶</a></h2>
<p>This provides more detail on the application including live and failed instances</p>
-<h3 id="statuslive-live-role-instances-by-container"><code>/status/live</code>: live role instances by container</h3>
+<h3 id="statuslive-live-role-instances-by-container"><code>/status/live</code>: live role instances by container<a class="headerlink" href="#statuslive-live-role-instances-by-container" title="Permanent link">¶</a></h3>
<div class="codehilite"><pre>"<span class="n">cluster</span>"<span class="p">:</span> <span class="p">{</span>
"<span class="n">live</span>"<span class="p">:</span> <span class="p">{</span>
"<span class="n">worker</span>"<span class="p">:</span> <span class="p">{</span>
@@ -528,7 +539,7 @@ for debugging.</p>
<p>Failed clusters MAY be listed in the <code>/status/failed</code> section, specifically,
a limited set of recently failed clusters SHOULD be provided.</p>
<p>Future versions of this document may introduce more sections under <code>/status</code>.</p>
-<h3 id="statusrolestatus-role-status-information"><code>/status/rolestatus</code>: role status information</h3>
+<h3 id="statusrolestatus-role-status-information"><code>/status/rolestatus</code>: role status information<a class="headerlink" href="#statusrolestatus-role-status-information" title="Permanent link">¶</a></h3>
<p>This lists the current status of the roles:
How many are running vs requested, how many are being
released.</p>
@@ -562,14 +573,14 @@ released.</p>
</pre></div>
-<h3 id="statusprovider-provider-specific-information"><code>/status/provider</code>: provider-specific information</h3>
+<h3 id="statusprovider-provider-specific-information"><code>/status/provider</code>: provider-specific information<a class="headerlink" href="#statusprovider-provider-specific-information" title="Permanent link">¶</a></h3>
<p>Providers MAY publish information to the <code>/status/provider</code> section.</p>
<ol>
<li>There's no restriction on what JSON is permitted in this section.</li>
<li>Providers may make their own updates to the application state to read and
write this block -operations that are asynchronous to any status queries.</li>
</ol>
-<h2 id="statistics-aggregate-statistics"><code>/statistics</code>: aggregate statistics</h2>
+<h2 id="statistics-aggregate-statistics"><code>/statistics</code>: aggregate statistics<a class="headerlink" href="#statistics-aggregate-statistics" title="Permanent link">¶</a></h2>
<p>Statistics on the cluster and each role in the cluster </p>
<p>Better to have a specific <code>/statistics/cluster</code> element,
and to move the roles' statistics under <code>/statistics/roles</code>:</p>
@@ -612,14 +623,14 @@ and to move the roles' statistics under
<p><code>/statistics/cluster</code> provides aggregate statistics for the entire cluster.</p>
<p>Under <code>/statistics/roles</code> MUST come an entry for each role in the cluster.</p>
<p>All simple values in statistics section are integers.</p>
-<h3 id="clientproperties"><code>/clientProperties</code></h3>
+<h3 id="clientproperties"><code>/clientProperties</code><a class="headerlink" href="#clientproperties" title="Permanent link">¶</a></h3>
<p>The <code>/clientProperties</code> section contains key-val pairs of type
string, the expectation being this is where providers can insert specific
single attributes for client applications.</p>
<p>These values can be converted to application-specific files on the client,
in code -as done today in the Slider CLI-, or via template expansion (beyond
the scope of this document.</p>
-<h3 id="clientfiles"><code>/clientfiles</code></h3>
+<h3 id="clientfiles"><code>/clientfiles</code><a class="headerlink" href="#clientfiles" title="Permanent link">¶</a></h3>
<p>This section list all files that an application instance MAY generate
for clients, along with with a description.</p>
<div class="codehilite"><pre>"<span class="o">/</span><span class="n">clientfiles</span><span class="o">/</span><span class="n">hbase</span><span class="o">-</span><span class="n">site</span><span class="p">.</span><span class="n">xml</span>"<span class="p">:</span> "<span class="n">site</span> <span class="n">information</span> <span class="k">for</span> <span class="n">HBase</span>"
@@ -629,7 +640,7 @@ for clients, along with with a descripti
<p>Client configuration file retrieval is by other means; this
status operation merely lists files that are available;</p>
-<h3 id="aggregate-example">Aggregate Example</h3>
+<h3 id="aggregate-example">Aggregate Example<a class="headerlink" href="#aggregate-example" title="Permanent link">¶</a></h3>
</div>
<div id="footer">
Modified: websites/staging/slider/trunk/content/docs/examples.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/examples.html (original)
+++ websites/staging/slider/trunk/content/docs/examples.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,16 +196,16 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="apache-slider-examples">Apache Slider Examples</h1>
+<h1 id="apache-slider-examples">Apache Slider Examples<a class="headerlink" href="#apache-slider-examples" title="Permanent link">¶</a></h1>
<p><code>Note: These examples use HBase providers written for Slider.</code>
To use the HBase application package refer to <a href="/docs/getting_started.html">Getting Started</a>.</p>
-<h2 id="setup">Setup</h2>
-<h3 id="setting-up-a-yarn-cluster">Setting up a YARN cluster</h3>
+<h2 id="setup">Setup<a class="headerlink" href="#setup" title="Permanent link">¶</a></h2>
+<h3 id="setting-up-a-yarn-cluster">Setting up a YARN cluster<a class="headerlink" href="#setting-up-a-yarn-cluster" title="Permanent link">¶</a></h3>
<p>For simple local demos, a Hadoop pseudo-distributed cluster will suffice -if on a VM then
its configuration should be changed to use a public (machine public) IP.</p>
-<h1 id="the-examples-below-all-assume-there-is-a-cluster-node-called-master-which">The examples below all assume there is a cluster node called 'master', which</h1>
+<h1 id="the-examples-below-all-assume-there-is-a-cluster-node-called-master-which">The examples below all assume there is a cluster node called 'master', which<a class="headerlink" href="#the-examples-below-all-assume-there-is-a-cluster-node-called-master-which" title="Permanent link">¶</a></h1>
<p>hosts the HDFS NameNode and the YARN Resource Manager</p>
-<h1 id="preamble">preamble</h1>
+<h1 id="preamble">preamble<a class="headerlink" href="#preamble" title="Permanent link">¶</a></h1>
<div class="codehilite"><pre><span class="n">export</span> <span class="n">HADOOP_CONF_DIR</span><span class="p">=</span><span class="o">~/</span><span class="n">conf</span>
<span class="n">export</span> <span class="n">PATH</span><span class="p">=</span><span class="o">~/</span><span class="n">hadoop</span><span class="o">/</span><span class="n">bin</span><span class="p">:</span><span class="o">/~/</span><span class="n">hadoop</span><span class="o">/</span><span class="n">sbin</span><span class="p">:</span><span class="o">~/</span><span class="n">zookeeper</span><span class="o">-</span>3<span class="p">.</span>4<span class="p">.</span>5<span class="o">/</span><span class="n">bin</span><span class="p">:</span>$<span class="n">PATH</span>
@@ -202,7 +213,7 @@ its configuration should be changed to u
</pre></div>
-<h1 id="start-all-the-services">start all the services</h1>
+<h1 id="start-all-the-services">start all the services<a class="headerlink" href="#start-all-the-services" title="Permanent link">¶</a></h1>
<div class="codehilite"><pre><span class="n">nohup</span> <span class="n">hdfs</span> <span class="o">--</span><span class="n">config</span> $<span class="n">HADOOP_CONF_DIR</span> <span class="n">namenode</span> <span class="o">&</span>
<span class="n">nohup</span> <span class="n">hdfs</span> <span class="o">--</span><span class="n">config</span> $<span class="n">HADOOP_CONF_DIR</span> <span class="n">datanode</span> <span class="o">&</span>
@@ -212,7 +223,7 @@ its configuration should be changed to u
</pre></div>
-<h1 id="using-hadoopsbin-service-launchers">using hadoop/sbin service launchers</h1>
+<h1 id="using-hadoopsbin-service-launchers">using hadoop/sbin service launchers<a class="headerlink" href="#using-hadoopsbin-service-launchers" title="Permanent link">¶</a></h1>
<div class="codehilite"><pre><span class="n">hadoop</span><span class="o">-</span><span class="n">daemon</span><span class="p">.</span><span class="n">sh</span> <span class="o">--</span><span class="n">config</span> $<span class="n">HADOOP_CONF_DIR</span> <span class="o">--</span><span class="n">script</span> <span class="n">hdfs</span> <span class="n">start</span> <span class="n">namenode</span>
<span class="n">hadoop</span><span class="o">-</span><span class="n">daemon</span><span class="p">.</span><span class="n">sh</span> <span class="o">--</span><span class="n">config</span> $<span class="n">HADOOP_CONF_DIR</span> <span class="o">--</span><span class="n">script</span> <span class="n">hdfs</span> <span class="n">start</span> <span class="n">datanode</span>
<span class="n">yarn</span><span class="o">-</span><span class="n">daemon</span><span class="p">.</span><span class="n">sh</span> <span class="o">--</span><span class="n">config</span> $<span class="n">HADOOP_CONF_DIR</span> <span class="n">start</span> <span class="n">resourcemanager</span>
@@ -222,7 +233,7 @@ its configuration should be changed to u
</pre></div>
-<h1 id="stop-them">stop them</h1>
+<h1 id="stop-them">stop them<a class="headerlink" href="#stop-them" title="Permanent link">¶</a></h1>
<div class="codehilite"><pre><span class="n">hadoop</span><span class="o">-</span><span class="n">daemon</span><span class="p">.</span><span class="n">sh</span> <span class="o">--</span><span class="n">config</span> $<span class="n">HADOOP_CONF_DIR</span> <span class="o">--</span><span class="n">script</span> <span class="n">hdfs</span> <span class="n">stop</span> <span class="n">namenode</span>
<span class="n">hadoop</span><span class="o">-</span><span class="n">daemon</span><span class="p">.</span><span class="n">sh</span> <span class="o">--</span><span class="n">config</span> $<span class="n">HADOOP_CONF_DIR</span> <span class="o">--</span><span class="n">script</span> <span class="n">hdfs</span> <span class="n">stop</span> <span class="n">datanode</span>
@@ -243,7 +254,7 @@ RM yarn-daemon.sh --config $HADOOP_CONF_
<p>Tip: after a successful run on a local cluster, do a quick <code>rm -rf $HADOOP_HOME/logs</code>
to keep the log bloat under control.</p>
-<h2 id="get-hbase-in">get hbase in</h2>
+<h2 id="get-hbase-in">get hbase in<a class="headerlink" href="#get-hbase-in" title="Permanent link">¶</a></h2>
<p>copy to local </p>
<div class="codehilite"><pre><span class="n">get</span> <span class="n">hbase</span><span class="o">-</span>0<span class="p">.</span>98<span class="p">.</span>0<span class="o">-</span><span class="n">bin</span><span class="p">.</span><span class="n">tar</span> <span class="n">on</span>
@@ -259,18 +270,18 @@ to keep the log bloat under control.</p>
</pre></div>
-<h3 id="optional-point-binslider-at-your-chosen-cluster-configuration">Optional: point bin/slider at your chosen cluster configuration</h3>
+<h3 id="optional-point-binslider-at-your-chosen-cluster-configuration">Optional: point bin/slider at your chosen cluster configuration<a class="headerlink" href="#optional-point-binslider-at-your-chosen-cluster-configuration" title="Permanent link">¶</a></h3>
<div class="codehilite"><pre><span class="n">export</span> <span class="n">SLIDER_CONF_DIR</span><span class="p">=</span><span class="o">~/</span><span class="n">Projects</span><span class="o">/</span><span class="n">slider</span><span class="o">/</span><span class="n">slider</span><span class="o">-</span><span class="n">core</span><span class="o">/</span><span class="n">src</span><span class="o">/</span><span class="n">test</span><span class="o">/</span><span class="n">configs</span><span class="o">/</span><span class="n">ubuntu</span><span class="o">-</span><span class="n">secure</span><span class="o">/</span><span class="n">slider</span>
</pre></div>
-<h2 id="optional-clean-up-any-existing-slider-cluster-details">Optional: Clean up any existing slider cluster details</h2>
+<h2 id="optional-clean-up-any-existing-slider-cluster-details">Optional: Clean up any existing slider cluster details<a class="headerlink" href="#optional-clean-up-any-existing-slider-cluster-details" title="Permanent link">¶</a></h2>
<p>This is for demos only, otherwise you lose the clusters and their databases.</p>
<div class="codehilite"><pre><span class="n">hdfs</span> <span class="n">dfs</span> <span class="o">-</span><span class="n">rm</span> <span class="o">-</span><span class="n">r</span> <span class="n">hdfs</span><span class="p">:</span><span class="o">//</span><span class="n">master</span><span class="p">:</span>9090<span class="o">/</span><span class="n">user</span><span class="o">/</span><span class="n">home</span><span class="o">/</span><span class="n">stevel</span><span class="o">/</span><span class="p">.</span><span class="n">slider</span>
</pre></div>
-<h2 id="create-a-slider-cluster">Create a Slider Cluster</h2>
+<h2 id="create-a-slider-cluster">Create a Slider Cluster<a class="headerlink" href="#create-a-slider-cluster" title="Permanent link">¶</a></h2>
<div class="codehilite"><pre><span class="n">slider</span> <span class="n">create</span> <span class="n">cl1</span> <span class="o">\</span>
<span class="o">--</span><span class="n">component</span> <span class="n">worker</span> 1 <span class="o">--</span><span class="n">component</span> <span class="n">master</span> 1 <span class="o">\</span>
<span class="o">--</span><span class="n">manager</span> <span class="n">master</span><span class="p">:</span>8032 <span class="o">--</span><span class="n">filesystem</span> <span class="n">hdfs</span><span class="p">:</span><span class="o">//</span><span class="n">master</span><span class="p">:</span>9090 <span class="o">\</span>
Modified: websites/staging/slider/trunk/content/docs/exitcodes.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/exitcodes.html (original)
+++ websites/staging/slider/trunk/content/docs/exitcodes.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,7 +196,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="apache-slider-client-exit-codes">Apache Slider Client Exit Codes</h1>
+<h1 id="apache-slider-client-exit-codes">Apache Slider Client Exit Codes<a class="headerlink" href="#apache-slider-client-exit-codes" title="Permanent link">¶</a></h1>
<p>Here are the exit codes returned by slider</p>
<p>Many of the exit codes are designed to resemble HTTP error codes,
squashed into a single byte. e.g 44 , "not found" is the equivalent
@@ -412,7 +423,7 @@ of 404</p>
</pre></div>
-<h2 id="other-exit-codes">Other exit codes</h2>
+<h2 id="other-exit-codes">Other exit codes<a class="headerlink" href="#other-exit-codes" title="Permanent link">¶</a></h2>
<p>YARN itself can fail containers, here are some of the causes we've seen</p>
<div class="codehilite"><pre><span class="mi">143</span><span class="o">:</span> <span class="n">Appears</span> <span class="n">to</span> <span class="n">be</span> <span class="n">triggered</span> <span class="n">by</span> <span class="n">the</span> <span class="n">container</span> <span class="n">exceeding</span> <span class="n">its</span> <span class="n">cgroup</span> <span class="n">memory</span>
<span class="n">limits</span>
Modified: websites/staging/slider/trunk/content/docs/getting_started.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/getting_started.html (original)
+++ websites/staging/slider/trunk/content/docs/getting_started.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,9 +196,9 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="apache-slider-getting-started">Apache Slider: Getting Started</h1>
+<h1 id="apache-slider-getting-started">Apache Slider: Getting Started<a class="headerlink" href="#apache-slider-getting-started" title="Permanent link">¶</a></h1>
<p>This page is updated to reflect the latest code in "develop".</p>
-<h2 id="introduction">Introduction</h2>
+<h2 id="introduction">Introduction<a class="headerlink" href="#introduction" title="Permanent link">¶</a></h2>
<p>The following provides the steps required for setting up a cluster and deploying a YARN hosted application using Slider.</p>
<ul>
<li>
@@ -215,7 +226,7 @@ Latest release: <strong>0.80.0-incubatin
<p><a href="#installapp">Install, Configure, Start and Verify Sample Application</a></p>
</li>
</ul>
-<h2 id="system-requirements"><a name="sysreqs"></a>System Requirements</h2>
+<h2 id="system-requirements"><a name="sysreqs"></a>System Requirements<a class="headerlink" href="#system-requirements" title="Permanent link">¶</a></h2>
<p>The Slider deployment has the following minimum system requirements:</p>
<ul>
<li>
@@ -234,7 +245,7 @@ Latest release: <strong>0.80.0-incubatin
<p>openssl</p>
</li>
</ul>
-<h2 id="setup-the-cluster"><a name="setup"></a>Setup the Cluster</h2>
+<h2 id="setup-the-cluster"><a name="setup"></a>Setup the Cluster<a class="headerlink" href="#setup-the-cluster" title="Permanent link">¶</a></h2>
<p>Set up your Hadoop cluster with the services listed above.</p>
<p><em>Note:</em> Ensure the debug delay config is set to a non-zero value to allow easy debugging.
If you are using a single VM or a sandbox then you may need to modify your YARN
@@ -256,15 +267,15 @@ In <code>yarn-site.xml</code> make the f
</table>
<p>There are other options detailed in the Troubleshooting file available <a href="/docs/troubleshooting.html">here</a>.</p>
-<h2 id="download-slider-packages"><a name="download"></a>Download Slider Packages</h2>
+<h2 id="download-slider-packages"><a name="download"></a>Download Slider Packages<a class="headerlink" href="#download-slider-packages" title="Permanent link">¶</a></h2>
<p><em>You can build it as described below.</em></p>
-<h2 id="build-slider"><a name="build"></a>Build Slider</h2>
+<h2 id="build-slider"><a name="build"></a>Build Slider<a class="headerlink" href="#build-slider" title="Permanent link">¶</a></h2>
<ul>
<li>From the top level directory, execute <code>mvn clean site:site site:stage package -DskipTests</code></li>
<li>Use the generated compressed tar file in slider-assembly/target directory (e.g. slider-0.80.0-incubating-all.tar.gz or slider-0.80.0-incubating-all.zip) for the subsequent steps</li>
<li>If you are cloning the Slider git repo, go to <code>releases/slider-0.80.0-incubating</code> branch for the latest released or <code>develop</code> for the latest under development</li>
</ul>
-<h2 id="install-slider"><a name="install"></a>Install Slider</h2>
+<h2 id="install-slider"><a name="install"></a>Install Slider<a class="headerlink" href="#install-slider" title="Permanent link">¶</a></h2>
<p>Slider is installed on a client machine that can access the hadoop cluster. Follow the following steps to expand/install Slider:</p>
<div class="codehilite"><pre>mkdir <span class="cp">${</span><span class="n">slider</span><span class="o">-</span><span class="n">install</span><span class="o">-</span><span class="nb">dir</span><span class="cp">}</span>
@@ -276,7 +287,7 @@ cd <span class="cp">${</span><span class
<p>Login as the "yarn" user (assuming this is a host associated with the installed cluster). E.g., <code>su yarn</code>
<em>This assumes that all apps are being run as âyarnâ user</em></p>
<p>Expand the tar file: <code>tar -xvf slider-0.80.0-incubating-all.tar.gz</code> or <code>unzip slider-0.80.0-incubating-all.zip</code></p>
-<h2 id="configure-slider">Configure Slider</h2>
+<h2 id="configure-slider">Configure Slider<a class="headerlink" href="#configure-slider" title="Permanent link">¶</a></h2>
<p>Browse to the Slider directory: <code>cd slider-0.80.0-incubating/conf</code></p>
<p>Edit <code>slider-env.sh</code> and specify correct values.</p>
<div class="codehilite"><pre><span class="n">export</span> <span class="n">JAVA_HOME</span><span class="p">=</span><span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">jdk64</span><span class="o">/</span><span class="n">jdk1</span><span class="p">.</span>7<span class="p">.</span>0<span class="n">_67</span>
@@ -337,9 +348,9 @@ directory containing the <code>slider-cl
<p>Ensure there are no errors and you can see "Compiled against Hadoop 2.6.0". <strong>This ensures that Slider is installed correctly.</strong></p>
-<h2 id="deploy-slider-resources"><a name="deploy"></a>Deploy Slider Resources</h2>
+<h2 id="deploy-slider-resources"><a name="deploy"></a>Deploy Slider Resources<a class="headerlink" href="#deploy-slider-resources" title="Permanent link">¶</a></h2>
<p>Ensure that all file folders are accessible to the user creating the application instance. The example assumes "yarn" to be that user. The default yarn user may be different e.g. hadoop.</p>
-<h3 id="ensure-hdfs-home-folder-for-the-user">Ensure HDFS home folder for the User</h3>
+<h3 id="ensure-hdfs-home-folder-for-the-user">Ensure HDFS home folder for the User<a class="headerlink" href="#ensure-hdfs-home-folder-for-the-user" title="Permanent link">¶</a></h3>
<p>Perform the following steps to create the user home folder with the appropriate permissions:</p>
<div class="codehilite"><pre><span class="n">su</span> <span class="n">hdfs</span>
@@ -349,7 +360,7 @@ directory containing the <code>slider-cl
</pre></div>
-<h2 id="create-application-packages"><a name="downsample"></a>Create Application Packages</h2>
+<h2 id="create-application-packages"><a name="downsample"></a>Create Application Packages<a class="headerlink" href="#create-application-packages" title="Permanent link">¶</a></h2>
<p>There are several sample application packages available for use with Slider:</p>
<ul>
<li>
@@ -366,7 +377,7 @@ directory containing the <code>slider-cl
</li>
</ul>
<p>Create one or more Slider application packages and follow the steps below to install them.</p>
-<h2 id="install-configure-start-and-verify-sample-application"><a name="installapp"></a>Install, Configure, Start and Verify Sample Application</h2>
+<h2 id="install-configure-start-and-verify-sample-application"><a name="installapp"></a>Install, Configure, Start and Verify Sample Application<a class="headerlink" href="#install-configure-start-and-verify-sample-application" title="Permanent link">¶</a></h2>
<ul>
<li>
<p><a href="#load">Install Sample Application Package</a></p>
@@ -384,7 +395,7 @@ directory containing the <code>slider-cl
<p><a href="#manage">Manage the Application Lifecycle</a></p>
</li>
</ul>
-<h3 id="install-sample-application-package"><a name="load"></a>Install Sample Application Package</h3>
+<h3 id="install-sample-application-package"><a name="load"></a>Install Sample Application Package<a class="headerlink" href="#install-sample-application-package" title="Permanent link">¶</a></h3>
<div class="codehilite"><pre><span class="n">slider</span> <span class="n">install</span><span class="o">-</span><span class="n">package</span> <span class="o">--</span><span class="n">name</span> <span class="o">*</span><span class="n">package</span> <span class="n">name</span><span class="o">*</span> <span class="o">--</span><span class="n">package</span> <span class="o">*</span><span class="n">sample</span><span class="o">-</span><span class="n">application</span><span class="o">-</span><span class="n">package</span><span class="o">*</span>
</pre></div>
@@ -396,25 +407,25 @@ This path is also reflected in appConfig
</pre></div>
-<h3 id="create-application-specifications"><a name="create"></a>Create Application Specifications</h3>
+<h3 id="create-application-specifications"><a name="create"></a>Create Application Specifications<a class="headerlink" href="#create-application-specifications" title="Permanent link">¶</a></h3>
<p>Configuring a Slider application consists of two parts: the <a href="#resspec">Resource Specification</a>,
and the *<a href="#appconfig">Application Configuration</a>.</p>
<p><strong>Note</strong>: There are sample Resource Specifications (<strong>resources-default.json</strong>) and Application Configuration
(<strong>appConfig-default.json</strong>) files in the application packages zip files.</p>
-<h4 id="resource-specification"><a name="resspec"></a>Resource Specification</h4>
+<h4 id="resource-specification"><a name="resspec"></a>Resource Specification<a class="headerlink" href="#resource-specification" title="Permanent link">¶</a></h4>
<p>Slider needs to know what components (and how many components) to deploy.</p>
<p>As Slider creates each instance of a component in its own YARN container, it also needs to know what to ask YARN for in terms of <strong>memory</strong> and <strong>CPU</strong> for those containers. </p>
<p>All this information goes into the <strong>Resources Specification</strong> file ("Resource Spec") named <code>resources.json</code>. The Resource Spec tells Slider how many instances of each component in the application (such as an HBase RegionServer) to deploy and the parameters for YARN.</p>
<p>An application package should contain the default resources.json and you can start from there. Or you can create one based on <a href="/docs/configuration/resources.html">Resource Specification</a>).</p>
<p>Store the Resource Spec file on your local disk (e.g. <code>/tmp/resources.json</code>).</p>
-<h4 id="application-configuration"><a name="appconfig"></a>Application Configuration</h4>
+<h4 id="application-configuration"><a name="appconfig"></a>Application Configuration<a class="headerlink" href="#application-configuration" title="Permanent link">¶</a></h4>
<p>Alongside the Resource Spec there is the <strong>Application Configuration</strong> file ("App Config") which includes parameters that are specific to the application, rather than YARN. The App Config is a file that contains all application configuration. This configuration is applied to the default configuration provided by the application definition and then handed off to the associated component agent.</p>
<p>For example, the heap sizes of the JVMs, The App Config defines the configuration details <strong>specific to the application and component</strong> instances. An application package should contain the default resources.json and you can start from there.</p>
<p>Ensure the following variables are accurate:</p>
<ul>
<li>"application.def": "application_package.zip" (path to uploaded application package)</li>
</ul>
-<h3 id="start-the-application"><a name="start"></a>Start the Application</h3>
+<h3 id="start-the-application"><a name="start"></a>Start the Application<a class="headerlink" href="#start-the-application" title="Permanent link">¶</a></h3>
<p>Once the steps above are completed, the application can be started through the <strong>Slider Command Line Interface (CLI)</strong>.</p>
<p>Change directory to the "bin" directory under the slider installation</p>
<div class="codehilite"><pre>cd <span class="cp">${</span><span class="n">slider</span><span class="o">-</span><span class="n">install</span><span class="o">-</span><span class="nb">dir</span><span class="cp">}</span>/slider-0.80.0-incubating/bin
@@ -426,12 +437,12 @@ This path is also reflected in appConfig
</pre></div>
-<h3 id="verify-the-application"><a name="verify"></a>Verify the Application</h3>
+<h3 id="verify-the-application"><a name="verify"></a>Verify the Application<a class="headerlink" href="#verify-the-application" title="Permanent link">¶</a></h3>
<p>The successful launch of the application can be verified via the YARN Resource Manager Web UI. In most instances, this UI is accessible via a web browser at port 8088 of the Resource Manager Host:</p>
<p><img alt="image alt text" src="../images/image_1_apps_rm_ui-lrg.png" /></p>
<p>The specific information for the running application is accessible via the "ApplicationMaster" link that can be seen in the far right column of the row associated with the running application (probably the top row):</p>
<p><img alt="image alt text" src="../images/image_0_app_master_ui_lrg.png" /></p>
-<h3 id="obtain-client-config"><a name="client-config"></a>Obtain client config</h3>
+<h3 id="obtain-client-config"><a name="client-config"></a>Obtain client config<a class="headerlink" href="#obtain-client-config" title="Permanent link">¶</a></h3>
<p>An application publishes several useful details that can be used to manage and communicate with the application instance.</p>
<p>You can use <code>registry</code> commands to get all published data.</p>
<p><strong>All published data</strong></p>
@@ -461,30 +472,30 @@ Client configurations are at <code>/ws/v
<p><strong>Quick links</strong></p>
<p>An application may publish some quick links for the monitoring UI, JMX endpoint, etc. as the listening ports are dynamically allocated and thus need to be published. These information is available at <code>/ws/v1/slider/publisher/slider/quicklinks</code>.</p>
-<h3 id="manage-the-application-lifecycle"><a name="manage"></a>Manage the Application Lifecycle</h3>
+<h3 id="manage-the-application-lifecycle"><a name="manage"></a>Manage the Application Lifecycle<a class="headerlink" href="#manage-the-application-lifecycle" title="Permanent link">¶</a></h3>
<p>Once started, applications can be stopped, (re)started, and destroyed as follows:</p>
-<h4 id="stopped">Stopped:</h4>
+<h4 id="stopped">Stopped:<a class="headerlink" href="#stopped" title="Permanent link">¶</a></h4>
<div class="codehilite"><pre><span class="o">./</span><span class="n">slider</span> <span class="n">stop</span> <span class="n">cl1</span>
</pre></div>
-<h4 id="started">Started:</h4>
+<h4 id="started">Started:<a class="headerlink" href="#started" title="Permanent link">¶</a></h4>
<div class="codehilite"><pre><span class="o">./</span><span class="n">slider</span> <span class="n">start</span> <span class="n">cl1</span>
</pre></div>
-<h4 id="destroyed">Destroyed:</h4>
+<h4 id="destroyed">Destroyed:<a class="headerlink" href="#destroyed" title="Permanent link">¶</a></h4>
<div class="codehilite"><pre><span class="o">./</span><span class="n">slider</span> <span class="n">destroy</span>
</pre></div>
-<h4 id="flexed">Flexed:</h4>
+<h4 id="flexed">Flexed:<a class="headerlink" href="#flexed" title="Permanent link">¶</a></h4>
<div class="codehilite"><pre><span class="o">./</span><span class="n">slider</span> <span class="n">flex</span> <span class="n">cl1</span> <span class="o">--</span><span class="n">component</span> <span class="n">worker</span> 5
</pre></div>
-<h1 id="appendix-a-debugging-slider-agent"><a name="appendixa"></a>Appendix A: Debugging Slider-Agent</h1>
-<h2 id="create-and-deploy-slider-agent-configuration">Create and deploy Slider Agent configuration</h2>
+<h1 id="appendix-a-debugging-slider-agent"><a name="appendixa"></a>Appendix A: Debugging Slider-Agent<a class="headerlink" href="#appendix-a-debugging-slider-agent" title="Permanent link">¶</a></h1>
+<h2 id="create-and-deploy-slider-agent-configuration">Create and deploy Slider Agent configuration<a class="headerlink" href="#create-and-deploy-slider-agent-configuration" title="Permanent link">¶</a></h2>
<p>Create an agent config file (agent.ini) based on the sample available at:</p>
<div class="codehilite"><pre><span class="cp">${</span><span class="n">slider</span><span class="o">-</span><span class="n">install</span><span class="o">-</span><span class="nb">dir</span><span class="cp">}</span>/slider-0.80.0-incubating/agent/conf/agent.ini
</pre></div>
Modified: websites/staging/slider/trunk/content/docs/high_availability.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/high_availability.html (original)
+++ websites/staging/slider/trunk/content/docs/high_availability.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
@@ -182,13 +193,13 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License. See accompanying LICENSE file.
-->
-<h1 id="high-availability">High Availability:</h1>
-<h3 id="setup-resource-manager-ha-name-node-ha-work-preserving-resource-manager-restart-and-work-preserving-node-manager-restart">Setup Resource Manager HA, Name Node HA, Work Preserving Resource Manager restart and Work Preserving Node Manager restart</h3>
+<h1 id="high-availability">High Availability:<a class="headerlink" href="#high-availability" title="Permanent link">¶</a></h1>
+<h3 id="setup-resource-manager-ha-name-node-ha-work-preserving-resource-manager-restart-and-work-preserving-node-manager-restart">Setup Resource Manager HA, Name Node HA, Work Preserving Resource Manager restart and Work Preserving Node Manager restart<a class="headerlink" href="#setup-resource-manager-ha-name-node-ha-work-preserving-resource-manager-restart-and-work-preserving-node-manager-restart" title="Permanent link">¶</a></h3>
<p>Setting up High Availability ensures uninterrupted service provided by long running applications
installed by Slider in the event of any or all of the following YARN component failures - Resource Manager,
Name Node and Node Manager. This document provides details on how to configure YARN properties to
achieve the corresponding high availability setup.</p>
-<h3 id="resource-manager-ha-automatic">Resource Manager HA (automatic)</h3>
+<h3 id="resource-manager-ha-automatic">Resource Manager HA (automatic)<a class="headerlink" href="#resource-manager-ha-automatic" title="Permanent link">¶</a></h3>
<div class="codehilite"><pre> <span class="nt"><property></span>
<span class="nt"><name></span>yarn.resourcemanager.ha.enabled<span class="nt"></name></span>
<span class="nt"><value></span>true<span class="nt"></value></span>
@@ -257,11 +268,11 @@ achieve the corresponding high availabil
</pre></div>
-<h3 id="name-node-ha">Name Node HA</h3>
+<h3 id="name-node-ha">Name Node HA<a class="headerlink" href="#name-node-ha" title="Permanent link">¶</a></h3>
<p>Setting up NN HA is outside the scope of this document. Please refer to this
<a href="http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html">HDFS HA document (r2.6.0)</a>
for setup details.</p>
-<h3 id="work-preserving-rm-restart">Work Preserving RM Restart</h3>
+<h3 id="work-preserving-rm-restart">Work Preserving RM Restart<a class="headerlink" href="#work-preserving-rm-restart" title="Permanent link">¶</a></h3>
<div class="codehilite"><pre> <span class="nt"><property></span>
<span class="nt"><description></span>Enable RM to recover state after starting. If true, then
yarn.resourcemanager.store.class must be specified
@@ -297,7 +308,7 @@ for setup details.</p>
</pre></div>
-<h3 id="work-preserving-nm-restart">Work Preserving NM Restart</h3>
+<h3 id="work-preserving-nm-restart">Work Preserving NM Restart<a class="headerlink" href="#work-preserving-nm-restart" title="Permanent link">¶</a></h3>
<div class="codehilite"><pre> <span class="nt"><property></span>
<span class="nt"><description></span>Enable the node manager to recover after starting<span class="nt"></description></span>
<span class="nt"><name></span>yarn.nodemanager.recovery.enabled<span class="nt"></name></span>
Modified: websites/staging/slider/trunk/content/docs/index.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/index.html (original)
+++ websites/staging/slider/trunk/content/docs/index.html Mon Nov 2 17:24:47 2015
@@ -155,7 +155,7 @@
<div style="text-align: center">
<h1><a href="/index.html">Apache Slider (incubating)</a></h1>
<hr>
-Latest release: <strong>0.80.0-incubating</strong><br>
+Latest release: <strong>0.81.1-incubating</strong><br>
<br>
<a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
</div>
@@ -168,7 +168,18 @@ Latest release: <strong>0.80.0-incubatin
<h1 class="title"></h1>
- <!---
+ <style type="text/css">
+/* The following code is added by mdx_elementid.py
+ It was originally lifted from http://subversion.apache.org/style/site.css */
+/*
+ * Hide class="elementid-permalink", except when an enclosing heading
+ * has the :hover property.
+ */
+.headerlink, .elementid-permalink {
+ visibility: hidden;
+}
+h2:hover > .headerlink, h3:hover > .headerlink, h1:hover > .headerlink, h6:hover > .headerlink, h4:hover > .headerlink, h5:hover > .headerlink, dt:hover > .elementid-permalink { visibility: visible }</style>
+<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
@@ -185,7 +196,7 @@ Latest release: <strong>0.80.0-incubatin
limitations under the License.
-->
-<h1 id="using-slider">Using Slider</h1>
+<h1 id="using-slider">Using Slider<a class="headerlink" href="#using-slider" title="Permanent link">¶</a></h1>
<ul>
<li><a href="getting_started.html">Getting Started</a></li>
<li><a href="manpage.html">Man Page</a></li>