You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by bu...@apache.org on 2014/10/09 03:35:41 UTC
svn commit: r925112 - in /websites/staging/slider/trunk/content: ./
docs/getting_started.html docs/manpage.html
docs/slider_specs/resource_specification.html
Author: buildbot
Date: Thu Oct 9 01:35:41 2014
New Revision: 925112
Log:
Staging update by buildbot for slider
Modified:
websites/staging/slider/trunk/content/ (props changed)
websites/staging/slider/trunk/content/docs/getting_started.html
websites/staging/slider/trunk/content/docs/manpage.html
websites/staging/slider/trunk/content/docs/slider_specs/resource_specification.html
Propchange: websites/staging/slider/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Thu Oct 9 01:35:41 2014
@@ -1 +1 @@
-1630264
+1630266
Modified: websites/staging/slider/trunk/content/docs/getting_started.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/getting_started.html (original)
+++ websites/staging/slider/trunk/content/docs/getting_started.html Thu Oct 9 01:35:41 2014
@@ -223,10 +223,11 @@ Latest release: <strong>0.40-incubating<
</li>
</ul>
<h2 id="wzxhzdk22wzxhzdk23setup-the-cluster"><a name="setup"></a>Setup the Cluster</h2>
-<p>Setting up your Hadoop cluster with the services listed above.</p>
-<p>If you are using a one-box or a sandbox then you may need to modify your YARN
-configuration to allow for multiple, modify your YARN configuration to allow for multiple
-containers on a single host. In <code>yarn-site.xml</code> make the following modifications:</p>
+<p>Set up your Hadoop cluster with the services listed above.</p>
+<p><em>Note:</em> Ensure the debug delay config is set to a non-zero value to allow easy debugging.
+If you are using a single VM or a sandbox then you may need to modify your YARN
+configuration to allow for multiple containers on a single host.
+In <code>yarn-site.xml</code> make the following modifications:</p>
<table>
<tr>
<td>Property</td>
Modified: websites/staging/slider/trunk/content/docs/manpage.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/manpage.html (original)
+++ websites/staging/slider/trunk/content/docs/manpage.html Thu Oct 9 01:35:41 2014
@@ -268,7 +268,7 @@ If the application instance is live, the
<h3 id="create-name"><code>create <name></code></h3>
<p>Build and run an application instance of the given name </p>
<p>The <code>--wait</code> parameter, if provided, specifies the time to wait until the YARN application is actually running. Even after the YARN application has started, there may be some delay for the instance to start up.</p>
-<h3 id="arguments-for-build-and-create">Arguments for <code>build</code> and <code>create</code></h3>
+<h4 id="arguments-for-build-and-create"><strong>Arguments for <code>build</code> and <code>create</code></strong></h4>
<h5 id="-package-uri-to-package"><code>--package <uri-to-package></code></h5>
<p>This define the slider application package to be deployed.</p>
<h5 id="-option-name-value"><code>--option <name> <value></code></h5>
Modified: websites/staging/slider/trunk/content/docs/slider_specs/resource_specification.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/slider_specs/resource_specification.html (original)
+++ websites/staging/slider/trunk/content/docs/slider_specs/resource_specification.html Thu Oct 9 01:35:41 2014
@@ -214,7 +214,7 @@ Latest release: <strong>0.40-incubating<
</pre></div>
-<h2 id="wzxhzdk5wzxhzdk6container-failure-policy"><a name="failurepolicy"></a>Container Failure Policy</h2>
+<h2 id="wzxhzdk4wzxhzdk5container-failure-policy"><a name="failurepolicy"></a>Container Failure Policy</h2>
<p>YARN containers hosting component instances may fail. This can happen because of</p>
<ol>
<li>A problem in the configuration of the instance.</li>
@@ -317,51 +317,11 @@ which are frequently failing due to conf
<p>In a production application, large failure thresholds and/or shorter windows
ensures that the application is resilient to transient failures of the underlying
YARN cluster and hardware.</p>
-<h2 id="wzxhzdk7wzxhzdk8using-labels"><a name="labels"></a>Using Labels</h2>
+<h2 id="wzxhzdk6wzxhzdk7using-labels"><a name="labels"></a>Using Labels</h2>
<p>The resources.json file can be used to specify the labels to be used when allocating containers for the components. The details of the YARN Label feature can be found at <a href="https://issues.apache.org/jira/browse/YARN-796">YARN-796</a>.</p>
-<p>In summary:
-<em> Nodes can be assigned one or more labels
-</em> Capacity Queues can be defined with access to one or more labels
-<em> Ensure application components are associated with appropriate label expressions
-</em> Create the application using specific queue</p>
-<p>This way, you can guarantee that a certain set of nodes are reserved for an application or for a component within an application.</p>
-<p>Label expression is specified through property "yarn.label.expression". When no label expression is specified then it is assumed that only non-labeled nodes are used when allocating containers for component instances.</p>
-<p>If label expression is specified for slider-appmaster then it also becomes the default label expression for all component. To take advantage of default label expression leave out the property (see HBASE_REGIONSERVER in the example). Label expression with empty string ("yarn.label.expression":"") means nodes without labels. </p>
-<h3 id="example_1">Example</h3>
-<p>Here is a <code>resource.json</code> file for an HBase cluster which uses labels. The label for the application instance is "hbase1" and the label expression for the HBASE_MASTER components is "hbase1_master". HBASE_REGIONSERVER instances will automatically use label "hbase1". Alternatively, if you specify ("yarn.label.expression":"") for HBASE_REGIONSERVER then the containers will only be allocated on nodes with no labels.</p>
-<div class="codehilite"><pre><span class="p">{</span>
- "<span class="n">schema</span>"<span class="p">:</span> "<span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">example</span><span class="p">.</span><span class="n">org</span><span class="o">/</span><span class="n">specification</span><span class="o">/</span><span class="n">v2</span><span class="p">.</span>0<span class="p">.</span>0"<span class="p">,</span>
- "<span class="n">metadata</span>"<span class="p">:</span> <span class="p">{</span>
- <span class="p">},</span>
- "<span class="k">global</span>"<span class="p">:</span> <span class="p">{</span>
- <span class="p">},</span>
- "<span class="n">components</span>"<span class="p">:</span> <span class="p">{</span>
- "<span class="n">HBASE_MASTER</span>"<span class="p">:</span> <span class="p">{</span>
- "<span class="n">yarn</span><span class="p">.</span><span class="n">role</span><span class="p">.</span><span class="n">priority</span>"<span class="p">:</span> "1"<span class="p">,</span>
- "<span class="n">yarn</span><span class="p">.</span><span class="n">component</span><span class="p">.</span><span class="n">instances</span>"<span class="p">:</span> "1"<span class="p">,</span>
- "<span class="n">yarn</span><span class="p">.</span><span class="n">label</span><span class="p">.</span><span class="n">expression</span>"<span class="p">:</span>"<span class="n">hbase1_master</span>"
- <span class="p">},</span>
- "<span class="n">HBASE_REGIONSERVER</span>"<span class="p">:</span> <span class="p">{</span>
- "<span class="n">yarn</span><span class="p">.</span><span class="n">role</span><span class="p">.</span><span class="n">priority</span>"<span class="p">:</span> "1"<span class="p">,</span>
- "<span class="n">yarn</span><span class="p">.</span><span class="n">component</span><span class="p">.</span><span class="n">instances</span>"<span class="p">:</span> "1"<span class="p">,</span>
- <span class="p">},</span>
- "<span class="n">slider</span><span class="o">-</span><span class="n">appmaster</span>"<span class="p">:</span> <span class="p">{</span>
- "<span class="n">yarn</span><span class="p">.</span><span class="n">label</span><span class="p">.</span><span class="n">expression</span>"<span class="p">:</span>"<span class="n">hbase1</span>"
- <span class="p">}</span>
- <span class="p">}</span>
-<span class="p">}</span>
-</pre></div>
-
-
-<p>Specifically, for the above example you will need:
-<em> Create two labels, <code>hbase1</code> and <code>hbase1_master</code> (use yarn rmadmin commands)
-</em> Assign the labels to nodes (use yarn rmadmin commands)
-<em> Perform refresh queue (yarn -refreshqueue)
-</em> Create a queue by defining it in the capacity scheduler config
-<em> Allow the queue to access to the labels and ensure that appropriate min/max capacity is assigned
-</em> Perform refresh queue (yarn -refreshqueue)
-* Create the Slider application against the above queue using parameter <code>--queue</code> while creating the application</p>
-<h2 id="wzxhzdk9wzxhzdk10using-log-aggregation"><a name="logagg"></a>Using Log Aggregation</h2>
+<p>The feature is not yet committed.</p>
+<h2 id="wzxhzdk8wzxhzdk9using-log-aggregation"><a name="logagg"></a>Using Log Aggregation</h2>
+<p>The feature is not yet committed.</p>
</div>
<div id="footer">