You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by bu...@apache.org on 2016/02/05 22:47:29 UTC
svn commit: r979608 - in /websites/staging/slider/trunk/content: ./
docs/manpage.html docs/slider_specs/writing_app_command_scripts.html
Author: buildbot
Date: Fri Feb 5 21:47:29 2016
New Revision: 979608
Log:
Staging update by buildbot for slider
Modified:
websites/staging/slider/trunk/content/ (props changed)
websites/staging/slider/trunk/content/docs/manpage.html
websites/staging/slider/trunk/content/docs/slider_specs/writing_app_command_scripts.html
Propchange: websites/staging/slider/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Fri Feb 5 21:47:29 2016
@@ -1 +1 @@
-1726136
+1728735
Modified: websites/staging/slider/trunk/content/docs/manpage.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/manpage.html (original)
+++ websites/staging/slider/trunk/content/docs/manpage.html Fri Feb 5 21:47:29 2016
@@ -265,6 +265,24 @@ If not, the changes will be picked up wh
<p>Use the specific filesystem URI as an argument to the operation.</p>
<!--- ======================================================================= -->
+<h2 id="instance-naming">Instance Naming<a class="headerlink" href="#instance-naming" title="Permanent link">¶</a></h2>
+<p>Application instance names must:</p>
+<ol>
+<li>be at least one character long</li>
+<li>begin with a lower case letter</li>
+<li>All other characters must be in the range [a-z,0-9,_]</li>
+<li>All upper case characters are converted to lower case</li>
+</ol>
+<p>Example valid names:</p>
+<div class="codehilite"><pre><span class="n">slider1</span>
+<span class="n">storm4</span>
+<span class="n">hbase_instance</span>
+<span class="n">accumulo_m1_tserve4</span>
+</pre></div>
+
+
+<!--- ======================================================================= -->
+
<!--- ======================================================================= -->
<h2 id="actions">Actions<a class="headerlink" href="#actions" title="Permanent link">¶</a></h2>
@@ -525,7 +543,7 @@ and the principal to log in as. The file
If the JVM does not support this length, the command will fail.
The default value is to 256, as needed for the <code>AES256</code> encryption scheme.
A JVM without the Java Cryptography Extensions installed does not support
-<code>--keylength 256</code>: Kerberos will not work Kerberos itself is configured to use
+a key length of 256 bits: Kerberos will unless configured to use
an encryption scheme with a shorter key length.</p>
</li>
<li>
@@ -536,14 +554,22 @@ That is: if the hadoop authentication me
<p>Although there is a <code>--out outfile</code> option, much of the output can come from the JRE
(to <code>stderr</code>) and via log4j (to <code>stdout</code>). To get all the output, it is best
to redirect both these output streams to the same file, and omit the <code>--out</code> option.</p>
-<div class="codehilite"><pre><span class="n">slider</span> <span class="n">kdiag</span> <span class="o">--</span><span class="n">keytab</span> <span class="n">zk</span><span class="p">.</span><span class="n">service</span><span class="p">.</span><span class="n">keytab</span> <span class="o">--</span><span class="n">principal</span> <span class="n">zookeeper</span><span class="o">/</span><span class="n">devix</span><span class="p">@</span><span class="n">REALM</span> <span class="o">></span> <span class="n">out</span><span class="p">.</span><span class="n">txt</span> 2<span class="o">>&</span>1
+<div class="codehilite"><pre><span class="n">slider</span> <span class="n">kdiag</span> <span class="o">--</span><span class="n">keytab</span> <span class="n">zk</span><span class="p">.</span><span class="n">service</span><span class="p">.</span><span class="n">keytab</span> <span class="o">--</span><span class="n">principal</span> <span class="n">zookeeper</span><span class="o">/</span><span class="n">devix</span><span class="p">.</span><span class="n">example</span><span class="p">.</span><span class="n">org</span><span class="p">@</span><span class="n">REALM</span> <span class="o">></span> <span class="n">out</span><span class="p">.</span><span class="n">txt</span> 2<span class="o">>&</span>1
</pre></div>
<p>For extra logging during the operation</p>
<ol>
-<li>Set the environment variable <code>HADOOP_JAAS_DEBUG</code> to <code>true</code>.</li>
-<li>Edit the <code>log4j.properties</code> file for the slider client:<div class="codehilite"><pre><span class="n">log4j</span><span class="p">.</span><span class="n">logger</span><span class="p">.</span><span class="n">org</span><span class="p">.</span><span class="n">apache</span><span class="p">.</span><span class="n">hadoop</span><span class="p">.</span><span class="n">security</span><span class="p">=</span><span class="n">DEBUG</span>
+<li>
+<p>Set the environment variable <code>HADOOP_JAAS_DEBUG</code> to <code>true</code>.</p>
+<div class="codehilite"><pre><span class="n">export</span> <span class="n">HADOOP_JAAS_DEBUG</span><span class="p">=</span><span class="n">true</span>
+</pre></div>
+
+
+</li>
+<li>
+<p>Edit the <code>log4j.properties</code> file for the slider client:</p>
+<div class="codehilite"><pre><span class="n">log4j</span><span class="p">.</span><span class="n">logger</span><span class="p">.</span><span class="n">org</span><span class="p">.</span><span class="n">apache</span><span class="p">.</span><span class="n">hadoop</span><span class="p">.</span><span class="n">security</span><span class="p">=</span><span class="n">DEBUG</span>
</pre></div>
@@ -741,8 +767,7 @@ the version of Hadoop that is currently
<p>Note that this is the client-side Hadoop version, not that running on the server, though
that can be obtained in the status operation</p>
<h2 id="commands-for-testing">Commands for testing<a class="headerlink" href="#commands-for-testing" title="Permanent link">¶</a></h2>
-<p>These are clearly abnormal operations; they are here primarily for testing
--and documented for completeness.</p>
+<p>These operations are here primarily for testing.</p>
<h3 id="kill-container-name-id-container-id"><code>kill-container <name> --id container-id</code><a class="headerlink" href="#kill-container-name-id-container-id" title="Permanent link">¶</a></h3>
<p>Kill a YARN container belong to the application. This is useful primarily for testing the
resilience to failures.</p>
@@ -758,22 +783,22 @@ operation that does not make any attempt
</pre></div>
-<!--- ======================================================================= -->
-
-<h2 id="instance-naming">Instance Naming<a class="headerlink" href="#instance-naming" title="Permanent link">¶</a></h2>
-<p>Application instance names must:</p>
-<ol>
-<li>be at least one character long</li>
-<li>begin with a lower case letter</li>
-<li>All other characters must be in the range [a-z,0-9,_]</li>
-<li>All upper case characters are converted to lower case</li>
-</ol>
-<p>Example valid names:</p>
-<div class="codehilite"><pre><span class="n">slider1</span>
-<span class="n">storm4</span>
-<span class="n">hbase_instance</span>
-<span class="n">accumulo_m1_tserve4</span>
-</pre></div>
+<h3 id="tokens-source-file-out-file-keytab-keytab-principal-principal"><code>tokens [--source <file>] [--out file] [--keytab <keytab> --principal <principal>]</code><a class="headerlink" href="#tokens-source-file-out-file-keytab-keytab-principal-principal" title="Permanent link">¶</a></h3>
+<p>Lists current delegation tokens, or, on a secure cluster creates new ones.</p>
+<p>This is useful for testing
+the delegation token mechanism offered by Oozie, in which Oozie collects
+the tokens needed by slider, saves them to a file, then starts slider in
+with the environment variable <code>HADOOP_TOKEN_FILE_LOCATION</code> set to the location
+of this file. For ease of doing that, the bash command to set the property is
+printed.</p>
+<p>For reference the tokens needed are:</p>
+<ul>
+<li>An HDFS token.</li>
+<li>A YARN client token to interact with the RM.</li>
+<li>If the timeline server is enabled, a timeline server delegation token.</li>
+</ul>
+<p>If the <code>--keytab</code> and <code>--principal</code> arguments are supplied, then the credentials
+will be generated with the named principal logged in from the specific keytab.</p>
</div>
<div id="footer">
Modified: websites/staging/slider/trunk/content/docs/slider_specs/writing_app_command_scripts.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/slider_specs/writing_app_command_scripts.html (original)
+++ websites/staging/slider/trunk/content/docs/slider_specs/writing_app_command_scripts.html Fri Feb 5 21:47:29 2016
@@ -203,12 +203,13 @@ h2:hover > .headerlink, h3:hover > .head
<ul>
-<li>SCRIPT is the top level script that implements the commands for the component. </li>
-<li>COMMAND is one of the following default commands - START, STOP, INSTALL, CONFIG, RESTART, STATUS or any custom commands. </li>
-<li>JSON_FILE includes all configuration parameters and the values. </li>
-<li>PACKAGE_ROOT is the root folder of the package. From this folder, its possible to access files, scripts, templates, packages (e.g. tarballs), etc. The Yarn-App author has complete control over the structure of the package as long as the PACKAGE_ROOT and SCRIPT path is known to the management tool. </li>
-<li>STRUCTURED_OUT_FILE is the file where the script can output structured data. </li>
-<li>LOG_LEVEL is the log level to use when executing the script - Slider defaults to INFO, use other levels while debugging</li>
+<li><code>SCRIPT</code> is the top level script that implements the commands for the component. </li>
+<li><code>COMMAND</code> is one of the following default commands - <code>START</code>, <code>STOP</code>, <code>INSTALL</code>, <code>CONFIG</code>, <code>RESTART</code>, <code>STATUS</code> or any custom commands. </li>
+<li><code>JSON_FILE</code> includes all configuration parameters and the values. </li>
+<li><code>PACKAGE_ROOT</code> is the root folder of the package. From this folder, its possible to access files, scripts, templates, packages (e.g. tarballs), etc.
+The Yarn-App author has complete control over the structure of the package as long as the <code>PACKAGE_ROOT</code> and <code>SCRIPT</code> path is known to the management tool. </li>
+<li><code>STRUCTURED_OUT_FILE</code> is the file where the script can output structured data. </li>
+<li><code>LOG_LEVEL</code> is the log level to use when executing the script - Slider defaults to INFO, use other levels while debugging</li>
</ul>
<p>The management infrastructure is expected to automatically reports back STD_OUT and STD_ERR.</p>
<p>Sample:</p>
@@ -250,7 +251,7 @@ h2:hover > .headerlink, h3:hover > .head
<h3 id="sample-configuration-parameters">Sample configuration parameters<a class="headerlink" href="#sample-configuration-parameters" title="Permanent link">¶</a></h3>
-<p>While a script can access all the config parameters the key property bags are <strong>hostLevelParams</strong> and <strong>configurations</strong>.</p>
+<p>While a script can access all the config parameters the key property bags are <strong><code>hostLevelParams</code></strong> and <strong><code>configurations</code></strong>.</p>
<div class="codehilite"><pre>{
"roleCommand": "START",
"commandParams": {
@@ -351,10 +352,10 @@ h2:hover > .headerlink, h3:hover > .head
<p>At this point, you are free to implement as you wish. Slider provides a rich library for basic operations needed for application commands. So the structure we suggest is:</p>
<ul>
-<li>Create a params.py file to read the json config file into variables. This will allow you to provide initial validation, provide default values, or create complex values</li>
+<li>Create a <code>params.py</code> file to read the json config file into variables. This will allow you to provide initial validation, provide default values, or create complex values</li>
<li>Use the above parameters in your scripts</li>
</ul>
-<p>A params.py file may be as follows:</p>
+<p>A <code>params.py</code> file may be as follows:</p>
<div class="codehilite"><pre><span class="n">from</span> <span class="n">resource_management</span> <span class="n">import</span> <span class="o">*</span>
# <span class="n">server</span> <span class="n">configurations</span>