You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by sm...@apache.org on 2014/07/08 02:49:35 UTC
svn commit: r1608633 [4/7] - in /incubator/slider/site: content/design/
content/design/registry/ content/design/specification/ content/developing/
content/docs/ content/docs/configuration/ content/docs/slider_specs/
trunk/content/developing/
Modified: incubator/slider/site/content/developing/releasing.html
URL: http://svn.apache.org/viewvc/incubator/slider/site/content/developing/releasing.html?rev=1608633&r1=1608632&r2=1608633&view=diff
==============================================================================
--- incubator/slider/site/content/developing/releasing.html (original)
+++ incubator/slider/site/content/developing/releasing.html Tue Jul 8 00:49:34 2014
@@ -189,33 +189,39 @@ create HBase and Accumulo clusters in th
starting to make a release</em></p>
<p><strong>Step #1:</strong> Create a JIRA for the release, estimate 3h
(so you don't try to skip the tests)</p>
-<pre class="codehilite"><code>export SLIDER_RELEASE_JIRA=SLIDER-13927</code></pre>
+<div class="codehilite"><pre><span class="n">export</span> <span class="n">SLIDER_RELEASE_JIRA</span><span class="o">=</span><span class="n">SLIDER</span><span class="o">-</span><span class="mi">13927</span>
+</pre></div>
<p><strong>Step #2:</strong> Check everything in. Git flow won't let you progress without this.</p>
<p><strong>Step #3:</strong> Git flow: create a release branch</p>
-<pre class="codehilite"><code>export SLIDER_RELEASE=0.5.2
+<div class="codehilite"><pre><span class="n">export</span> <span class="n">SLIDER_RELEASE</span><span class="o">=</span><span class="mf">0.5.2</span>
-git flow release start slider-$SLIDER_RELEASE</code></pre>
+<span class="n">git</span> <span class="n">flow</span> <span class="n">release</span> <span class="n">start</span> <span class="n">slider</span><span class="o">-</span><span class="err">$</span><span class="n">SLIDER_RELEASE</span>
+</pre></div>
<p><strong>Step #4:</strong> in the new branch, increment those version numbers using (the maven
versions plugin)[http://mojo.codehaus.org/versions-maven-plugin/]</p>
-<pre class="codehilite"><code>mvn versions:set -DnewVersion=$SLIDER_RELEASE</code></pre>
+<div class="codehilite"><pre><span class="n">mvn</span> <span class="n">versions</span><span class="o">:</span><span class="n">set</span> <span class="o">-</span><span class="n">DnewVersion</span><span class="o">=</span><span class="err">$</span><span class="n">SLIDER_RELEASE</span>
+</pre></div>
<p><strong>Step #5:</strong> commit the changed POM files</p>
-<pre class="codehilite"><code>git add <changed files>
-git commit -m "$SLIDER_RELEASE_JIRA updating release POMs for $SLIDER_RELEASE"</code></pre>
+<div class="codehilite"><pre><span class="n">git</span> <span class="n">add</span> <span class="o"><</span><span class="n">changed</span> <span class="n">files</span><span class="o">></span>
+<span class="n">git</span> <span class="n">commit</span> <span class="o">-</span><span class="n">m</span> <span class="s">"$SLIDER_RELEASE_JIRA updating release POMs for $SLIDER_RELEASE"</span>
+</pre></div>
<p><strong>Step #6:</strong> Do a final test run to make sure nothing is broken</p>
<p>In the <code>slider</code> directory, run:</p>
-<pre class="codehilite"><code>mvn clean install -DskipTests</code></pre>
+<div class="codehilite"><pre><span class="n">mvn</span> <span class="n">clean</span> <span class="n">install</span> <span class="o">-</span><span class="n">DskipTests</span>
+</pre></div>
<p>Once everything is built- including .tar files, run the tests</p>
-<pre class="codehilite"><code>mvn test</code></pre>
+<div class="codehilite"><pre><span class="n">mvn</span> <span class="n">test</span>
+</pre></div>
<p>This will run the functional tests as well as the <code>slider-core</code> tests.</p>
@@ -226,7 +232,8 @@ the tests time out.</p>
finalizing the release notes.</p>
<p><strong>Step #7:</strong> Build the release package</p>
<p>Run</p>
-<pre class="codehilite"><code>mvn clean site:site site:stage package -DskipTests</code></pre>
+<div class="codehilite"><pre><span class="n">mvn</span> <span class="n">clean</span> <span class="n">site</span><span class="o">:</span><span class="n">site</span> <span class="n">site</span><span class="o">:</span><span class="n">stage</span> <span class="n">package</span> <span class="o">-</span><span class="n">DskipTests</span>
+</pre></div>
<p><strong>Step #8:</strong> validate the tar file</p>
@@ -237,19 +244,22 @@ look good too: there must be no <code>-S
<p><strong>Step #9:</strong> Build the release notes</p>
<p>Create a a one-line plain text release note for commits and tags
And a multi-line markdown release note, which will be used for artifacts.</p>
-<pre class="codehilite"><code>Release against hadoop 2.4.0, HBase-0.98.1 and Accumulo 1.5.1 artifacts.</code></pre>
+<div class="codehilite"><pre><span class="n">Release</span> <span class="n">against</span> <span class="n">hadoop</span> <span class="mf">2.4.0</span><span class="p">,</span> <span class="n">HBase</span><span class="o">-</span><span class="mf">0.98.1</span> <span class="n">and</span> <span class="n">Accumulo</span> <span class="mf">1.5.1</span> <span class="n">artifacts</span><span class="p">.</span>
+</pre></div>
<p>The multi-line release notes should go into <code>slider/src/site/markdown/release_notes</code>.</p>
<p>These should be committed</p>
-<pre class="codehilite"><code>git add --all
-git commit -m "$SLIDER_RELEASE_JIRA updating release notes"</code></pre>
+<div class="codehilite"><pre><span class="n">git</span> <span class="n">add</span> <span class="o">--</span><span class="n">all</span>
+<span class="n">git</span> <span class="n">commit</span> <span class="o">-</span><span class="n">m</span> <span class="s">"$SLIDER_RELEASE_JIRA updating release notes"</span>
+</pre></div>
<p><strong>Step #10:</strong> End the git flow</p>
<p>Finish the git flow release, either in the SourceTree GUI or
the command line:</p>
-<pre class="codehilite"><code>git flow release finish slider-$SLIDER_RELEASE</code></pre>
+<div class="codehilite"><pre><span class="n">git</span> <span class="n">flow</span> <span class="n">release</span> <span class="n">finish</span> <span class="n">slider</span><span class="o">-</span><span class="err">$</span><span class="n">SLIDER_RELEASE</span>
+</pre></div>
<p>On the command line you have to enter the one-line release description
@@ -258,13 +268,15 @@ prepared earlier.</p>
<p><strong>Step #11:</strong> update mvn versions</p>
<p>Switch back to <code>develop</code> and update its version number past
the release number</p>
-<pre class="codehilite"><code>export SLIDER_RELEASE=0.6.0-SNAPSHOT
-mvn versions:set -DnewVersion=$SLIDER_RELEASE
-git commit -a -m "$SLIDER_RELEASE_JIRA updating development POMs to $SLIDER_RELEASE"</code></pre>
+<div class="codehilite"><pre><span class="n">export</span> <span class="n">SLIDER_RELEASE</span><span class="o">=</span><span class="mf">0.6.0</span><span class="o">-</span><span class="n">SNAPSHOT</span>
+<span class="n">mvn</span> <span class="n">versions</span><span class="o">:</span><span class="n">set</span> <span class="o">-</span><span class="n">DnewVersion</span><span class="o">=</span><span class="err">$</span><span class="n">SLIDER_RELEASE</span>
+<span class="n">git</span> <span class="n">commit</span> <span class="o">-</span><span class="n">a</span> <span class="o">-</span><span class="n">m</span> <span class="s">"$SLIDER_RELEASE_JIRA updating development POMs to $SLIDER_RELEASE"</span>
+</pre></div>
<p><strong>Step #12:</strong> Push the release and develop branches to github </p>
-<pre class="codehilite"><code>git push origin master develop</code></pre>
+<div class="codehilite"><pre><span class="n">git</span> <span class="n">push</span> <span class="n">origin</span> <span class="n">master</span> <span class="n">develop</span>
+</pre></div>
<p>(assuming that <code>origin</code> maps to <code>git@github.com:hortonworks/slider.git</code>;
@@ -290,9 +302,10 @@ tar file's URL.</p>
sprint -so wrap that up too.</p>
<p><strong>Step #17:</strong> Get back to developing!</p>
<p>Check out the develop branch and purge all release artifacts</p>
-<pre class="codehilite"><code>git checkout develop
-git pull origin
-mvn clean</code></pre>
+<div class="codehilite"><pre><span class="n">git</span> <span class="n">checkout</span> <span class="n">develop</span>
+<span class="n">git</span> <span class="n">pull</span> <span class="n">origin</span>
+<span class="n">mvn</span> <span class="n">clean</span>
+</pre></div>
</div>
<div id="footer">
Modified: incubator/slider/site/content/developing/testing.html
URL: http://svn.apache.org/viewvc/incubator/slider/site/content/developing/testing.html?rev=1608633&r1=1608632&r2=1608633&view=diff
==============================================================================
--- incubator/slider/site/content/developing/testing.html (original)
+++ incubator/slider/site/content/developing/testing.html Tue Jul 8 00:49:34 2014
@@ -171,10 +171,11 @@ Latest release: <strong>0.30-incubating<
-->
<h1 id="testing-apache-slider">Testing Apache Slider</h1>
-<pre class="codehilite"><code> The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL
- NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and
- "OPTIONAL" in this document are to be interpreted as described in
- RFC 2119.</code></pre>
+<div class="codehilite"><pre> <span class="n">The</span> <span class="n">key</span> <span class="n">words</span> <span class="s">"MUST"</span><span class="p">,</span> <span class="s">"MUST NOT"</span><span class="p">,</span> <span class="s">"REQUIRED"</span><span class="p">,</span> <span class="s">"SHALL"</span><span class="p">,</span> <span class="s">"SHALL</span>
+ <span class="n">NOT</span><span class="s">", "</span><span class="n">SHOULD</span><span class="s">", "</span><span class="n">SHOULD</span> <span class="n">NOT</span><span class="s">", "</span><span class="n">RECOMMENDED</span><span class="s">", "</span><span class="n">MAY</span><span class="s">", and</span>
+ <span class="s">"OPTIONAL"</span> <span class="n">in</span> <span class="n">this</span> <span class="n">document</span> <span class="n">are</span> <span class="n">to</span> <span class="n">be</span> <span class="n">interpreted</span> <span class="n">as</span> <span class="n">described</span> <span class="n">in</span>
+ <span class="n">RFC</span> <span class="mf">2119.</span>
+</pre></div>
<h2 id="standalone-tests">Standalone Tests</h2>
@@ -194,64 +195,65 @@ AM and spawned processeses run independe
</ul>
<p>All of these need to be defined in the file <code>slider-core/src/test/resources/slider-test.xml</code></p>
<p>Example:</p>
-<pre class="codehilite"><code><configuration>
+<div class="codehilite"><pre><span class="nt"><configuration></span>
- <property>
- <name>slider.test.hbase.enabled</name>
- <description>Flag to enable/disable HBase tests</description>
- <value>true</value>
- </property>
-
- <property>
- <name>slider.test.hbase.home</name>
- <value>/home/slider/hbase-0.98.0</value>
- <description>HBASE Home</description>
- </property>
-
- <property>
- <name>slider.test.hbase.tar</name>
- <value>/home/slider/Projects/hbase-0.98.0-bin.tar.gz</value>
- <description>HBASE archive URI</description>
- </property>
-
- <property>
- <name>slider.test.accumulo.enabled</name>
- <description>Flag to enable/disable Accumulo tests</description>
- <value>true</value>
- </property>
-
- <property>
- <name>slider.test.accumulo.home</name>
- <value>
- /home/slider/accumulo-1.6.0-SNAPSHOT/</value>
- <description>Accumulo Home</description>
- </property>
-
- <property>
- <name>slider.test.accumulo.tar</name>
- <value>/home/slider/accumulo-1.6.0-SNAPSHOT-bin.tar</value>
- <description>Accumulo archive URI</description>
- </property>
-
- <property>
- <name>slider.test.am.restart.time</name>
- <description>Time in millis to await an AM restart</description>
- <value>30000</value>
- </property>
-
- <property>
- <name>zk.home</name>
- <value>/home/slider/zookeeper</value>
- <description>Zookeeper home dir on target systems</description>
- </property>
-
- <property>
- <name>hadoop.home</name>
- <value>/home/slider/hadoop-2.2.0</value>
- <description>Hadoop home dir on target systems</description>
- </property>
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.hbase.enabled<span class="nt"></name></span>
+ <span class="nt"><description></span>Flag to enable/disable HBase tests<span class="nt"></description></span>
+ <span class="nt"><value></span>true<span class="nt"></value></span>
+ <span class="nt"></property></span>
+
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.hbase.home<span class="nt"></name></span>
+ <span class="nt"><value></span>/home/slider/hbase-0.98.0<span class="nt"></value></span>
+ <span class="nt"><description></span>HBASE Home<span class="nt"></description></span>
+ <span class="nt"></property></span>
+
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.hbase.tar<span class="nt"></name></span>
+ <span class="nt"><value></span>/home/slider/Projects/hbase-0.98.0-bin.tar.gz<span class="nt"></value></span>
+ <span class="nt"><description></span>HBASE archive URI<span class="nt"></description></span>
+ <span class="nt"></property></span>
+
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.accumulo.enabled<span class="nt"></name></span>
+ <span class="nt"><description></span>Flag to enable/disable Accumulo tests<span class="nt"></description></span>
+ <span class="nt"><value></span>true<span class="nt"></value></span>
+ <span class="nt"></property></span>
+
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.accumulo.home<span class="nt"></name></span>
+ <span class="nt"><value></span>
+ /home/slider/accumulo-1.6.0-SNAPSHOT/<span class="nt"></value></span>
+ <span class="nt"><description></span>Accumulo Home<span class="nt"></description></span>
+ <span class="nt"></property></span>
+
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.accumulo.tar<span class="nt"></name></span>
+ <span class="nt"><value></span>/home/slider/accumulo-1.6.0-SNAPSHOT-bin.tar<span class="nt"></value></span>
+ <span class="nt"><description></span>Accumulo archive URI<span class="nt"></description></span>
+ <span class="nt"></property></span>
+
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.am.restart.time<span class="nt"></name></span>
+ <span class="nt"><description></span>Time in millis to await an AM restart<span class="nt"></description></span>
+ <span class="nt"><value></span>30000<span class="nt"></value></span>
+ <span class="nt"></property></span>
+
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>zk.home<span class="nt"></name></span>
+ <span class="nt"><value></span>/home/slider/zookeeper<span class="nt"></value></span>
+ <span class="nt"><description></span>Zookeeper home dir on target systems<span class="nt"></description></span>
+ <span class="nt"></property></span>
+
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>hadoop.home<span class="nt"></name></span>
+ <span class="nt"><value></span>/home/slider/hadoop-2.2.0<span class="nt"></value></span>
+ <span class="nt"><description></span>Hadoop home dir on target systems<span class="nt"></description></span>
+ <span class="nt"></property></span>
-</configuration></code></pre>
+<span class="nt"></configuration></span>
+</pre></div>
<p><em>Important:</em> For the local tests, a simple local filesystem path is used for
@@ -271,50 +273,55 @@ away</li>
<h3 id="agent-tests">Agent Tests</h3>
<h3 id="hbase-tests">HBase Tests</h3>
<p>The HBase tests can be enabled or disabled</p>
-<pre class="codehilite"><code><property>
- <name>slider.test.hbase.enabled</name>
- <description>Flag to enable/disable HBase tests</description>
- <value>true</value>
-</property></code></pre>
+<div class="codehilite"><pre><span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.hbase.enabled<span class="nt"></name></span>
+ <span class="nt"><description></span>Flag to enable/disable HBase tests<span class="nt"></description></span>
+ <span class="nt"><value></span>true<span class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
<p>Mandatory test parameters must be added to <code>slider-client.xml</code></p>
-<pre class="codehilite"><code><property>
- <name>slider.test.hbase.tar</name>
- <description>Path to the HBase Tar file in HDFS</description>
- <value>hdfs://sandbox:8020/user/slider/hbase.tar.gz</value>
-</property>
-
-<property>
- <name>slider.test.hbase.appconf</name>
- <description>Path to the directory containing the HBase application config</description>
- <value>file://${user.dir}/src/test/configs/sandbox/hbase</value>
-</property></code></pre>
+<div class="codehilite"><pre><span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.hbase.tar<span class="nt"></name></span>
+ <span class="nt"><description></span>Path to the HBase Tar file in HDFS<span class="nt"></description></span>
+ <span class="nt"><value></span>hdfs://sandbox:8020/user/slider/hbase.tar.gz<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.hbase.appconf<span class="nt"></name></span>
+ <span class="nt"><description></span>Path to the directory containing the HBase application config<span class="nt"></description></span>
+ <span class="nt"><value></span>file://<span class="cp">${</span><span class="n">user</span><span class="o">.</span><span class="n">dir</span><span class="cp">}</span>/src/test/configs/sandbox/hbase<span class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
<p>Optional parameters: </p>
-<pre class="codehilite"><code> <property>
- <name>slider.test.hbase.launch.wait.seconds</name>
- <description>Time to wait in seconds for HBase to start</description>
- <value>180000</value>
-</property></code></pre>
+<div class="codehilite"><pre> <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.hbase.launch.wait.seconds<span class="nt"></name></span>
+ <span class="nt"><description></span>Time to wait in seconds for HBase to start<span class="nt"></description></span>
+ <span class="nt"><value></span>180000<span class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
<h4 id="accumulo-configuration-options">Accumulo configuration options</h4>
<p>Enable/disable the tests</p>
-<pre class="codehilite"><code> <property>
- <name>slider.test.accumulo.enabled</name>
- <description>Flag to enable/disable Accumulo tests</description>
- <value>true</value>
- </property></code></pre>
+<div class="codehilite"><pre> <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.accumulo.enabled<span class="nt"></name></span>
+ <span class="nt"><description></span>Flag to enable/disable Accumulo tests<span class="nt"></description></span>
+ <span class="nt"><value></span>true<span class="nt"></value></span>
+ <span class="nt"></property></span>
+</pre></div>
<p>Optional parameters</p>
-<pre class="codehilite"><code> <property>
- <name>slider.test.accumulo.launch.wait.seconds</name>
- <description>Time to wait in seconds for Accumulo to start</description>
- <value>180000</value>
- </property></code></pre>
+<div class="codehilite"><pre> <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.accumulo.launch.wait.seconds<span class="nt"></name></span>
+ <span class="nt"><description></span>Time to wait in seconds for Accumulo to start<span class="nt"></description></span>
+ <span class="nt"><value></span>180000<span class="nt"></value></span>
+ <span class="nt"></property></span>
+</pre></div>
</div>
<div id="footer">
Modified: incubator/slider/site/content/docs/client-configuration.html
URL: http://svn.apache.org/viewvc/incubator/slider/site/content/docs/client-configuration.html?rev=1608633&r1=1608632&r2=1608633&view=diff
==============================================================================
--- incubator/slider/site/content/docs/client-configuration.html (original)
+++ incubator/slider/site/content/docs/client-configuration.html Tue Jul 8 00:49:34 2014
@@ -203,7 +203,8 @@ if unset the <code>bin/slider</code> scr
current when that version of Slider was released. These values may change
across versions, and may in fact be.</p>
<p>At the time of writing, the default values were:</p>
-<pre class="codehilite"><code>"-Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Xmx256m -Dslider.confdir=${confdir}"</code></pre>
+<div class="codehilite"><pre>"-Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Xmx256m -Dslider.confdir=<span class="cp">${</span><span class="n">confdir</span><span class="cp">}</span>"
+</pre></div>
<p>To allow some Java system properties to be set without editing this
@@ -211,11 +212,13 @@ environment variable, such system proper
line through the <code>-S</code> parameter. For example, the following two operations are
equivalent in terms of setting the system property <code>java.security.krb5.realm</code>
to the value <code>LOCAL</code>.</p>
-<pre class="codehilite"><code>export SLIDER_JVM_OPTS="-Djava.security.krb5.realm=LOCAL"</code></pre>
+<div class="codehilite"><pre><span class="n">export</span> <span class="n">SLIDER_JVM_OPTS</span><span class="o">=</span><span class="s">"-Djava.security.krb5.realm=LOCAL"</span>
+</pre></div>
<p>and</p>
-<pre class="codehilite"><code>slider -S java.security.krb5.realm=LOCAL</code></pre>
+<div class="codehilite"><pre><span class="n">slider</span> <span class="o">-</span><span class="n">S</span> <span class="n">java</span><span class="p">.</span><span class="n">security</span><span class="p">.</span><span class="n">krb5</span><span class="p">.</span><span class="n">realm</span><span class="o">=</span><span class="n">LOCAL</span>
+</pre></div>
<p>Note that the first declaration invalidates all default JVM options; if any of
@@ -234,15 +237,18 @@ are again set as Hadoop configuration pa
<p>All Hadoop and Slider options can be set on the command line using the <code>-D</code>
parameter followed by the appropriate <code>key=value</code> argument</p>
<p>For example, here is a definition of the default Hadoop filesystem:</p>
-<pre class="codehilite"><code>-D fs.defaultFS=hdfs://namenode:9000</code></pre>
+<div class="codehilite"><pre><span class="o">-</span><span class="n">D</span> <span class="n">fs</span><span class="p">.</span><span class="n">defaultFS</span><span class="o">=</span><span class="n">hdfs</span><span class="o">:</span><span class="c1">//namenode:9000</span>
+</pre></div>
<p>Multiple definitions are of course allowed on the command line </p>
-<pre class="codehilite"><code>-D fs.defaultFS=hdfs://namenode:9000 -D dfs.namenode.kerberos.principal=hdfs/namenode@LOCAL</code></pre>
+<div class="codehilite"><pre><span class="o">-</span><span class="n">D</span> <span class="n">fs</span><span class="p">.</span><span class="n">defaultFS</span><span class="o">=</span><span class="n">hdfs</span><span class="o">:</span><span class="c1">//namenode:9000 -D dfs.namenode.kerberos.principal=hdfs/namenode@LOCAL</span>
+</pre></div>
<p>Slider-specific options can be made the same way</p>
-<pre class="codehilite"><code>-D slider.kerberos.principal=</code></pre>
+<div class="codehilite"><pre><span class="o">-</span><span class="n">D</span> <span class="n">slider</span><span class="p">.</span><span class="n">kerberos</span><span class="p">.</span><span class="n">principal</span><span class="o">=</span>
+</pre></div>
<p>If duplicate declarations are made the order of assignment is undefined.</p>
@@ -268,20 +274,21 @@ client and the server.</p>
<p>The <code>slider-client.xml</code> file is a hadoop-formatted XML options file, which
is read by the Slider client -but not by they Slider Application Master.</p>
<p>Here is an example file:</p>
-<pre class="codehilite"><code><property>
- <name>yarn.resourcemanager.address</name>
- <value>namenode:8033</value>
-</property>
-
-<property>
- <name>fs.defaultFS</name>
- <value>hdfs://namenode:9000</value>
-</property>
-
-<property>
- <name>ipc.client.fallback-to-simple-auth-allowed</name>
- <value>false</value>
-</property></code></pre>
+<div class="codehilite"><pre><span class="nt"><property></span>
+ <span class="nt"><name></span>yarn.resourcemanager.address<span class="nt"></name></span>
+ <span class="nt"><value></span>namenode:8033<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>fs.defaultFS<span class="nt"></name></span>
+ <span class="nt"><value></span>hdfs://namenode:9000<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>ipc.client.fallback-to-simple-auth-allowed<span class="nt"></name></span>
+ <span class="nt"><value></span>false<span class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
<p>This defines both the filesystem and the YARN RM, and so obviates the need
@@ -296,46 +303,47 @@ as the source of the <code>log4j.propert
<h2 id="slider-client-configuration-options">Slider Client Configuration options</h2>
<p>As well as standard YARN and Hadoop configuration options, Slider supports
a limited number of slider-specific configuration parameters.</p>
-<pre class="codehilite"><code><property>
- <name>slider.zookeeper.quorum</name>
- <value>localhost:2181,zookeeper2:4545</value>
-</property>
-
-<property>
- <name>slider.yarn.queue</name>
- <value>default</value>
-</property>
-
-<property>
- <name>slider.security.enabled</name>
- <value>false</value>
-</property>
-
-<property>
- <name>slider.yarn.queue</name>
- <value>default</value>
-</property>
-
-<property>
- <name>slider.yarn.queue.priority</name>
- <value>1</value>
-</property>
-
-<property>
- <name>slider.yarn.restart.limit</name>
- <value>5</value>
- <description>How many times to start/restart the Slider AM</description>
-</property>
-
-<property>
- <name>slider.cluster.directory.permissions</name>
- <value>750</value>
-</property>
-
-<property>
- <name>slider.data.directory.permissions</name>
- <value>750</value>
-</property></code></pre>
+<div class="codehilite"><pre><span class="nt"><property></span>
+ <span class="nt"><name></span>slider.zookeeper.quorum<span class="nt"></name></span>
+ <span class="nt"><value></span>localhost:2181,zookeeper2:4545<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>slider.yarn.queue<span class="nt"></name></span>
+ <span class="nt"><value></span>default<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>slider.security.enabled<span class="nt"></name></span>
+ <span class="nt"><value></span>false<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>slider.yarn.queue<span class="nt"></name></span>
+ <span class="nt"><value></span>default<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>slider.yarn.queue.priority<span class="nt"></name></span>
+ <span class="nt"><value></span>1<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>slider.yarn.restart.limit<span class="nt"></name></span>
+ <span class="nt"><value></span>5<span class="nt"></value></span>
+ <span class="nt"><description></span>How many times to start/restart the Slider AM<span class="nt"></description></span>
+<span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>slider.cluster.directory.permissions<span class="nt"></name></span>
+ <span class="nt"><value></span>750<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>slider.data.directory.permissions<span class="nt"></name></span>
+ <span class="nt"><value></span>750<span class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
<h3 id="sliderzookeeperquorum-the-zookeeper-quorum"><code>slider.zookeeper.quorum</code> - the zookeeper quorum.</h3>
@@ -369,25 +377,28 @@ containers created in the Slider cluster
<p>This identifies the priority within the queue. The lower the value, the higher the
priority</p>
<p>Default value: <code>1</code>.</p>
-<pre class="codehilite"><code>bin/slider thaw cl1 -D slider.yarn.queue.priority=5</code></pre>
+<div class="codehilite"><pre><span class="n">bin</span><span class="o">/</span><span class="n">slider</span> <span class="n">thaw</span> <span class="n">cl1</span> <span class="o">-</span><span class="n">D</span> <span class="n">slider</span><span class="p">.</span><span class="n">yarn</span><span class="p">.</span><span class="n">queue</span><span class="p">.</span><span class="n">priority</span><span class="o">=</span><span class="mi">5</span>
+</pre></div>
<h4 id="sliderclusterdirectorypermissions"><code>slider.cluster.directory.permissions</code></h4>
<p>An octal-format (<code>chmod</code>-style) permissions mask for the directory
that contains the cluster specification <code>${user.home}/.slider/clusters/${clustername}</code></p>
-<pre class="codehilite"><code><property>
- <name>slider.cluster.directory.permissions</name>
- <value>750</value>
-</property></code></pre>
+<div class="codehilite"><pre><span class="nt"><property></span>
+ <span class="nt"><name></span>slider.cluster.directory.permissions<span class="nt"></name></span>
+ <span class="nt"><value></span>750<span class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
<h4 id="sliderdatadirectorypermissions"><code>slider.data.directory.permissions</code></h4>
<p>An octal-format (<code>chmod</code>-style) permissions mask for the directory
that contains the application data <code>${user.home}/.slider/clusters/${clustername}/database</code></p>
-<pre class="codehilite"><code><property>
- <name>slider.data.directory.permissions</name>
- <value>750</value>
-</property></code></pre>
+<div class="codehilite"><pre><span class="nt"><property></span>
+ <span class="nt"><name></span>slider.data.directory.permissions<span class="nt"></name></span>
+ <span class="nt"><value></span>750<span class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
<h2 id="debugging-configuration-issues">Debugging configuration issues</h2>
Modified: incubator/slider/site/content/docs/configuration/core.html
URL: http://svn.apache.org/viewvc/incubator/slider/site/content/docs/configuration/core.html?rev=1608633&r1=1608632&r2=1608633&view=diff
==============================================================================
--- incubator/slider/site/content/docs/configuration/core.html (original)
+++ incubator/slider/site/content/docs/configuration/core.html Tue Jul 8 00:49:34 2014
@@ -197,7 +197,8 @@ and what their resource requirements are
<ol>
<li>
<p>A <code>schema</code> string indicating version. Currently this is temporarily set to</p>
-<pre class="codehilite"><code>"http://example.org/specification/v2.0.0"</code></pre>
+<div class="codehilite"><pre><span class="s">"http://example.org/specification/v2.0.0"</span>
+</pre></div>
</li>
@@ -211,14 +212,15 @@ and what their resource requirements are
author, or any other information that is not parsed or processed directly).</li>
</ol>
<p>The simplest valid specification file is </p>
-<pre class="codehilite"><code>{
- "schema": "http://example.org/specification/v2.0.0",
+<div class="codehilite"><pre><span class="p">{</span>
+ <span class="s">"schema"</span><span class="o">:</span> <span class="s">"http://example.org/specification/v2.0.0"</span><span class="p">,</span>
- "global": {
- },
- "components": {
- }
-}</code></pre>
+ <span class="s">"global"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="p">},</span>
+ <span class="s">"components"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
<h2 id="property-inheritance-model-and-resolution">Property inheritance model and <em>resolution</em></h2>
@@ -240,59 +242,64 @@ an application configuration.</li>
</ol>
<h3 id="example">Example</h3>
<p>Here is an example configuration</p>
-<pre class="codehilite"><code>{
- "schema": "http://example.org/specification/v2.0.0",
+<div class="codehilite"><pre><span class="p">{</span>
+ <span class="s">"schema"</span><span class="o">:</span> <span class="s">"http://example.org/specification/v2.0.0"</span><span class="p">,</span>
- "global": {
- "g1": "a",
- "g2": "b"
- },
- "components": {
- "simple": {
- },
- "master": {
- "name": "m",
- "g1": "overridden"
-
- },
- "worker": {
- "name": "w",
- "g1": "overridden-by-worker",
- "timeout": "1000"
- }
- }
-}</code></pre>
+ <span class="s">"global"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"g1"</span><span class="o">:</span> <span class="s">"a"</span><span class="p">,</span>
+ <span class="s">"g2"</span><span class="o">:</span> <span class="s">"b"</span>
+ <span class="p">},</span>
+ <span class="s">"components"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"simple"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="p">},</span>
+ <span class="s">"master"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"name"</span><span class="o">:</span> <span class="s">"m"</span><span class="p">,</span>
+ <span class="s">"g1"</span><span class="o">:</span> <span class="s">"overridden"</span>
+
+ <span class="p">},</span>
+ <span class="s">"worker"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"name"</span><span class="o">:</span> <span class="s">"w"</span><span class="p">,</span>
+ <span class="s">"g1"</span><span class="o">:</span> <span class="s">"overridden-by-worker"</span><span class="p">,</span>
+ <span class="s">"timeout"</span><span class="o">:</span> <span class="s">"1000"</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
<p>The <code>/global</code> section defines two properties</p>
-<pre class="codehilite"><code>g1="a"
-g2="b"</code></pre>
+<div class="codehilite"><pre><span class="n">g1</span><span class="o">=</span><span class="s">"a"</span>
+<span class="n">g2</span><span class="o">=</span><span class="s">"b"</span>
+</pre></div>
<p>These are the values visible to any part of the application which is
not itself one of the components. </p>
<p>There are three components defined, <code>simple</code>, <code>master</code> and <code>worker</code>.</p>
<h4 id="component-simple">component <code>simple</code>:</h4>
-<pre class="codehilite"><code>g1="a"
-g2="b"</code></pre>
+<div class="codehilite"><pre><span class="n">g1</span><span class="o">=</span><span class="s">"a"</span>
+<span class="n">g2</span><span class="o">=</span><span class="s">"b"</span>
+</pre></div>
<p>No settings have been defined specifically for the component; the global
settings are applied.</p>
<h4 id="component-master">component <code>master</code>:</h4>
-<pre class="codehilite"><code>name="m",
-g1="overridden"
-g2="b"</code></pre>
+<div class="codehilite"><pre><span class="n">name</span><span class="o">=</span><span class="s">"m"</span><span class="p">,</span>
+<span class="n">g1</span><span class="o">=</span><span class="s">"overridden"</span>
+<span class="n">g2</span><span class="o">=</span><span class="s">"b"</span>
+</pre></div>
<p>A new attribute, <code>name</code>, has been defined with the value <code>"m"</code>, and the
global property <code>g1</code> has been overridden with the new value, <code>"overridden"</code>.
The global property <code>g2</code> is passed down unchanged.</p>
<h4 id="component-worker">component <code>worker</code>:</h4>
-<pre class="codehilite"><code>name="w",
-g1="overridden-by-worker"
-g2="b"
-timeout: "1000"</code></pre>
+<div class="codehilite"><pre><span class="n">name</span><span class="o">=</span><span class="s">"w"</span><span class="p">,</span>
+<span class="n">g1</span><span class="o">=</span><span class="s">"overridden-by-worker"</span>
+<span class="n">g2</span><span class="o">=</span><span class="s">"b"</span>
+<span class="nl">timeout:</span> <span class="s">"1000"</span>
+</pre></div>
<p>A new attribute, <code>name</code>, has been defined with the value <code>"w"</code>, and another,
@@ -308,31 +315,32 @@ timeout: "1000"</code></pre>
<li>new properties defined in a component are not visible to any other component.</li>
</ul>
<p>The final <em>resolved</em> model is as follows</p>
-<pre class="codehilite"><code>{
- "schema": "http://example.org/specification/v2.0.0",
+<div class="codehilite"><pre><span class="p">{</span>
+ <span class="s">"schema"</span><span class="o">:</span> <span class="s">"http://example.org/specification/v2.0.0"</span><span class="p">,</span>
- "global": {
- "g1": "a",
- "g2": "b"
- },
- "components": {
- "simple": {
- "g1": "a",
- "g2": "b"
- },
- "master": {
- "name": "m",
- "g1": "overridden",
- "g2": "b"
- },
- "worker": {
- "name": "m",
- "g1": "overridden-by-worker",
- "g2": "b",
- "timeout": "1000"
- }
- }
-}</code></pre>
+ <span class="s">"global"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"g1"</span><span class="o">:</span> <span class="s">"a"</span><span class="p">,</span>
+ <span class="s">"g2"</span><span class="o">:</span> <span class="s">"b"</span>
+ <span class="p">},</span>
+ <span class="s">"components"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"simple"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"g1"</span><span class="o">:</span> <span class="s">"a"</span><span class="p">,</span>
+ <span class="s">"g2"</span><span class="o">:</span> <span class="s">"b"</span>
+ <span class="p">},</span>
+ <span class="s">"master"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"name"</span><span class="o">:</span> <span class="s">"m"</span><span class="p">,</span>
+ <span class="s">"g1"</span><span class="o">:</span> <span class="s">"overridden"</span><span class="p">,</span>
+ <span class="s">"g2"</span><span class="o">:</span> <span class="s">"b"</span>
+ <span class="p">},</span>
+ <span class="s">"worker"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"name"</span><span class="o">:</span> <span class="s">"m"</span><span class="p">,</span>
+ <span class="s">"g1"</span><span class="o">:</span> <span class="s">"overridden-by-worker"</span><span class="p">,</span>
+ <span class="s">"g2"</span><span class="o">:</span> <span class="s">"b"</span><span class="p">,</span>
+ <span class="s">"timeout"</span><span class="o">:</span> <span class="s">"1000"</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
<p>This the specification JSON that would have generate exactly the same result as
@@ -351,56 +359,58 @@ of an application.</p>
<p><code>instances</code>: the number of instances of a role desired.
<code>yarn.vcores</code>: number of "virtual" required by a component.
<code>yarn.memory</code>: the number of megabytes required by a component.</p>
-<pre class="codehilite"><code>{
- "schema": "http://example.org/specification/v2.0.0",
+<div class="codehilite"><pre><span class="p">{</span>
+ <span class="s">"schema"</span><span class="o">:</span> <span class="s">"http://example.org/specification/v2.0.0"</span><span class="p">,</span>
- "metadata": {
- "description": "example of a resources file"
- },
-
- "global": {
- "yarn.vcores": "1",
- "yarn.memory": "512"
- },
-
- "components": {
- "master": {
- "instances": "1",
- "yarn.memory": "1024"
- },
- "worker": {
- "instances":"5"
- }
- }
-}</code></pre>
+ <span class="s">"metadata"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"description"</span><span class="o">:</span> <span class="s">"example of a resources file"</span>
+ <span class="p">},</span>
+
+ <span class="s">"global"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"yarn.vcores"</span><span class="o">:</span> <span class="s">"1"</span><span class="p">,</span>
+ <span class="s">"yarn.memory"</span><span class="o">:</span> <span class="s">"512"</span>
+ <span class="p">},</span>
+
+ <span class="s">"components"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"master"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"instances"</span><span class="o">:</span> <span class="s">"1"</span><span class="p">,</span>
+ <span class="s">"yarn.memory"</span><span class="o">:</span> <span class="s">"1024"</span>
+ <span class="p">},</span>
+ <span class="s">"worker"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"instances"</span><span class="o">:</span><span class="s">"5"</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
<p>The resolved file would be</p>
-<pre class="codehilite"><code>{
- "schema": "http://example.org/specification/v2.0.0",
+<div class="codehilite"><pre><span class="p">{</span>
+ <span class="s">"schema"</span><span class="o">:</span> <span class="s">"http://example.org/specification/v2.0.0"</span><span class="p">,</span>
- "metadata": {
- "description": "example of a resources file"
- },
-
- "global": {
- "yarn.vcores": "1",
- "yarn.memory": "512"
- },
-
- "components": {
- "master": {
- "instances": "1",
- "yarn.vcores": "1",
- "yarn.memory": "1024"
- },
- "worker": {
- "instances":"5",
- "yarn.vcores": "1",
- "yarn.memory": "512"
- }
- }
-}</code></pre>
+ <span class="s">"metadata"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"description"</span><span class="o">:</span> <span class="s">"example of a resources file"</span>
+ <span class="p">},</span>
+
+ <span class="s">"global"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"yarn.vcores"</span><span class="o">:</span> <span class="s">"1"</span><span class="p">,</span>
+ <span class="s">"yarn.memory"</span><span class="o">:</span> <span class="s">"512"</span>
+ <span class="p">},</span>
+
+ <span class="s">"components"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"master"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"instances"</span><span class="o">:</span> <span class="s">"1"</span><span class="p">,</span>
+ <span class="s">"yarn.vcores"</span><span class="o">:</span> <span class="s">"1"</span><span class="p">,</span>
+ <span class="s">"yarn.memory"</span><span class="o">:</span> <span class="s">"1024"</span>
+ <span class="p">},</span>
+ <span class="s">"worker"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"instances"</span><span class="o">:</span><span class="s">"5"</span><span class="p">,</span>
+ <span class="s">"yarn.vcores"</span><span class="o">:</span> <span class="s">"1"</span><span class="p">,</span>
+ <span class="s">"yarn.memory"</span><span class="o">:</span> <span class="s">"512"</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
<p>This declares this deployment of the application to consist of one instance of
@@ -413,102 +423,105 @@ intended for manual editing.</p>
diagnostic information for support calls, and MUST NOT be interpreted
during application deployment, (though it may be included in the generation
of diagnostics reports)</p>
-<pre class="codehilite"><code>{
- "schema": "http://example.org/specification/v2.0.0",
+<div class="codehilite"><pre><span class="p">{</span>
+ <span class="s">"schema"</span><span class="o">:</span> <span class="s">"http://example.org/specification/v2.0.0"</span><span class="p">,</span>
- "metadata": {
- "description": "Internal configuration DO NOT EDIT"
- },
- "global": {
- "name": "small_cluster",
- "application": "hdfs://cluster:8020/apps/hbase/v/1.0.0/application.tar"
- },
- "components": {
-
- "diagnostics": {
- "create.hadoop.deployed.info": "(release-2.3.0) @dfe463",
- "create.hadoop.build.info": "2.3.0",
- "create.time.millis": "1393512091276",
- "create.time": "27 Feb 2014 14:41:31 GMT"
- }
- }
-}</code></pre>
+ <span class="s">"metadata"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"description"</span><span class="o">:</span> <span class="s">"Internal configuration DO NOT EDIT"</span>
+ <span class="p">},</span>
+ <span class="s">"global"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"name"</span><span class="o">:</span> <span class="s">"small_cluster"</span><span class="p">,</span>
+ <span class="s">"application"</span><span class="o">:</span> <span class="s">"hdfs://cluster:8020/apps/hbase/v/1.0.0/application.tar"</span>
+ <span class="p">},</span>
+ <span class="s">"components"</span><span class="o">:</span> <span class="p">{</span>
+
+ <span class="s">"diagnostics"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"create.hadoop.deployed.info"</span><span class="o">:</span> <span class="s">"(release-2.3.0) @dfe463"</span><span class="p">,</span>
+ <span class="s">"create.hadoop.build.info"</span><span class="o">:</span> <span class="s">"2.3.0"</span><span class="p">,</span>
+ <span class="s">"create.time.millis"</span><span class="o">:</span> <span class="s">"1393512091276"</span><span class="p">,</span>
+ <span class="s">"create.time"</span><span class="o">:</span> <span class="s">"27 Feb 2014 14:41:31 GMT"</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
<h2 id="deployment-specification-app_configurationjson">Deployment specification: <code>app_configuration.json</code></h2>
<p>This defines parameters that are to be used when creating the instance of the
application, and instances of the individual components.</p>
-<pre class="codehilite"><code>{
- "schema": "http://example.org/specification/v2.0.0",
+<div class="codehilite"><pre><span class="p">{</span>
+ <span class="s">"schema"</span><span class="o">:</span> <span class="s">"http://example.org/specification/v2.0.0"</span><span class="p">,</span>
- "global": {
+ <span class="s">"global"</span><span class="o">:</span> <span class="p">{</span>
- "zookeeper.port": "2181",
- "zookeeper.path": "/yarnapps_small_cluster",
- "zookeeper.hosts": "zoo1,zoo2,zoo3",
- "env.MALLOC_ARENA_MAX": "4",
- "site.hbase.master.startup.retainassign": "true",
- "site.fs.defaultFS": "hdfs://cluster:8020",
- "site.fs.default.name": "hdfs://cluster:8020",
- "site.hbase.master.info.port": "0",
- "site.hbase.regionserver.info.port": "0"
- },
- "components": {
-
- "worker": {
- "jvm.heapsize": "512M"
- },
- "master": {
- "jvm.heapsize": "512M"
- }
- }
-}</code></pre>
+ <span class="s">"zookeeper.port"</span><span class="o">:</span> <span class="s">"2181"</span><span class="p">,</span>
+ <span class="s">"zookeeper.path"</span><span class="o">:</span> <span class="s">"/yarnapps_small_cluster"</span><span class="p">,</span>
+ <span class="s">"zookeeper.hosts"</span><span class="o">:</span> <span class="s">"zoo1,zoo2,zoo3"</span><span class="p">,</span>
+ <span class="s">"env.MALLOC_ARENA_MAX"</span><span class="o">:</span> <span class="s">"4"</span><span class="p">,</span>
+ <span class="s">"site.hbase.master.startup.retainassign"</span><span class="o">:</span> <span class="s">"true"</span><span class="p">,</span>
+ <span class="s">"site.fs.defaultFS"</span><span class="o">:</span> <span class="s">"hdfs://cluster:8020"</span><span class="p">,</span>
+ <span class="s">"site.fs.default.name"</span><span class="o">:</span> <span class="s">"hdfs://cluster:8020"</span><span class="p">,</span>
+ <span class="s">"site.hbase.master.info.port"</span><span class="o">:</span> <span class="s">"0"</span><span class="p">,</span>
+ <span class="s">"site.hbase.regionserver.info.port"</span><span class="o">:</span> <span class="s">"0"</span>
+ <span class="p">},</span>
+ <span class="s">"components"</span><span class="o">:</span> <span class="p">{</span>
+
+ <span class="s">"worker"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"jvm.heapsize"</span><span class="o">:</span> <span class="s">"512M"</span>
+ <span class="p">},</span>
+ <span class="s">"master"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"jvm.heapsize"</span><span class="o">:</span> <span class="s">"512M"</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
<p>The resolved specification defines the values that are passed to the
different components.</p>
-<pre class="codehilite"><code>{
- "schema": "http://example.org/specification/v2.0.0",
+<div class="codehilite"><pre><span class="p">{</span>
+ <span class="s">"schema"</span><span class="o">:</span> <span class="s">"http://example.org/specification/v2.0.0"</span><span class="p">,</span>
- "global": {
- "zookeeper.port": "2181",
- "zookeeper.path": "/yarnapps_small_cluster",
- "zookeeper.hosts": "zoo1,zoo2,zoo3",
- "env.MALLOC_ARENA_MAX": "4",
- "site.hbase.master.startup.retainassign": "true",
- "site.fs.defaultFS": "hdfs://cluster:8020",
- "site.fs.default.name": "hdfs://cluster:8020",
- "site.hbase.master.info.port": "0",
- "site.hbase.regionserver.info.port": "0"
- },
- "components": {
-
- "worker": {
- "zookeeper.port": "2181",
- "zookeeper.path": "/yarnapps_small_cluster",
- "zookeeper.hosts": "zoo1,zoo2,zoo3",
- "env.MALLOC_ARENA_MAX": "4",
- "site.hbase.master.startup.retainassign": "true",
- "site.fs.defaultFS": "hdfs://cluster:8020",
- "site.fs.default.name": "hdfs://cluster:8020",
- "site.hbase.master.info.port": "0",
- "site.hbase.regionserver.info.port": "0",
- "jvm.heapsize": "512M"
- },
- "master": {
- "zookeeper.port": "2181",
- "zookeeper.path": "/yarnapps_small_cluster",
- "zookeeper.hosts": "zoo1,zoo2,zoo3",
- "env.MALLOC_ARENA_MAX": "4",
- "site.hbase.master.startup.retainassign": "true",
- "site.fs.defaultFS": "hdfs://cluster:8020",
- "site.fs.default.name": "hdfs://cluster:8020",
- "site.hbase.master.info.port": "0",
- "site.hbase.regionserver.info.port": "0",
- "jvm.heapsize": "512M"
- }
- }
-}</code></pre>
+ <span class="s">"global"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"zookeeper.port"</span><span class="o">:</span> <span class="s">"2181"</span><span class="p">,</span>
+ <span class="s">"zookeeper.path"</span><span class="o">:</span> <span class="s">"/yarnapps_small_cluster"</span><span class="p">,</span>
+ <span class="s">"zookeeper.hosts"</span><span class="o">:</span> <span class="s">"zoo1,zoo2,zoo3"</span><span class="p">,</span>
+ <span class="s">"env.MALLOC_ARENA_MAX"</span><span class="o">:</span> <span class="s">"4"</span><span class="p">,</span>
+ <span class="s">"site.hbase.master.startup.retainassign"</span><span class="o">:</span> <span class="s">"true"</span><span class="p">,</span>
+ <span class="s">"site.fs.defaultFS"</span><span class="o">:</span> <span class="s">"hdfs://cluster:8020"</span><span class="p">,</span>
+ <span class="s">"site.fs.default.name"</span><span class="o">:</span> <span class="s">"hdfs://cluster:8020"</span><span class="p">,</span>
+ <span class="s">"site.hbase.master.info.port"</span><span class="o">:</span> <span class="s">"0"</span><span class="p">,</span>
+ <span class="s">"site.hbase.regionserver.info.port"</span><span class="o">:</span> <span class="s">"0"</span>
+ <span class="p">},</span>
+ <span class="s">"components"</span><span class="o">:</span> <span class="p">{</span>
+
+ <span class="s">"worker"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"zookeeper.port"</span><span class="o">:</span> <span class="s">"2181"</span><span class="p">,</span>
+ <span class="s">"zookeeper.path"</span><span class="o">:</span> <span class="s">"/yarnapps_small_cluster"</span><span class="p">,</span>
+ <span class="s">"zookeeper.hosts"</span><span class="o">:</span> <span class="s">"zoo1,zoo2,zoo3"</span><span class="p">,</span>
+ <span class="s">"env.MALLOC_ARENA_MAX"</span><span class="o">:</span> <span class="s">"4"</span><span class="p">,</span>
+ <span class="s">"site.hbase.master.startup.retainassign"</span><span class="o">:</span> <span class="s">"true"</span><span class="p">,</span>
+ <span class="s">"site.fs.defaultFS"</span><span class="o">:</span> <span class="s">"hdfs://cluster:8020"</span><span class="p">,</span>
+ <span class="s">"site.fs.default.name"</span><span class="o">:</span> <span class="s">"hdfs://cluster:8020"</span><span class="p">,</span>
+ <span class="s">"site.hbase.master.info.port"</span><span class="o">:</span> <span class="s">"0"</span><span class="p">,</span>
+ <span class="s">"site.hbase.regionserver.info.port"</span><span class="o">:</span> <span class="s">"0"</span><span class="p">,</span>
+ <span class="s">"jvm.heapsize"</span><span class="o">:</span> <span class="s">"512M"</span>
+ <span class="p">},</span>
+ <span class="s">"master"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"zookeeper.port"</span><span class="o">:</span> <span class="s">"2181"</span><span class="p">,</span>
+ <span class="s">"zookeeper.path"</span><span class="o">:</span> <span class="s">"/yarnapps_small_cluster"</span><span class="p">,</span>
+ <span class="s">"zookeeper.hosts"</span><span class="o">:</span> <span class="s">"zoo1,zoo2,zoo3"</span><span class="p">,</span>
+ <span class="s">"env.MALLOC_ARENA_MAX"</span><span class="o">:</span> <span class="s">"4"</span><span class="p">,</span>
+ <span class="s">"site.hbase.master.startup.retainassign"</span><span class="o">:</span> <span class="s">"true"</span><span class="p">,</span>
+ <span class="s">"site.fs.defaultFS"</span><span class="o">:</span> <span class="s">"hdfs://cluster:8020"</span><span class="p">,</span>
+ <span class="s">"site.fs.default.name"</span><span class="o">:</span> <span class="s">"hdfs://cluster:8020"</span><span class="p">,</span>
+ <span class="s">"site.hbase.master.info.port"</span><span class="o">:</span> <span class="s">"0"</span><span class="p">,</span>
+ <span class="s">"site.hbase.regionserver.info.port"</span><span class="o">:</span> <span class="s">"0"</span><span class="p">,</span>
+ <span class="s">"jvm.heapsize"</span><span class="o">:</span> <span class="s">"512M"</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
<p>The <code>site.</code> properties have been passed down to each component, components