You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by sm...@apache.org on 2014/07/17 08:53:54 UTC
svn commit: r1611262 [1/2] - in /incubator/slider/site:
content/docs/slider_specs/ trunk/content/docs/slider_specs/
Author: smohanty
Date: Thu Jul 17 06:53:53 2014
New Revision: 1611262
URL: http://svn.apache.org/r1611262
Log:
SLIDER-121 manually generate slider site from CMS content for now
Modified:
incubator/slider/site/content/docs/slider_specs/application_package.html
incubator/slider/site/content/docs/slider_specs/creating_app_definitions.html
incubator/slider/site/content/docs/slider_specs/hello_world_slider_app.html
incubator/slider/site/content/docs/slider_specs/writing_app_command_scripts.html
incubator/slider/site/trunk/content/docs/slider_specs/application_package.md
incubator/slider/site/trunk/content/docs/slider_specs/creating_app_definitions.md
incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md
incubator/slider/site/trunk/content/docs/slider_specs/writing_app_command_scripts.md
Modified: incubator/slider/site/content/docs/slider_specs/application_package.html
URL: http://svn.apache.org/viewvc/incubator/slider/site/content/docs/slider_specs/application_package.html?rev=1611262&r1=1611261&r2=1611262&view=diff
==============================================================================
--- incubator/slider/site/content/docs/slider_specs/application_package.html (original)
+++ incubator/slider/site/content/docs/slider_specs/application_package.html Thu Jul 17 06:53:53 2014
@@ -233,29 +233,28 @@ other scripts, txt files, tarballs, etc.
</li>
</ol>
<p>The script specified in the metainfo is expected to understand the command. It can choose to call other scripts based on how the application author organizes the code base. For example:</p>
-<p>```
-class HbaseMaster(Script):
- def install(self, env):
- self.install_packages(env)</p>
-<p>def configure(self, env):
- import params
- env.set_params(params)</p>
-<div class="codehilite"><pre><span class="n">hbase</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="err">'</span><span class="n">master</span><span class="err">'</span><span class="p">)</span>
+<div class="codehilite"><pre><span class="n">class</span> <span class="n">HbaseMaster</span><span class="p">(</span><span class="n">Script</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">def</span> <span class="n">install</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">self</span><span class="p">.</span><span class="n">install_packages</span><span class="p">(</span><span class="n">env</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">configure</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+
+ <span class="n">hbase</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="err">'</span><span class="n">master</span><span class="err">'</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">start</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+ <span class="n">self</span><span class="p">.</span><span class="n">configure</span><span class="p">(</span><span class="n">env</span><span class="p">)</span> <span class="err">#</span> <span class="k">for</span> <span class="n">security</span>
+
+ <span class="n">hbase_service</span><span class="p">(</span> <span class="err">'</span><span class="n">master</span><span class="err">'</span><span class="p">,</span>
+ <span class="n">action</span> <span class="o">=</span> <span class="err">'</span><span class="n">start</span><span class="err">'</span>
+ <span class="p">)</span>
+<span class="p">...</span>
</pre></div>
-<p>def start(self, env):
- import params
- env.set_params(params)
- self.configure(env) # for security</p>
-<div class="codehilite"><pre><span class="n">hbase_service</span><span class="p">(</span> <span class="err">'</span><span class="n">master</span><span class="err">'</span><span class="p">,</span>
- <span class="n">action</span> <span class="o">=</span> <span class="err">'</span><span class="n">start</span><span class="err">'</span>
-<span class="p">)</span>
-</pre></div>
-
-
-<p>... <br />
-```</p>
<p><a href="/docs/slider_specs/writing_app_command_scripts.html">How to write scripts</a> discusses how the scripts are developed and the structure of the JSON_FILE containing the parameters.</p>
<h3 id="templates-folder">templates folder</h3>
<p>templates are configurable text files that are NOT regular config files. <em>A library has been developed that can materialize a complete site configuration file from a property bag and therefore are not generated from templates.</em> Other files such as env sh files, log4j properties file, etc. may be derived from a template. Again, the implementor can choose to create these files from scratch and not use templates. The following operations are allowed during template expansion:</p>
Modified: incubator/slider/site/content/docs/slider_specs/creating_app_definitions.html
URL: http://svn.apache.org/viewvc/incubator/slider/site/content/docs/slider_specs/creating_app_definitions.html?rev=1611262&r1=1611261&r2=1611262&view=diff
==============================================================================
--- incubator/slider/site/content/docs/slider_specs/creating_app_definitions.html (original)
+++ incubator/slider/site/content/docs/slider_specs/creating_app_definitions.html Thu Jul 17 06:53:53 2014
@@ -223,48 +223,51 @@ Latest release: <strong>0.30-incubating<
<ul>
<li>cd slider/app-packages/hbase</li>
<li>zip -r hbase-v098.zip .</li>
-<li>Looking at the content through unzip -l "$@" hbase-v098.zip</li>
</ul>
-<p><code>Archive: hbase-v098.zip
- Length Date Time Name
---------- ---------- ----- ----
- 3158 06-19-2014 22:08 appConfig.json
- 0 06-19-2014 22:15 configuration/
- 5077 06-19-2014 22:15 configuration/global.xml
- 5248 06-19-2014 22:15 configuration/hbase-log4j.xml
- 2250 06-19-2014 22:15 configuration/hbase-policy.xml
- 14705 06-19-2014 22:15 configuration/hbase-site.xml
- 2142 06-19-2014 22:17 jmx_metrics.json
- 3991 06-19-2014 22:45 metainfo.xml
- 0 06-19-2014 22:33 package/
- 0 06-19-2014 22:31 package/files/
- 83278591 06-19-2014 22:31 package/files/hbase-0.98.3-hadoop2-bin.tar.gz
- 0 06-19-2014 22:29 package/scripts/
- 787 06-19-2014 22:27 package/scripts/__init__.py
- 1378 06-19-2014 22:27 package/scripts/functions.py
- 3738 06-19-2014 22:27 package/scripts/hbase.py
- 1205 06-19-2014 22:27 package/scripts/hbase_client.py
- 1640 06-19-2014 22:27 package/scripts/hbase_master.py
- 1764 06-19-2014 22:27 package/scripts/hbase_regionserver.py
- 1482 06-19-2014 22:27 package/scripts/hbase_service.py
- 5089 06-19-2014 22:27 package/scripts/params.py
- 973 06-19-2014 22:27 package/scripts/status_params.py
- 0 06-19-2014 22:34 package/templates/
- 2799 06-19-2014 22:34 package/templates/hadoop-metrics2-hbase.properties-GANGLIA-MASTER.j2
- 2799 06-19-2014 22:34 package/templates/hadoop-metrics2-hbase.properties-GANGLIA-RS.j2
- 3878 06-19-2014 22:34 package/templates/hbase-env.sh.j2
- 909 06-19-2014 22:34 package/templates/hbase_client_jaas.conf.j2
- 989 06-19-2014 22:34 package/templates/hbase_master_jaas.conf.j2
- 1001 06-19-2014 22:34 package/templates/hbase_regionserver_jaas.conf.j2
- 837 06-19-2014 22:34 package/templates/regionservers.j2
- 3486 06-19-2014 22:39 pom.xml
- 1921 06-19-2014 21:53 README.txt
- 358 06-19-2014 22:36 resources.json
- 0 06-19-2014 22:40 src/
- 0 06-19-2014 23:02 src/assembly/
- 2400 06-19-2014 22:45 src/assembly/hbase.xml
---------- -------
- 83354595 35 files</code></p>
+<p>Looking at the content through unzip -l "$@" hbase-v098.zip</p>
+<div class="codehilite"><pre><span class="n">Archive</span><span class="o">:</span> <span class="n">hbase</span><span class="o">-</span><span class="n">v098</span><span class="o">.</span><span class="na">zip</span>
+ <span class="n">Length</span> <span class="n">Date</span> <span class="n">Time</span> <span class="n">Name</span>
+<span class="o">---------</span> <span class="o">----------</span> <span class="o">-----</span> <span class="o">----</span>
+ <span class="mi">3158</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">08</span> <span class="n">appConfig</span><span class="o">.</span><span class="na">json</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">15</span> <span class="n">configuration</span><span class="o">/</span>
+ <span class="mi">5077</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">15</span> <span class="n">configuration</span><span class="o">/</span><span class="n">global</span><span class="o">.</span><span class="na">xml</span>
+ <span class="mi">5248</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">15</span> <span class="n">configuration</span><span class="o">/</span><span class="n">hbase</span><span class="o">-</span><span class="n">log4j</span><span class="o">.</span><span class="na">xml</span>
+ <span class="mi">2250</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">15</span> <span class="n">configuration</span><span class="o">/</span><span class="n">hbase</span><span class="o">-</span><span class="n">policy</span><span class="o">.</span><span class="na">xml</span>
+<span class="mi">14705</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">15</span> <span class="n">configuration</span><span class="o">/</span><span class="n">hbase</span><span class="o">-</span><span class="n">site</span><span class="o">.</span><span class="na">xml</span>
+ <span class="mi">2142</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">17</span> <span class="n">jmx_metrics</span><span class="o">.</span><span class="na">json</span>
+ <span class="mi">3991</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">45</span> <span class="n">metainfo</span><span class="o">.</span><span class="na">xml</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">33</span> <span class="kd">package</span><span class="o">/</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">31</span> <span class="kd">package</span><span class="sr">/files/</span>
+ <span class="mi">83278591</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">31</span> <span class="kd">package</span><span class="sr">/files/</span><span class="n">hbase</span><span class="o">-</span><span class="mf">0.98</span><span class="o">.</span><span class="mi">3</span><span class="o">-</span><span class="n">hadoop2</span><span class="o">-</span><span class="n">bin</span><span class="o">.</span><span class="na">tar</span><span class="o">.</span><span class="na">gz</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">29</span> <span class="kd">package</span><span class="sr">/scripts/</span>
+ <span class="mi">787</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">__init__</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">1378</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">functions</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">3738</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">hbase</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">1205</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">hbase_client</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">1640</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">hbase_master</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">1764</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">hbase_regionserver</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">1482</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">hbase_service</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">5089</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">params</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">973</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/s</span><span class="n">tatus_params</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span>
+ <span class="mi">2799</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">hadoop</span><span class="o">-</span><span class="n">metrics2</span><span class="o">-</span><span class="n">hbase</span><span class="o">.</span><span class="na">properties</span><span class="o">-</span><span class="n">GANGLIA</span><span class="o">-</span><span class="n">MASTER</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">2799</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">hadoop</span><span class="o">-</span><span class="n">metrics2</span><span class="o">-</span><span class="n">hbase</span><span class="o">.</span><span class="na">properties</span><span class="o">-</span><span class="n">GANGLIA</span><span class="o">-</span><span class="n">RS</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">3878</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">hbase</span><span class="o">-</span><span class="n">env</span><span class="o">.</span><span class="na">sh</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">909</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">hbase_client_jaas</span><span class="o">.</span><span class="na">conf</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">989</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">hbase_master_jaas</span><span class="o">.</span><span class="na">conf</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">1001</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">hbase_regionserver_jaas</span><span class="o">.</span><span class="na">conf</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">837</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">regionservers</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">3486</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">39</span> <span class="n">pom</span><span class="o">.</span><span class="na">xml</span>
+ <span class="mi">1921</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">21</span><span class="o">:</span><span class="mi">53</span> <span class="n">README</span><span class="o">.</span><span class="na">txt</span>
+ <span class="mi">358</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">36</span> <span class="n">resources</span><span class="o">.</span><span class="na">json</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">40</span> <span class="n">src</span><span class="o">/</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">23</span><span class="o">:</span><span class="mi">02</span> <span class="n">src</span><span class="sr">/assembly/</span>
+ <span class="mi">2400</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">45</span> <span class="n">src</span><span class="sr">/assembly/</span><span class="n">hbase</span><span class="o">.</span><span class="na">xml</span>
+<span class="o">---------</span> <span class="o">-------</span>
+ <span class="mi">83354595</span> <span class="mi">35</span> <span class="n">files</span>
+</pre></div>
+
+
<p>Sample <strong>resources.json</strong> and <strong>appConfig.json</strong> files are also included in the enlistment. These are samples and are typically tested on one node test installations.</p>
<h3 id="-template-appconfigjson">--template appConfig.json</h3>
<p>An appConfig.json contains the application configuration. See <a href="application_instance_configuration.html">Specifications InstanceConfiguration</a> for details on how to create a template config file. The enlistment includes sample config files for HBase, Accumulo, and Storm.</p>
Modified: incubator/slider/site/content/docs/slider_specs/hello_world_slider_app.html
URL: http://svn.apache.org/viewvc/incubator/slider/site/content/docs/slider_specs/hello_world_slider_app.html?rev=1611262&r1=1611261&r2=1611262&view=diff
==============================================================================
--- incubator/slider/site/content/docs/slider_specs/hello_world_slider_app.html (original)
+++ incubator/slider/site/content/docs/slider_specs/hello_world_slider_app.html Thu Jul 17 06:53:53 2014
@@ -178,81 +178,90 @@ Latest release: <strong>0.30-incubating<
<p>The basic version of the app will allow creation of one or more memcached daemons on default ports. Some memory settings may be configured. Later examples will enhance the application package to add more capabilities.</p>
<p>The structure of an app package is discussed <a href="/docs/slider_specs/application_package.html">here</a>.</p>
<p>In this example, the application package created looks as follows:</p>
-<p><code>unzip -l "$@" ~/enlistments/ambari-vagrant/centos6.4/memcached/jmemcached-1.0.0.zip
-Archive: /jmemcached-1.0.0.zip
- Length Date Time Name
- -------- ---- ---- ----
- 637 07-15-14 19:17 appConfig.json
- 1673 07-15-14 17:58 metainfo.xml
- 0 07-15-14 17:54 package/
- 0 07-15-14 18:03 package/files/
- 122880 07-15-14 18:03 package/files/jmemcached-1.0.0.tar
- 0 07-15-14 19:31 package/scripts/
- 1530 07-15-14 19:31 package/scripts/memcached.py
- 1287 07-15-14 18:46 package/scripts/params.py
- 1581 07-15-14 19:16 README.txt
- 252 07-15-14 17:58 resources.json</code></p>
+<div class="codehilite"><pre><span class="n">unzip</span> <span class="o">-</span><span class="n">l</span> <span class="s">"$@"</span> <span class="o">~/</span><span class="n">enlistments</span><span class="o">/</span><span class="n">ambari</span><span class="o">-</span><span class="n">vagrant</span><span class="o">/</span><span class="n">centos6</span><span class="mf">.4</span><span class="o">/</span><span class="n">memcached</span><span class="o">/</span><span class="n">jmemcached</span><span class="o">-</span><span class="mf">1.0.0</span><span class="p">.</span><span class="n">zip</span>
+<span class="nl">Archive:</span> <span class="o">/</span><span class="n">jmemcached</span><span class="o">-</span><span class="mf">1.0.0</span><span class="p">.</span><span class="n">zip</span>
+ <span class="n">Length</span> <span class="n">Date</span> <span class="n">Time</span> <span class="n">Name</span>
+ <span class="o">--------</span> <span class="o">----</span> <span class="o">----</span> <span class="o">----</span>
+ <span class="mi">637</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">19</span><span class="o">:</span><span class="mi">17</span> <span class="n">appConfig</span><span class="p">.</span><span class="n">json</span>
+ <span class="mi">1673</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">17</span><span class="o">:</span><span class="mi">58</span> <span class="n">metainfo</span><span class="p">.</span><span class="n">xml</span>
+ <span class="mi">0</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">17</span><span class="o">:</span><span class="mi">54</span> <span class="n">package</span><span class="o">/</span>
+ <span class="mi">0</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">18</span><span class="o">:</span><span class="mo">03</span> <span class="n">package</span><span class="o">/</span><span class="n">files</span><span class="o">/</span>
+ <span class="mi">122880</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">18</span><span class="o">:</span><span class="mo">03</span> <span class="n">package</span><span class="o">/</span><span class="n">files</span><span class="o">/</span><span class="n">jmemcached</span><span class="o">-</span><span class="mf">1.0.0</span><span class="p">.</span><span class="n">tar</span>
+ <span class="mi">0</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">19</span><span class="o">:</span><span class="mi">31</span> <span class="n">package</span><span class="o">/</span><span class="n">scripts</span><span class="o">/</span>
+ <span class="mi">1530</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">19</span><span class="o">:</span><span class="mi">31</span> <span class="n">package</span><span class="o">/</span><span class="n">scripts</span><span class="o">/</span><span class="n">memcached</span><span class="p">.</span><span class="n">py</span>
+ <span class="mi">1287</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">18</span><span class="o">:</span><span class="mi">46</span> <span class="n">package</span><span class="o">/</span><span class="n">scripts</span><span class="o">/</span><span class="n">params</span><span class="p">.</span><span class="n">py</span>
+ <span class="mi">1581</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">19</span><span class="o">:</span><span class="mi">16</span> <span class="n">README</span><span class="p">.</span><span class="n">txt</span>
+ <span class="mi">252</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">17</span><span class="o">:</span><span class="mi">58</span> <span class="n">resources</span><span class="p">.</span><span class="n">json</span>
+</pre></div>
+
+
<h2 id="step-1-create-metainfoxml">Step 1: Create metainfo.xml</h2>
<p>The minimal metainfo contains some information about the application (name, comment, version), at least one component type, in this case its MEMCACHED, and information about the tarball. More details are available <a href="/docs/slider_specs/application_definition.html">here</a>.</p>
-<p>```
-<metainfo>
- <schemaVersion>2.0</schemaVersion>
- <application>
- <name>MEMCACHED</name>
- <comment>Memcache is a network accessible key/value storage system, often used as a distributed cache.</comment>
- <version>1.0.0</version></p>
-<div class="codehilite"><pre><span class="nt"><components></span>
- <span class="nt"><component></span>
+<div class="codehilite"><pre><span class="nt"><metainfo></span>
+ <span class="nt"><schemaVersion></span>2.0<span class="nt"></schemaVersion></span>
+ <span class="nt"><application></span>
<span class="nt"><name></span>MEMCACHED<span class="nt"></name></span>
- <span class="nt"><category></span>MASTER<span class="nt"></category></span>
- <span class="nt"><commandScript></span>
- <span class="nt"><script></span>scripts/memcached.py<span class="nt"></script></span>
- <span class="nt"><scriptType></span>PYTHON<span class="nt"></scriptType></span>
- <span class="nt"></commandScript></span>
- <span class="nt"></component></span>
-<span class="nt"></components></span>
-
-<span class="nt"><osSpecifics></span>
- <span class="nt"><osSpecific></span>
- <span class="nt"><osType></span>any<span class="nt"></osType></span>
- <span class="nt"><packages></span>
- <span class="nt"><package></span>
- <span class="nt"><type></span>tarball<span class="nt"></type></span>
- <span class="nt"><name></span>files/jmemcached-1.0.0.tar<span class="nt"></name></span>
- <span class="nt"></package></span>
- <span class="nt"></packages></span>
- <span class="nt"></osSpecific></span>
-<span class="nt"></osSpecifics></span>
+ <span class="nt"><comment></span>Memcache is a network accessible key/value storage system, often used as a distributed cache.<span class="nt"></comment></span>
+ <span class="nt"><version></span>1.0.0<span class="nt"></version></span>
+
+ <span class="nt"><components></span>
+ <span class="nt"><component></span>
+ <span class="nt"><name></span>MEMCACHED<span class="nt"></name></span>
+ <span class="nt"><category></span>MASTER<span class="nt"></category></span>
+ <span class="nt"><commandScript></span>
+ <span class="nt"><script></span>scripts/memcached.py<span class="nt"></script></span>
+ <span class="nt"><scriptType></span>PYTHON<span class="nt"></scriptType></span>
+ <span class="nt"></commandScript></span>
+ <span class="nt"></component></span>
+ <span class="nt"></components></span>
+
+ <span class="nt"><osSpecifics></span>
+ <span class="nt"><osSpecific></span>
+ <span class="nt"><osType></span>any<span class="nt"></osType></span>
+ <span class="nt"><packages></span>
+ <span class="nt"><package></span>
+ <span class="nt"><type></span>tarball<span class="nt"></type></span>
+ <span class="nt"><name></span>files/jmemcached-1.0.0.tar<span class="nt"></name></span>
+ <span class="nt"></package></span>
+ <span class="nt"></packages></span>
+ <span class="nt"></osSpecific></span>
+ <span class="nt"></osSpecifics></span>
+
+ <span class="nt"></application></span>
+<span class="nt"></metainfo></span>
</pre></div>
-<p></application>
-</metainfo>
-``` </p>
<h2 id="step-2-ensure-application-tarball">Step 2: Ensure application tarball</h2>
<p>Most applications release a tarball that you can download otherwise you can create one. For this sample, we created a simple tarball that contains the cli and core jar from jmemcached.</p>
-<p><code>tar tvf jmemcached-1.0.0.tar
-drwxr-xr-x 0 yarn hadoop 0 Jul 15 17:51 jmemcached-1.0.0/
--rw-r--r-- 0 yarn hadoop 13537 Jul 15 17:51 jmemcached-1.0.0/jmemcached-cli-1.0.0.jar
--rwxr-xr-x 0 yarn hadoop 101467 Jul 15 17:51 jmemcached-1.0.0/jmemcached-core-1.0.0.jar</code></p>
+<div class="codehilite"><pre><span class="n">tar</span> <span class="n">tvf</span> <span class="n">jmemcached</span><span class="o">-</span><span class="mf">1.0.0</span><span class="p">.</span><span class="n">tar</span>
+<span class="n">drwxr</span><span class="o">-</span><span class="n">xr</span><span class="o">-</span><span class="n">x</span> <span class="mi">0</span> <span class="n">yarn</span> <span class="n">hadoop</span> <span class="mi">0</span> <span class="n">Jul</span> <span class="mi">15</span> <span class="mi">17</span><span class="o">:</span><span class="mi">51</span> <span class="n">jmemcached</span><span class="o">-</span><span class="mf">1.0.0</span><span class="o">/</span>
+<span class="o">-</span><span class="n">rw</span><span class="o">-</span><span class="n">r</span><span class="o">--</span><span class="n">r</span><span class="o">--</span> <span class="mi">0</span> <span class="n">yarn</span> <span class="n">hadoop</span> <span class="mi">13537</span> <span class="n">Jul</span> <span class="mi">15</span> <span class="mi">17</span><span class="o">:</span><span class="mi">51</span> <span class="n">jmemcached</span><span class="o">-</span><span class="mf">1.0.0</span><span class="o">/</span><span class="n">jmemcached</span><span class="o">-</span><span class="n">cli</span><span class="o">-</span><span class="mf">1.0.0</span><span class="p">.</span><span class="n">jar</span>
+<span class="o">-</span><span class="n">rwxr</span><span class="o">-</span><span class="n">xr</span><span class="o">-</span><span class="n">x</span> <span class="mi">0</span> <span class="n">yarn</span> <span class="n">hadoop</span> <span class="mi">101467</span> <span class="n">Jul</span> <span class="mi">15</span> <span class="mi">17</span><span class="o">:</span><span class="mi">51</span> <span class="n">jmemcached</span><span class="o">-</span><span class="mf">1.0.0</span><span class="o">/</span><span class="n">jmemcached</span><span class="o">-</span><span class="n">core</span><span class="o">-</span><span class="mf">1.0.0</span><span class="p">.</span><span class="n">jar</span>
+</pre></div>
+
+
<h2 id="step-3-create-a-default-resources-file-resourcesjson">Step 3: Create a default resources file (resources.json)</h2>
<p>By default all resources.json file must include slider-appmaster. Add one more entry for the component MEMCACHED and assign a unique priority and default number of instances. Ensure, that a suitable default value is provided for yarn.memory. Additional details are available <a href="/docs/slider_specs/resource_specification.html">here</a>.</p>
-<p><code>{
- "schema" : "http://example.org/specification/v2.0.0",
- "metadata" : {
- },
- "global" : {
- },
- "components": {
- "slider-appmaster": {
- },
- "MEMCACHED": {
- "yarn.role.priority": "1",
- "yarn.component.instances": "1",
- "yarn.memory": "256"
- }
- }
-}</code></p>
+<div class="codehilite"><pre><span class="p">{</span>
+ <span class="s">"schema"</span> <span class="o">:</span> <span class="s">"http://example.org/specification/v2.0.0"</span><span class="p">,</span>
+ <span class="s">"metadata"</span> <span class="o">:</span> <span class="p">{</span>
+ <span class="p">},</span>
+ <span class="s">"global"</span> <span class="o">:</span> <span class="p">{</span>
+ <span class="p">},</span>
+ <span class="s">"components"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"slider-appmaster"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="p">},</span>
+ <span class="s">"MEMCACHED"</span><span class="o">:</span> <span class="p">{</span>
+ <span class="s">"yarn.role.priority"</span><span class="o">:</span> <span class="s">"1"</span><span class="p">,</span>
+ <span class="s">"yarn.component.instances"</span><span class="o">:</span> <span class="s">"1"</span><span class="p">,</span>
+ <span class="s">"yarn.memory"</span><span class="o">:</span> <span class="s">"256"</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
+
+
<h2 id="step-4-create-a-default-configuration-template-appconfigjson">Step 4: Create a default configuration template (appConfig.json)</h2>
<p>The config template has few mandatory parameters such as </p>
<ul>
@@ -269,76 +278,87 @@ drwxr-xr-x 0 yarn hadoop 0 Jul 1
<li>site.global.xms_val - value of Xms</li>
<li>site.global.memory_val - value of memory</li>
</ul>
-<p><code>{
- "schema": "http://example.org/specification/v2.0.0",
- "metadata": {
+<p>You can add additional parameters as needed.</p>
+<div class="codehilite"><pre>{
+ "schema": "http://example.org/specification/v2.0.0",
+ "metadata": {
},
- "global": {
- "application.def": "/slider/jmemcached-1.0.0.zip",
- "java_home": "/usr/jdk64/jdk1.7.0_45",
- "package_list": "files/jmemcached-1.0.0.tar",
- "config_types": "",
- "site.global.app_user": "yarn",
- "site.global.app_root": "${AGENT_WORK_ROOT}/app/install/jmemcached-1.0.0",
- "site.global.additional_cp": "/usr/lib/hadoop/lib/*",
- "site.global.xmx_val": "256m",
- "site.global.xms_val": "128m",
- "site.global.memory_val": "200M"
+ "global": {
+ "application.def": "/slider/jmemcached-1.0.0.zip",
+ "java_home": "/usr/jdk64/jdk1.7.0_45",
+ "package_list": "files/jmemcached-1.0.0.tar",
+ "config_types": "",
+ "site.global.app_user": "yarn",
+ "site.global.app_root": "<span class="cp">${</span><span class="n">AGENT_WORK_ROOT</span><span class="cp">}</span>/app/install/jmemcached-1.0.0",
+ "site.global.additional_cp": "/usr/lib/hadoop/lib/*",
+ "site.global.xmx_val": "256m",
+ "site.global.xms_val": "128m",
+ "site.global.memory_val": "200M"
},
- "components": {
- "slider-appmaster": {
- "jvm.heapsize": "256M"
+ "components": {
+ "slider-appmaster": {
+ "jvm.heapsize": "256M"
},
- "MEMCACHED": {
+ "MEMCACHED": {
}
}
-}</code></p>
+}
+</pre></div>
+
+
<p>Additional details on how to define a configuration template is <a href="/docs/slider_specs/application_instance_configuration.html">here</a>.</p>
<h2 id="step-5-implement-the-basic-commands">Step 5: Implement the basic commands</h2>
<p>All Slider applications are expected to implement INSTALL/CONFIGURE/START/STOP/STATUS for each component. Some of the implementations can be NOP - in our case, we will implement only INSTALL and START and leave rest of the code as default. Note, Slider has an extensive library that can be used to implement the commands. More details can be found <a href="/docs/slider_specs/writing_app_command_scripts">here</a>.</p>
<p>The parameters file we will use is:</p>
-<p>```
-from resource_management import *</p>
-<p>config = Script.get_config()</p>
-<p>app_root = config['configurations']['global']['app_root']
-java64_home = config['hostLevelParams']['java_home']
-app_user = config['configurations']['global']['app_user']
-additional_cp = config['configurations']['global']['additional_cp']
-xmx_val = config['configurations']['global']['xmx_val']
-xms_val = config['configurations']['global']['xms_val']
-memory_val = config['configurations']['global']['memory_val']
-```</p>
+<div class="codehilite"><pre><span class="n">from</span> <span class="n">resource_management</span> <span class="n">import</span> <span class="o">*</span>
+
+<span class="n">config</span> <span class="o">=</span> <span class="n">Script</span><span class="p">.</span><span class="n">get_config</span><span class="p">()</span>
+
+<span class="n">app_root</span> <span class="o">=</span> <span class="n">config</span><span class="p">[</span><span class="err">'</span><span class="n">configurations</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">global</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">app_root</span><span class="err">'</span><span class="p">]</span>
+<span class="n">java64_home</span> <span class="o">=</span> <span class="n">config</span><span class="p">[</span><span class="err">'</span><span class="n">hostLevelParams</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">java_home</span><span class="err">'</span><span class="p">]</span>
+<span class="n">app_user</span> <span class="o">=</span> <span class="n">config</span><span class="p">[</span><span class="err">'</span><span class="n">configurations</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">global</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">app_user</span><span class="err">'</span><span class="p">]</span>
+<span class="n">additional_cp</span> <span class="o">=</span> <span class="n">config</span><span class="p">[</span><span class="err">'</span><span class="n">configurations</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">global</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">additional_cp</span><span class="err">'</span><span class="p">]</span>
+<span class="n">xmx_val</span> <span class="o">=</span> <span class="n">config</span><span class="p">[</span><span class="err">'</span><span class="n">configurations</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">global</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">xmx_val</span><span class="err">'</span><span class="p">]</span>
+<span class="n">xms_val</span> <span class="o">=</span> <span class="n">config</span><span class="p">[</span><span class="err">'</span><span class="n">configurations</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">global</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">xms_val</span><span class="err">'</span><span class="p">]</span>
+<span class="n">memory_val</span> <span class="o">=</span> <span class="n">config</span><span class="p">[</span><span class="err">'</span><span class="n">configurations</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">global</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">memory_val</span><span class="err">'</span><span class="p">]</span>
+</pre></div>
+
+
<p>Note that the parameter.py file only reads the parameters needed by the command implementations. </p>
-<p>```
-import sys
-from resource_management import *</p>
-<p>class Memcached(Script):
- def install(self, env):
- self.install_packages(env)</p>
-<p>def configure(self, env):
- import params
- env.set_params(params)</p>
-<p>def start(self, env):
- import params
- env.set_params(params)
- self.configure(env)
- process_cmd = format("{java64_home}/bin/java -Xmx{xmx_val} -Xms{xms_val} -classpath {app_root}/*:{additional_cp} com.thimbleware.jmemcached.Main --memory={memory_val}")</p>
-<div class="codehilite"><pre><span class="n">Execute</span><span class="p">(</span><span class="n">process_cmd</span><span class="p">,</span>
- <span class="n">user</span><span class="o">=</span><span class="n">params</span><span class="p">.</span><span class="n">app_user</span><span class="p">,</span>
- <span class="n">logoutput</span><span class="o">=</span><span class="n">False</span><span class="p">,</span>
- <span class="n">wait_for_finish</span><span class="o">=</span><span class="n">False</span>
-<span class="p">)</span>
+<div class="codehilite"><pre><span class="n">import</span> <span class="n">sys</span>
+<span class="n">from</span> <span class="n">resource_management</span> <span class="n">import</span> <span class="o">*</span>
+
+<span class="n">class</span> <span class="n">Memcached</span><span class="p">(</span><span class="n">Script</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">def</span> <span class="n">install</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">self</span><span class="p">.</span><span class="n">install_packages</span><span class="p">(</span><span class="n">env</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">configure</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">start</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+ <span class="n">self</span><span class="p">.</span><span class="n">configure</span><span class="p">(</span><span class="n">env</span><span class="p">)</span>
+ <span class="n">process_cmd</span> <span class="o">=</span> <span class="n">format</span><span class="p">(</span><span class="s">"{java64_home}/bin/java -Xmx{xmx_val} -Xms{xms_val} -classpath {app_root}/*:{additional_cp} com.thimbleware.jmemcached.Main --memory={memory_val}"</span><span class="p">)</span>
+
+ <span class="n">Execute</span><span class="p">(</span><span class="n">process_cmd</span><span class="p">,</span>
+ <span class="n">user</span><span class="o">=</span><span class="n">params</span><span class="p">.</span><span class="n">app_user</span><span class="p">,</span>
+ <span class="n">logoutput</span><span class="o">=</span><span class="n">False</span><span class="p">,</span>
+ <span class="n">wait_for_finish</span><span class="o">=</span><span class="n">False</span>
+ <span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">stop</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">status</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
</pre></div>
-<p>def stop(self, env):
- import params
- env.set_params(params)</p>
-<p>def status(self, env):
- import params
- env.set_params(params)
-```
-Thats pretty much it. The script basically does the following:</p>
+<p>Thats pretty much it. The script basically does the following:</p>
<ul>
<li>Expand the given tarball</li>
<li>Reads the provided configuration and creates the command string</li>
Modified: incubator/slider/site/content/docs/slider_specs/writing_app_command_scripts.html
URL: http://svn.apache.org/viewvc/incubator/slider/site/content/docs/slider_specs/writing_app_command_scripts.html?rev=1611262&r1=1611261&r2=1611262&view=diff
==============================================================================
--- incubator/slider/site/content/docs/slider_specs/writing_app_command_scripts.html (original)
+++ incubator/slider/site/content/docs/slider_specs/writing_app_command_scripts.html Thu Jul 17 06:53:53 2014
@@ -226,154 +226,158 @@ Latest release: <strong>0.30-incubating<
<h3 id="sample-configuration-parameters">Sample configuration parameters</h3>
-<p>While a script can access all the config parameters the key property bags are</p>
-<ul>
-<li>hostLevelParams</li>
-<li>configurations</li>
-</ul>
-<p><code>{
- "roleCommand": "START",
- "commandParams": {
- "record_config": "true",
- "service_package_folder": "${AGENT_WORK_ROOT}/work/app/definition/package",
- "script": "scripts/hbase_regionserver.py",
- "schema_version": "2.0",
- "command_timeout": "300",
- "script_type": "PYTHON"
+<p>While a script can access all the config parameters the key property bags are <strong>hostLevelParams</strong> and <strong>configurations</strong>.</p>
+<div class="codehilite"><pre>{
+ "roleCommand": "START",
+ "commandParams": {
+ "record_config": "true",
+ "service_package_folder": "<span class="cp">${</span><span class="n">AGENT_WORK_ROOT</span><span class="cp">}</span>/work/app/definition/package",
+ "script": "scripts/hbase_regionserver.py",
+ "schema_version": "2.0",
+ "command_timeout": "300",
+ "script_type": "PYTHON"
},
- "componentName": "HBASE_REGIONSERVER",
- "hostname": "c6403.ambari.apache.org",
- "hostLevelParams": {
- "java_home": "/usr/jdk64/jdk1.7.0_45",
- "container_id": "container_1405048900371_0015_02_000003"
+ "componentName": "HBASE_REGIONSERVER",
+ "hostname": "c6403.ambari.apache.org",
+ "hostLevelParams": {
+ "java_home": "/usr/jdk64/jdk1.7.0_45",
+ "container_id": "container_1405048900371_0015_02_000003"
},
- "commandType": "EXECUTION_COMMAND",
- "clusterName": "cl1",
- "serviceName": "cl1",
- "role": "HBASE_REGIONSERVER",
- "taskId": 4,
- "public_hostname": "c6403.ambari.apache.org",
- "commandId": "4-1",
- "configurations": {
- "hbase-site": {
- "hbase.hstore.flush.retries.number": "120",
- "hbase.client.keyvalue.maxsize": "10485760",
- "hbase.hstore.compactionThreshold": "3",
- "hbase.rootdir": "hdfs://c6403.ambari.apache.org:8020/user/yarn/.slider/cluster/cl1/database/data",
- "hbase.regionserver.handler.count": "60",
- "hbase.regionserver.global.memstore.lowerLimit": "0.38",
- "hbase.hregion.memstore.block.multiplier": "2",
- "hbase.hregion.memstore.flush.size": "134217728",
- "hbase.superuser": "yarn",
- "hbase.zookeeper.property.clientPort": "2181",
- "hbase.regionserver.global.memstore.upperLimit": "0.4",
- "zookeeper.session.timeout": "30000",
- "hbase.tmp.dir": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/work/app/tmp",
- "hfile.block.cache.size": "0.40",
- "hbase.stagingdir": "hdfs://c6403.ambari.apache.org:8020/user/yarn/.slider/cluster/cl1/database/staging",
- "hbase.hregion.max.filesize": "10737418240",
- "hbase.regionserver.port": "0",
- "hbase.security.authentication": "simple",
- "hbase.defaults.for.version.skip": "true",
- "hbase.master.info.port": "52502",
- "hbase.zookeeper.quorum": "c6403.ambari.apache.org",
- "hbase.regionserver.info.port": "0",
- "zookeeper.znode.parent": "/services/slider/users/yarn/cl1",
- "hbase.hstore.blockingStoreFiles": "10",
- "hbase.hregion.majorcompaction": "86400000",
- "hbase.security.authorization": "false",
- "hbase.local.dir": "${hbase.tmp.dir}/local",
- "hbase.cluster.distributed": "true",
- "hbase.hregion.memstore.mslab.enabled": "true",
- "hbase.client.scanner.caching": "100",
- "hbase.zookeeper.useMulti": "true"
+ "commandType": "EXECUTION_COMMAND",
+ "clusterName": "cl1",
+ "serviceName": "cl1",
+ "role": "HBASE_REGIONSERVER",
+ "taskId": 4,
+ "public_hostname": "c6403.ambari.apache.org",
+ "commandId": "4-1",
+ "configurations": {
+ "hbase-site": {
+ "hbase.hstore.flush.retries.number": "120",
+ "hbase.client.keyvalue.maxsize": "10485760",
+ "hbase.hstore.compactionThreshold": "3",
+ "hbase.rootdir": "hdfs://c6403.ambari.apache.org:8020/user/yarn/.slider/cluster/cl1/database/data",
+ "hbase.regionserver.handler.count": "60",
+ "hbase.regionserver.global.memstore.lowerLimit": "0.38",
+ "hbase.hregion.memstore.block.multiplier": "2",
+ "hbase.hregion.memstore.flush.size": "134217728",
+ "hbase.superuser": "yarn",
+ "hbase.zookeeper.property.clientPort": "2181",
+ "hbase.regionserver.global.memstore.upperLimit": "0.4",
+ "zookeeper.session.timeout": "30000",
+ "hbase.tmp.dir": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/work/app/tmp",
+ "hfile.block.cache.size": "0.40",
+ "hbase.stagingdir": "hdfs://c6403.ambari.apache.org:8020/user/yarn/.slider/cluster/cl1/database/staging",
+ "hbase.hregion.max.filesize": "10737418240",
+ "hbase.regionserver.port": "0",
+ "hbase.security.authentication": "simple",
+ "hbase.defaults.for.version.skip": "true",
+ "hbase.master.info.port": "52502",
+ "hbase.zookeeper.quorum": "c6403.ambari.apache.org",
+ "hbase.regionserver.info.port": "0",
+ "zookeeper.znode.parent": "/services/slider/users/yarn/cl1",
+ "hbase.hstore.blockingStoreFiles": "10",
+ "hbase.hregion.majorcompaction": "86400000",
+ "hbase.security.authorization": "false",
+ "hbase.local.dir": "<span class="cp">${</span><span class="n">hbase</span><span class="o">.</span><span class="n">tmp</span><span class="o">.</span><span class="n">dir</span><span class="cp">}</span>/local",
+ "hbase.cluster.distributed": "true",
+ "hbase.hregion.memstore.mslab.enabled": "true",
+ "hbase.client.scanner.caching": "100",
+ "hbase.zookeeper.useMulti": "true"
},
- "global": {
- "security_enabled": "false",
- "ganglia_server_port": "8667",
- "hbase_master_heapsize": "1024m",
- "app_root": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/install/hbase-0.98.3-hadoop2",
- "app_log_dir": "/hadoop/yarn/log/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/log",
- "ganglia_server_host": "c6403.ambari.apache.org",
- "ganglia_server_id": "Application1",
- "hbase_regionserver_heapsize": "1024m",
- "user_group": "hadoop",
- "app_pid_dir": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/run",
- "app_install_dir": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/install",
- "app_user": "yarn"
+ "global": {
+ "security_enabled": "false",
+ "ganglia_server_port": "8667",
+ "hbase_master_heapsize": "1024m",
+ "app_root": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/install/hbase-0.98.3-hadoop2",
+ "app_log_dir": "/hadoop/yarn/log/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/log",
+ "ganglia_server_host": "c6403.ambari.apache.org",
+ "ganglia_server_id": "Application1",
+ "hbase_regionserver_heapsize": "1024m",
+ "user_group": "hadoop",
+ "app_pid_dir": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/run",
+ "app_install_dir": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/install",
+ "app_user": "yarn"
},
- "hdfs-site": {},
- "core-site": {}
+ "hdfs-site": {},
+ "core-site": {}
}
-}</code></p>
+}
+</pre></div>
+
+
<h2 id="command-scripts">Command scripts</h2>
<p>A command script essentially implements commands. The basic structure of a script is as follows:</p>
-<p>```
-class HbaseMaster(Script):
- def install(self, env):
- ...</p>
-<p>def configure(self, env):
- ...</p>
-<p>def start(self, env):
- ...</p>
-<p>def stop(self, env):
- ...</p>
-<p>def status(self, env):
- ...
-```
-At this point, you are free to implement as you wish. Slider provides a rich library for basic operations needed for application commands. So the structure we suggest is:</p>
+<div class="codehilite"><pre><span class="n">class</span> <span class="n">HbaseMaster</span><span class="p">(</span><span class="n">Script</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">def</span> <span class="n">install</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="p">...</span>
+
+ <span class="n">def</span> <span class="n">configure</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="p">...</span>
+
+ <span class="n">def</span> <span class="n">start</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="p">...</span>
+
+ <span class="n">def</span> <span class="n">stop</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="p">...</span>
+
+ <span class="n">def</span> <span class="n">status</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="p">...</span>
+</pre></div>
+
+
+<p>At this point, you are free to implement as you wish. Slider provides a rich library for basic operations needed for application commands. So the structure we suggest is:</p>
<ul>
<li>Create a params.py file to read the json config file into variables. This will allow you to provide initial validation, provide default values, or create complex values</li>
<li>Use the above parameters in your scripts</li>
</ul>
<p>A params.py file may be as follows:</p>
-<p>```
-from resource_management import *</p>
-<h1 id="server-configurations">server configurations</h1>
-<p>config = Script.get_config()</p>
-<p>hbase_root = config['configurations']['global']['app_root']
-conf_dir = format("{hbase_root}/conf")
-daemon_script = format("{hbase_root}/bin/hbase-daemon.sh")
-...
-```</p>
-<h3 id="sample-command-script">Sample command script</h3>
-<p>In this example, hbase master command implementation is using the above parameter file and calling into several helper functions. Some are provided by Slider (e.g. install_package, check_process_status) and some are developed for HBase (e.g. habse_service called by start)</p>
-<p>```
-class HbaseMaster(Script):
- def install(self, env):
- self.install_packages(env)</p>
-<p>def configure(self, env):
- import params
- env.set_params(params)</p>
-<div class="codehilite"><pre><span class="n">hbase</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="err">'</span><span class="n">master</span><span class="err">'</span><span class="p">)</span>
-</pre></div>
+<div class="codehilite"><pre><span class="n">from</span> <span class="n">resource_management</span> <span class="n">import</span> <span class="o">*</span>
+<span class="cp"># server configurations</span>
+<span class="n">config</span> <span class="o">=</span> <span class="n">Script</span><span class="p">.</span><span class="n">get_config</span><span class="p">()</span>
-<p>def start(self, env):
- import params
- env.set_params(params)
- self.configure(env) # for security</p>
-<div class="codehilite"><pre><span class="n">hbase_service</span><span class="p">(</span> <span class="err">'</span><span class="n">master</span><span class="err">'</span><span class="p">,</span>
- <span class="n">action</span> <span class="o">=</span> <span class="err">'</span><span class="n">start</span><span class="err">'</span>
-<span class="p">)</span>
+<span class="n">hbase_root</span> <span class="o">=</span> <span class="n">config</span><span class="p">[</span><span class="err">'</span><span class="n">configurations</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">global</span><span class="err">'</span><span class="p">][</span><span class="err">'</span><span class="n">app_root</span><span class="err">'</span><span class="p">]</span>
+<span class="n">conf_dir</span> <span class="o">=</span> <span class="n">format</span><span class="p">(</span><span class="s">"{hbase_root}/conf"</span><span class="p">)</span>
+<span class="n">daemon_script</span> <span class="o">=</span> <span class="n">format</span><span class="p">(</span><span class="s">"{hbase_root}/bin/hbase-daemon.sh"</span><span class="p">)</span>
+<span class="p">...</span>
</pre></div>
-<p>def stop(self, env):
- import params
- env.set_params(params)</p>
-<div class="codehilite"><pre><span class="n">hbase_service</span><span class="p">(</span> <span class="err">'</span><span class="n">master</span><span class="err">'</span><span class="p">,</span>
- <span class="n">action</span> <span class="o">=</span> <span class="err">'</span><span class="n">stop</span><span class="err">'</span>
-<span class="p">)</span>
+<h3 id="sample-command-script">Sample command script</h3>
+<p>In this example, hbase master command implementation is using the above parameter file and calling into several helper functions. Some are provided by Slider (e.g. install_package, check_process_status) and some are developed for HBase (e.g. habse_service called by start)</p>
+<div class="codehilite"><pre><span class="n">class</span> <span class="n">HbaseMaster</span><span class="p">(</span><span class="n">Script</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">def</span> <span class="n">install</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">self</span><span class="p">.</span><span class="n">install_packages</span><span class="p">(</span><span class="n">env</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">configure</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+
+ <span class="n">hbase</span><span class="p">(</span><span class="n">name</span><span class="o">=</span><span class="err">'</span><span class="n">master</span><span class="err">'</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">start</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+ <span class="n">self</span><span class="p">.</span><span class="n">configure</span><span class="p">(</span><span class="n">env</span><span class="p">)</span> <span class="err">#</span> <span class="k">for</span> <span class="n">security</span>
+
+ <span class="n">hbase_service</span><span class="p">(</span> <span class="err">'</span><span class="n">master</span><span class="err">'</span><span class="p">,</span>
+ <span class="n">action</span> <span class="o">=</span> <span class="err">'</span><span class="n">start</span><span class="err">'</span>
+ <span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">stop</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+
+ <span class="n">hbase_service</span><span class="p">(</span> <span class="err">'</span><span class="n">master</span><span class="err">'</span><span class="p">,</span>
+ <span class="n">action</span> <span class="o">=</span> <span class="err">'</span><span class="n">stop</span><span class="err">'</span>
+ <span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">status</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">)</span><span class="o">:</span>
+ <span class="n">import</span> <span class="n">status_params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">status_params</span><span class="p">)</span>
+ <span class="n">pid_file</span> <span class="o">=</span> <span class="n">format</span><span class="p">(</span><span class="s">"{pid_dir}/hbase-{hbase_user}-master.pid"</span><span class="p">)</span>
+ <span class="n">check_process_status</span><span class="p">(</span><span class="n">pid_file</span><span class="p">)</span>
</pre></div>
-
-
-<p>def status(self, env):
- import status_params
- env.set_params(status_params)
- pid_file = format("{pid_dir}/hbase-{hbase_user}-master.pid")
- check_process_status(pid_file)
-```</p>
</div>
<div id="footer">
Modified: incubator/slider/site/trunk/content/docs/slider_specs/application_package.md
URL: http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/slider_specs/application_package.md?rev=1611262&r1=1611261&r2=1611262&view=diff
==============================================================================
--- incubator/slider/site/trunk/content/docs/slider_specs/application_package.md (original)
+++ incubator/slider/site/trunk/content/docs/slider_specs/application_package.md Thu Jul 17 06:53:53 2014
@@ -76,27 +76,27 @@ Scripts are the implementation of manage
The script specified in the metainfo is expected to understand the command. It can choose to call other scripts based on how the application author organizes the code base. For example:
-```
-class HbaseMaster(Script):
- def install(self, env):
- self.install_packages(env)
+
+ class HbaseMaster(Script):
+ def install(self, env):
+ self.install_packages(env)
- def configure(self, env):
- import params
- env.set_params(params)
+ def configure(self, env):
+ import params
+ env.set_params(params)
- hbase(name='master')
+ hbase(name='master')
- def start(self, env):
- import params
- env.set_params(params)
- self.configure(env) # for security
-
- hbase_service( 'master',
- action = 'start'
- )
-...
-```
+ def start(self, env):
+ import params
+ env.set_params(params)
+ self.configure(env) # for security
+
+ hbase_service( 'master',
+ action = 'start'
+ )
+ ...
+
[How to write scripts](/docs/slider_specs/writing_app_command_scripts.html) discusses how the scripts are developed and the structure of the JSON_FILE containing the parameters.
Modified: incubator/slider/site/trunk/content/docs/slider_specs/creating_app_definitions.md
URL: http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/slider_specs/creating_app_definitions.md?rev=1611262&r1=1611261&r2=1611262&view=diff
==============================================================================
--- incubator/slider/site/trunk/content/docs/slider_specs/creating_app_definitions.md (original)
+++ incubator/slider/site/trunk/content/docs/slider_specs/creating_app_definitions.md Thu Jul 17 06:53:53 2014
@@ -72,14 +72,15 @@ For other app packages, follow this exam
* cd slider/app-packages/hbase
* zip -r hbase-v098.zip .
-* Looking at the content through unzip -l "$@" hbase-v098.zip
-```
-Archive: hbase-v098.zip
- Length Date Time Name
---------- ---------- ----- ----
- 3158 06-19-2014 22:08 appConfig.json
- 0 06-19-2014 22:15 configuration/
+Looking at the content through unzip -l "$@" hbase-v098.zip
+
+
+ Archive: hbase-v098.zip
+ Length Date Time Name
+ --------- ---------- ----- ----
+ 3158 06-19-2014 22:08 appConfig.json
+ 0 06-19-2014 22:15 configuration/
5077 06-19-2014 22:15 configuration/global.xml
5248 06-19-2014 22:15 configuration/hbase-log4j.xml
2250 06-19-2014 22:15 configuration/hbase-policy.xml
@@ -88,34 +89,34 @@ Archive: hbase-v098.zip
3991 06-19-2014 22:45 metainfo.xml
0 06-19-2014 22:33 package/
0 06-19-2014 22:31 package/files/
- 83278591 06-19-2014 22:31 package/files/hbase-0.98.3-hadoop2-bin.tar.gz
- 0 06-19-2014 22:29 package/scripts/
- 787 06-19-2014 22:27 package/scripts/__init__.py
- 1378 06-19-2014 22:27 package/scripts/functions.py
- 3738 06-19-2014 22:27 package/scripts/hbase.py
- 1205 06-19-2014 22:27 package/scripts/hbase_client.py
- 1640 06-19-2014 22:27 package/scripts/hbase_master.py
- 1764 06-19-2014 22:27 package/scripts/hbase_regionserver.py
- 1482 06-19-2014 22:27 package/scripts/hbase_service.py
- 5089 06-19-2014 22:27 package/scripts/params.py
- 973 06-19-2014 22:27 package/scripts/status_params.py
- 0 06-19-2014 22:34 package/templates/
- 2799 06-19-2014 22:34 package/templates/hadoop-metrics2-hbase.properties-GANGLIA-MASTER.j2
- 2799 06-19-2014 22:34 package/templates/hadoop-metrics2-hbase.properties-GANGLIA-RS.j2
- 3878 06-19-2014 22:34 package/templates/hbase-env.sh.j2
- 909 06-19-2014 22:34 package/templates/hbase_client_jaas.conf.j2
- 989 06-19-2014 22:34 package/templates/hbase_master_jaas.conf.j2
- 1001 06-19-2014 22:34 package/templates/hbase_regionserver_jaas.conf.j2
- 837 06-19-2014 22:34 package/templates/regionservers.j2
- 3486 06-19-2014 22:39 pom.xml
- 1921 06-19-2014 21:53 README.txt
- 358 06-19-2014 22:36 resources.json
- 0 06-19-2014 22:40 src/
- 0 06-19-2014 23:02 src/assembly/
- 2400 06-19-2014 22:45 src/assembly/hbase.xml
---------- -------
- 83354595 35 files
-```
+ 83278591 06-19-2014 22:31 package/files/hbase-0.98.3-hadoop2-bin.tar.gz
+ 0 06-19-2014 22:29 package/scripts/
+ 787 06-19-2014 22:27 package/scripts/__init__.py
+ 1378 06-19-2014 22:27 package/scripts/functions.py
+ 3738 06-19-2014 22:27 package/scripts/hbase.py
+ 1205 06-19-2014 22:27 package/scripts/hbase_client.py
+ 1640 06-19-2014 22:27 package/scripts/hbase_master.py
+ 1764 06-19-2014 22:27 package/scripts/hbase_regionserver.py
+ 1482 06-19-2014 22:27 package/scripts/hbase_service.py
+ 5089 06-19-2014 22:27 package/scripts/params.py
+ 973 06-19-2014 22:27 package/scripts/status_params.py
+ 0 06-19-2014 22:34 package/templates/
+ 2799 06-19-2014 22:34 package/templates/hadoop-metrics2-hbase.properties-GANGLIA-MASTER.j2
+ 2799 06-19-2014 22:34 package/templates/hadoop-metrics2-hbase.properties-GANGLIA-RS.j2
+ 3878 06-19-2014 22:34 package/templates/hbase-env.sh.j2
+ 909 06-19-2014 22:34 package/templates/hbase_client_jaas.conf.j2
+ 989 06-19-2014 22:34 package/templates/hbase_master_jaas.conf.j2
+ 1001 06-19-2014 22:34 package/templates/hbase_regionserver_jaas.conf.j2
+ 837 06-19-2014 22:34 package/templates/regionservers.j2
+ 3486 06-19-2014 22:39 pom.xml
+ 1921 06-19-2014 21:53 README.txt
+ 358 06-19-2014 22:36 resources.json
+ 0 06-19-2014 22:40 src/
+ 0 06-19-2014 23:02 src/assembly/
+ 2400 06-19-2014 22:45 src/assembly/hbase.xml
+ --------- -------
+ 83354595 35 files
+
Sample **resources.json** and **appConfig.json** files are also included in the enlistment. These are samples and are typically tested on one node test installations.
Modified: incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md
URL: http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md?rev=1611262&r1=1611261&r2=1611262&view=diff
==============================================================================
--- incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md (original)
+++ incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md Thu Jul 17 06:53:53 2014
@@ -27,92 +27,92 @@ The structure of an app package is discu
In this example, the application package created looks as follows:
-```
-unzip -l "$@" ~/enlistments/ambari-vagrant/centos6.4/memcached/jmemcached-1.0.0.zip
-Archive: /jmemcached-1.0.0.zip
- Length Date Time Name
- -------- ---- ---- ----
- 637 07-15-14 19:17 appConfig.json
- 1673 07-15-14 17:58 metainfo.xml
- 0 07-15-14 17:54 package/
- 0 07-15-14 18:03 package/files/
- 122880 07-15-14 18:03 package/files/jmemcached-1.0.0.tar
- 0 07-15-14 19:31 package/scripts/
- 1530 07-15-14 19:31 package/scripts/memcached.py
- 1287 07-15-14 18:46 package/scripts/params.py
- 1581 07-15-14 19:16 README.txt
- 252 07-15-14 17:58 resources.json
-```
+
+ unzip -l "$@" ~/enlistments/ambari-vagrant/centos6.4/memcached/jmemcached-1.0.0.zip
+ Archive: /jmemcached-1.0.0.zip
+ Length Date Time Name
+ -------- ---- ---- ----
+ 637 07-15-14 19:17 appConfig.json
+ 1673 07-15-14 17:58 metainfo.xml
+ 0 07-15-14 17:54 package/
+ 0 07-15-14 18:03 package/files/
+ 122880 07-15-14 18:03 package/files/jmemcached-1.0.0.tar
+ 0 07-15-14 19:31 package/scripts/
+ 1530 07-15-14 19:31 package/scripts/memcached.py
+ 1287 07-15-14 18:46 package/scripts/params.py
+ 1581 07-15-14 19:16 README.txt
+ 252 07-15-14 17:58 resources.json
+
## Step 1: Create metainfo.xml
The minimal metainfo contains some information about the application (name, comment, version), at least one component type, in this case its MEMCACHED, and information about the tarball. More details are available [here](/docs/slider_specs/application_definition.html).
-```
-<metainfo>
- <schemaVersion>2.0</schemaVersion>
- <application>
- <name>MEMCACHED</name>
- <comment>Memcache is a network accessible key/value storage system, often used as a distributed cache.</comment>
- <version>1.0.0</version>
- <components>
- <component>
+ <metainfo>
+ <schemaVersion>2.0</schemaVersion>
+ <application>
<name>MEMCACHED</name>
- <category>MASTER</category>
- <commandScript>
- <script>scripts/memcached.py</script>
- <scriptType>PYTHON</scriptType>
- </commandScript>
- </component>
- </components>
-
- <osSpecifics>
- <osSpecific>
- <osType>any</osType>
- <packages>
- <package>
- <type>tarball</type>
- <name>files/jmemcached-1.0.0.tar</name>
- </package>
- </packages>
- </osSpecific>
- </osSpecifics>
-
- </application>
-</metainfo>
-```
+ <comment>Memcache is a network accessible key/value storage system, often used as a distributed cache.</comment>
+ <version>1.0.0</version>
+
+ <components>
+ <component>
+ <name>MEMCACHED</name>
+ <category>MASTER</category>
+ <commandScript>
+ <script>scripts/memcached.py</script>
+ <scriptType>PYTHON</scriptType>
+ </commandScript>
+ </component>
+ </components>
+
+ <osSpecifics>
+ <osSpecific>
+ <osType>any</osType>
+ <packages>
+ <package>
+ <type>tarball</type>
+ <name>files/jmemcached-1.0.0.tar</name>
+ </package>
+ </packages>
+ </osSpecific>
+ </osSpecifics>
+
+ </application>
+ </metainfo>
+
## Step 2: Ensure application tarball
Most applications release a tarball that you can download otherwise you can create one. For this sample, we created a simple tarball that contains the cli and core jar from jmemcached.
-```
-tar tvf jmemcached-1.0.0.tar
-drwxr-xr-x 0 yarn hadoop 0 Jul 15 17:51 jmemcached-1.0.0/
--rw-r--r-- 0 yarn hadoop 13537 Jul 15 17:51 jmemcached-1.0.0/jmemcached-cli-1.0.0.jar
--rwxr-xr-x 0 yarn hadoop 101467 Jul 15 17:51 jmemcached-1.0.0/jmemcached-core-1.0.0.jar
-```
+
+ tar tvf jmemcached-1.0.0.tar
+ drwxr-xr-x 0 yarn hadoop 0 Jul 15 17:51 jmemcached-1.0.0/
+ -rw-r--r-- 0 yarn hadoop 13537 Jul 15 17:51 jmemcached-1.0.0/jmemcached-cli-1.0.0.jar
+ -rwxr-xr-x 0 yarn hadoop 101467 Jul 15 17:51 jmemcached-1.0.0/jmemcached-core-1.0.0.jar
+
## Step 3: Create a default resources file (resources.json)
By default all resources.json file must include slider-appmaster. Add one more entry for the component MEMCACHED and assign a unique priority and default number of instances. Ensure, that a suitable default value is provided for yarn.memory. Additional details are available [here](/docs/slider_specs/resource_specification.html).
-```
-{
- "schema" : "http://example.org/specification/v2.0.0",
- "metadata" : {
- },
- "global" : {
- },
- "components": {
- "slider-appmaster": {
- },
- "MEMCACHED": {
- "yarn.role.priority": "1",
- "yarn.component.instances": "1",
- "yarn.memory": "256"
+
+ {
+ "schema" : "http://example.org/specification/v2.0.0",
+ "metadata" : {
+ },
+ "global" : {
+ },
+ "components": {
+ "slider-appmaster": {
+ },
+ "MEMCACHED": {
+ "yarn.role.priority": "1",
+ "yarn.component.instances": "1",
+ "yarn.memory": "256"
+ }
+ }
}
- }
-}
-```
+
## Step 4: Create a default configuration template (appConfig.json)
The config template has few mandatory parameters such as
@@ -130,32 +130,33 @@ Add other parameters needed by the appli
* site.global.xms_val - value of Xms
* site.global.memory_val - value of memory
-```
-{
- "schema": "http://example.org/specification/v2.0.0",
- "metadata": {
- },
- "global": {
- "application.def": "/slider/jmemcached-1.0.0.zip",
- "java_home": "/usr/jdk64/jdk1.7.0_45",
- "package_list": "files/jmemcached-1.0.0.tar",
- "config_types": "",
- "site.global.app_user": "yarn",
- "site.global.app_root": "${AGENT_WORK_ROOT}/app/install/jmemcached-1.0.0",
- "site.global.additional_cp": "/usr/lib/hadoop/lib/*",
- "site.global.xmx_val": "256m",
- "site.global.xms_val": "128m",
- "site.global.memory_val": "200M"
- },
- "components": {
- "slider-appmaster": {
- "jvm.heapsize": "256M"
- },
- "MEMCACHED": {
+You can add additional parameters as needed.
+
+ {
+ "schema": "http://example.org/specification/v2.0.0",
+ "metadata": {
+ },
+ "global": {
+ "application.def": "/slider/jmemcached-1.0.0.zip",
+ "java_home": "/usr/jdk64/jdk1.7.0_45",
+ "package_list": "files/jmemcached-1.0.0.tar",
+ "config_types": "",
+ "site.global.app_user": "yarn",
+ "site.global.app_root": "${AGENT_WORK_ROOT}/app/install/jmemcached-1.0.0",
+ "site.global.additional_cp": "/usr/lib/hadoop/lib/*",
+ "site.global.xmx_val": "256m",
+ "site.global.xms_val": "128m",
+ "site.global.memory_val": "200M"
+ },
+ "components": {
+ "slider-appmaster": {
+ "jvm.heapsize": "256M"
+ },
+ "MEMCACHED": {
+ }
+ }
}
- }
-}
-```
+
Additional details on how to define a configuration template is [here](/docs/slider_specs/application_instance_configuration.html).
@@ -164,54 +165,54 @@ All Slider applications are expected to
The parameters file we will use is:
-```
-from resource_management import *
-config = Script.get_config()
+ from resource_management import *
+
+ config = Script.get_config()
+
+ app_root = config['configurations']['global']['app_root']
+ java64_home = config['hostLevelParams']['java_home']
+ app_user = config['configurations']['global']['app_user']
+ additional_cp = config['configurations']['global']['additional_cp']
+ xmx_val = config['configurations']['global']['xmx_val']
+ xms_val = config['configurations']['global']['xms_val']
+ memory_val = config['configurations']['global']['memory_val']
-app_root = config['configurations']['global']['app_root']
-java64_home = config['hostLevelParams']['java_home']
-app_user = config['configurations']['global']['app_user']
-additional_cp = config['configurations']['global']['additional_cp']
-xmx_val = config['configurations']['global']['xmx_val']
-xms_val = config['configurations']['global']['xms_val']
-memory_val = config['configurations']['global']['memory_val']
-```
Note that the parameter.py file only reads the parameters needed by the command implementations.
-```
-import sys
-from resource_management import *
-
-class Memcached(Script):
- def install(self, env):
- self.install_packages(env)
-
- def configure(self, env):
- import params
- env.set_params(params)
-
- def start(self, env):
- import params
- env.set_params(params)
- self.configure(env)
- process_cmd = format("{java64_home}/bin/java -Xmx{xmx_val} -Xms{xms_val} -classpath {app_root}/*:{additional_cp} com.thimbleware.jmemcached.Main --memory={memory_val}")
-
- Execute(process_cmd,
- user=params.app_user,
- logoutput=False,
- wait_for_finish=False
- )
-
- def stop(self, env):
- import params
- env.set_params(params)
-
- def status(self, env):
- import params
- env.set_params(params)
-```
+
+ import sys
+ from resource_management import *
+
+ class Memcached(Script):
+ def install(self, env):
+ self.install_packages(env)
+
+ def configure(self, env):
+ import params
+ env.set_params(params)
+
+ def start(self, env):
+ import params
+ env.set_params(params)
+ self.configure(env)
+ process_cmd = format("{java64_home}/bin/java -Xmx{xmx_val} -Xms{xms_val} -classpath {app_root}/*:{additional_cp} com.thimbleware.jmemcached.Main --memory={memory_val}")
+
+ Execute(process_cmd,
+ user=params.app_user,
+ logoutput=False,
+ wait_for_finish=False
+ )
+
+ def stop(self, env):
+ import params
+ env.set_params(params)
+
+ def status(self, env):
+ import params
+ env.set_params(params)
+
Thats pretty much it. The script basically does the following:
* Expand the given tarball