You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by bu...@apache.org on 2014/07/17 08:54:00 UTC
svn commit: r916463 - in /websites/staging/slider/trunk/content: ./
docs/slider_specs/application_package.html
docs/slider_specs/creating_app_definitions.html
docs/slider_specs/hello_world_slider_app.html
docs/slider_specs/writing_app_command_scripts.html
Author: buildbot
Date: Thu Jul 17 06:54:00 2014
New Revision: 916463
Log:
Staging update by buildbot for slider
Added:
websites/staging/slider/trunk/content/docs/slider_specs/creating_app_definitions.html
Modified:
websites/staging/slider/trunk/content/ (props changed)
websites/staging/slider/trunk/content/docs/slider_specs/application_package.html
websites/staging/slider/trunk/content/docs/slider_specs/hello_world_slider_app.html
websites/staging/slider/trunk/content/docs/slider_specs/writing_app_command_scripts.html
Propchange: websites/staging/slider/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Thu Jul 17 06:54:00 2014
@@ -1 +1 @@
-1611257
+1611262
Modified: websites/staging/slider/trunk/content/docs/slider_specs/application_package.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/slider_specs/application_package.html (original)
+++ websites/staging/slider/trunk/content/docs/slider_specs/application_package.html Thu Jul 17 06:54:00 2014
@@ -233,29 +233,28 @@ other scripts, txt files, tarballs, etc.
</li>
</ol>
<p>The script specified in the metainfo is expected to understand the command. It can choose to call other scripts based on how the application author organizes the code base. For example:</p>
-<p>```
-class HbaseMaster(Script):
- def install(self, env):
- self.install_packages(env)</p>
-<p>def configure(self, env):
- import params
- env.set_params(params)</p>
-<div class="codehilite"><pre><span class="n">hbase</span><span class="p">(</span><span class="n">name</span><span class="p">=</span><span class="s">'master'</span><span class="p">)</span>
+<div class="codehilite"><pre><span class="n">class</span> <span class="n">HbaseMaster</span><span class="p">(</span><span class="n">Script</span><span class="p">):</span>
+ <span class="n">def</span> <span class="n">install</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">self</span><span class="p">.</span><span class="n">install_packages</span><span class="p">(</span><span class="n">env</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">configure</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+
+ <span class="n">hbase</span><span class="p">(</span><span class="n">name</span><span class="p">=</span><span class="s">'master'</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">start</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+ <span class="n">self</span><span class="p">.</span><span class="n">configure</span><span class="p">(</span><span class="n">env</span><span class="p">)</span> # <span class="k">for</span> <span class="n">security</span>
+
+ <span class="n">hbase_service</span><span class="p">(</span> <span class="s">'master'</span><span class="p">,</span>
+ <span class="n">action</span> <span class="p">=</span> <span class="s">'start'</span>
+ <span class="p">)</span>
+<span class="p">...</span>
</pre></div>
-<p>def start(self, env):
- import params
- env.set_params(params)
- self.configure(env) # for security</p>
-<div class="codehilite"><pre><span class="n">hbase_service</span><span class="p">(</span> <span class="s">'master'</span><span class="p">,</span>
- <span class="n">action</span> <span class="p">=</span> <span class="s">'start'</span>
-<span class="p">)</span>
-</pre></div>
-
-
-<p>... <br />
-```</p>
<p><a href="/docs/slider_specs/writing_app_command_scripts.html">How to write scripts</a> discusses how the scripts are developed and the structure of the JSON_FILE containing the parameters.</p>
<h3 id="templates-folder">templates folder</h3>
<p>templates are configurable text files that are NOT regular config files. <em>A library has been developed that can materialize a complete site configuration file from a property bag and therefore are not generated from templates.</em> Other files such as env sh files, log4j properties file, etc. may be derived from a template. Again, the implementor can choose to create these files from scratch and not use templates. The following operations are allowed during template expansion:</p>
Added: websites/staging/slider/trunk/content/docs/slider_specs/creating_app_definitions.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/slider_specs/creating_app_definitions.html (added)
+++ websites/staging/slider/trunk/content/docs/slider_specs/creating_app_definitions.html Thu Jul 17 06:54:00 2014
@@ -0,0 +1,303 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href="/css/bootstrap.min.css" rel="stylesheet">
+ <link href="/css/bootstrap-theme.min.css" rel="stylesheet">
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
+ <script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
+ <script src="/js/bootstrap.min.js"></script>
+ <link href="/css/slider.css" rel="stylesheet" type="text/css">
+ <link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
+ <title></title>
+
+</head>
+
+<body>
+<nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+<div class="container-fluid">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar-items">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/index.html">Slider</a>
+ </div>
+ <div class="collapse navbar-collapse" id="navbar-items">
+ <ul class="nav navbar-nav">
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Project <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_index"><a href="/index.html">Home</a></li>
+<li id="nav_downloads"><a href="/downloads">Downloads</a></li>
+<li id="nav_source"><a href="https://git-wip-us.apache.org/repos/asf/incubator-slider.git"><i class="fa fa-external-link"></i> Source</a></li>
+<li><a href="http://www.apache.org/licenses/LICENSE-2.0"><i class="fa fa-external-link"></i> License</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Community</li>
+<li id="nav_team"><a href="/team.html">Team</a></li>
+<li id="nav_mailinglists"><a href="/mailing_lists.html">Mailing Lists</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Tools</li>
+<li id="nav_jira"><a href="https://issues.apache.org/jira/browse/SLIDER"><i class="fa fa-external-link"></i> Issues</a></li>
+<li id="nav_builds"><a href="https://builds.apache.org/job/Slider-develop"><i class="fa fa-external-link"></i> Builds</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/design">
+ Design <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_design_overview"><a href="/design/architecture.html">Overview</a></li>
+<li id="nav_app_needs"><a href="/docs/slider_specs/application_needs.html">Application Needs</a></li>
+<li id="nav_spec"><a href="/design/specification/index.html">Expected Behavior</a></li>
+<li id="nav_registry"><a href="/design/registry/index.html">Service Registry</a></li>
+<li id="nav_ssl_impl"><a href="/design/ssl_implementation.html">SSL Implementation</a></li>
+<li id="nav_role_history"><a href="/design/rolehistory.html">Role History</a></li>
+</ul>
+</li>
+
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/developing">
+ Developing <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_dev_building"><a href="/developing/building.html">Building</a></li>
+<li id="nav_dev_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_dev_testing"><a href="/developing/testing.html">Testing</a></li>
+<li id="nav_dev_funtesting"><a href="/developing/functional_tests.html">Functional Testing</a></li>
+<li id="nav_dev_manual_testing"><a href="/developing/manual_testing.html">Manual Testing</a></li>
+<li id="nav_releasing"><a href="/developing/releasing.html">Releasing</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Using <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_use_getting_started"><a href="/docs/getting_started.html">Getting Started</a></li>
+<li id="nav_use_manpage"><a href="/docs/manpage.html">Man Page</a></li>
+<li id="nav_use_examples"><a href="/docs/examples.html">Examples</a></li>
+<li id="nav_use_client_config"><a href="/docs/client-configuration.html">Client Configuration</a></li>
+<li id="nav_use_exitcodes"><a href="/docs/exitcodes.html">Client Exit Codes</a></li>
+<li id="nav_use_security"><a href="/docs/security.html">Security</a></li>
+<li id="nav_am_agent_ssl"><a href="/docs/ssl.html">Agent to AM SSL</a></li>
+<li id="nav_use_app_package"><a href="/docs/slider_specs/index.html">Creating App Packages</a></li>
+<li id="nav_use_app_configuration"><a href="/docs/configuration/index.html">App Configuration Model</a></li>
+<li id="nav_use_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_use_troubleshooting"><a href="/docs/troubleshooting.html">Troubleshooting</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ ASF Links <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li><a href="http://www.apache.org"><i class="fa fa-external-link"></i> Apache Software Foundation</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html"><i class="fa fa-external-link"></i> Sponsorship</a></li>
+<li><a href="http://www.apache.org/security/"><i class="fa fa-external-link"></i> Security</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html"><i class="fa fa-external-link"></i> Thanks</a></li>
+</ul>
+</li>
+</ul>
+
+ </div>
+
+</div>
+</nav>
+
+
+<div class="container-fluid">
+<div class="row">
+ <div class="col-md-2" id="sidebar">
+ <div style="text-align: center">
+ <h1><a href="/index.html">Apache Slider (incubating)</a></h1>
+ <hr>
+Latest release: <strong>0.30-incubating</strong><br>
+ <br>
+ <a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
+ </div>
+ <hr>
+ <a href="http://incubator.apache.org"><img id="logo" alt="Apache Incubator ™" class="img-responsive" src="http://incubator.apache.org/images/apache-incubator-logo.png"/></a>
+ </div>
+ <div class="col-md-8 col-md-offset-1">
+
+ <div id="content">
+
+ <h1 class="title"></h1>
+
+ <!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<h1 id="apache-slider-apppackage">Apache Slider AppPackage</h1>
+<p>Slider AppPackages are a declarative definition of an application for application management. AppPackage is not a packaging scheme for application binaries and artifacts. Tarball, zip files, rpms etc. are available for that purpose. Instead AppPackage includes the application binaries along with other artifacts necessary for application management.</p>
+<p>An application instance consists of several active component such as one or more masters and several slaves. There may be a number of accompanying processes in addition to the basic master/slave processes - lets refer to all processes as app component instances. When run in the context of Yarn, the application specific processes are activated within individual Yarn Container. If you pry into an Yarn container (created through Slider) it will be apparent as to what is the role of Slider-Agent and the actual application components. The following image provides an high-level view. Within a container there are at least two processes - and instance of a slider agent and an instance of an application component. The application can itself spawn more procsses if needed.</p>
+<p><img alt="Image" src="../../resources/images/slider-container.png?raw=true" /></p>
+<p>Figure 1 - High-level view of a container</p>
+<p>For example:</p>
+<div class="codehilite"><pre><span class="n">yarn</span> 8849 <span class="o">--</span> <span class="n">python</span> <span class="o">./</span><span class="n">infra</span><span class="o">/</span><span class="n">agent</span><span class="o">/</span><span class="n">slider</span><span class="o">-</span><span class="n">agent</span><span class="o">/</span><span class="n">agent</span><span class="o">/</span><span class="n">main</span><span class="p">.</span><span class="n">py</span> <span class="o">--</span><span class="n">label</span> <span class="n">container_1397675825552_0011_01_000003___HBASE_REGIONSERVER</span> <span class="o">--</span><span class="n">host</span> <span class="n">AM_HOST</span> <span class="o">--</span><span class="n">port</span> 47830
+<span class="n">yarn</span> 9085 <span class="o">--</span> <span class="n">bash</span> <span class="o">/</span><span class="n">hadoop</span><span class="o">/</span><span class="n">yarn</span><span class="o">/</span><span class="n">local</span><span class="o">/</span><span class="n">usercache</span><span class="o">/</span><span class="n">yarn</span><span class="o">/</span><span class="n">appcache</span><span class="o">/</span><span class="n">application_1397675825552_0011</span><span class="o">/</span> <span class="p">...</span> <span class="n">internal_start</span> <span class="n">regionserver</span>
+<span class="n">yarn</span> 9114 <span class="o">--</span> <span class="o">/</span><span class="n">usr</span><span class="o">/</span><span class="n">jdk64</span><span class="o">/</span><span class="n">jdk1</span><span class="p">.</span>7<span class="p">.</span>0<span class="n">_45</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">java</span> <span class="o">-</span><span class="n">Dproc_regionserver</span> <span class="o">-</span><span class="n">XX</span><span class="p">:</span><span class="n">OnOutOfMemoryError</span><span class="p">=...</span>
+</pre></div>
+
+
+<p>The above list shows three processes, the Slider-Agent process, the bash script to start HBase Region Server and the HBase Region server itself. <em>Three of these together constitute the container</em>. </p>
+<h2 id="using-an-apppackage">Using an AppPackage</h2>
+<p>The following command creates an HBase application using the AppPackage for HBase.</p>
+<div class="codehilite"><pre> <span class="o">./</span><span class="n">slider</span> <span class="n">create</span> <span class="n">cl1</span> <span class="o">--</span><span class="n">image</span> <span class="n">hdfs</span><span class="p">:</span><span class="o">//</span><span class="n">NN</span><span class="p">:</span>8020<span class="o">/</span><span class="n">slider</span><span class="o">/</span><span class="n">agent</span><span class="o">/</span><span class="n">slider</span><span class="o">-</span><span class="n">agent</span><span class="p">.</span><span class="n">tar</span><span class="p">.</span><span class="n">gz</span> <span class="o">--</span><span class="n">template</span> <span class="o">/</span><span class="n">work</span><span class="o">/</span><span class="n">appConf</span><span class="p">.</span><span class="n">json</span> <span class="o">--</span><span class="n">resources</span> <span class="o">/</span><span class="n">work</span><span class="o">/</span><span class="n
">resources</span><span class="p">.</span><span class="n">json</span>
+</pre></div>
+
+
+<p>Lets analyze various parameters from the perspective of app creation:</p>
+<ul>
+<li><code>--image</code>: its the slider agent tarball</li>
+<li><code>--template</code>: app configuration</li>
+<li><code>--resources</code>: yarn resource requests</li>
+<li>⦠other parameters are described in accompanying docs. </li>
+</ul>
+<h3 id="apppackage">AppPackage</h3>
+<p>The structure of an AppPackage is described at <a href="application_package.html">AppPackage</a>.</p>
+<p>In the enlistment, there are three example AppPackages:</p>
+<ul>
+<li><code>app-packages/hbase-v0_96</code></li>
+<li><code>app-packages/accumulo</code></li>
+<li><code>app-packages/storm-v0_91</code></li>
+</ul>
+<p>The above folders, with minor edits, can be packaged as <em>zip</em> files to get the corresponding AppPackages. The application tarball file, containing the binaries/artifacts of the application itself is a component within the AppPackage. They are:</p>
+<ul>
+<li>For hbase - <code>app-packages/hbase-v0_96/package/files/hbase-0.96.1-hadoop2-bin.tar.gz.REPLACE</code></li>
+<li>For accumulo - <code>app-packages/accumulo/package/files/</code> (no file present)</li>
+<li>For storm - <code>app-packages/storm-v0_91/package/files/apache-storm-0.9.1.2.1.1.0-237.tar.gz.placeholder</code></li>
+</ul>
+<p><strong>They are placehoder files</strong>, mostly because the files themselves are too large as well as users are free to use their own version of the package. To create a Slider AppPackage - replace the file with an actual application tarball and then ensure that the metainfo.xml has the correct file name. After that create a zip file using standard zip commands and ensure that the package has the metainfo.xml file at the root folder.</p>
+<p>For Accumulo, this is automated in maven by running</p>
+<div class="codehilite"><pre><span class="n">mvn</span> <span class="n">clean</span> <span class="n">package</span> <span class="o">-</span><span class="n">DskipTests</span> <span class="o">-</span><span class="n">Paccumulo</span><span class="o">-</span><span class="n">app</span><span class="o">-</span><span class="n">package</span>
+</pre></div>
+
+
+<p>Additional configuration parameters are described in <code>app-packages/accumulo/README.txt</code>.</p>
+<p>For other app packages, follow this example:</p>
+<ul>
+<li>cd slider/app-packages/hbase</li>
+<li>zip -r hbase-v098.zip .</li>
+</ul>
+<p>Looking at the content through unzip -l "$@" hbase-v098.zip</p>
+<div class="codehilite"><pre><span class="n">Archive</span><span class="o">:</span> <span class="n">hbase</span><span class="o">-</span><span class="n">v098</span><span class="o">.</span><span class="na">zip</span>
+ <span class="n">Length</span> <span class="n">Date</span> <span class="n">Time</span> <span class="n">Name</span>
+<span class="o">---------</span> <span class="o">----------</span> <span class="o">-----</span> <span class="o">----</span>
+ <span class="mi">3158</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">08</span> <span class="n">appConfig</span><span class="o">.</span><span class="na">json</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">15</span> <span class="n">configuration</span><span class="o">/</span>
+ <span class="mi">5077</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">15</span> <span class="n">configuration</span><span class="o">/</span><span class="n">global</span><span class="o">.</span><span class="na">xml</span>
+ <span class="mi">5248</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">15</span> <span class="n">configuration</span><span class="o">/</span><span class="n">hbase</span><span class="o">-</span><span class="n">log4j</span><span class="o">.</span><span class="na">xml</span>
+ <span class="mi">2250</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">15</span> <span class="n">configuration</span><span class="o">/</span><span class="n">hbase</span><span class="o">-</span><span class="n">policy</span><span class="o">.</span><span class="na">xml</span>
+<span class="mi">14705</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">15</span> <span class="n">configuration</span><span class="o">/</span><span class="n">hbase</span><span class="o">-</span><span class="n">site</span><span class="o">.</span><span class="na">xml</span>
+ <span class="mi">2142</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">17</span> <span class="n">jmx_metrics</span><span class="o">.</span><span class="na">json</span>
+ <span class="mi">3991</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">45</span> <span class="n">metainfo</span><span class="o">.</span><span class="na">xml</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">33</span> <span class="kd">package</span><span class="o">/</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">31</span> <span class="kd">package</span><span class="sr">/files/</span>
+ <span class="mi">83278591</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">31</span> <span class="kd">package</span><span class="sr">/files/</span><span class="n">hbase</span><span class="o">-</span><span class="mf">0.98</span><span class="o">.</span><span class="mi">3</span><span class="o">-</span><span class="n">hadoop2</span><span class="o">-</span><span class="n">bin</span><span class="o">.</span><span class="na">tar</span><span class="o">.</span><span class="na">gz</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">29</span> <span class="kd">package</span><span class="sr">/scripts/</span>
+ <span class="mi">787</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">__init__</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">1378</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">functions</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">3738</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">hbase</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">1205</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">hbase_client</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">1640</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">hbase_master</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">1764</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">hbase_regionserver</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">1482</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">hbase_service</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">5089</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/</span><span class="n">params</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">973</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">27</span> <span class="kd">package</span><span class="sr">/scripts/s</span><span class="n">tatus_params</span><span class="o">.</span><span class="na">py</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span>
+ <span class="mi">2799</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">hadoop</span><span class="o">-</span><span class="n">metrics2</span><span class="o">-</span><span class="n">hbase</span><span class="o">.</span><span class="na">properties</span><span class="o">-</span><span class="n">GANGLIA</span><span class="o">-</span><span class="n">MASTER</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">2799</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">hadoop</span><span class="o">-</span><span class="n">metrics2</span><span class="o">-</span><span class="n">hbase</span><span class="o">.</span><span class="na">properties</span><span class="o">-</span><span class="n">GANGLIA</span><span class="o">-</span><span class="n">RS</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">3878</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">hbase</span><span class="o">-</span><span class="n">env</span><span class="o">.</span><span class="na">sh</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">909</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">hbase_client_jaas</span><span class="o">.</span><span class="na">conf</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">989</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">hbase_master_jaas</span><span class="o">.</span><span class="na">conf</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">1001</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">hbase_regionserver_jaas</span><span class="o">.</span><span class="na">conf</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">837</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">34</span> <span class="kd">package</span><span class="sr">/templates/</span><span class="n">regionservers</span><span class="o">.</span><span class="na">j2</span>
+ <span class="mi">3486</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">39</span> <span class="n">pom</span><span class="o">.</span><span class="na">xml</span>
+ <span class="mi">1921</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">21</span><span class="o">:</span><span class="mi">53</span> <span class="n">README</span><span class="o">.</span><span class="na">txt</span>
+ <span class="mi">358</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">36</span> <span class="n">resources</span><span class="o">.</span><span class="na">json</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">40</span> <span class="n">src</span><span class="o">/</span>
+ <span class="mi">0</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">23</span><span class="o">:</span><span class="mi">02</span> <span class="n">src</span><span class="sr">/assembly/</span>
+ <span class="mi">2400</span> <span class="mi">06</span><span class="o">-</span><span class="mi">19</span><span class="o">-</span><span class="mi">2014</span> <span class="mi">22</span><span class="o">:</span><span class="mi">45</span> <span class="n">src</span><span class="sr">/assembly/</span><span class="n">hbase</span><span class="o">.</span><span class="na">xml</span>
+<span class="o">---------</span> <span class="o">-------</span>
+ <span class="mi">83354595</span> <span class="mi">35</span> <span class="n">files</span>
+</pre></div>
+
+
+<p>Sample <strong>resources.json</strong> and <strong>appConfig.json</strong> files are also included in the enlistment. These are samples and are typically tested on one node test installations.</p>
+<h3 id="-template-appconfigjson">--template appConfig.json</h3>
+<p>An appConfig.json contains the application configuration. See <a href="application_instance_configuration.html">Specifications InstanceConfiguration</a> for details on how to create a template config file. The enlistment includes sample config files for HBase, Accumulo, and Storm.</p>
+<h3 id="-resources-resourcesjson">--resources resources.json</h3>
+<p>Resource specification is an input to Slider to specify the Yarn resource needs for each component type that belong to the application. <a href="resource_specification.html">Specification of Resources</a> describes how to write a resource config json file. The enlistment includes sample config files for HBase, Accumulo, and Storm.</p>
+<h2 id="scripting-for-apppackage">Scripting for AppPackage</h2>
+<p>Refer to <a href="writing_app_command_scripts">App Command Scripts</a> for details on how to write scripts for a AppPackage. These scripts are in the package/script folder within the AppPackage. <em>Use the checked in samples for HBase/Storm/Accumulo as reference for script development.</em></p>
+ </div>
+
+ <div id="footer">
+ <a alt="Apache Software Foundation" href="http://www.apache.org">
+ <img id="asf-logo" alt="Apache Software Foundation" src="/images/feather-small.gif" width="100">
+ </a>
+ <div class="copyright">
+ <p>
+ Copyright © 2011-2014 The Apache Software Foundation, Licensed under
+ the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.
+ Apache Slider, Slider, Apache, the Apache feather logo, and the Apache Incubator
+ project logo are trademarks of the <a href="http://www.apache.org">Apache Software Foundation</a>.<br />
+ Site created with <a href="http://getbootstrap.com/">Bootstrap</a> including icons from <a href="http://glyphicons.com/">GLYPHICONS</a> and <a href="http://fontawesome.io/">Font Awesome</a>.
+ </p>
+ </div>
+ </div>
+ </div>
+
+</div>
+</div>
+
+<script type="text/javascript">
+
+</script>
+</body>
+</html>
Modified: websites/staging/slider/trunk/content/docs/slider_specs/hello_world_slider_app.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/slider_specs/hello_world_slider_app.html (original)
+++ websites/staging/slider/trunk/content/docs/slider_specs/hello_world_slider_app.html Thu Jul 17 06:54:00 2014
@@ -178,81 +178,90 @@ Latest release: <strong>0.30-incubating<
<p>The basic version of the app will allow creation of one or more memcached daemons on default ports. Some memory settings may be configured. Later examples will enhance the application package to add more capabilities.</p>
<p>The structure of an app package is discussed <a href="/docs/slider_specs/application_package.html">here</a>.</p>
<p>In this example, the application package created looks as follows:</p>
-<p><code>unzip -l "$@" ~/enlistments/ambari-vagrant/centos6.4/memcached/jmemcached-1.0.0.zip
-Archive: /jmemcached-1.0.0.zip
- Length Date Time Name
- -------- ---- ---- ----
- 637 07-15-14 19:17 appConfig.json
- 1673 07-15-14 17:58 metainfo.xml
- 0 07-15-14 17:54 package/
- 0 07-15-14 18:03 package/files/
- 122880 07-15-14 18:03 package/files/jmemcached-1.0.0.tar
- 0 07-15-14 19:31 package/scripts/
- 1530 07-15-14 19:31 package/scripts/memcached.py
- 1287 07-15-14 18:46 package/scripts/params.py
- 1581 07-15-14 19:16 README.txt
- 252 07-15-14 17:58 resources.json</code></p>
+<div class="codehilite"><pre><span class="n">unzip</span> <span class="o">-</span><span class="n">l</span> <span class="s">"$@"</span> <span class="o">~/</span><span class="n">enlistments</span><span class="o">/</span><span class="n">ambari</span><span class="o">-</span><span class="n">vagrant</span><span class="o">/</span><span class="n">centos6</span><span class="mf">.4</span><span class="o">/</span><span class="n">memcached</span><span class="o">/</span><span class="n">jmemcached</span><span class="o">-</span><span class="mf">1.0.0</span><span class="p">.</span><span class="n">zip</span>
+<span class="nl">Archive:</span> <span class="o">/</span><span class="n">jmemcached</span><span class="o">-</span><span class="mf">1.0.0</span><span class="p">.</span><span class="n">zip</span>
+ <span class="n">Length</span> <span class="n">Date</span> <span class="n">Time</span> <span class="n">Name</span>
+ <span class="o">--------</span> <span class="o">----</span> <span class="o">----</span> <span class="o">----</span>
+ <span class="mi">637</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">19</span><span class="o">:</span><span class="mi">17</span> <span class="n">appConfig</span><span class="p">.</span><span class="n">json</span>
+ <span class="mi">1673</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">17</span><span class="o">:</span><span class="mi">58</span> <span class="n">metainfo</span><span class="p">.</span><span class="n">xml</span>
+ <span class="mi">0</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">17</span><span class="o">:</span><span class="mi">54</span> <span class="n">package</span><span class="o">/</span>
+ <span class="mi">0</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">18</span><span class="o">:</span><span class="mo">03</span> <span class="n">package</span><span class="o">/</span><span class="n">files</span><span class="o">/</span>
+ <span class="mi">122880</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">18</span><span class="o">:</span><span class="mo">03</span> <span class="n">package</span><span class="o">/</span><span class="n">files</span><span class="o">/</span><span class="n">jmemcached</span><span class="o">-</span><span class="mf">1.0.0</span><span class="p">.</span><span class="n">tar</span>
+ <span class="mi">0</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">19</span><span class="o">:</span><span class="mi">31</span> <span class="n">package</span><span class="o">/</span><span class="n">scripts</span><span class="o">/</span>
+ <span class="mi">1530</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">19</span><span class="o">:</span><span class="mi">31</span> <span class="n">package</span><span class="o">/</span><span class="n">scripts</span><span class="o">/</span><span class="n">memcached</span><span class="p">.</span><span class="n">py</span>
+ <span class="mi">1287</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">18</span><span class="o">:</span><span class="mi">46</span> <span class="n">package</span><span class="o">/</span><span class="n">scripts</span><span class="o">/</span><span class="n">params</span><span class="p">.</span><span class="n">py</span>
+ <span class="mi">1581</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">19</span><span class="o">:</span><span class="mi">16</span> <span class="n">README</span><span class="p">.</span><span class="n">txt</span>
+ <span class="mi">252</span> <span class="mo">07</span><span class="o">-</span><span class="mi">15</span><span class="o">-</span><span class="mi">14</span> <span class="mi">17</span><span class="o">:</span><span class="mi">58</span> <span class="n">resources</span><span class="p">.</span><span class="n">json</span>
+</pre></div>
+
+
<h2 id="step-1-create-metainfoxml">Step 1: Create metainfo.xml</h2>
<p>The minimal metainfo contains some information about the application (name, comment, version), at least one component type, in this case its MEMCACHED, and information about the tarball. More details are available <a href="/docs/slider_specs/application_definition.html">here</a>.</p>
-<p>```
-<metainfo>
- <schemaVersion>2.0</schemaVersion>
- <application>
- <name>MEMCACHED</name>
- <comment>Memcache is a network accessible key/value storage system, often used as a distributed cache.</comment>
- <version>1.0.0</version></p>
-<div class="codehilite"><pre><span class="nt"><components></span>
- <span class="nt"><component></span>
+<div class="codehilite"><pre><span class="nt"><metainfo></span>
+ <span class="nt"><schemaVersion></span>2.0<span class="nt"></schemaVersion></span>
+ <span class="nt"><application></span>
<span class="nt"><name></span>MEMCACHED<span class="nt"></name></span>
- <span class="nt"><category></span>MASTER<span class="nt"></category></span>
- <span class="nt"><commandScript></span>
- <span class="nt"><script></span>scripts/memcached.py<span class="nt"></script></span>
- <span class="nt"><scriptType></span>PYTHON<span class="nt"></scriptType></span>
- <span class="nt"></commandScript></span>
- <span class="nt"></component></span>
-<span class="nt"></components></span>
-
-<span class="nt"><osSpecifics></span>
- <span class="nt"><osSpecific></span>
- <span class="nt"><osType></span>any<span class="nt"></osType></span>
- <span class="nt"><packages></span>
- <span class="nt"><package></span>
- <span class="nt"><type></span>tarball<span class="nt"></type></span>
- <span class="nt"><name></span>files/jmemcached-1.0.0.tar<span class="nt"></name></span>
- <span class="nt"></package></span>
- <span class="nt"></packages></span>
- <span class="nt"></osSpecific></span>
-<span class="nt"></osSpecifics></span>
+ <span class="nt"><comment></span>Memcache is a network accessible key/value storage system, often used as a distributed cache.<span class="nt"></comment></span>
+ <span class="nt"><version></span>1.0.0<span class="nt"></version></span>
+
+ <span class="nt"><components></span>
+ <span class="nt"><component></span>
+ <span class="nt"><name></span>MEMCACHED<span class="nt"></name></span>
+ <span class="nt"><category></span>MASTER<span class="nt"></category></span>
+ <span class="nt"><commandScript></span>
+ <span class="nt"><script></span>scripts/memcached.py<span class="nt"></script></span>
+ <span class="nt"><scriptType></span>PYTHON<span class="nt"></scriptType></span>
+ <span class="nt"></commandScript></span>
+ <span class="nt"></component></span>
+ <span class="nt"></components></span>
+
+ <span class="nt"><osSpecifics></span>
+ <span class="nt"><osSpecific></span>
+ <span class="nt"><osType></span>any<span class="nt"></osType></span>
+ <span class="nt"><packages></span>
+ <span class="nt"><package></span>
+ <span class="nt"><type></span>tarball<span class="nt"></type></span>
+ <span class="nt"><name></span>files/jmemcached-1.0.0.tar<span class="nt"></name></span>
+ <span class="nt"></package></span>
+ <span class="nt"></packages></span>
+ <span class="nt"></osSpecific></span>
+ <span class="nt"></osSpecifics></span>
+
+ <span class="nt"></application></span>
+<span class="nt"></metainfo></span>
</pre></div>
-<p></application>
-</metainfo>
-``` </p>
<h2 id="step-2-ensure-application-tarball">Step 2: Ensure application tarball</h2>
<p>Most applications release a tarball that you can download otherwise you can create one. For this sample, we created a simple tarball that contains the cli and core jar from jmemcached.</p>
-<p><code>tar tvf jmemcached-1.0.0.tar
-drwxr-xr-x 0 yarn hadoop 0 Jul 15 17:51 jmemcached-1.0.0/
--rw-r--r-- 0 yarn hadoop 13537 Jul 15 17:51 jmemcached-1.0.0/jmemcached-cli-1.0.0.jar
--rwxr-xr-x 0 yarn hadoop 101467 Jul 15 17:51 jmemcached-1.0.0/jmemcached-core-1.0.0.jar</code></p>
+<div class="codehilite"><pre><span class="n">tar</span> <span class="n">tvf</span> <span class="n">jmemcached</span><span class="o">-</span>1<span class="p">.</span>0<span class="p">.</span>0<span class="p">.</span><span class="n">tar</span>
+<span class="n">drwxr</span><span class="o">-</span><span class="n">xr</span><span class="o">-</span><span class="n">x</span> 0 <span class="n">yarn</span> <span class="n">hadoop</span> 0 <span class="n">Jul</span> 15 17<span class="p">:</span>51 <span class="n">jmemcached</span><span class="o">-</span>1<span class="p">.</span>0<span class="p">.</span>0<span class="o">/</span>
+<span class="o">-</span><span class="n">rw</span><span class="o">-</span><span class="n">r</span><span class="o">--</span><span class="n">r</span><span class="o">--</span> 0 <span class="n">yarn</span> <span class="n">hadoop</span> 13537 <span class="n">Jul</span> 15 17<span class="p">:</span>51 <span class="n">jmemcached</span><span class="o">-</span>1<span class="p">.</span>0<span class="p">.</span>0<span class="o">/</span><span class="n">jmemcached</span><span class="o">-</span><span class="n">cli</span><span class="o">-</span>1<span class="p">.</span>0<span class="p">.</span>0<span class="p">.</span><span class="n">jar</span>
+<span class="o">-</span><span class="n">rwxr</span><span class="o">-</span><span class="n">xr</span><span class="o">-</span><span class="n">x</span> 0 <span class="n">yarn</span> <span class="n">hadoop</span> 101467 <span class="n">Jul</span> 15 17<span class="p">:</span>51 <span class="n">jmemcached</span><span class="o">-</span>1<span class="p">.</span>0<span class="p">.</span>0<span class="o">/</span><span class="n">jmemcached</span><span class="o">-</span><span class="n">core</span><span class="o">-</span>1<span class="p">.</span>0<span class="p">.</span>0<span class="p">.</span><span class="n">jar</span>
+</pre></div>
+
+
<h2 id="step-3-create-a-default-resources-file-resourcesjson">Step 3: Create a default resources file (resources.json)</h2>
<p>By default all resources.json file must include slider-appmaster. Add one more entry for the component MEMCACHED and assign a unique priority and default number of instances. Ensure, that a suitable default value is provided for yarn.memory. Additional details are available <a href="/docs/slider_specs/resource_specification.html">here</a>.</p>
-<p><code>{
- "schema" : "http://example.org/specification/v2.0.0",
- "metadata" : {
- },
- "global" : {
- },
- "components": {
- "slider-appmaster": {
- },
- "MEMCACHED": {
- "yarn.role.priority": "1",
- "yarn.component.instances": "1",
- "yarn.memory": "256"
- }
- }
-}</code></p>
+<div class="codehilite"><pre><span class="p">{</span>
+ "<span class="n">schema</span>" <span class="p">:</span> "<span class="n">http</span><span class="p">:</span><span class="o">//</span><span class="n">example</span><span class="p">.</span><span class="n">org</span><span class="o">/</span><span class="n">specification</span><span class="o">/</span><span class="n">v2</span><span class="p">.</span>0<span class="p">.</span>0"<span class="p">,</span>
+ "<span class="n">metadata</span>" <span class="p">:</span> <span class="p">{</span>
+ <span class="p">},</span>
+ "<span class="k">global</span>" <span class="p">:</span> <span class="p">{</span>
+ <span class="p">},</span>
+ "<span class="n">components</span>"<span class="p">:</span> <span class="p">{</span>
+ "<span class="n">slider</span><span class="o">-</span><span class="n">appmaster</span>"<span class="p">:</span> <span class="p">{</span>
+ <span class="p">},</span>
+ "<span class="n">MEMCACHED</span>"<span class="p">:</span> <span class="p">{</span>
+ "<span class="n">yarn</span><span class="p">.</span><span class="n">role</span><span class="p">.</span><span class="n">priority</span>"<span class="p">:</span> "1"<span class="p">,</span>
+ "<span class="n">yarn</span><span class="p">.</span><span class="n">component</span><span class="p">.</span><span class="n">instances</span>"<span class="p">:</span> "1"<span class="p">,</span>
+ "<span class="n">yarn</span><span class="p">.</span><span class="n">memory</span>"<span class="p">:</span> "256"
+ <span class="p">}</span>
+ <span class="p">}</span>
+<span class="p">}</span>
+</pre></div>
+
+
<h2 id="step-4-create-a-default-configuration-template-appconfigjson">Step 4: Create a default configuration template (appConfig.json)</h2>
<p>The config template has few mandatory parameters such as </p>
<ul>
@@ -269,76 +278,87 @@ drwxr-xr-x 0 yarn hadoop 0 Jul 1
<li>site.global.xms_val - value of Xms</li>
<li>site.global.memory_val - value of memory</li>
</ul>
-<p><code>{
- "schema": "http://example.org/specification/v2.0.0",
- "metadata": {
+<p>You can add additional parameters as needed.</p>
+<div class="codehilite"><pre>{
+ "schema": "http://example.org/specification/v2.0.0",
+ "metadata": {
},
- "global": {
- "application.def": "/slider/jmemcached-1.0.0.zip",
- "java_home": "/usr/jdk64/jdk1.7.0_45",
- "package_list": "files/jmemcached-1.0.0.tar",
- "config_types": "",
- "site.global.app_user": "yarn",
- "site.global.app_root": "${AGENT_WORK_ROOT}/app/install/jmemcached-1.0.0",
- "site.global.additional_cp": "/usr/lib/hadoop/lib/*",
- "site.global.xmx_val": "256m",
- "site.global.xms_val": "128m",
- "site.global.memory_val": "200M"
+ "global": {
+ "application.def": "/slider/jmemcached-1.0.0.zip",
+ "java_home": "/usr/jdk64/jdk1.7.0_45",
+ "package_list": "files/jmemcached-1.0.0.tar",
+ "config_types": "",
+ "site.global.app_user": "yarn",
+ "site.global.app_root": "<span class="cp">${</span><span class="n">AGENT_WORK_ROOT</span><span class="cp">}</span>/app/install/jmemcached-1.0.0",
+ "site.global.additional_cp": "/usr/lib/hadoop/lib/*",
+ "site.global.xmx_val": "256m",
+ "site.global.xms_val": "128m",
+ "site.global.memory_val": "200M"
},
- "components": {
- "slider-appmaster": {
- "jvm.heapsize": "256M"
+ "components": {
+ "slider-appmaster": {
+ "jvm.heapsize": "256M"
},
- "MEMCACHED": {
+ "MEMCACHED": {
}
}
-}</code></p>
+}
+</pre></div>
+
+
<p>Additional details on how to define a configuration template is <a href="/docs/slider_specs/application_instance_configuration.html">here</a>.</p>
<h2 id="step-5-implement-the-basic-commands">Step 5: Implement the basic commands</h2>
<p>All Slider applications are expected to implement INSTALL/CONFIGURE/START/STOP/STATUS for each component. Some of the implementations can be NOP - in our case, we will implement only INSTALL and START and leave rest of the code as default. Note, Slider has an extensive library that can be used to implement the commands. More details can be found <a href="/docs/slider_specs/writing_app_command_scripts">here</a>.</p>
<p>The parameters file we will use is:</p>
-<p>```
-from resource_management import *</p>
-<p>config = Script.get_config()</p>
-<p>app_root = config['configurations']['global']['app_root']
-java64_home = config['hostLevelParams']['java_home']
-app_user = config['configurations']['global']['app_user']
-additional_cp = config['configurations']['global']['additional_cp']
-xmx_val = config['configurations']['global']['xmx_val']
-xms_val = config['configurations']['global']['xms_val']
-memory_val = config['configurations']['global']['memory_val']
-```</p>
+<div class="codehilite"><pre><span class="n">from</span> <span class="n">resource_management</span> <span class="n">import</span> <span class="o">*</span>
+
+<span class="n">config</span> <span class="p">=</span> <span class="n">Script</span><span class="p">.</span><span class="n">get_config</span><span class="p">()</span>
+
+<span class="n">app_root</span> <span class="p">=</span> <span class="n">config</span><span class="p">[</span><span class="s">'configurations'</span><span class="p">][</span><span class="s">'global'</span><span class="p">][</span><span class="s">'app_root'</span><span class="p">]</span>
+<span class="n">java64_home</span> <span class="p">=</span> <span class="n">config</span><span class="p">[</span><span class="s">'hostLevelParams'</span><span class="p">][</span><span class="s">'java_home'</span><span class="p">]</span>
+<span class="n">app_user</span> <span class="p">=</span> <span class="n">config</span><span class="p">[</span><span class="s">'configurations'</span><span class="p">][</span><span class="s">'global'</span><span class="p">][</span><span class="s">'app_user'</span><span class="p">]</span>
+<span class="n">additional_cp</span> <span class="p">=</span> <span class="n">config</span><span class="p">[</span><span class="s">'configurations'</span><span class="p">][</span><span class="s">'global'</span><span class="p">][</span><span class="s">'additional_cp'</span><span class="p">]</span>
+<span class="n">xmx_val</span> <span class="p">=</span> <span class="n">config</span><span class="p">[</span><span class="s">'configurations'</span><span class="p">][</span><span class="s">'global'</span><span class="p">][</span><span class="s">'xmx_val'</span><span class="p">]</span>
+<span class="n">xms_val</span> <span class="p">=</span> <span class="n">config</span><span class="p">[</span><span class="s">'configurations'</span><span class="p">][</span><span class="s">'global'</span><span class="p">][</span><span class="s">'xms_val'</span><span class="p">]</span>
+<span class="n">memory_val</span> <span class="p">=</span> <span class="n">config</span><span class="p">[</span><span class="s">'configurations'</span><span class="p">][</span><span class="s">'global'</span><span class="p">][</span><span class="s">'memory_val'</span><span class="p">]</span>
+</pre></div>
+
+
<p>Note that the parameter.py file only reads the parameters needed by the command implementations. </p>
-<p>```
-import sys
-from resource_management import *</p>
-<p>class Memcached(Script):
- def install(self, env):
- self.install_packages(env)</p>
-<p>def configure(self, env):
- import params
- env.set_params(params)</p>
-<p>def start(self, env):
- import params
- env.set_params(params)
- self.configure(env)
- process_cmd = format("{java64_home}/bin/java -Xmx{xmx_val} -Xms{xms_val} -classpath {app_root}/*:{additional_cp} com.thimbleware.jmemcached.Main --memory={memory_val}")</p>
-<div class="codehilite"><pre><span class="n">Execute</span><span class="p">(</span><span class="n">process_cmd</span><span class="p">,</span>
- <span class="n">user</span><span class="p">=</span><span class="n">params</span><span class="p">.</span><span class="n">app_user</span><span class="p">,</span>
- <span class="n">logoutput</span><span class="p">=</span><span class="n">False</span><span class="p">,</span>
- <span class="n">wait_for_finish</span><span class="p">=</span><span class="n">False</span>
-<span class="p">)</span>
+<div class="codehilite"><pre><span class="n">import</span> <span class="n">sys</span>
+<span class="n">from</span> <span class="n">resource_management</span> <span class="n">import</span> <span class="o">*</span>
+
+<span class="n">class</span> <span class="n">Memcached</span><span class="p">(</span><span class="n">Script</span><span class="p">):</span>
+ <span class="n">def</span> <span class="n">install</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">self</span><span class="p">.</span><span class="n">install_packages</span><span class="p">(</span><span class="n">env</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">configure</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">start</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+ <span class="n">self</span><span class="p">.</span><span class="n">configure</span><span class="p">(</span><span class="n">env</span><span class="p">)</span>
+ <span class="n">process_cmd</span> <span class="p">=</span> <span class="n">format</span><span class="p">(</span>"<span class="p">{</span><span class="n">java64_home</span><span class="p">}</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">java</span> <span class="o">-</span><span class="n">Xmx</span><span class="p">{</span><span class="n">xmx_val</span><span class="p">}</span> <span class="o">-</span><span class="n">Xms</span><span class="p">{</span><span class="n">xms_val</span><span class="p">}</span> <span class="o">-</span><span class="n">classpath</span> <span class="p">{</span><span class="n">app_root</span><span class="p">}</span><span class="o">/*</span><span class="p">:{</span><span class="n">additional_cp</span><span class="p">}</span> <span class="n">com</span><span class="p">.</span><span class="n">thimbleware</span><span class="p">.</span><span class="n">jmemcached</span><span class="p">.</span><span class="n">Main
</span> <span class="o">--</span><span class="n">memory</span><span class="p">={</span><span class="n">memory_val</span><span class="p">}</span>"<span class="p">)</span>
+
+ <span class="n">Execute</span><span class="p">(</span><span class="n">process_cmd</span><span class="p">,</span>
+ <span class="n">user</span><span class="p">=</span><span class="n">params</span><span class="p">.</span><span class="n">app_user</span><span class="p">,</span>
+ <span class="n">logoutput</span><span class="p">=</span><span class="n">False</span><span class="p">,</span>
+ <span class="n">wait_for_finish</span><span class="p">=</span><span class="n">False</span>
+ <span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">stop</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">status</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
</pre></div>
-<p>def stop(self, env):
- import params
- env.set_params(params)</p>
-<p>def status(self, env):
- import params
- env.set_params(params)
-```
-Thats pretty much it. The script basically does the following:</p>
+<p>Thats pretty much it. The script basically does the following:</p>
<ul>
<li>Expand the given tarball</li>
<li>Reads the provided configuration and creates the command string</li>
Modified: websites/staging/slider/trunk/content/docs/slider_specs/writing_app_command_scripts.html
==============================================================================
--- websites/staging/slider/trunk/content/docs/slider_specs/writing_app_command_scripts.html (original)
+++ websites/staging/slider/trunk/content/docs/slider_specs/writing_app_command_scripts.html Thu Jul 17 06:54:00 2014
@@ -226,154 +226,158 @@ Latest release: <strong>0.30-incubating<
<h3 id="sample-configuration-parameters">Sample configuration parameters</h3>
-<p>While a script can access all the config parameters the key property bags are</p>
-<ul>
-<li>hostLevelParams</li>
-<li>configurations</li>
-</ul>
-<p><code>{
- "roleCommand": "START",
- "commandParams": {
- "record_config": "true",
- "service_package_folder": "${AGENT_WORK_ROOT}/work/app/definition/package",
- "script": "scripts/hbase_regionserver.py",
- "schema_version": "2.0",
- "command_timeout": "300",
- "script_type": "PYTHON"
+<p>While a script can access all the config parameters the key property bags are <strong>hostLevelParams</strong> and <strong>configurations</strong>.</p>
+<div class="codehilite"><pre>{
+ "roleCommand": "START",
+ "commandParams": {
+ "record_config": "true",
+ "service_package_folder": "<span class="cp">${</span><span class="n">AGENT_WORK_ROOT</span><span class="cp">}</span>/work/app/definition/package",
+ "script": "scripts/hbase_regionserver.py",
+ "schema_version": "2.0",
+ "command_timeout": "300",
+ "script_type": "PYTHON"
},
- "componentName": "HBASE_REGIONSERVER",
- "hostname": "c6403.ambari.apache.org",
- "hostLevelParams": {
- "java_home": "/usr/jdk64/jdk1.7.0_45",
- "container_id": "container_1405048900371_0015_02_000003"
+ "componentName": "HBASE_REGIONSERVER",
+ "hostname": "c6403.ambari.apache.org",
+ "hostLevelParams": {
+ "java_home": "/usr/jdk64/jdk1.7.0_45",
+ "container_id": "container_1405048900371_0015_02_000003"
},
- "commandType": "EXECUTION_COMMAND",
- "clusterName": "cl1",
- "serviceName": "cl1",
- "role": "HBASE_REGIONSERVER",
- "taskId": 4,
- "public_hostname": "c6403.ambari.apache.org",
- "commandId": "4-1",
- "configurations": {
- "hbase-site": {
- "hbase.hstore.flush.retries.number": "120",
- "hbase.client.keyvalue.maxsize": "10485760",
- "hbase.hstore.compactionThreshold": "3",
- "hbase.rootdir": "hdfs://c6403.ambari.apache.org:8020/user/yarn/.slider/cluster/cl1/database/data",
- "hbase.regionserver.handler.count": "60",
- "hbase.regionserver.global.memstore.lowerLimit": "0.38",
- "hbase.hregion.memstore.block.multiplier": "2",
- "hbase.hregion.memstore.flush.size": "134217728",
- "hbase.superuser": "yarn",
- "hbase.zookeeper.property.clientPort": "2181",
- "hbase.regionserver.global.memstore.upperLimit": "0.4",
- "zookeeper.session.timeout": "30000",
- "hbase.tmp.dir": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/work/app/tmp",
- "hfile.block.cache.size": "0.40",
- "hbase.stagingdir": "hdfs://c6403.ambari.apache.org:8020/user/yarn/.slider/cluster/cl1/database/staging",
- "hbase.hregion.max.filesize": "10737418240",
- "hbase.regionserver.port": "0",
- "hbase.security.authentication": "simple",
- "hbase.defaults.for.version.skip": "true",
- "hbase.master.info.port": "52502",
- "hbase.zookeeper.quorum": "c6403.ambari.apache.org",
- "hbase.regionserver.info.port": "0",
- "zookeeper.znode.parent": "/services/slider/users/yarn/cl1",
- "hbase.hstore.blockingStoreFiles": "10",
- "hbase.hregion.majorcompaction": "86400000",
- "hbase.security.authorization": "false",
- "hbase.local.dir": "${hbase.tmp.dir}/local",
- "hbase.cluster.distributed": "true",
- "hbase.hregion.memstore.mslab.enabled": "true",
- "hbase.client.scanner.caching": "100",
- "hbase.zookeeper.useMulti": "true"
+ "commandType": "EXECUTION_COMMAND",
+ "clusterName": "cl1",
+ "serviceName": "cl1",
+ "role": "HBASE_REGIONSERVER",
+ "taskId": 4,
+ "public_hostname": "c6403.ambari.apache.org",
+ "commandId": "4-1",
+ "configurations": {
+ "hbase-site": {
+ "hbase.hstore.flush.retries.number": "120",
+ "hbase.client.keyvalue.maxsize": "10485760",
+ "hbase.hstore.compactionThreshold": "3",
+ "hbase.rootdir": "hdfs://c6403.ambari.apache.org:8020/user/yarn/.slider/cluster/cl1/database/data",
+ "hbase.regionserver.handler.count": "60",
+ "hbase.regionserver.global.memstore.lowerLimit": "0.38",
+ "hbase.hregion.memstore.block.multiplier": "2",
+ "hbase.hregion.memstore.flush.size": "134217728",
+ "hbase.superuser": "yarn",
+ "hbase.zookeeper.property.clientPort": "2181",
+ "hbase.regionserver.global.memstore.upperLimit": "0.4",
+ "zookeeper.session.timeout": "30000",
+ "hbase.tmp.dir": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/work/app/tmp",
+ "hfile.block.cache.size": "0.40",
+ "hbase.stagingdir": "hdfs://c6403.ambari.apache.org:8020/user/yarn/.slider/cluster/cl1/database/staging",
+ "hbase.hregion.max.filesize": "10737418240",
+ "hbase.regionserver.port": "0",
+ "hbase.security.authentication": "simple",
+ "hbase.defaults.for.version.skip": "true",
+ "hbase.master.info.port": "52502",
+ "hbase.zookeeper.quorum": "c6403.ambari.apache.org",
+ "hbase.regionserver.info.port": "0",
+ "zookeeper.znode.parent": "/services/slider/users/yarn/cl1",
+ "hbase.hstore.blockingStoreFiles": "10",
+ "hbase.hregion.majorcompaction": "86400000",
+ "hbase.security.authorization": "false",
+ "hbase.local.dir": "<span class="cp">${</span><span class="n">hbase</span><span class="o">.</span><span class="n">tmp</span><span class="o">.</span><span class="n">dir</span><span class="cp">}</span>/local",
+ "hbase.cluster.distributed": "true",
+ "hbase.hregion.memstore.mslab.enabled": "true",
+ "hbase.client.scanner.caching": "100",
+ "hbase.zookeeper.useMulti": "true"
},
- "global": {
- "security_enabled": "false",
- "ganglia_server_port": "8667",
- "hbase_master_heapsize": "1024m",
- "app_root": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/install/hbase-0.98.3-hadoop2",
- "app_log_dir": "/hadoop/yarn/log/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/log",
- "ganglia_server_host": "c6403.ambari.apache.org",
- "ganglia_server_id": "Application1",
- "hbase_regionserver_heapsize": "1024m",
- "user_group": "hadoop",
- "app_pid_dir": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/run",
- "app_install_dir": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/install",
- "app_user": "yarn"
+ "global": {
+ "security_enabled": "false",
+ "ganglia_server_port": "8667",
+ "hbase_master_heapsize": "1024m",
+ "app_root": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/install/hbase-0.98.3-hadoop2",
+ "app_log_dir": "/hadoop/yarn/log/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/log",
+ "ganglia_server_host": "c6403.ambari.apache.org",
+ "ganglia_server_id": "Application1",
+ "hbase_regionserver_heapsize": "1024m",
+ "user_group": "hadoop",
+ "app_pid_dir": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/run",
+ "app_install_dir": "/hadoop/yarn/local/usercache/yarn/appcache/application_1405048900371_0015/container_1405048900371_0015_02_000003/app/install",
+ "app_user": "yarn"
},
- "hdfs-site": {},
- "core-site": {}
+ "hdfs-site": {},
+ "core-site": {}
}
-}</code></p>
+}
+</pre></div>
+
+
<h2 id="command-scripts">Command scripts</h2>
<p>A command script essentially implements commands. The basic structure of a script is as follows:</p>
-<p>```
-class HbaseMaster(Script):
- def install(self, env):
- ...</p>
-<p>def configure(self, env):
- ...</p>
-<p>def start(self, env):
- ...</p>
-<p>def stop(self, env):
- ...</p>
-<p>def status(self, env):
- ...
-```
-At this point, you are free to implement as you wish. Slider provides a rich library for basic operations needed for application commands. So the structure we suggest is:</p>
+<div class="codehilite"><pre><span class="n">class</span> <span class="n">HbaseMaster</span><span class="p">(</span><span class="n">Script</span><span class="p">):</span>
+ <span class="n">def</span> <span class="n">install</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="p">...</span>
+
+ <span class="n">def</span> <span class="n">configure</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="p">...</span>
+
+ <span class="n">def</span> <span class="n">start</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="p">...</span>
+
+ <span class="n">def</span> <span class="n">stop</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="p">...</span>
+
+ <span class="n">def</span> <span class="n">status</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="p">...</span>
+</pre></div>
+
+
+<p>At this point, you are free to implement as you wish. Slider provides a rich library for basic operations needed for application commands. So the structure we suggest is:</p>
<ul>
<li>Create a params.py file to read the json config file into variables. This will allow you to provide initial validation, provide default values, or create complex values</li>
<li>Use the above parameters in your scripts</li>
</ul>
<p>A params.py file may be as follows:</p>
-<p>```
-from resource_management import *</p>
-<h1 id="server-configurations">server configurations</h1>
-<p>config = Script.get_config()</p>
-<p>hbase_root = config['configurations']['global']['app_root']
-conf_dir = format("{hbase_root}/conf")
-daemon_script = format("{hbase_root}/bin/hbase-daemon.sh")
-...
-```</p>
-<h3 id="sample-command-script">Sample command script</h3>
-<p>In this example, hbase master command implementation is using the above parameter file and calling into several helper functions. Some are provided by Slider (e.g. install_package, check_process_status) and some are developed for HBase (e.g. habse_service called by start)</p>
-<p>```
-class HbaseMaster(Script):
- def install(self, env):
- self.install_packages(env)</p>
-<p>def configure(self, env):
- import params
- env.set_params(params)</p>
-<div class="codehilite"><pre><span class="n">hbase</span><span class="p">(</span><span class="n">name</span><span class="p">=</span><span class="s">'master'</span><span class="p">)</span>
-</pre></div>
+<div class="codehilite"><pre><span class="n">from</span> <span class="n">resource_management</span> <span class="n">import</span> <span class="o">*</span>
+# <span class="n">server</span> <span class="n">configurations</span>
+<span class="n">config</span> <span class="p">=</span> <span class="n">Script</span><span class="p">.</span><span class="n">get_config</span><span class="p">()</span>
-<p>def start(self, env):
- import params
- env.set_params(params)
- self.configure(env) # for security</p>
-<div class="codehilite"><pre><span class="n">hbase_service</span><span class="p">(</span> <span class="s">'master'</span><span class="p">,</span>
- <span class="n">action</span> <span class="p">=</span> <span class="s">'start'</span>
-<span class="p">)</span>
+<span class="n">hbase_root</span> <span class="p">=</span> <span class="n">config</span><span class="p">[</span><span class="s">'configurations'</span><span class="p">][</span><span class="s">'global'</span><span class="p">][</span><span class="s">'app_root'</span><span class="p">]</span>
+<span class="n">conf_dir</span> <span class="p">=</span> <span class="n">format</span><span class="p">(</span>"<span class="p">{</span><span class="n">hbase_root</span><span class="p">}</span><span class="o">/</span><span class="n">conf</span>"<span class="p">)</span>
+<span class="n">daemon_script</span> <span class="p">=</span> <span class="n">format</span><span class="p">(</span>"<span class="p">{</span><span class="n">hbase_root</span><span class="p">}</span><span class="o">/</span><span class="n">bin</span><span class="o">/</span><span class="n">hbase</span><span class="o">-</span><span class="n">daemon</span><span class="p">.</span><span class="n">sh</span>"<span class="p">)</span>
+<span class="p">...</span>
</pre></div>
-<p>def stop(self, env):
- import params
- env.set_params(params)</p>
-<div class="codehilite"><pre><span class="n">hbase_service</span><span class="p">(</span> <span class="s">'master'</span><span class="p">,</span>
- <span class="n">action</span> <span class="p">=</span> <span class="s">'stop'</span>
-<span class="p">)</span>
+<h3 id="sample-command-script">Sample command script</h3>
+<p>In this example, hbase master command implementation is using the above parameter file and calling into several helper functions. Some are provided by Slider (e.g. install_package, check_process_status) and some are developed for HBase (e.g. habse_service called by start)</p>
+<div class="codehilite"><pre><span class="n">class</span> <span class="n">HbaseMaster</span><span class="p">(</span><span class="n">Script</span><span class="p">):</span>
+ <span class="n">def</span> <span class="n">install</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">self</span><span class="p">.</span><span class="n">install_packages</span><span class="p">(</span><span class="n">env</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">configure</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+
+ <span class="n">hbase</span><span class="p">(</span><span class="n">name</span><span class="p">=</span><span class="s">'master'</span><span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">start</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+ <span class="n">self</span><span class="p">.</span><span class="n">configure</span><span class="p">(</span><span class="n">env</span><span class="p">)</span> # <span class="k">for</span> <span class="n">security</span>
+
+ <span class="n">hbase_service</span><span class="p">(</span> <span class="s">'master'</span><span class="p">,</span>
+ <span class="n">action</span> <span class="p">=</span> <span class="s">'start'</span>
+ <span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">stop</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">import</span> <span class="n">params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">params</span><span class="p">)</span>
+
+ <span class="n">hbase_service</span><span class="p">(</span> <span class="s">'master'</span><span class="p">,</span>
+ <span class="n">action</span> <span class="p">=</span> <span class="s">'stop'</span>
+ <span class="p">)</span>
+
+ <span class="n">def</span> <span class="n">status</span><span class="p">(</span><span class="n">self</span><span class="p">,</span> <span class="n">env</span><span class="p">):</span>
+ <span class="n">import</span> <span class="n">status_params</span>
+ <span class="n">env</span><span class="p">.</span><span class="n">set_params</span><span class="p">(</span><span class="n">status_params</span><span class="p">)</span>
+ <span class="n">pid_file</span> <span class="p">=</span> <span class="n">format</span><span class="p">(</span>"<span class="p">{</span><span class="n">pid_dir</span><span class="p">}</span><span class="o">/</span><span class="n">hbase</span><span class="o">-</span><span class="p">{</span><span class="n">hbase_user</span><span class="p">}</span><span class="o">-</span><span class="n">master</span><span class="p">.</span><span class="n">pid</span>"<span class="p">)</span>
+ <span class="n">check_process_status</span><span class="p">(</span><span class="n">pid_file</span><span class="p">)</span>
</pre></div>
-
-
-<p>def status(self, env):
- import status_params
- env.set_params(status_params)
- pid_file = format("{pid_dir}/hbase-{hbase_user}-master.pid")
- check_process_status(pid_file)
-```</p>
</div>
<div id="footer">