You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by sm...@apache.org on 2014/05/20 06:59:17 UTC
svn commit: r1596111 - in /incubator/slider/trunk:
app-packages/accumulo-v1_5/ src/site/markdown/slider_specs/
Author: smohanty
Date: Tue May 20 04:59:17 2014
New Revision: 1596111
URL: http://svn.apache.org/r1596111
Log:
SLIDER 73. Accumulo package description is too long
Modified:
incubator/slider/trunk/app-packages/accumulo-v1_5/appConfig.json
incubator/slider/trunk/app-packages/accumulo-v1_5/metainfo.xml
incubator/slider/trunk/src/site/markdown/slider_specs/application_configuration.md
incubator/slider/trunk/src/site/markdown/slider_specs/application_package.md
incubator/slider/trunk/src/site/markdown/slider_specs/creating_app_definitions.md
Modified: incubator/slider/trunk/app-packages/accumulo-v1_5/appConfig.json
URL: http://svn.apache.org/viewvc/incubator/slider/trunk/app-packages/accumulo-v1_5/appConfig.json?rev=1596111&r1=1596110&r2=1596111&view=diff
==============================================================================
--- incubator/slider/trunk/app-packages/accumulo-v1_5/appConfig.json (original)
+++ incubator/slider/trunk/app-packages/accumulo-v1_5/appConfig.json Tue May 20 04:59:17 2014
@@ -45,22 +45,17 @@
},
"components": {
"ACCUMULO_MASTER": {
- "wait.heartbeat": "3"
},
"slider-appmaster": {
"jvm.heapsize": "256M"
},
"ACCUMULO_TSERVER": {
- "wait.heartbeat": "6"
},
"ACCUMULO_MONITOR": {
- "wait.heartbeat": "8"
},
"ACCUMULO_GC": {
- "wait.heartbeat": "6"
},
"ACCUMULO_TRACER": {
- "wait.heartbeat": "8"
}
}
}
Modified: incubator/slider/trunk/app-packages/accumulo-v1_5/metainfo.xml
URL: http://svn.apache.org/viewvc/incubator/slider/trunk/app-packages/accumulo-v1_5/metainfo.xml?rev=1596111&r1=1596110&r2=1596111&view=diff
==============================================================================
--- incubator/slider/trunk/app-packages/accumulo-v1_5/metainfo.xml (original)
+++ incubator/slider/trunk/app-packages/accumulo-v1_5/metainfo.xml Tue May 20 04:59:17 2014
@@ -21,13 +21,27 @@
<service>
<name>ACCUMULO</name>
<comment>
- The Apache Accumulo sorted, distributed key/value store is a robust,
- scalable, high performance data storage system that features cell-based
- access control and customizable server-side processing. It is based on
- Google's BigTable design and is built on top of Apache Hadoop,
- Zookeeper, and Thrift.
+ The Apache Accumulo is a robust, scalable, high performance data storage system.
</comment>
<version>1.5.1</version>
+ <commandOrders>
+ <commandOrder>
+ <command>ACCUMULO_TSERVER-START</command>
+ <requires>ACCUMULO_MASTER-STARTED</requires>
+ </commandOrder>
+ <commandOrder>
+ <command>ACCUMULO_MONITOR-START</command>
+ <requires>ACCUMULO_TSERVER-STARTED</requires>
+ </commandOrder>
+ <commandOrder>
+ <command>ACCUMULO_GC-START</command>
+ <requires>ACCUMULO_TSERVER-STARTED</requires>
+ </commandOrder>
+ <commandOrder>
+ <command>ACCUMULO_TRACER-START</command>
+ <requires>ACCUMULO_TSERVER-STARTED</requires>
+ </commandOrder>
+ </commandOrders>
<components>
<component>
<name>ACCUMULO_MASTER</name>
Modified: incubator/slider/trunk/src/site/markdown/slider_specs/application_configuration.md
URL: http://svn.apache.org/viewvc/incubator/slider/trunk/src/site/markdown/slider_specs/application_configuration.md?rev=1596111&r1=1596110&r2=1596111&view=diff
==============================================================================
--- incubator/slider/trunk/src/site/markdown/slider_specs/application_configuration.md (original)
+++ incubator/slider/trunk/src/site/markdown/slider_specs/application_configuration.md Tue May 20 04:59:17 2014
@@ -23,7 +23,7 @@ App Configuration is a folder containing
In addition, application may have configuration parameters that do not necessarily go into a config files. Such configurations may be used during template expansion (parameters in env.sh files), as environment variables (e.g. JAVA_HOME), customize user names (for runas). These configurations can be provided as user inputs or are automatically inferred from the environment. Such configurations are stored in a file named "app_config.xml".
-
+
A config file is of the form:
Modified: incubator/slider/trunk/src/site/markdown/slider_specs/application_package.md
URL: http://svn.apache.org/viewvc/incubator/slider/trunk/src/site/markdown/slider_specs/application_package.md?rev=1596111&r1=1596110&r2=1596111&view=diff
==============================================================================
--- incubator/slider/trunk/src/site/markdown/slider_specs/application_package.md (original)
+++ incubator/slider/trunk/src/site/markdown/slider_specs/application_package.md Tue May 20 04:59:17 2014
@@ -21,7 +21,7 @@ All application artifacts, app definitio
## Overall Structure
-App package is a tarball containing all application artifacts. App package contains the following items:
+App package is a zip package containing all application artifacts. App package contains the following items:
* **app definition file**
application structure, content, definition, supported platforms, version, etc.
@@ -42,7 +42,7 @@ various templates used by the applicatio
other scripts, txt files, tarballs, etc.
-
+
The example above shows a semi-expanded view of an application "HBASE-YARN-APP" and the package structure for OOZIE command scripts.
Modified: incubator/slider/trunk/src/site/markdown/slider_specs/creating_app_definitions.md
URL: http://svn.apache.org/viewvc/incubator/slider/trunk/src/site/markdown/slider_specs/creating_app_definitions.md?rev=1596111&r1=1596110&r2=1596111&view=diff
==============================================================================
--- incubator/slider/trunk/src/site/markdown/slider_specs/creating_app_definitions.md (original)
+++ incubator/slider/trunk/src/site/markdown/slider_specs/creating_app_definitions.md Tue May 20 04:59:17 2014
@@ -21,7 +21,7 @@ Slider AppPackages are a declarative def
An application instance consists of several active component such as one or more masters and several slaves. There may be a number of accompanying processes in addition to the basic master/slave processes - lets refer to all processes as app component instances. When run in the context of Yarn, the application specific processes are activated within individual Yarn Container. If you pry into an Yarn container (created through Slider) it will be apparent as to what is the role of Slider-Agent and the actual application components. The following image provides an high-level view. Within a container there are at least two processes - and instance of a slider agent and an instance of an application component. The application can itself spawn more procsses if needed.
-
+
Figure 1 - High-level view of a container
@@ -36,7 +36,7 @@ Shows three processes, the Slider-Agent
## Using an AppPackage
The following command creates an HBase application using the AppPackage for HBase.
- ./slider create cl1 --zkhosts zk1,zk2 --image hdfs://NN:8020/slider/agent/slider-agent-0.21.tar --option agent.conf hdfs://NN:8020/slider/agent/conf/agent.ini --template /work/appConf.json --resources /work/resources.json --option application.def hdfs://NN:8020/slider/hbase_v096.tar
+ ./slider create cl1 --zkhosts zk1,zk2 --image hdfs://NN:8020/slider/agent/slider-agent-0.21.tar --option agent.conf hdfs://NN:8020/slider/agent/conf/agent.ini --template /work/appConf.json --resources /work/resources.json --option application.def hdfs://NN:8020/slider/hbase_v096.zip
Lets analyze various parameters from the perspective of app creation:
@@ -56,13 +56,59 @@ In the enlistment there are three exampl
* `app-packages/accumulo-v1_5`
* `app-packages/storm-v0_91`
-The application tarball, containing the binaries/artifacts of the application itself is a component within the AppPackage. They are:
+The application zip file, containing the binaries/artifacts of the application itself is a component within the AppPackage. They are:
* For hbase - `app-packages/hbase-v0_96/package/files/hbase-0.96.1-hadoop2-bin.tar.gz.REPLACE`
* For accumulo - `app-packages/accumulo-v1_5/package/files/accumulo-1.5.1-bin.tar.gz.REPLACE`
* For storm - `app-packages/storm-v0_91/package/files/apache-storm-0.9.1.2.1.1.0-237.tar.gz.placeholder`
-They are placehoder files, mostly because the files themselves are too large as well as users are free to use their own version of the package. To create a Slider AppPackage - replace the file with an actual application tarball and then ensure that the metainfo.xml has the correct file name. After that create a tarball using standard tar commands and ensure that the package has the metainfo.xml file at the root folder.
+**They are placehoder files**, mostly because the files themselves are too large as well as users are free to use their own version of the package. To create a Slider AppPackage - replace the file with an actual application tarball and then ensure that the metainfo.xml has the correct file name. After that create a zip file using standard zip commands and ensure that the package has the metainfo.xml file at the root folder.
+
+Sample **resources.json** and **appConfig.json** files are also included in the enlistment.
+
+For example:
+
+* cd slider/app-packages/hbase-v0_96
+* zip -r hbase_v096.zip .
+* Looking at the content through unzip -l "$@" hbase_v096.zip
+
+```
+Archive: hbase_v096.zip
+ Length Date Time Name
+ -------- ---- ---- ----
+ 3163 05-16-14 16:32 appConfig.json
+ 0 05-02-14 07:51 configuration/
+ 5077 05-02-14 07:51 configuration/global.xml
+ 5248 05-02-14 07:51 configuration/hbase-log4j.xml
+ 2250 05-02-14 07:51 configuration/hbase-policy.xml
+ 14705 05-02-14 07:51 configuration/hbase-site.xml
+ 3332 05-16-14 16:33 metainfo.xml
+ 0 05-02-14 07:51 package/
+ 0 05-19-14 20:36 package/files/
+ 83154798 05-19-14 20:36 package/files/hbase-0.96.1-hadoop2-bin.tar.gz
+ 0 05-02-14 07:51 package/scripts/
+ 787 05-02-14 07:51 package/scripts/__init__.py
+ 1378 05-02-14 07:51 package/scripts/functions.py
+ 3599 05-02-14 07:51 package/scripts/hbase.py
+ 1205 05-02-14 07:51 package/scripts/hbase_client.py
+ 1640 05-02-14 07:51 package/scripts/hbase_master.py
+ 1764 05-02-14 07:51 package/scripts/hbase_regionserver.py
+ 1482 05-02-14 07:51 package/scripts/hbase_service.py
+ 4924 05-02-14 07:51 package/scripts/params.py
+ 973 05-02-14 07:51 package/scripts/status_params.py
+ 0 05-02-14 07:51 package/templates/
+ 2723 05-02-14 07:51 package/templates/hadoop-metrics2-hbase.properties-GANGLIA-MASTER.j2
+ 2723 05-02-14 07:51 package/templates/hadoop-metrics2-hbase.properties-GANGLIA-RS.j2
+ 3878 05-02-14 07:51 package/templates/hbase-env.sh.j2
+ 909 05-02-14 07:51 package/templates/hbase_client_jaas.conf.j2
+ 989 05-02-14 07:51 package/templates/hbase_master_jaas.conf.j2
+ 1001 05-02-14 07:51 package/templates/hbase_regionserver_jaas.conf.j2
+ 837 05-02-14 07:51 package/templates/regionservers.j2
+ 357 05-12-14 12:04 resources.json
+ -------- -------
+ 83219742 29 files
+```
+
### appConf.json
An appConf.json contains the application configuration. The sample below shows configuration for HBase.