You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by go...@apache.org on 2015/05/22 20:31:03 UTC
svn commit: r1681180 -
/incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md
Author: gourksaha
Date: Fri May 22 18:31:02 2015
New Revision: 1681180
URL: http://svn.apache.org/r1681180
Log:
SLIDER-245 Update sample app jmemcached documentation with sample add-on package app
Modified:
incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md
Modified: incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md
URL: http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md?rev=1681180&r1=1681179&r2=1681180&view=diff
==============================================================================
--- incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md (original)
+++ incubator/slider/site/trunk/content/docs/slider_specs/hello_world_slider_app.md Fri May 22 18:31:02 2015
@@ -225,8 +225,8 @@ Note that the `parameter.py` file only r
env.set_params(params)
check_process_status(params.pid_file)
- if __name__ == "__main__":
- Memcached().execute()
+ if __name__ == "__main__":
+ Memcached().execute()
Thats pretty much it. The script basically does the following:
@@ -246,4 +246,182 @@ which, expands to
port and memory are based on the input provided. And, class path includes YARN container location for the active container where the tarball got expanded.
-So, without Slider/YARN, one should be able to execute the command above and see memcached up and running. It might be a better option to try the command first to ensure that you have the right tarballs and environment to run memcached.
\ No newline at end of file
+So, without Slider/YARN, one should be able to execute the command above and see memcached up and running. It might be a better option to try the command first to ensure that you have the right tarballs and environment to run memcached.
+
+## Add on package
+
+You can deploy the application with add on packages as well. Add on packages are extension libraries, configurations, or scripts that the master application can use. For example, HBase has Phoenix providing SQL access to it and Ranger providing Authorization added to it. With Slider, you can deploy HBase as the master application, and Phoenix and/or Ranger as the add on package to it
+
+To do that, you need to package your add on packages in a similar way to the master package. For example, in order to deploy Phoenix with HBase, you need to create an add on package as below:
+
+```
+unzip -l "$@" Phoenix.zip
+Archive: Archive.zip
+ Length Date Time Name
+ -------- ---- ---- ----
+ 2143 04-19-15 12:03 metainfo.xml
+ 0 11-19-14 15:17 package/
+ 0 04-19-15 11:41 package/files/
+ 1908 12-04-14 17:07 package/files/end2endTest.py
+ 840 12-04-14 17:07 package/files/hadoop-metrics2-hbase.properties
+ 2271 12-04-14 17:07 package/files/hadoop-metrics2-phoenix.properties
+ 1136 12-04-14 17:07 package/files/hbase-site.xml
+ 2342 12-04-14 17:07 package/files/log4j.properties
+ 3762 12-04-14 17:07 package/files/performance.py
+ 38015961 12-04-14 17:07 package/files/phoenix-4.2.2-client.jar
+ 4039324 12-04-14 17:07 package/files/phoenix-4.2.2-server.jar
+ 3171 12-04-14 17:07 package/files/phoenix_utils.py
+ 1669 12-04-14 17:07 package/files/psql.py
+ 1820 12-04-14 17:07 package/files/readme.txt
+ 2314 12-04-14 17:07 package/files/sqlline.py
+ 0 04-19-15 21:57 package/scripts/
+ 2287 04-19-15 21:57 package/scripts/addon_hbase_master.py
+ 1659 04-19-15 11:45 package/scripts/addon_hbase_regionserver.py
+ 1079 04-18-15 17:36 package/scripts/params.py
+ 0 04-17-15 18:42 package/templates/
+ -------- -------
+ 42106790 21 files
+```
+
+Please note that different from the master package, the add on package doesn't have appConfig.json and resources.json, because appConfig.json is not supported for now, while you can define all the variables you need in the master package's appConfig.json.
+For add on package, you should/can not add any new components to the master application, so resources.json is not needed either.
+
+What is remaining is the metainfo.xml, in which you should/can not add any new components. However, you can apply some components in the add on package's metainfo.json belonging to all components in the master package's metainfo.json by specifying 'ALL' in the component name
+
+```
+<metainfo>
+ <schemaVersion>2.0</schemaVersion>
+ <applicationPackage>
+ <name>PHOENIX</name>
+ <comment>
+ Apache Phoenix is a relational database layer over HBase delivered as a client-embedded JDBC driver targeting low latency queries over HBase data.
+ </comment>
+ <version>2.0</version>
+ <type>ADDON-PACKAGE</type>
+ <minHadoopVersion>1.0</minHadoopVersion>
+
+ <components>
+ <component>
+ <name>HBASE_REGIONSERVER</name>
+ <commandScript>
+ <script>scripts/addon_hbase_regionserver.py</script>
+ <scriptType>PYTHON</scriptType>
+ <timeout>600</timeout>
+ </commandScript>
+ </component>
+ <component>
+ <name>HBASE_MASTER</name>
+ <commandScript>
+ <script>scripts/addon_hbase_master.py</script>
+ <scriptType>PYTHON</scriptType>
+ <timeout>600</timeout>
+ </commandScript>
+ </component>
+ </components>
+
+ <osSpecifics>
+ <osSpecific>
+ <osType>any</osType>
+ <packages>
+ <package>
+ <type>tarball</type>
+ <name>files/phoenix-4.2.2-server.jar</name>
+ </package>
+ </packages>
+ </osSpecific>
+ </osSpecifics>
+ </applicationPackage>
+</metainfo>
+```
+
+Please note the type of the application package is specified as 'ADDON-PACKAGE'.
+
+The add on package is expected to implement INSTALL for each component, without other commands defined for the master application. Note, you can still use Slider's extensive library that can be used to implement the commands. More details can be found [here](/docs/slider_specs/writing_app_command_scripts).
+
+Below is an example of the python script for HbaseMaster:
+```
+import sys
+import os
+from shutil import copyfile
+from resource_management import *
+
+
+class HbaseMaster(Script):
+ def install(self, env):
+ config = Script.get_config()
+
+ src = config['commandParams']['addonPackageRoot'] + "/package/files/phoenix-4.2.2-client.jar"
+ dst = config['configurations']['global']['app_root'] + "/lib/phoenix-4.2.2-client.jar"
+ copyfile(src, dst)
+
+ filestocopytobin = ["end2endTest.py","hadoop-metrics2-hbase.properties","hadoop-metrics2-phoenix.properties","hbase-site.xml","log4j.properties","performance.py","phoenix_utils.py","psql.py","readme.txt","sqlline.py"]
+
+ for file in filestocopytobin:
+ src = config['commandParams']['addonPackageRoot'] + "/package/files/" + file
+ dst = config['configurations']['global']['app_root'] + "/bin/" + file
+ copyfile(src, dst)
+ bin_file = config['configurations']['global']['app_root'] + "/bin/" + "sqlline.py"
+ os.chmod(bin_file, 0555)
+
+if __name__ == "__main__":
+ HbaseMaster().execute()
+ pass
+```
+
+Below is an example of the python script for HbaseRegionServer:
+```
+import sys
+from shutil import copyfile
+from resource_management import *
+
+
+class HbaseRegionserver(Script):
+ def install(self, env):
+ config = Script.get_config()
+ src = config['commandParams']['addonPackageRoot'] + "/package/files/phoenix-4.2.2-server.jar"
+ dst = config['configurations']['global']['app_root'] + "/lib/phoenix-4.2.2-server.jar"
+ copyfile(src, dst)
+
+ def configure(self, env):
+ import params
+ env.set_params(params)
+
+
+ def start(self, env):
+ import params
+ env.set_params(params)
+ self.configure(env) # for security
+
+
+ def stop(self, env):
+ import params
+ env.set_params(params)
+
+
+ def status(self, env):
+ import status_params
+ env.set_params(status_params)
+
+if __name__ == "__main__":
+ HbaseRegionserver().execute()
+ pass
+```
+
+Similarly, you can provide params.py to provide all configuration needed to run the install command
+
+```
+from resource_management import *
+
+# server configurations
+config = Script.get_config()
+
+hbase_root = config['configurations']['global']['app_root']
+jar_location = config['commandParams']['addonPackageRoot'] + "/package/files/phoenix-4.2.2-server.jar"
+
+print ('jar_location' + jar_location)
+```
+
+When submitting an application with add on package with Slider, you need to use '--addon' option to specify the add on package name and path to its zipped package
+```
+slider create [application_name] --template [path to appConfig.json] --resources [path to resources.json] --addon PHOENIX [path to Phoenix.zip]
+```