You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by sm...@apache.org on 2014/10/09 03:03:03 UTC
svn commit: r1630264 - in /incubator/slider/site/trunk/content/docs:
getting_started.md getting_started_40.md manpage.md
slider_specs/resource_specification.md
Author: smohanty
Date: Thu Oct 9 01:03:03 2014
New Revision: 1630264
URL: http://svn.apache.org/r1630264
Log:
Doc update for latest develop branch and some corrections
Added:
incubator/slider/site/trunk/content/docs/getting_started_40.md
Modified:
incubator/slider/site/trunk/content/docs/getting_started.md
incubator/slider/site/trunk/content/docs/manpage.md
incubator/slider/site/trunk/content/docs/slider_specs/resource_specification.md
Modified: incubator/slider/site/trunk/content/docs/getting_started.md
URL: http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/getting_started.md?rev=1630264&r1=1630263&r2=1630264&view=diff
==============================================================================
--- incubator/slider/site/trunk/content/docs/getting_started.md (original)
+++ incubator/slider/site/trunk/content/docs/getting_started.md Thu Oct 9 01:03:03 2014
@@ -16,6 +16,9 @@
-->
# Apache Slider: Getting Started
+This page is updated to reflect the latest code in "develop".
+
+For the version applicable to the last release go to [Slider-40](/docs/getting_started_40.html).
## Introduction
@@ -43,7 +46,7 @@ The following provides the steps require
The Slider deployment has the following minimum system requirements:
-* Hadoop 2.4+
+* Hadoop 2.6+
* Required Services: HDFS, YARN and ZooKeeper
@@ -55,8 +58,10 @@ The Slider deployment has the following
## <a name="setup"></a>Setup the Cluster
-After setting up your Hadoop cluster (using Ambari or other means) with the
-services listed above, modify your YARN configuration to allow for multiple
+Setting up your Hadoop cluster with the services listed above.
+
+If you are using a one-box or a sandbox then you may need to modify your YARN
+configuration to allow for multiple, modify your YARN configuration to allow for multiple
containers on a single host. In `yarn-site.xml` make the following modifications:
<table>
@@ -80,15 +85,12 @@ There are other options detailed in the
## <a name="download"></a>Download Slider Packages
-Slider releases are available at
-[https://www.apache.org/dyn/closer.cgi/incubator/slider](https://www.apache.org/dyn/closer.cgi/incubator/slider).
-
-*Or, you can build it as described below.*
+*You can build it as described below.*
## <a name="build"></a>Build Slider
* From the top level directory, execute `mvn clean site:site site:stage package -DskipTests`
-* Use the generated compressed tar file in slider-assembly/target directory (e.g. slider-0.40.0-all.tar.gz or slider-0.40.0-all.zip) for the subsequent steps
+* Use the generated compressed tar file in slider-assembly/target directory (e.g. slider-0.50.0-all.tar.gz or slider-0.50.0-all.zip) for the subsequent steps
* If you are cloning the Slider git repo, go to `master` branch for the latest released or `develop` for the latest under development
## <a name="install"></a>Install Slider
@@ -99,43 +101,46 @@ Slider is installed on a client machine
cd ${slider-install-dir}
+You can run Slider app as any user. Only requirement is that the user should have a home directory on HDFS. For the reminder of the doc, it is assumed that "yarn" user is being used.
+
Login as the "yarn" user (assuming this is a host associated with the installed cluster). E.g., `su yarn`
-*This assumes that all apps are being run as âyarnâ user. Any other user can be used to run the apps - ensure that file permission is granted as required.*
+*This assumes that all apps are being run as âyarnâ user*
+
+Expand the tar file: `tar -xvf slider-0.50.0-incubating-all.tar.gz` or `unzip slider-0.50.0-incubating-all.zip`
-Expand the tar file: `tar -xvf slider-0.40.0-incubating-all.tar.gz` or `unzip slider-0.40.0-incubating-all.zip`
+## Configure Slider
-Browse to the Slider directory: `cd slider-0.40.0/bin`
+Browse to the Slider directory: `cd slider-0.50.0/conf`
-*Add java to the PATH.*
+Edit slider-env.sh and specify correct values.
- export PATH=$PATH:/usr/jdk64/jdk1.7.0_45/bin
+ export JAVA_HOME=/usr/jdk64/jdk1.7.0_67
+ export HADOOP_CONF_DIR=/etc/hadoop/conf
-(or the path to the JDK bin directory)
+If you are on a node that does not have the hadoop conf folder then you can add the relevant configurations into slider-client.xml.
-Modify Slider configuration file `${slider-install-dir}/slider-0.40.0/conf/slider-client.xml` to add the following properties:
+For example, if you are targeting a non-secure cluster with no HA for NameNode or ResourceManager, modify Slider configuration file `${slider-install-dir}/slider-0.50.0/conf/slider-client.xml` to add the following properties:
<property>
<name>yarn.application.classpath</name>
<value>/etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*</value>
</property>
-
<property>
<name>slider.zookeeper.quorum</name>
<value>yourZooKeeperHost:port</value>
</property>
-
- <property>
+ <property>
<name>yarn.resourcemanager.address</name>
<value>yourResourceManagerHost:8050</value>
- </property>
- <property>
+ </property>
+ <property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>yourResourceManagerHost:8030</value>
- </property>
- <property>
+ </property>
+ <property>
<name>fs.defaultFS</name>
<value>hdfs://yourNameNodeHost:8020</value>
- </property>
+ </property>
If the target cluster is on Windows then make sure that the `yarn.application.classpath` is set accordingly.
@@ -148,11 +153,11 @@ If the target cluster is on Windows then
Execute:
- ${slider-install-dir}/slider-0.40.0/bin/slider version
+ ${slider-install-dir}/slider-0.50.0/bin/slider version
OR
- python %slider-install-dir%/slider-0.40.0/bin/slider.py version
+ python %slider-install-dir%/slider-0.50.0/bin/slider.py version
-Ensure there are no errors and you can see "Compiled against Hadoop 2.4.0". **This ensures that Slider is installed correctly.**
+Ensure there are no errors and you can see "Compiled against Hadoop 2.6.0". **This ensures that Slider is installed correctly.**
## <a name="deploy"></a>Deploy Slider Resources
@@ -168,15 +173,6 @@ Perform the following steps to create th
hdfs dfs -chown yarn:hdfs /user/yarn
-### Load Slider Agent
-
- su yarn
-
- hdfs dfs -mkdir /user/yarn/agent
-
- hdfs dfs -copyFromLocal ${slider-install-dir}/slider-0.40.0/agent/slider-agent.tar.gz /user/yarn/agent
-
-
## <a name="downsample"></a>Create Application Packages
Slider "master" branch has several sample application packages available for use with Slider:
@@ -187,13 +183,13 @@ Slider "master" branch has several sampl
* app-packages/accumulo: The README.txt file describes how to create a Slider Application Package for Accumulo.
-* app-packages/storm: The README.txt describes how to create the Slider Application package for Storm.
+* app-packages/storm or storm-win: The README.txt describes how to create the Slider Application package for Storm.
Create one or more Slider application packages and follow the steps below to install them.
## <a name="installapp"></a>Install, Configure, Start and Verify Sample Application
-* [Load Sample Application Package](#load)
+* [Install Sample Application Package](#load)
* [Create Application Specifications](#create)
@@ -203,21 +199,26 @@ Create one or more Slider application pa
* [Manage the Application Lifecycle](#manage)
-### <a name="load"></a>Load Sample Application Package
+### <a name="load"></a>Install Sample Application Package
+
+ slider install-package --name *package name* --package *sample-application-package*
+
+For example, you can use the following command for HBase application package.
+
+ slider install-package --name HBASE --package path-to-hbase-package.zip
- hdfs dfs -copyFromLocal *sample-application-package* /user/yarn
### <a name="create"></a>Create Application Specifications
Configuring a Slider application consists of two parts: the [Resource Specification](#resspec),
and the *[Application Configuration](#appconfig).
-**Note**: There are sample Resource Specifications (**resources.json**) and Application Configuration
-(**appConfig.json**) files in the application packages zip files (e.g. /**resources.json** and /**appConfig.json**).
+**Note**: There are sample Resource Specifications (**resources-default.json**) and Application Configuration
+(**appConfig-default.json**) files in the application packages zip files.
#### <a name="resspec"></a>Resource Specification
-Slider needs to know what components (and how many components) are in an application package to deploy.
+Slider needs to know what components (and how many components) to deploy.
As Slider creates each instance of a component in its own YARN container, it also needs to know what to ask YARN for in terms of **memory** and **CPU** for those containers.
@@ -245,11 +246,11 @@ Once the steps above are completed, the
Change directory to the "bin" directory under the slider installation
- cd ${slider-install-dir}/slider-0.40.0/bin
+ cd ${slider-install-dir}/slider-0.50.0/bin
Execute the following command:
- ./slider create cl1 --image hdfs://yourNameNodeHost:8020/user/yarn/agent/slider-agent.tar.gz --template appConfig.json --resources resources.json
+ ./slider create cl1 --template appConfig.json --resources resources.json
### <a name="verify"></a>Verify the Application
@@ -265,6 +266,8 @@ The specific information for the running
An application publishes several useful details that can be used to manage and communicate with the application instance.
+You can use `registry` commands to get all published data.
+
**All published data**
All published data are available at the `/ws/v1/slider/publisher` end point of the AppMaster (e.g. http://c6403.ambari.apache.org:34701/ws/v1/slider/publisher). Its obtainable through `slider-client status <app name>` where field `info.am.web.url` specifies the base address. The URL is also advertised in the AppMaster tracking UI.
@@ -300,15 +303,13 @@ An application may publish some quick li
Once started, applications can be frozen/stopped, thawed/restarted, and destroyed/removed as follows:
-#### Frozen:
-(*on develop branch freeze has been renamed to stop*)
+#### Stopped:
- ./slider freeze cl1
+ ./slider stop cl1
-#### Thawed:
-(*on develop branch thaw has been renamed to start*)
+#### Started:
- ./slider thaw cl1
+ ./slider start cl1
#### Destroyed:
@@ -325,7 +326,7 @@ Once started, applications can be frozen
Create an agent config file (agent.ini) based on the sample available at:
- ${slider-install-dir}/slider-0.40.0/agent/conf/agent.ini
+ ${slider-install-dir}/slider-0.50.0/agent/conf/agent.ini
The sample agent.ini file can be used as is (see below). Some of the parameters of interest are:
Added: incubator/slider/site/trunk/content/docs/getting_started_40.md
URL: http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/getting_started_40.md?rev=1630264&view=auto
==============================================================================
--- incubator/slider/site/trunk/content/docs/getting_started_40.md (added)
+++ incubator/slider/site/trunk/content/docs/getting_started_40.md Thu Oct 9 01:03:03 2014
@@ -0,0 +1,352 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+# Apache Slider: Getting Started
+This page is updated to reflect the latest code in "develop".
+
+For the version applicable to the last release go to [Slider-40](#slider40).
+
+
+## Introduction
+
+The following provides the steps required for setting up a cluster and deploying a YARN hosted application using Slider.
+
+* [System Requirements](#sysreqs)
+
+* [Setup the Cluster](#setup)
+
+* [Download Slider Packages](#download)
+
+* [Build Slider](#build)
+
+* [Install Slider](#install)
+
+* [Deploy Slider Resources](#deploy)
+
+* [Create Application Packages](#downsample)
+
+* [Install, Configure, Start and Verify Sample Application](#installapp)
+
+
+## <a name="sysreqs"></a>System Requirements
+
+The Slider deployment has the following minimum system requirements:
+
+* Hadoop 2.4+
+
+* Required Services: HDFS, YARN and ZooKeeper
+
+* Oracle JDK 1.6 (64-bit)
+
+* Python 2.6
+
+* openssl
+
+## <a name="setup"></a>Setup the Cluster
+
+After setting up your Hadoop cluster (using Ambari or other means) with the
+services listed above, modify your YARN configuration to allow for multiple
+containers on a single host. In `yarn-site.xml` make the following modifications:
+
+<table>
+ <tr>
+ <td>Property</td>
+ <td>Value</td>
+ </tr>
+ <tr>
+ <td>yarn.scheduler.minimum-allocation-mb</td>
+ <td>>= 256 (ensure that YARN can allocate sufficient number of containers)</td>
+ </tr>
+ <tr>
+ <td>yarn.nodemanager.delete.debug-delay-sec</td>
+ <td>>= 3600 (to retain for an hour)</td>
+ </tr>
+</table>
+
+
+There are other options detailed in the Troubleshooting file available [here](/docs/troubleshooting.html).
+
+
+## <a name="download"></a>Download Slider Packages
+
+Slider releases are available at
+[https://www.apache.org/dyn/closer.cgi/incubator/slider](https://www.apache.org/dyn/closer.cgi/incubator/slider).
+
+*Or, you can build it as described below.*
+
+## <a name="build"></a>Build Slider
+
+* From the top level directory, execute `mvn clean site:site site:stage package -DskipTests`
+* Use the generated compressed tar file in slider-assembly/target directory (e.g. slider-0.40.0-all.tar.gz or slider-0.40.0-all.zip) for the subsequent steps
+* If you are cloning the Slider git repo, go to `master` branch for the latest released or `develop` for the latest under development
+
+## <a name="install"></a>Install Slider
+
+Slider is installed on a client machine that can access the hadoop cluster. Follow the following steps to expand/install Slider:
+
+ mkdir ${slider-install-dir}
+
+ cd ${slider-install-dir}
+
+Login as the "yarn" user (assuming this is a host associated with the installed cluster). E.g., `su yarn`
+*This assumes that all apps are being run as âyarnâ user. Any other user can be used to run the apps - ensure that file permission is granted as required.*
+
+Expand the tar file: `tar -xvf slider-0.40.0-incubating-all.tar.gz` or `unzip slider-0.40.0-incubating-all.zip`
+
+Browse to the Slider directory: `cd slider-0.40.0/bin`
+
+*Add java to the PATH.*
+
+ export PATH=$PATH:/usr/jdk64/jdk1.7.0_45/bin
+
+(or the path to the JDK bin directory)
+
+Modify Slider configuration file `${slider-install-dir}/slider-0.40.0/conf/slider-client.xml` to add the following properties:
+
+ <property>
+ <name>yarn.application.classpath</name>
+ <value>/etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*</value>
+ </property>
+
+ <property>
+ <name>slider.zookeeper.quorum</name>
+ <value>yourZooKeeperHost:port</value>
+ </property>
+
+ <property>
+ <name>yarn.resourcemanager.address</name>
+ <value>yourResourceManagerHost:8050</value>
+ </property>
+ <property>
+ <name>yarn.resourcemanager.scheduler.address</name>
+ <value>yourResourceManagerHost:8030</value>
+ </property>
+ <property>
+ <name>fs.defaultFS</name>
+ <value>hdfs://yourNameNodeHost:8020</value>
+ </property>
+
+If the target cluster is on Windows then make sure that the `yarn.application.classpath` is set accordingly.
+
+ <property>
+ <name>yarn.application.classpath</name>
+ <value>
+ %HADOOP_CONF_DIR%,%HADOOP_COMMON_HOME%/share/hadoop/common/*,%HADOOP_YARN_HOME%/share/hadoop/yarn/*,...
+ </value>
+ </property>
+
+Execute:
+
+ ${slider-install-dir}/slider-0.40.0/bin/slider version
+ OR
+ python %slider-install-dir%/slider-0.40.0/bin/slider.py version
+
+Ensure there are no errors and you can see "Compiled against Hadoop 2.4.0". **This ensures that Slider is installed correctly.**
+
+## <a name="deploy"></a>Deploy Slider Resources
+
+Ensure that all file folders are accessible to the user creating the application instance. The example assumes "yarn" to be that user. The default yarn user may be different e.g. hadoop.
+
+### Ensure HDFS home folder for the User
+
+Perform the following steps to create the user home folder with the appropriate permissions:
+
+ su hdfs
+
+ hdfs dfs -mkdir /user/yarn
+
+ hdfs dfs -chown yarn:hdfs /user/yarn
+
+### Load Slider Agent
+
+ su yarn
+
+ hdfs dfs -mkdir /user/yarn/agent
+
+ hdfs dfs -copyFromLocal ${slider-install-dir}/slider-0.40.0/agent/slider-agent.tar.gz /user/yarn/agent
+
+
+## <a name="downsample"></a>Create Application Packages
+
+Slider "master" branch has several sample application packages available for use with Slider:
+
+* app-packages/memcached-win or memcached: The README.txt describes how to create the Slider Application package for memcached for linux or windows. **memcached** app packages are a good place to [start](slider_specs/hello_world_slider_app.html)
+
+* app-packages/hbase or hbase-win: The README.txt file describes how to create a Slider Application Package for HBase. By default it will create a package for HBase 0.98.3 but you can create the same for other versions.
+
+* app-packages/accumulo: The README.txt file describes how to create a Slider Application Package for Accumulo.
+
+* app-packages/storm: The README.txt describes how to create the Slider Application package for Storm.
+
+Create one or more Slider application packages and follow the steps below to install them.
+
+## <a name="installapp"></a>Install, Configure, Start and Verify Sample Application
+
+* [Load Sample Application Package](#load)
+
+* [Create Application Specifications](#create)
+
+* [Start the Application](#start)
+
+* [Verify the Application](#verify)
+
+* [Manage the Application Lifecycle](#manage)
+
+### <a name="load"></a>Load Sample Application Package
+
+ hdfs dfs -copyFromLocal *sample-application-package* /user/yarn
+
+### <a name="create"></a>Create Application Specifications
+
+Configuring a Slider application consists of two parts: the [Resource Specification](#resspec),
+ and the *[Application Configuration](#appconfig).
+
+**Note**: There are sample Resource Specifications (**resources.json**) and Application Configuration
+(**appConfig.json**) files in the application packages zip files (e.g. /**resources.json** and /**appConfig.json**).
+
+#### <a name="resspec"></a>Resource Specification
+
+Slider needs to know what components (and how many components) are in an application package to deploy.
+
+As Slider creates each instance of a component in its own YARN container, it also needs to know what to ask YARN for in terms of **memory** and **CPU** for those containers.
+
+All this information goes into the **Resources Specification** file ("Resource Spec") named `resources.json`. The Resource Spec tells Slider how many instances of each component in the application (such as an HBase RegionServer) to deploy and the parameters for YARN.
+
+An application package should contain the default resources.json and you can start from there. Or you can create one based on [Resource Specification](slider_specs/resource_specification.html).
+
+Store the Resource Spec file on your local disk (e.g. `/tmp/resources.json`).
+
+
+#### <a name="appconfig"></a>Application Configuration
+
+Alongside the Resource Spec there is the **Application Configuration** file ("App Config") which includes parameters that are specific to the application, rather than YARN. The App Config is a file that contains all application configuration. This configuration is applied to the default configuration provided by the application definition and then handed off to the associated component agent.
+
+For example, the heap sizes of the JVMs, The App Config defines the configuration details **specific to the application and component** instances. An application package should contain the default resources.json and you can start from there.
+
+Ensure the following variables are accurate:
+
+* "application.def": "application_package.zip" (path to uploaded application package)
+
+
+### <a name="start"></a>Start the Application
+
+Once the steps above are completed, the application can be started through the **Slider Command Line Interface (CLI)**.
+
+Change directory to the "bin" directory under the slider installation
+
+ cd ${slider-install-dir}/slider-0.40.0/bin
+
+Execute the following command:
+
+ ./slider create cl1 --image hdfs://yourNameNodeHost:8020/user/yarn/agent/slider-agent.tar.gz --template appConfig.json --resources resources.json
+
+### <a name="verify"></a>Verify the Application
+
+The successful launch of the application can be verified via the YARN Resource Manager Web UI. In most instances, this UI is accessible via a web browser at port 8088 of the Resource Manager Host:
+
+
+
+The specific information for the running application is accessible via the "ApplicationMaster" link that can be seen in the far right column of the row associated with the running application (probably the top row):
+
+
+
+### <a name="client-config"></a>Obtain client config
+
+An application publishes several useful details that can be used to manage and communicate with the application instance.
+
+**All published data**
+
+All published data are available at the `/ws/v1/slider/publisher` end point of the AppMaster (e.g. http://c6403.ambari.apache.org:34701/ws/v1/slider/publisher). Its obtainable through `slider-client status <app name>` where field `info.am.web.url` specifies the base address. The URL is also advertised in the AppMaster tracking UI.
+
+**Client configuration**
+Client configurations are at `/ws/v1/slider/publisher/slider/<config name>` where *config name* can be
+
+* site config file name without extension, such as `hbase-site` (e.g. http://c6403.ambari.apache.org:34701/ws/v1/slider/publisher/slider/hbase-site) where the output is json formatted name-value pairs
+* or, with extension such as `hbase-site.xml` (http://c6403.ambari.apache.org:34701/ws/v1/slider/publisher/slider/hbase-site.xml) to get an XML formatted output that can be consumed by the clients
+
+**Log locations**
+
+The log locations for various containers in the application instance are at `ws/v1/slider/publisher/slider/logfolders`
+
+ {
+
+ "description": "LogFolders",
+ "entries": {
+ "c6403.ambari.apache.org-container_1404783501621_0002_01_000002-AGENT_LOG_ROOT": "/hadoop/yarn/log/application_1404783501621_0002/container_1404783501621_0002_01_000002",
+ "c6403.ambari.apache.org-container_1404783501621_0002_01_000003-AGENT_LOG_ROOT": "/hadoop/yarn/log/application_1404783501621_0002/container_1404783501621_0002_01_000003",
+ "c6403.ambari.apache.org-container_1404783501621_0002_01_000003-AGENT_WORK_ROOT": "/hadoop/yarn/local/usercache/yarn/appcache/application_1404783501621_0002/container_1404783501621_0002_01_000003",
+ "c6403.ambari.apache.org-container_1404783501621_0002_01_000002-AGENT_WORK_ROOT": "/hadoop/yarn/local/usercache/yarn/appcache/application_1404783501621_0002/container_1404783501621_0002_01_000002"
+ },
+ "updated": 0,
+ "empty": false
+ }
+
+**Quick links**
+
+An application may publish some quick links for the monitoring UI, JMX endpoint, etc. as the listening ports are dynamically allocated and thus need to be published. These information is available at `/ws/v1/slider/publisher/slider/quicklinks`.
+
+### <a name="manage"></a>Manage the Application Lifecycle
+
+Once started, applications can be frozen/stopped, thawed/restarted, and destroyed/removed as follows:
+
+#### Frozen:
+(*on develop branch freeze has been renamed to stop*)
+
+ ./slider freeze cl1
+
+#### Thawed:
+(*on develop branch thaw has been renamed to start*)
+
+ ./slider thaw cl1
+
+#### Destroyed:
+
+ ./slider destroy
+
+#### Flexed:
+
+ ./slider flex cl1 --component worker 5
+
+
+# <a name="appendixa"></a>Appendix A: Debugging Slider-Agent
+
+## Create and deploy Slider Agent configuration
+
+Create an agent config file (agent.ini) based on the sample available at:
+
+ ${slider-install-dir}/slider-0.40.0/agent/conf/agent.ini
+
+The sample agent.ini file can be used as is (see below). Some of the parameters of interest are:
+
+`log_level` = INFO or DEBUG, to control the verbosity of log
+
+`app_log_dir` = the relative location of the application log file
+
+`log_dir` = the relative location of the agent and command log file
+
+Once created, deploy the agent.ini file to HDFS (it can be deployed in any location accessible to application instance):
+
+ su yarn
+
+ hdfs dfs -copyFromLocal agent.ini /user/yarn/agent/conf
+
+Modify the --template json file (appConfig.json) and add the location of the agent.ini file.
+
+ "agent.conf": "/user/yarn/agent/conf/agent.ini"
+
+
+
Modified: incubator/slider/site/trunk/content/docs/manpage.md
URL: http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/manpage.md?rev=1630264&r1=1630263&r2=1630264&view=diff
==============================================================================
--- incubator/slider/site/trunk/content/docs/manpage.md (original)
+++ incubator/slider/site/trunk/content/docs/manpage.md Thu Oct 9 01:03:03 2014
@@ -120,7 +120,7 @@ It is not started; this can be done late
### `create <name>`
-Build and run an applicaton instance of the given name
+Build and run an application instance of the given name
The `--wait` parameter, if provided, specifies the time to wait until the YARN application is actually running. Even after the YARN application has started, there may be some delay for the instance to start up.
@@ -269,7 +269,7 @@ Create an application by providing `temp
### `destroy <name>`
-Destroy a (stopped) applicaton instance .
+Destroy a (stopped) application instance .
Important: This deletes all persistent data
Modified: incubator/slider/site/trunk/content/docs/slider_specs/resource_specification.md
URL: http://svn.apache.org/viewvc/incubator/slider/site/trunk/content/docs/slider_specs/resource_specification.md?rev=1630264&r1=1630263&r2=1630264&view=diff
==============================================================================
--- incubator/slider/site/trunk/content/docs/slider_specs/resource_specification.md (original)
+++ incubator/slider/site/trunk/content/docs/slider_specs/resource_specification.md Thu Oct 9 01:03:03 2014
@@ -28,7 +28,7 @@ An example resource requirement for an a
Some parameters that can be specified for a component instance include:
-* `yarn.memory`: amount of memory requried for the component instance
+* `yarn.memory`: amount of memory required for the component instance
* `yarn.vcores`: number of vcores requested
* `yarn.role.priority`: each component must be assigned unique priority. Component with higher priority come up earlier than components with lower priority
* `yarn.component.instances`: number of instances for this component type
@@ -190,9 +190,9 @@ In summary:
* Ensure application components are associated with appropriate label expressions
* Create the application using specific queue
-This way, you can gurantee that a certain set of nodes are reserved for an application or for a component within an application.
+This way, you can guarantee that a certain set of nodes are reserved for an application or for a component within an application.
-Label expression is specified through property "yarn.label.expression". When no label expression is specified then it is assummed that only non-labeled nodes are used when allocating containers for component instances.
+Label expression is specified through property "yarn.label.expression". When no label expression is specified then it is assumed that only non-labeled nodes are used when allocating containers for component instances.
If label expression is specified for slider-appmaster then it also becomes the default label expression for all component. To take advantage of default label expression leave out the property (see HBASE_REGIONSERVER in the example). Label expression with empty string ("yarn.label.expression":"") means nodes without labels.