You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@slider.apache.org by bu...@apache.org on 2014/07/17 02:24:55 UTC
svn commit: r916440 - in /websites/staging/slider/trunk/content: ./
developing/ developing/functional_tests.html
Author: buildbot
Date: Thu Jul 17 00:24:54 2014
New Revision: 916440
Log:
Staging update by buildbot for slider
Added:
websites/staging/slider/trunk/content/ (with props)
websites/staging/slider/trunk/content/developing/
websites/staging/slider/trunk/content/developing/functional_tests.html
Propchange: websites/staging/slider/trunk/content/
------------------------------------------------------------------------------
cms:source-revision = 1611224
Added: websites/staging/slider/trunk/content/developing/functional_tests.html
==============================================================================
--- websites/staging/slider/trunk/content/developing/functional_tests.html (added)
+++ websites/staging/slider/trunk/content/developing/functional_tests.html Thu Jul 17 00:24:54 2014
@@ -0,0 +1,556 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE- 2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+ <meta charset="utf-8">
+ <meta http-equiv="X-UA-Compatible" content="IE=edge">
+ <meta name="viewport" content="width=device-width, initial-scale=1">
+ <link href="/css/bootstrap.min.css" rel="stylesheet">
+ <link href="/css/bootstrap-theme.min.css" rel="stylesheet">
+ <!-- WARNING: Respond.js doesn't work if you view the page via file:// -->
+ <!--[if lt IE 9]>
+ <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
+ <script src="https://oss.maxcdn.com/libs/respond.js/1.4.2/respond.min.js"></script>
+ <![endif]-->
+ <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js"></script>
+ <script src="/js/bootstrap.min.js"></script>
+ <link href="/css/slider.css" rel="stylesheet" type="text/css">
+ <link href="//netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
+ <title></title>
+
+</head>
+
+<body>
+<nav class="navbar navbar-default navbar-fixed-top" role="navigation">
+<div class="container-fluid">
+ <div class="navbar-header">
+ <button type="button" class="navbar-toggle" data-toggle="collapse" data-target="#navbar-items">
+ <span class="sr-only">Toggle navigation</span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ <span class="icon-bar"></span>
+ </button>
+ <a class="navbar-brand" href="/index.html">Slider</a>
+ </div>
+ <div class="collapse navbar-collapse" id="navbar-items">
+ <ul class="nav navbar-nav">
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Project <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_index"><a href="/index.html">Home</a></li>
+<li id="nav_downloads"><a href="/downloads">Downloads</a></li>
+<li id="nav_source"><a href="https://git-wip-us.apache.org/repos/asf/incubator-slider.git"><i class="fa fa-external-link"></i> Source</a></li>
+<li><a href="http://www.apache.org/licenses/LICENSE-2.0"><i class="fa fa-external-link"></i> License</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Community</li>
+<li id="nav_team"><a href="/team.html">Team</a></li>
+<li id="nav_mailinglists"><a href="/mailing_lists.html">Mailing Lists</a></li>
+<li class="divider"></li>
+<li class="dropdown-header">Tools</li>
+<li id="nav_jira"><a href="https://issues.apache.org/jira/browse/SLIDER"><i class="fa fa-external-link"></i> Issues</a></li>
+<li id="nav_builds"><a href="https://builds.apache.org/job/Slider-develop"><i class="fa fa-external-link"></i> Builds</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/design">
+ Design <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_design_overview"><a href="/design/architecture.html">Overview</a></li>
+<li id="nav_app_needs"><a href="/docs/slider_specs/application_needs.html">Application Needs</a></li>
+<li id="nav_spec"><a href="/design/specification/index.html">Expected Behavior</a></li>
+<li id="nav_registry"><a href="/design/registry/index.html">Service Registry</a></li>
+<li id="nav_ssl_impl"><a href="/design/ssl_implementation.html">SSL Implementation</a></li>
+<li id="nav_role_history"><a href="/design/rolehistory.html">Role History</a></li>
+</ul>
+</li>
+
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="/developing">
+ Developing <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_dev_building"><a href="/developing/building.html">Building</a></li>
+<li id="nav_dev_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_dev_testing"><a href="/developing/testing.html">Testing</a></li>
+<li id="nav_dev_funtesting"><a href="/developing/functional_tests.html">Functional Testing</a></li>
+<li id="nav_dev_manual_testing"><a href="/developing/manual_testing.html">Manual Testing</a></li>
+<li id="nav_releasing"><a href="/developing/releasing.html">Releasing</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ Using <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li id="nav_use_getting_started"><a href="/docs/getting_started.html">Getting Started</a></li>
+<li id="nav_use_manpage"><a href="/docs/manpage.html">Man Page</a></li>
+<li id="nav_use_examples"><a href="/docs/examples.html">Examples</a></li>
+<li id="nav_use_client_config"><a href="/docs/client-configuration.html">Client Configuration</a></li>
+<li id="nav_use_exitcodes"><a href="/docs/exitcodes.html">Client Exit Codes</a></li>
+<li id="nav_use_security"><a href="/docs/security.html">Security</a></li>
+<li id="nav_am_agent_ssl"><a href="/docs/ssl.html">Agent to AM SSL</a></li>
+<li id="nav_use_app_package"><a href="/docs/slider_specs/index.html">Creating App Packages</a></li>
+<li id="nav_use_app_configuration"><a href="/docs/configuration/index.html">App Configuration Model</a></li>
+<li id="nav_use_debugging"><a href="/docs/debugging.html">Debugging</a></li>
+<li id="nav_use_troubleshooting"><a href="/docs/troubleshooting.html">Troubleshooting</a></li>
+</ul>
+</li>
+
+ <li class="dropdown">
+ <a class="dropdown-toggle" data-toggle="dropdown" href="#">
+ ASF Links <span class="caret"></span>
+ </a>
+<ul class="dropdown-menu">
+<li><a href="http://www.apache.org"><i class="fa fa-external-link"></i> Apache Software Foundation</a></li>
+<li><a href="http://www.apache.org/foundation/sponsorship.html"><i class="fa fa-external-link"></i> Sponsorship</a></li>
+<li><a href="http://www.apache.org/security/"><i class="fa fa-external-link"></i> Security</a></li>
+<li><a href="http://www.apache.org/foundation/thanks.html"><i class="fa fa-external-link"></i> Thanks</a></li>
+</ul>
+</li>
+</ul>
+
+ </div>
+
+</div>
+</nav>
+
+
+<div class="container-fluid">
+<div class="row">
+ <div class="col-md-2" id="sidebar">
+ <div style="text-align: center">
+ <h1><a href="/index.html">Apache Slider (incubating)</a></h1>
+ <hr>
+Latest release: <strong>0.30-incubating</strong><br>
+ <br>
+ <a id="download-button-sidebar" class="btn btn-success btn-block" href="/downloads/" role="button">Download</a>
+ </div>
+ <hr>
+ <a href="http://incubator.apache.org"><img id="logo" alt="Apache Incubator ™" class="img-responsive" src="http://incubator.apache.org/images/apache-incubator-logo.png"/></a>
+ </div>
+ <div class="col-md-8 col-md-offset-1">
+
+ <div id="content">
+
+ <h1 class="title"></h1>
+
+ <!---
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<h1 id="testing-apache-slider">Testing Apache Slider</h1>
+<div class="codehilite"><pre> <span class="n">The</span> <span class="n">key</span> <span class="n">words</span> "<span class="n">MUST</span>"<span class="p">,</span> "<span class="n">MUST</span> <span class="n">NOT</span>"<span class="p">,</span> "<span class="n">REQUIRED</span>"<span class="p">,</span> "<span class="n">SHALL</span>"<span class="p">,</span> "<span class="n">SHALL</span>
+ <span class="n">NOT</span>"<span class="p">,</span> "<span class="n">SHOULD</span>"<span class="p">,</span> "<span class="n">SHOULD</span> <span class="n">NOT</span>"<span class="p">,</span> "<span class="n">RECOMMENDED</span>"<span class="p">,</span> "<span class="n">MAY</span>"<span class="p">,</span> <span class="n">and</span>
+ "<span class="n">OPTIONAL</span>" <span class="n">in</span> <span class="n">this</span> <span class="n">document</span> <span class="n">are</span> <span class="n">to</span> <span class="n">be</span> <span class="n">interpreted</span> <span class="n">as</span> <span class="n">described</span> <span class="n">in</span>
+ <span class="n">RFC</span> 2119<span class="p">.</span>
+</pre></div>
+
+
+<h1 id="functional-tests">Functional Tests</h1>
+<p>The functional test suite is designed to test slider against
+a live cluster. </p>
+<p>For these to work you need</p>
+<ol>
+<li>A YARN Cluster -secure or insecure</li>
+<li>A <code>slider-client.xml</code> file configured to interact with the cluster</li>
+<li>Agent tests and Accumulo Agent tests: nothing additional</li>
+<li>HBase provider tests: HBase <code>.tar.gz</code> uploaded to HDFS, and a local or remote hbase conf
+directory</li>
+<li>Accumulo provider tests: Accumulo <code>.tar.gz</code> uploaded to HDFS, and a local or remote accumulo conf
+directory</li>
+</ol>
+<h2 id="configuration-of-functional-tests">Configuration of functional tests</h2>
+<p>Maven needs to be given</p>
+<ol>
+<li>A path to the expanded test archive</li>
+<li>A path to a slider configuration directory for the cluster</li>
+</ol>
+<p>The path for the expanded test is automatically calculated as being the directory under
+<code>..\slider-assembly\target</code> where an untarred slider distribution can be found.
+If it is not present, the tests will fail</p>
+<p>The path to the configuration directory must be supplied in the property
+<code>slider.conf.dir</code> which can be set on the command line</p>
+<div class="codehilite"><pre><span class="n">mvn</span> <span class="n">clean</span> <span class="n">verify</span> <span class="o">-</span><span class="n">Dslider</span><span class="p">.</span><span class="n">conf</span><span class="p">.</span><span class="n">dir</span><span class="p">=</span><span class="n">src</span><span class="o">/</span><span class="n">test</span><span class="o">/</span><span class="n">clusters</span><span class="o">/</span><span class="n">sandbox</span><span class="o">/</span><span class="n">slider</span>
+</pre></div>
+
+
+<p>It can also be set in the (optional) file <code>slider-funtest/build.properties</code>:</p>
+<div class="codehilite"><pre><span class="n">slider</span><span class="p">.</span><span class="n">conf</span><span class="p">.</span><span class="n">dir</span><span class="p">=</span><span class="n">src</span><span class="o">/</span><span class="n">test</span><span class="o">/</span><span class="n">clusters</span><span class="o">/</span><span class="n">sandbox</span><span class="o">/</span><span class="n">slider</span>
+</pre></div>
+
+
+<p>This file is loaded whenever a slider build or test run takes place</p>
+<h2 id="configuration-of-slider-clientxml">Configuration of <code>slider-client.xml</code></h2>
+<p>The <code>slider-client.xml</code> must have extra configuration options for both the HBase and
+Accumulo tests, as well as a common set for actually talking to a YARN cluster.</p>
+<h2 id="disabling-the-functional-tests-entirely">Disabling the functional tests entirely</h2>
+<p>All functional tests which require a live YARN cluster
+are run during the integration-test phase. They are executed with the command
+<code>mvn verify</code>.</p>
+<p>If you do not wish to run the functional tests, simply use the <code>mvn package</code> command
+and only those tests which do not require a live YARN cluster will run.</p>
+<h3 id="non-mandatory-options">Non-mandatory options</h3>
+<p>The following test options may be added to <code>slider-client.xml</code> if the defaults
+need to be changed</p>
+<div class="codehilite"><pre><span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.thaw.wait.seconds<span class="nt"></name></span>
+ <span class="nt"><description></span>Time to wait in seconds for a thaw to result in a running AM<span class="nt"></description></span>
+ <span class="nt"><value></span>60000<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.freeze.wait.seconds<span class="nt"></name></span>
+ <span class="nt"><description></span>Time to wait in seconds for a freeze to halt the cluster<span class="nt"></description></span>
+ <span class="nt"><value></span>60000<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.timeout.millisec<span class="nt"></name></span>
+ <span class="nt"><description></span>Time out in milliseconds before a test is considered to have failed.
+ There are some maven properties which also define limits and may need adjusting<span class="nt"></description></span>
+ <span class="nt"><value></span>180000<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.yarn.ram<span class="nt"></name></span>
+ <span class="nt"><description></span>Size in MB to ask for containers<span class="nt"></description></span>
+ <span class="nt"><value></span>192<span class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
+
+
+<p>Note that while the same properties need to be set in
+<code>slider-core/src/test/resources/slider-client.xml</code>, those tests take a file in the local
+filesystem -here a URI to a path visible across all nodes in the cluster are required
+the tests do not copy the .tar/.tar.gz files over. The application configuration
+directories may be local or remote -they are copied into the <code>.slider</code> directory
+during cluster creation.</p>
+<h2 id="provider-specific-parameters">Provider-specific parameters</h2>
+<p>An individual provider can pick up settings from their own
+<code>src/test/resources/slider-client.xml</code> file, or the one in <code>slider-core</code>.
+We strongly advice placing all the values in the <code>slider-core</code> file.</p>
+<ol>
+<li>All uncertainty about which file is picked up on the class path first goes
+away</li>
+<li>There's one place to keep all the configuration values in sync.</li>
+</ol>
+<h3 id="agent-tests">Agent Tests</h3>
+<p>Agent tests are executed through the following mvn command executed at slider/slider-funtest:</p>
+<p><code>cd slider-funtest
+mvn verify -Dslider.conf.dir=../src/test/clusters/remote/slider -Dit.test=AppsThroughAgentIT -DfailIfNoTests=false</code></p>
+<p><strong>Enable/Execute the tests</strong></p>
+<p>To enable the test ensure that <em>slider.test.agent.enabled</em> is set to <em>true</em>.</p>
+<div class="codehilite"><pre><span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.agent.enabled<span class="nt"></name></span>
+ <span class="nt"><description></span>Flag to enable/disable Agent tests<span class="nt"></description></span>
+ <span class="nt"><value></span>true<span class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
+
+
+<p><strong>Test setup</strong></p>
+<p>Edit config file src/test/clusters/remote/slider/slider-client.xml and ensure that the host names are accurate for the test cluster.</p>
+<p><strong>User setup</strong></p>
+<p>Ensure that the user, running the test, is present on the cluster against which you are running the tests. The user must be a member of the hadoop group.</p>
+<p>E.g. adduser <strong>testuser</strong> -d /home/<strong>testuser</strong> -G hadoop -m</p>
+<p><strong>HDFS Setup</strong></p>
+<p>Set up hdfs folders for test user</p>
+<ul>
+<li>su hdfs</li>
+<li>hdfs dfs -mkdir /user/testuser</li>
+<li>hdfs dfs -chown testuser:hdfs /user/testuser</li>
+</ul>
+<h3 id="hbase-tests">HBase Tests</h3>
+<p>The HBase tests can be enabled or disabled</p>
+<div class="codehilite"><pre><span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.hbase.enabled<span class="nt"></name></span>
+ <span class="nt"><description></span>Flag to enable/disable HBase tests<span class="nt"></description></span>
+ <span class="nt"><value></span>true<span class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
+
+
+<p>Mandatory test parameters must be added to <code>slider-client.xml</code></p>
+<div class="codehilite"><pre><span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.hbase.tar<span class="nt"></name></span>
+ <span class="nt"><description></span>Path to the HBase Tar file in HDFS<span class="nt"></description></span>
+ <span class="nt"><value></span>hdfs://sandbox:8020/user/slider/hbase.tar.gz<span class="nt"></value></span>
+<span class="nt"></property></span>
+
+<span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.hbase.appconf<span class="nt"></name></span>
+ <span class="nt"><description></span>Path to the directory containing the HBase application config<span class="nt"></description></span>
+ <span class="nt"><value></span>file://<span class="cp">${</span><span class="n">user</span><span class="o">.</span><span class="n">dir</span><span class="cp">}</span>/src/test/clusters/sandbox/hbase<span class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
+
+
+<p>Optional parameters: </p>
+<div class="codehilite"><pre> <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.hbase.launch.wait.seconds<span class="nt"></name></span>
+ <span class="nt"><description></span>Time to wait in seconds for HBase to start<span class="nt"></description></span>
+ <span class="nt"><value></span>1800<span class="nt"></value></span>
+<span class="nt"></property></span>
+</pre></div>
+
+
+<h4 id="accumulo-configuration-options">Accumulo configuration options</h4>
+<p>Enable/disable the tests</p>
+<div class="codehilite"><pre> <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.accumulo.enabled<span class="nt"></name></span>
+ <span class="nt"><description></span>Flag to enable/disable Accumulo tests<span class="nt"></description></span>
+ <span class="nt"><value></span>true<span class="nt"></value></span>
+ <span class="nt"></property></span>
+</pre></div>
+
+
+<p>Optional parameters</p>
+<div class="codehilite"><pre> <span class="nt"><property></span>
+ <span class="nt"><name></span>slider.test.accumulo.launch.wait.seconds<span class="nt"></name></span>
+ <span class="nt"><description></span>Time to wait in seconds for Accumulo to start<span class="nt"></description></span>
+ <span class="nt"><value></span>1800<span class="nt"></value></span>
+ <span class="nt"></property></span>
+</pre></div>
+
+
+<h3 id="configuring-the-yarn-cluster-for-tests">Configuring the YARN cluster for tests</h3>
+<p>Here are the configuration options we use in <code>yarn-site.xml</code> for testing:</p>
+<p>These tell YARN to ignore memory requirements in allocating VMs, and
+to keep the log files around after an application run. </p>
+<div class="codehilite"><pre> <span class="nt"><property></span>
+ <span class="nt"><name></span>yarn.scheduler.minimum-allocation-mb<span class="nt"></name></span>
+ <span class="nt"><value></span>1<span class="nt"></value></span>
+ <span class="nt"></property></span>
+ <span class="nt"><property></span>
+ <span class="nt"><description></span>Whether physical memory limits will be enforced for
+ containers.
+ <span class="nt"></description></span>
+ <span class="nt"><name></span>yarn.nodemanager.pmem-check-enabled<span class="nt"></name></span>
+ <span class="nt"><value></span>false<span class="nt"></value></span>
+ <span class="nt"></property></span>
+ <span class="c"><!-- we really don't want checking here--></span>
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>yarn.nodemanager.vmem-check-enabled<span class="nt"></name></span>
+ <span class="nt"><value></span>false<span class="nt"></value></span>
+ <span class="nt"></property></span>
+
+ <span class="c"><!-- how long after a failure to see what is left in the directory--></span>
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>yarn.nodemanager.delete.debug-delay-sec<span class="nt"></name></span>
+ <span class="nt"><value></span>60000<span class="nt"></value></span>
+ <span class="nt"></property></span>
+
+ <span class="c"><!--ten seconds before the process gets a -9 --></span>
+ <span class="nt"><property></span>
+ <span class="nt"><name></span>yarn.nodemanager.sleep-delay-before-sigkill.ms<span class="nt"></name></span>
+ <span class="nt"><value></span>30000<span class="nt"></value></span>
+ <span class="nt"></property></span>
+</pre></div>
+
+
+<h2 id="testing-against-a-secure-cluster">Testing against a secure cluster</h2>
+<p>To test against a secure cluster</p>
+<ol>
+<li><code>slider-client.xml</code> must be configured as per <a href="/docs/security.html">Security</a>.</li>
+<li>the client must have the kerberos tokens issued so that the user running
+the tests has access to HDFS and YARN.</li>
+</ol>
+<p>If there are problems authenticating (including the cluster being offline)
+the tests appear to hang</p>
+<h3 id="validating-the-configuration">Validating the configuration</h3>
+<div class="codehilite"><pre><span class="n">mvn</span> <span class="n">clean</span> <span class="n">verify</span> <span class="o">-</span><span class="n">Dit</span><span class="p">.</span><span class="n">test</span><span class="p">=</span><span class="n">BuildSetupIT</span>
+</pre></div>
+
+
+<h3 id="using-relative-paths-in-test-configurations">Using relative paths in test configurations</h3>
+<p>When you are sharing configurations across machines via SCM or similar,
+its impossible to have absolute paths in the configuration options to
+the location of items in the local filesystem (e.g. configuration directories).</p>
+<p>There's two techniques</p>
+<ol>
+<li>
+<p>Keep the data in HDFS and refer to it there. This works if there is a shared,
+persistent HDFS cluster.</p>
+</li>
+<li>
+<p>Use the special property <code>slider.test.conf.dir</code> that is set to the path
+of the directory, and which can then be used to create an absolute path
+from paths relative to the configuration dir:</p>
+<p><property>
+ <name>slider.test.hbase.appconf</name>
+ <description>Path to the directory containing the HBase application config</description>
+ <value>file://${slider.test.conf.dir}/../hbase</value>
+ </property></p>
+</li>
+</ol>
+<p>If the actual XML file path is required, a similar property
+<code>slider.test.conf.xml</code> is set.</p>
+<h2 id="parallel-execution">Parallel execution</h2>
+<p>Attempts to run test cases in parallel failed -even with a configuration
+to run methods in a class sequentially, but separate classes independently.</p>
+<p>Even after identifying and eliminating some unintended sharing of static
+mutable variables, trying to run test cases in parallel seemed to hang
+tests and produce timeouts.</p>
+<p>For this reason parallel tests have been disabled. To accelerate test runs
+through parallelization, run different tests on different hosts instead.</p>
+<h2 id="other-constraints">Other constraints</h2>
+<ul>
+<li>Port assignments SHOULD NOT be fixed, as this will cause clusters to fail if
+there are too many instances of a role on a same host, or if other tests are
+using the same port.</li>
+<li>If a test does need to fix a port, it MUST be for a single instance of a role,
+and it must be different from all others. The assignment should be set in
+<code>org.apache.slider.funtest.itest.PortAssignments</code> so as to ensure uniqueness
+over time. Otherwise: use the value of <code>0</code> to allow the OS to assign free ports
+on demand.</li>
+</ul>
+<h2 id="test-requirements">Test Requirements</h2>
+<ol>
+<li>Test cases should be written so that each class works with exactly one
+Slider-deployed cluster</li>
+<li>Every test MUST have its own cluster name -preferably derived from the
+classname.</li>
+<li>This cluster should be deployed in an <code>@BeforeClass</code> method.</li>
+<li>The <code>@AfterClass</code> method MUST tear this cluster down.</li>
+<li>Tests must skip their execution if functional tests -or the
+specific hbase or accumulo categories- are disabled.</li>
+<li>Tests within the suite (i.e. class) must be designed to be independent
+-to work irrespectively of the ordering of other tests.</li>
+</ol>
+<h2 id="running-and-debugging-the-functional-tests">Running and debugging the functional tests.</h2>
+<p>The functional tests all </p>
+<ol>
+<li>
+<p>In the root <code>slider</code> directory, build a complete Slider release</p>
+<div class="codehilite"><pre><span class="n">mvn</span> <span class="n">install</span> <span class="o">-</span><span class="n">DskipTests</span>
+</pre></div>
+
+
+</li>
+<li>
+<p>Start the YARN cluster/set up proxies to connect to it, etc.</p>
+</li>
+<li>
+<p>In the <code>slider-funtest</code> dir, run the tests</p>
+<div class="codehilite"><pre><span class="n">mvn</span> <span class="n">clean</span> <span class="n">verify</span>
+</pre></div>
+
+
+</li>
+</ol>
+<p>Slider does not need <code>mvn install</code> to be run before executing the
+functional tests. In fact, this may interfere with the tests picking
+up the most recent changes to the code. However, if you want to run tests
+for an individual module instead of running all Slider tests at once, you
+will need to install first. Make sure to install from the top level Slider
+directory so that all code changes are included. To run all functional
+tests, simply run <code>mvn clean verify</code> at the top level.</p>
+<p>If you are testing an individual module, and you want to
+propagate changes in slider-core through to the funtest classes for
+testing, you must build/install all the slider packages from the root assembly.
+A common mistake during development is to rebuild the <code>slider-core</code> JARs
+then the <code>slider-funtest</code> tests without rebuilding the <code>slider-assembly</code>.
+In this situation, the tests are in sync with the latest build of the code
+-including any bug fixes- but the scripts executed by those tests are
+of a previous build of <code>slider-core.jar</code>. As a result, the fixes are not picked
+up.</p>
+<h2 id="limitations-of-slider-funtest">Limitations of slider-funtest</h2>
+<ol>
+<li>All tests run from a single client -workload can't scale</li>
+<li>Output from failed AM and containers aren't collected</li>
+</ol>
+<h2 id="troubleshooting-the-functional-tests">Troubleshooting the functional tests</h2>
+<ol>
+<li>
+<p>If application instances fail to come up as there are still outstanding
+requests, it means that YARN didn't have the RAM/cores to spare for the number
+of containers. Edit the <code>slider.test.yarn.ram</code> to make it smaller.</p>
+</li>
+<li>
+<p>If you are testing in a local VM and stops responding, it'll have been
+swapped out to RAM. Rebooting can help, but for a long term fix go through
+all the Hadoop configurations (HDFS, YARN, Zookeeper) and set their heaps to
+smaller numbers, like 256M each. Also: turn off unused services (hcat, oozie,
+webHDFS)</p>
+</li>
+<li>
+<p>The YARN UI will list the cluster launches -look for the one
+with a name close to the test and view its logs</p>
+</li>
+<li>
+<p>Container logs will appear "elsewhere". The log lists
+the containers used -you may be able to track the logs
+down from the specific nodes.</p>
+</li>
+<li>
+<p>If you browse the filesystem, look for the specific test clusters
+in <code>~/.slider/cluster/$testname</code></p>
+</li>
+<li>
+<p>If you are using a secure cluster, make sure that the clocks
+are synchronized, and that you have a current token -<code>klist</code> will
+tell you this. In a VM: install and enable <code>ntp</code>, consider rebooting if ther
+are any problems. Check also that it has the same time zone settings
+as the host OS.</p>
+</li>
+</ol>
+ </div>
+
+ <div id="footer">
+ <a alt="Apache Software Foundation" href="http://www.apache.org">
+ <img id="asf-logo" alt="Apache Software Foundation" src="/images/feather-small.gif" width="100">
+ </a>
+ <div class="copyright">
+ <p>
+ Copyright © 2011-2014 The Apache Software Foundation, Licensed under
+ the <a href="http://www.apache.org/licenses/LICENSE-2.0">Apache License, Version 2.0</a>.
+ Apache Slider, Slider, Apache, the Apache feather logo, and the Apache Incubator
+ project logo are trademarks of the <a href="http://www.apache.org">Apache Software Foundation</a>.<br />
+ Site created with <a href="http://getbootstrap.com/">Bootstrap</a> including icons from <a href="http://glyphicons.com/">GLYPHICONS</a> and <a href="http://fontawesome.io/">Font Awesome</a>.
+ </p>
+ </div>
+ </div>
+ </div>
+
+</div>
+</div>
+
+<script type="text/javascript">
+
+</script>
+</body>
+</html>