You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by mm...@apache.org on 2019/06/06 22:11:30 UTC
[accumulo] branch master updated: Update TESTING.md (#1183)
This is an automated email from the ASF dual-hosted git repository.
mmiller pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/accumulo.git
The following commit(s) were added to refs/heads/master by this push:
new f7f18d7 Update TESTING.md (#1183)
f7f18d7 is described below
commit f7f18d7889b7c734fcf18052445a59527778e494
Author: Mike Miller <mm...@apache.org>
AuthorDate: Thu Jun 6 18:11:25 2019 -0400
Update TESTING.md (#1183)
* Made the doc more concise and added useful commands
---
TESTING.md | 124 +++++++++++++++++++++++++++----------------------------------
1 file changed, 54 insertions(+), 70 deletions(-)
diff --git a/TESTING.md b/TESTING.md
index 322759e..51d4e16 100644
--- a/TESTING.md
+++ b/TESTING.md
@@ -17,42 +17,41 @@ limitations under the License.
# Testing Apache Accumulo
-This document is meant to serve as a quick reference to the automated test suites included in Apache Accumulo for users
-to run which validate the product and developers to continue to iterate upon to ensure that the product is stable and as
-free of bugs as possible.
-
-The automated testing suite can be categorized as two sets of tests: unit tests and integration tests. These are the
-traditional unit and integrations tests as defined by the Apache Maven [lifecycle][3] phases.
+This document is meant to serve as a quick reference to the automated test suites of Accumulo.
# Unit tests
Unit tests can be run by invoking `mvn test` at the root of the Apache Accumulo source tree. For more information see
-the [maven-surefire-plugin docs][4].
+the [maven-surefire-plugin docs][surefire]. This command will run just the unit tests:
-The unit tests should run rather quickly (order of minutes for the entire project) and, in nearly all cases, do not
-require any noticable amount of computer resources (the compilation of the files typically exceeds running the tests).
-Maven will automatically generate a report for each unit test run and will give a summary at the end of each Maven
-module for the total run/failed/errored/skipped tests.
+```bash
+mvn clean test -Dspotbugs.skip -DskipITs
+```
-The Apache Accumulo developers expect that these tests are always passing on every revision of the code. If this is not
-the case, it is almost certainly in error.
+# SpotBugs (formerly findbugs)
-# Integration tests
+[SpotBugs] will run by default when building Accumulo (unless "-Dspotbugs.skip" is used) and does a thorough static code
+analysis of potential bugs. There is also a security findbugs plugin configured that can be run with this
+command:
-Integration tests can be run by invoking `mvn verify` at the root of the Apache Accumulo source tree. For more
-information see the [maven-failsafe-plugin docs][5].
+```bash
+mvn clean verify -Psec-bugs -DskipTests
+```
-The integration tests are medium length tests (order minutes for each test class and order hours for the complete suite)
-but are checking for regressions that were previously seen in the codebase. These tests do require a noticable amount of
-resources, at least another gigabyte of memory over what Maven itself requires. As such, it's recommended to have at
-least 3-4GB of free memory and 10GB of free disk space.
+# Integration Tests
-## Test Categories
+The integration tests are medium length tests that check for regressions. These tests do require more memory over what
+Maven itself requires. As such, it's recommended to have at least 3-4GB of free memory and 10GB of free disk space.
Accumulo uses JUnit Category annotations to categorize certain integration tests based on their runtime requirements.
-Presently there are several different categories:
+The different categories are listed below. To run a single IT use the following command. This command will run just
+the WriteAheadLogIT:
+
+```bash
+mvn clean verify -Dit.test=WriteAheadLogIT -Dtest=foo -Dspotbugs.skip
+```
-### SunnyDay (`SunnyDayTests`)
+## SunnyDay (`SunnyDayTests`)
This test category represents a minimal set of tests chosen to verify the basic
functionality of Accumulo. These would typically be run prior to submitting a
@@ -60,53 +59,42 @@ patch or pull request, or fixing a bug, to quickly ensure no basic functions
were broken by the change.
These tests will run by default during the `integration-test` lifecycle phase using `mvn verify`.
-To execute only these tests, use `mvn verify -Dfailsafe.groups=org.apache.accumulo.test.categories.SunnyDayTests`
-To execute everything except these tests, use `mvn verify -Dfailsafe.excludedGroups=org.apache.accumulo.test.categories.SunnyDayTests`
+To run all the Sunny day tests, run:
+
+```bash
+mvn clean verify -Psunny
+```
-### MiniAccumuloCluster (`MiniClusterOnlyTests`)
+## MiniAccumuloCluster (`MiniClusterOnlyTests`)
These tests use MiniAccumuloCluster (MAC) which is a multi-process "implementation" of Accumulo, managed
through Java APIs. This MiniAccumuloCluster has the ability to use the local filesystem or Apache Hadoop's
-MiniDFSCluster, as well as starting one to many tablet servers. MiniAccumuloCluster tends to be a very useful tool in
-that it can automatically provide a workable instance that mimics how an actual deployment functions.
-
-The downside of using MiniAccumuloCluster is that a significant portion of each test is now devoted to starting and
-stopping the MiniAccumuloCluster. While this is a surefire way to isolate tests from interferring with one another, it
-increases the actual runtime of the test by, on average, 10x. Some times the tests require the use of MAC because the
-test is being destructive or some special environment setup (e.g. Kerberos).
-
-These tests will run by default during the `integration-test` lifecycle phase using `mvn verify`.
-To execute only these tests, use `mvn verify -Dfailsafe.groups=org.apache.accumulo.test.categories.MiniClusterOnlyTests`
-To execute everything except these tests, use `mvn verify -Dfailsafe.excludedGroups=org.apache.accumulo.test.categories.MiniClusterOnlyTests`
-
-### Standalone Cluster (`StandaloneCapableClusterTests`)
+MiniDFSCluster, as well as starting one to many tablet servers. Most tests will be run in the local directory:
-An alternative to the MiniAccumuloCluster for testing, a standalone Accumulo cluster can also be configured for use by
-most tests. This requires a manual step of building and deploying the Accumulo cluster by hand. The build can then be
-configured to use this cluster instead of always starting a MiniAccumuloCluster. Not all of the integration tests are
-good candidates to run against a standalone Accumulo cluster, these tests will still launch a MiniAccumuloCluster for
-their use.
+```bash
+$ACCUMULO_HOME/test/target/mini-tests
+```
-Use of a standalone cluster can be enabled using system properties on the Maven command line or, more concisely, by
-providing a Java properties file on the Maven command line. The use of a properties file is recommended since it is
-typically a fixed file per standalone cluster you want to run the tests against.
+The downside of using MiniAccumuloCluster is the extra time it takes to start and stop the MAC.
These tests will run by default during the `integration-test` lifecycle phase using `mvn verify`.
-To execute only these tests, use `mvn verify -Dfailsafe.groups=org.apache.accumulo.test.categories.StandaloneCapableClusterTests`
-To execute everything except these tests, use `mvn verify -Dfailsafe.excludedGroups=org.apache.accumulo.test.categories.StandaloneCapableClusterTests`
+To run all the Mini tests, run:
+```bash
+mvn clean verify -Dspotbugs.skip
+```
-### Performance Tests (`PerformanceTests`)
+## Standalone Cluster (`StandaloneCapableClusterTests`)
-This category of tests refer to integration tests written specifically to
-exercise expected performance, which may be dependent on the available
-resources of the host machine. Normal integration tests should be capable of
-running anywhere with a lower-bound on available memory.
+A standalone Accumulo cluster can also be configured for use by most tests. Not all of the integration tests are good
+candidates to run against a standalone Accumulo cluster, these tests will still launch a MiniAccumuloCluster for their use.
-These tests will run by default during the `integration-test` lifecycle phase using `mvn verify`.
-To execute only these tests, use `mvn verify -Dfailsafe.groups=org.apache.accumulo.test.categories.PerformanceTests`
-To execute everything except these tests, use `mvn verify -Dfailsafe.excludedGroups=org.apache.accumulo.test.categories.PerformanceTests`
+These tests can be run by providing a system property. This command will run all tests against a standalone cluster:
-## Configuration for Standalone clusters
+```bash
+mvn clean verify -Dtest=foo -Daccumulo.it.properties=/home/user/my_cluster.properties -Dfailsafe.groups=org.apache.accumulo.test.categories.StandaloneCapableClusterTests -Dspotbugs.skip
+```
+
+### Configuration for Standalone clusters
The following properties can be used to configure a standalone cluster:
@@ -130,7 +118,7 @@ to use. When Kerberos is enabled, these are principal/username and a path to a k
installations, these are just principal/username and password pairs. It is not required to create the users
in Accumulo -- the provided admin user will be used to create the user accounts in Accumulo when necessary.
-Setting 5 users should be sufficient for all of the integration test's purposes. Each property is suffixed
+Setting 2 users should be sufficient for all of the integration test's purposes. Each property is suffixed
with an integer which groups the keytab or password with the username.
- `accumulo.it.cluster.standalone.users.$x` The principal name
@@ -139,14 +127,11 @@ with an integer which groups the keytab or password with the username.
Each of the above properties can be set on the commandline (-Daccumulo.it.cluster.standalone.principal=root), or the
collection can be placed into a properties file and referenced using "accumulo.it.cluster.properties". Properties
-specified on the command line override properties set in a file. For example, the following might be similar to
-what is executed for a standalone cluster.
-
- `mvn verify -Daccumulo.it.properties=/home/user/my_cluster.properties`
+specified on the command line override properties set in a file.
## MapReduce job for Integration tests
-[ACCUMULO-3871][6] (re)introduced the ability to parallelize the execution of the Integration Test suite by the use
+[ACCUMULO-3871][issue] (re)introduced the ability to parallelize the execution of the Integration Test suite by the use
of MapReduce/YARN. When a YARN cluster is available, this can drastically reduce the amount of time to run all tests.
To run the tests, you first need a list of the tests. A simple way to get a list, is to scan the accumulo-test jar file for them.
@@ -168,10 +153,9 @@ These files do exist in the build tree, but at different locations)
# Manual Distributed Testing
Apache Accumulo has a number of tests which are suitable for running against large clusters for hours to days at a time.
-These test suites exist in the [accumulo-testing repo][2].
+These test suites exist in the [accumulo-testing repo][testing].
-[2]: https://github.com/apache/accumulo-testing
-[3]: https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
-[4]: http://maven.apache.org/surefire/maven-surefire-plugin/
-[5]: http://maven.apache.org/surefire/maven-failsafe-plugin/
-[6]: https://issues.apache.org/jira/browse/ACCUMULO-3871
+[testing]: https://github.com/apache/accumulo-testing
+[surefire]: http://maven.apache.org/surefire/maven-surefire-plugin/
+[issue]: https://issues.apache.org/jira/browse/ACCUMULO-3871
+[SpotBugs]: https://spotbugs.github.io/