You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@accumulo.apache.org by bh...@apache.org on 2014/01/07 15:23:03 UTC
[04/10] git commit: ACCUMULO-2042 Get scalability test running
ACCUMULO-2042 Get scalability test running
Two problems prevented scalability tests from being run properly.
1. The Run class was looking for test classes under the "accumulo"
package instead of "org.apache.accumulo".
2. A results subdirectory on the local filesystem was not being
created, and so result data from HDFS could not be copied out.
This commit also updates the scalability test README with more
detail.
Project: http://git-wip-us.apache.org/repos/asf/accumulo/repo
Commit: http://git-wip-us.apache.org/repos/asf/accumulo/commit/a3d77e4f
Tree: http://git-wip-us.apache.org/repos/asf/accumulo/tree/a3d77e4f
Diff: http://git-wip-us.apache.org/repos/asf/accumulo/diff/a3d77e4f
Branch: refs/heads/master
Commit: a3d77e4ff2a12f161f1343490f42aa91d5b109d3
Parents: 8f9fe41
Author: Bill Havanki <bh...@cloudera.com>
Authored: Thu Dec 19 11:15:44 2013 -0500
Committer: Bill Havanki <bh...@cloudera.com>
Committed: Tue Jan 7 09:10:04 2014 -0500
----------------------------------------------------------------------
.../accumulo/server/test/scalability/Run.java | 2 +-
test/system/scalability/README | 35 ++++++++++++++------
test/system/scalability/run.py | 2 ++
3 files changed, 28 insertions(+), 11 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/accumulo/blob/a3d77e4f/src/server/src/main/java/org/apache/accumulo/server/test/scalability/Run.java
----------------------------------------------------------------------
diff --git a/src/server/src/main/java/org/apache/accumulo/server/test/scalability/Run.java b/src/server/src/main/java/org/apache/accumulo/server/test/scalability/Run.java
index 63861f8..7fb78c7 100644
--- a/src/server/src/main/java/org/apache/accumulo/server/test/scalability/Run.java
+++ b/src/server/src/main/java/org/apache/accumulo/server/test/scalability/Run.java
@@ -59,7 +59,7 @@ public class Run {
e.printStackTrace();
}
- ScaleTest test = (ScaleTest) Class.forName(String.format("accumulo.server.test.scalability.%s", testId)).newInstance();
+ ScaleTest test = (ScaleTest) Class.forName(String.format("org.apache.accumulo.server.test.scalability.%s", testId)).newInstance();
test.init(scaleProps, testProps, numTabletServers);
http://git-wip-us.apache.org/repos/asf/accumulo/blob/a3d77e4f/test/system/scalability/README
----------------------------------------------------------------------
diff --git a/test/system/scalability/README b/test/system/scalability/README
index 7206daa..cd7440c 100644
--- a/test/system/scalability/README
+++ b/test/system/scalability/README
@@ -1,23 +1,38 @@
-1. To run scalability tests, set up the site configuration (used by all tests)
+Apache Accumulo Scalability Tests
- cp conf/site.conf.example conf/site.conf
+The scalability test framework needs to be configured for your Accumulo
+instance by performing the following steps.
+
+WARNING: Each scalability test rewrites your conf/slaves file and reinitializes
+your Accumulo instance. Do not run these tests on a cluster holding essential
+data.
+
+1. Make sure you have both ACCUMULO_HOME and HADOOP_HOME set in your
+ $ACCUMULO_CONF_DIR/accumulo-env.sh.
-2. Modify site.conf to the correct configuration parameters for your Accumulo instance
+2. Create a 'site.conf' file in the conf directory containing settings
+ needed by test nodes to connect to Accumulo, and to guide the tests.
+
+ cp conf/site.conf.example conf/site.conf
-3. Set up the configuration file for the test you want to run
+3. Create an 'Ingest.conf' file in the conf directory containing performance
+ settings for the Ingest test. (This test is currently the only scalability
+ test available.)
cp conf/Ingest.conf.example conf/Ingest.conf
-4. Each test has a unique ID which correlates with its test code in:
+ Each test has a unique ID (e.g., "Ingest") which correlates with its test
+ code in:
org.apache.accumulo.server.test.scalability.tests.<ID>
- This ID correlates with a the following config file:
+ This ID correlates with a config file:
- ./conf/<ID>.conf
+ conf/<ID>.conf
-5. To run the test, just specifying its ID to the run.py script
+To run the test, specify its ID to the run.py script.
- export ACCUMULO_HOME=/opt/accumulo; nohup ./run.py Ingest > test1.log 2>&1 &
+ nohup ./run.py Ingest > test1.log 2>&1 &
-5. A directory will be create where logs and results are placed
+A timestamped directory will be created, and results are placed in it as each
+test completes.
http://git-wip-us.apache.org/repos/asf/accumulo/blob/a3d77e4f/test/system/scalability/run.py
----------------------------------------------------------------------
diff --git a/test/system/scalability/run.py b/test/system/scalability/run.py
index ac45aa2..938805a 100755
--- a/test/system/scalability/run.py
+++ b/test/system/scalability/run.py
@@ -189,8 +189,10 @@ def main():
log('Creating test directory structure')
testDir = 'test-%d' % time.time()
nodesDir = testDir+'/nodes'
+ resultsDir = testDir+'/results'
syscall('mkdir %s' % testDir)
syscall('mkdir %s' % nodesDir)
+ syscall('mkdir %s' % resultsDir)
log('Removing current /accumulo-scale directory')
syscall('hadoop fs -rmr /accumulo-scale')