You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@qpid.apache.org by ra...@apache.org on 2008/07/07 22:30:40 UTC
svn commit: r674622 - in /incubator/qpid/trunk/qpid/java/testkit: README
bin/soak_report.sh etc/test.log4j
src/main/java/org/apache/qpid/testkit/soak/MultiThreadedConsumer.java
src/main/java/org/apache/qpid/testkit/soak/SimpleConsumer.java
Author: rajith
Date: Mon Jul 7 13:30:40 2008
New Revision: 674622
URL: http://svn.apache.org/viewvc?rev=674622&view=rev
Log:
This is related to QPID-1162
Added a README file to describe what the tests are and how they can be run.
Modified to consumers to print the iteration number instead of the message id.
Added:
incubator/qpid/trunk/qpid/java/testkit/README
Modified:
incubator/qpid/trunk/qpid/java/testkit/bin/soak_report.sh
incubator/qpid/trunk/qpid/java/testkit/etc/test.log4j
incubator/qpid/trunk/qpid/java/testkit/src/main/java/org/apache/qpid/testkit/soak/MultiThreadedConsumer.java
incubator/qpid/trunk/qpid/java/testkit/src/main/java/org/apache/qpid/testkit/soak/SimpleConsumer.java
Added: incubator/qpid/trunk/qpid/java/testkit/README
URL: http://svn.apache.org/viewvc/incubator/qpid/trunk/qpid/java/testkit/README?rev=674622&view=auto
==============================================================================
--- incubator/qpid/trunk/qpid/java/testkit/README (added)
+++ incubator/qpid/trunk/qpid/java/testkit/README Mon Jul 7 13:30:40 2008
@@ -0,0 +1,153 @@
+Introduction
+============
+
+The Test kit for the java client consists of 2 components.
+
+1) A Simple Perf Test that can be used to,
+ a) Run a predefined perf report consisting of 8 use cases (see below)
+ b) Run a producer and a consumer with a number of different options
+
+2) Soak tests that can be run for longer durations (hours or days).
+
+I am planning to add some stress tests to this module as well.
+Please note this is not a replacement for the existing perf/systests etc.
+But rather a small test kit thats focused on providing a packaged set of tests that can be quickly deployed on an environment to do quick smoke testing or easily setup a soak test.
+
+Table of Contents
+=================
+1. Perf Kit
+2. Soak Kit
+3. Perf Test use cases
+4. Soak Test use cases
+5. Running the sample perf test report
+6. Running the sample soak test report
+
+1.0 Perf Kit
+------------
+1.1 The perf kit can be packaged as an RPM or a tar file and deploy on a target environment and run the perf report.
+Or else a perf report can be automated to run every day or so an record numbers to catch perf regressions.
+
+1.2 It calculates the following results in msg/sec.
+
+ System throuhgput : no_of_msgs / (time_last_msg_rcvd - time_first_msg_send)
+
+ Producer rate : no_of_msgs / (time_after_sending - time_before_sending)
+
+ Producer rate : no_of_msgs / (time_last_msg_rcvd - time_first_msg_rcvd)
+
+ Latency : time_msg_rcvd - time_msg_sent
+
+The test will print min, max and avg latency.
+
+1.3 The test assume that both producer and consumer are run on the same machine or different machines that are time synced.
+
+1.4 You can also use run_sub.sh and run_pub.sh to run different use cases with several options.
+ Please look at TestParams.java for all the configurable options.
+
+1.5 You can also use the test kit to benchmark against any vendor.
+
+
+2.0 Soak tests
+--------------
+2.0 This includes a set of soak tests that can be run for a longer duration.
+
+2.1 A typical test will send x-1 messages and the xth message will contain an "End" marker.
+ The producer will print the timestamp as soon as it sends the xth message.
+ The consumer will reply with an empty message to the replyTo destination given in the xth message.
+ The consumer prints the throuhgput for the iteration and the latency for the xth message.
+ A typical value for x is 100k
+
+2.2 The feedback loop prevents the producer from overrunning the consumer.
+ And the printout for every xth message will let you know how many iterations been completed at any given time.
+ (Ex a simple cat log | wc -l will give you the how many iterations have been completed so far).
+
+2.2 The following results can be calculated for these tests.
+
+ Memory, CPU for each producer/consumer - look at testkit/bin/run_soak_client.sh for an example
+
+ You can find the Avg, Min & Max for throughput, latency, CPU and memory for the entire test run.
+ (look at testkit/bin/soak_report.sh) for an example).
+
+ You could also graph throughput, latency, CPU and memory using the comma separated log files.
+
+2.2 If you use different machines for producer and consumer the machines have to be time synced to have meaningful latency samples.
+
+3.0 Perf Test report use cases
+-------------------------------
+3.1 Please check testkit/bin/perf_report.sh for more details
+
+3.2 A typical test run will send 1000 msgs during warmup and 200k msgs for result calculation.
+
+Test 1 Trans Queue
+
+Test 2 Dura Queue
+
+Test 3 Dura Queue Sync
+
+Test 4 Topic
+
+Test 5 Durable Topic
+
+Test 6 Fanout
+
+Test 7 Small TX (about 2 msgs per tx)
+
+Test 8 Large TX (about 1000 msgs per tx)
+
+
+4.0 Soak tests use cases
+-------------------------
+4.1 Following are the current tests available in the test kit.
+
+4.2 Please refer to the source to see the javadoc and options
+
+
+1. SimpleProducer/Consumer sends X messages at a time and will wait for confirmation from producer before proceeding with the next iteration. A no of options can be configured.
+
+2. MultiThreadedProducer/Consumer does the same thing as above but runs each session in a separate thread.
+ It can also send messages transactionally. Again a no of options can be configured.
+
+3. ResourceLeakTest will setup consumer/producers sends x messages and then teard down everything and continue again.
+
+
+5.0 Running the sample perf test report
+---------------------------------------
+The testkit/bin contains perf_report.sh.
+It runs the above 8 use cases against a broker and print the results in a tabular format.
+
+For example
+================================================================================================
+|Test |System throuput|Producer rate|Consumer Rate|Avg Latency|Min Latency|Max Latency|
+------------------------------------------------------------------------------------------------
+|Trans_Queue | xxxxx.xx| xxxxx.xx| xxxxx.xx| xx.xx| x| xx|
+
+
+5.1 running perf_report.sh
+
+5.1.1 set JAVA_HOME to point to Java 1.5 and above
+5.1.2 set QPID_TEST_HOME to point to the testkit dir
+5.1.3 set VENDOR_LIB to point to the Qpid (or other JMS providers) jar files.
+5.1.4 start a broker
+5.1.5 update the testkit/etc/jndi.properties to point to the correct broker
+5.1.6 execute perf_report.sh
+
+
+6.0 Running the sample soak test report
+---------------------------------------
+The testkit/bin contains soak_report.sh
+It runs MultiThreadedProducer/Consumer for the duration specified and prints a report for the following stats.
+Avg, Min and Max for System Throughput, letency, CPU and memory.
+
+6.1 running soak_report.sh
+
+5.1.1 set JAVA_HOME to point to Java 1.5 and above
+5.1.2 set QPID_TEST_HOME to point to the testkit dir
+5.1.3 set JAR_PATH to point to the Qpid jars
+5.1.4 start a broker
+5.1.5 execute soak_report.sh with correct params.
+ Ex sh soak_report.sh 1 36000 will run for 10 hours colllecting CPU, memory every second.
+
+5.1.6 Please note the total duration for the test is log_freq * log_iterations
+ So if you want to run the test for 10 hours and collect 10 second samples then do the following
+ sh soak_report.sh 10 3600
+
Modified: incubator/qpid/trunk/qpid/java/testkit/bin/soak_report.sh
URL: http://svn.apache.org/viewvc/incubator/qpid/trunk/qpid/java/testkit/bin/soak_report.sh?rev=674622&r1=674621&r2=674622&view=diff
==============================================================================
--- incubator/qpid/trunk/qpid/java/testkit/bin/soak_report.sh (original)
+++ incubator/qpid/trunk/qpid/java/testkit/bin/soak_report.sh Mon Jul 7 13:30:40 2008
@@ -130,12 +130,12 @@
cleanup()
{
- kill -9 `ps aux | grep soak_client | awk '{ print $2 }'`
+ kill -9 `ps aux | grep soak | awk '{ print $2 }'`
}
print_results()
{
- printf "\n======================================================= \n"
+ printf "\n======================================================= \n"
print_rates MultiThreadedConsumer_1.log
printf "\nConsumer process stats "
printf "\n----------------------- \n"
Modified: incubator/qpid/trunk/qpid/java/testkit/etc/test.log4j
URL: http://svn.apache.org/viewvc/incubator/qpid/trunk/qpid/java/testkit/etc/test.log4j?rev=674622&r1=674621&r2=674622&view=diff
==============================================================================
--- incubator/qpid/trunk/qpid/java/testkit/etc/test.log4j (original)
+++ incubator/qpid/trunk/qpid/java/testkit/etc/test.log4j Mon Jul 7 13:30:40 2008
@@ -21,7 +21,7 @@
log4j.logger.org.apache.qpid=ERROR, console
log4j.additivity.org.apache.qpid=false
-log4j.logger.org.apache.qpidity=ERROR, console
+log4j.logger.org.apache.qpidity=TRACE, console
log4j.additivity.org.apache.qpidity=false
log4j.appender.console=org.apache.log4j.ConsoleAppender
Modified: incubator/qpid/trunk/qpid/java/testkit/src/main/java/org/apache/qpid/testkit/soak/MultiThreadedConsumer.java
URL: http://svn.apache.org/viewvc/incubator/qpid/trunk/qpid/java/testkit/src/main/java/org/apache/qpid/testkit/soak/MultiThreadedConsumer.java?rev=674622&r1=674621&r2=674622&view=diff
==============================================================================
--- incubator/qpid/trunk/qpid/java/testkit/src/main/java/org/apache/qpid/testkit/soak/MultiThreadedConsumer.java (original)
+++ incubator/qpid/trunk/qpid/java/testkit/src/main/java/org/apache/qpid/testkit/soak/MultiThreadedConsumer.java Mon Jul 7 13:30:40 2008
@@ -80,6 +80,7 @@
private boolean startIteration = true;
private long startTime = 0;
+ private long iterations = 0;
public void onMessage(Message m)
{
@@ -98,9 +99,10 @@
long totalIterationTime = now - startTime;
double throughput = ((double)msg_count/(double)totalIterationTime) * 1000;
long latencySample = now - m.getJMSTimestamp();
+ iterations++;
StringBuilder sb = new StringBuilder();
- sb.append(m.getJMSMessageID()).append(",").
+ sb.append(iterations).append(",").
append(nf.format(throughput)).append(",").append(latencySample);
System.out.println(sb.toString());
Modified: incubator/qpid/trunk/qpid/java/testkit/src/main/java/org/apache/qpid/testkit/soak/SimpleConsumer.java
URL: http://svn.apache.org/viewvc/incubator/qpid/trunk/qpid/java/testkit/src/main/java/org/apache/qpid/testkit/soak/SimpleConsumer.java?rev=674622&r1=674621&r2=674622&view=diff
==============================================================================
--- incubator/qpid/trunk/qpid/java/testkit/src/main/java/org/apache/qpid/testkit/soak/SimpleConsumer.java (original)
+++ incubator/qpid/trunk/qpid/java/testkit/src/main/java/org/apache/qpid/testkit/soak/SimpleConsumer.java Mon Jul 7 13:30:40 2008
@@ -74,6 +74,7 @@
private boolean startIteration = true;
private long startTime = 0;
+ private long iterations = 0;
public void onMessage(Message m)
{
@@ -93,9 +94,10 @@
startIteration = true;
double throughput = ((double)msg_count/(double)totalIterationTime) * 1000;
long latencySample = now - m.getJMSTimestamp();
+ iterations++;
StringBuilder sb = new StringBuilder();
- sb.append(m.getJMSMessageID()).append(",").
+ sb.append(iterations).append(",").
append(nf.format(throughput)).append(",").append(latencySample);
System.out.println(sb.toString());