[jira] Commented: (QPID-1161) Test kit for java

["Rajith Attapattu (JIRA)" <qp...@incubator.apache.org>]

    [ https://issues.apache.org/jira/browse/QPID-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12642018#action_12642018 ] 

Rajith Attapattu commented on QPID-1161:
----------------------------------------

Added a latency test to the test kit.

Latency test sends an x number of messages in warmup mode and wait for a confirmation from the consumer that it has successfully consumed them and ready to start the test. It will start sending y number of messages and each message will contain a time stamp. This will be used at the receiving end to measure the latency.
It is important to have a sufficiently large number for the warmup count to ensure the system is in steady state before the test is started.

If you plan to plot the latencies then msg_count should be a smaller number (ex 500 or 1000) You also need to specify a file name using -Dfile=/home/rajith/latency.log.1
 The idea is to get a latency sample for the system once it achieves steady state.

I will create a wiki page on the usage of the test kit. 

> Test kit for java
> -----------------
>
>                 Key: QPID-1161
>                 URL: https://issues.apache.org/jira/browse/QPID-1161
>             Project: Qpid
>          Issue Type: Bug
>          Components: Java Tests
>    Affects Versions: M3
>            Reporter: Rajith Attapattu
>            Assignee: Rajith Attapattu
>             Fix For: M3
>
>         Attachments: testkit.patch
>
>
> The attached Test kit for the java client consists of 2 components.
> 1) A Simple Perf Test that can be used to,
>    a) Run a predefined perf report consisting of 8 use cases (see below)
>    b) Run a producer and a consumer with a number of different options
> 2) Soak tests that can be run for longer durations (hours or days).
> I am planning to add some stress tests to this module as well.
> Please note this is not a replacement for the existing perf/systests etc.
> But rather a small test kit thats focused on providing a packaged set of tests that can be quickly deployed on an environment to do quick smoke testing or easily setup a soak test.
> 1.0 Perf Kit
> ------------
> 1.1 The perf kit can be packaged as an RPM or a tar file and deploy on a target environment and run the perf report.
> Or else a perf report can be automated to run every day or so an record numbers to catch perf regressions.
> 1.2 It calculates the following results in msg/sec.
>     
>     System throuhgput : no_of_msgs / (time_last_msg_rcvd - time_first_msg_send)
>     Producer rate     : no_of_msgs / (time_after_sending - time_before_sending)
>     Producer rate     : no_of_msgs / (time_last_msg_rcvd - time_first_msg_rcvd) 
>     Latency           : time_msg_rcvd - time_msg_sent
> The test will print min, max and avg latency.
> 1.3 The test assume that both producer and consumer are run on the same machine or different machines that are time synced.
> 1.4 You can also use run_sub.sh and run_pub.sh to run different use cases with several options.
>     Please look at TestParams.java for all the configurable options.
> 1.5 You can also use the test kit to benchmark against any vendor.
> 2.0 Soak tests
> --------------
> 2.0 This includes a set of soak tests that can be run for a longer duration.
> 2.1 A typical test will send x-1 messages and the xth message will contain an "End" marker.
>     The producer will print the message_id and timestamp for the xth message.
>     The consumer will reply with an empty message to the replyTo destination given in the xth message.
>     The consumer also prints the message_id and timestamp for the xth message.
>     A typical value for x is 100k 
> 2.2 The feedback loop prevents the producer from overrunning the consumer. 
>     And the printout for every xth message will let you know how many iterations been completed at any given time.
> 2.2 The following results can be calculated for these tests. 
>     
>     Memory, CPU for each producer/consumer - scripts will be checked in
>     Using the message_id and timestamp you can get latency samples for the test run.
>     Using the timestamps you can calculate the system throughput rates.   
> 2.2 If you use different machines for producer and consumer the machines have to be time synced if you want latency samples. 
>  
> 3.0 Perf Test report use cases
> -------------------------------
> 3.1 Please check testkit/bin/perf_report.sh for more details
> 3.2 A typical test run will send 1000 msgs during warmup and 200k msgs for result calculation.
> Test 1 Trans Queue
> Test 2 Dura Queue
> Test 3 Dura Queue Sync
> Test 4 Topic
> Test 5 Durable Topic
> Test 6 Fanout
> Test 7 Small TX  (about 2 msgs per tx)
> Test 8 Large TX  (about 1000 msgs per tx) 
> 4.0 Soak tests use cases
> -------------------------
> 4.1 Following are the current tests available in the test kit.
> 4.2 Please refer to the source to see the javadoc and options
> 1. SimpleProducer/Consumer sends X messages at a time and will wait for confirmation from producer before proceeding with the next iteration. A no of options can be configured.
> 2. MultiThreadedProducer/Consumer does the same thing as above but runs each session in a separate thread.
>    It can also send messages transactionally. Again a no of options can be configured.
> 3. ResourceLeakTest will setup consumer/producers sends x messages and then teard down everything and continue again. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.