You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@qpid.apache.org by ph...@apache.org on 2013/07/25 22:53:51 UTC
svn commit: r1507122 - in /qpid/trunk/qpid: doc/book/ doc/book/src/
doc/book/src/java-perftests/ java/perftests/
Author: philharveyonline
Date: Thu Jul 25 20:53:50 2013
New Revision: 1507122
URL: http://svn.apache.org/r1507122
Log:
QPID-4989: moved Java perf test README into a docbook so it can be published if required.
Added:
qpid/trunk/qpid/doc/book/src/java-perftests/
qpid/trunk/qpid/doc/book/src/java-perftests/JMS-Performance-Test-Framework.xml
qpid/trunk/qpid/doc/book/src/java-perftests/Makefile
- copied, changed from r1507085, qpid/trunk/qpid/doc/book/Makefile
Removed:
qpid/trunk/qpid/java/perftests/README-java-perftests.txt
Modified:
qpid/trunk/qpid/doc/book/Makefile
qpid/trunk/qpid/doc/book/src/Makefile.inc
Modified: qpid/trunk/qpid/doc/book/Makefile
URL: http://svn.apache.org/viewvc/qpid/trunk/qpid/doc/book/Makefile?rev=1507122&r1=1507121&r2=1507122&view=diff
==============================================================================
--- qpid/trunk/qpid/doc/book/Makefile (original)
+++ qpid/trunk/qpid/doc/book/Makefile Thu Jul 25 20:53:50 2013
@@ -17,7 +17,7 @@
# under the License.
#
-DIRS = src/java-broker src/cpp-broker src/programming
+DIRS = src/java-broker src/java-perftests src/cpp-broker src/programming
.PHONY: all $(DIRS)
Modified: qpid/trunk/qpid/doc/book/src/Makefile.inc
URL: http://svn.apache.org/viewvc/qpid/trunk/qpid/doc/book/src/Makefile.inc?rev=1507122&r1=1507121&r2=1507122&view=diff
==============================================================================
--- qpid/trunk/qpid/doc/book/src/Makefile.inc (original)
+++ qpid/trunk/qpid/doc/book/src/Makefile.inc Thu Jul 25 20:53:50 2013
@@ -17,7 +17,7 @@
# under the License.
#
-BOOK=$(wildcard *Book.xml Programming-In-Apache-Qpid.xml)
+BOOK=$(wildcard *Book.xml Programming-In-Apache-Qpid.xml JMS-Performance-Test-Framework.xml)
XML=$(wildcard *.xml) $(wildcard ../common/*.xml)
IMAGES=$(wildcard images/*.png)
CSS=$(wilcard ../common/css/*.css)
Added: qpid/trunk/qpid/doc/book/src/java-perftests/JMS-Performance-Test-Framework.xml
URL: http://svn.apache.org/viewvc/qpid/trunk/qpid/doc/book/src/java-perftests/JMS-Performance-Test-Framework.xml?rev=1507122&view=auto
==============================================================================
--- qpid/trunk/qpid/doc/book/src/java-perftests/JMS-Performance-Test-Framework.xml (added)
+++ qpid/trunk/qpid/doc/book/src/java-perftests/JMS-Performance-Test-Framework.xml Thu Jul 25 20:53:50 2013
@@ -0,0 +1,304 @@
+<?xml version="1.0"?>
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+<book xmlns:xi="http://www.w3.org/2001/XInclude">
+ <title>JMS Performance Test Framework</title>
+ <chapter>
+ <para>
+ The distributed test (aka Perf Test) framework was written for
+ testing the performance of a JMS provider in various common scenarios.
+ Although it was originally written for the purpose of testing Qpid,
+ it can be used to test the performance of any JMS provider with minimal configuration changes.
+ <para>
+ </para>
+ This document explains how to use the framework.
+ </para>
+
+ <section id="how-it-works">
+ <title>How it works</title>
+ <para>
+ First, you need to <emphasis>run a message broker</emphasis>. This can be Qpid, ActiveMQ etc
+ (although see <xref linkend="caveats-for-non-qpid-jms-providers"/>).
+ All messages are sent using the JMS API.
+ <para>
+ </para>
+ Then run a <emphasis>Perf Test Controller</emphasis>, providing the details of the test in one or more
+ JSON or Javascript files (see <xref linkend="test-definitions">Test definitions</xref>).
+ <para>
+ </para>
+ Now <emphasis>run one or more Perf Test Client processes</emphasis>. These will be responsible for
+ sending/receiving the messages once the test starts. For convenience, you can
+ instead configure the Controller to start clients in-process. The clients and
+ the controller communicate using queues on the message broker.
+ <para>
+ </para>
+ The test results are written to CSV files or optionally to a
+ database (see <xref linkend="writing-results-to-a-jdbc-database"/>).
+ <para>
+ </para>
+ You can <emphasis>use the qpid-perftests-visualisation tool</emphasis>
+ (<xref linkend="visualising-test-results"/>) to create charts from the results.
+ </para>
+ </section>
+
+ <section id="test-definitions">
+ <title>Test definitions</title>
+ <para>
+ Test definition files specify details about the messages to send,
+ how many connections and sessions to use etc. There are a lot of options
+ available - see the .js and .json files under the <filename>perftests/etc/testdefs/</filename> folder for examples.
+ </para>
+ <para>
+ Each JSON file contains a list of tests, expressed as a JSON structure. Alternatively, JavaScript can be used
+ to generate this data structure. If a file has a .js extension it is parsed as JavaScript and the
+ resulting object <code>jsonObject</code> used as the test specification. JavaScript is useful for reducing
+ duplication in test specifications (e.g. by looping or by moving repeating literals into variables).
+ </para>
+ <para>
+ If the ControllerRunner is pointed at a directory instead of a file, each test specification file in that
+ directory is used.
+ </para>
+ </section>
+
+ <section id="example-usage">
+ <title>Example usage</title>
+ <para>
+ The <filename>perftests/etc/</filename> folder contains shell scripts that can be used to run the performance
+ tests and visualise the results. It also contains sub-folders for test config
+ and chart definitions.
+ </para>
+ </section>
+
+ <section id="instructions">
+ <title>Instructions</title>
+ <para>
+ <itemizedlist>
+ <listitem>Extract the perftests archive</listitem>
+ <listitem>Start your JMS broker</listitem>
+ <listitem>cd into the <filename>perftests/etc/</filename> folder</listitem>
+ <listitem>
+ <para>To run the Controller and clients in a single process, run the following command:
+ </para>
+ <screen>
+ $ java -cp ".:../lib/*:/path/to/your-jms-client-jars/*" \
+ -Dqpid.dest_syntax=BURL \ # used if the test specifications use Qpid's BURL format for queues
+ org.apache.qpid.disttest.ControllerRunner \
+ jndi-config=perftests-jndi.properties \
+ test-config=/path/to/test-config.json \
+ distributed=false
+ </screen>
+ <para>
+ When the test is complete, the CSV files containing the results are written to
+ the current directory.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+
+ <section id="running-clients-in-a-separate-process">
+ <title>Running the clients in a separate process</title>
+ <para>
+ When using a large number of clients, you may get more representative
+ performance results if the clients are distributed among multiple processes,
+ potentially on multiple machines. To do this:
+ <itemizedlist>
+ <listitem>Run the Controller, providing <envar>distributed=true</envar>.</listitem>
+ <listitem>
+ <para>Run your clients (assuming you want to launch 10 logical clients in this process):</para>
+ <screen>
+ $ cd perftests/etc
+ $ java -cp ".:../lib/*:/path/to/your-jms-client-jars/*" \
+ -Dqpid.dest_syntax=BURL \
+ org.apache.qpid.disttest.ClientRunner \
+ jndi-config=perftests-jndi.properties \
+ number-of-clients=10
+ </screen>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </section>
+ </section>
+
+ <section id="writing-results-to-a-jdbc-database">
+ <title>Writing results to a JDBC database</title>
+ <para>
+ For most use cases, writing results to a CSV file is acceptable. However, there are some cases
+ where it is desired to write to a database instead. For example, if you need to keep track of how
+ results have varied over time, writing to a database table allows this to be easily discovered by
+ running a SQL query and/or producing a time series chart with the visualisation tool
+ (see <xref linkend="visualising-test-results-queries"/>).
+ </para>
+ <para>
+ To write results to a database:
+ <itemizedlist>
+ <listitem>Add <code>writeToDb=true</code> to the <code>ControllerRunner</code> parameters</listitem>
+ <listitem>
+ Add <code>jdbcDriverClass</code> and <code>jdbcUrl</code> properties to your JNDI configuration file
+ (obviously adding the JDBC driver class to your classpath too).
+ </listitem>
+ <listitem>
+ Note that the framework automatically creates the results table <code>RESULTS</code>
+ if it does not already exist.
+ </listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id="caveats-for-non-qpid-jms-providers">
+ <title>Caveats for non-Qpid JMS providers</title>
+ <para>
+ If you are not using the Qpid broker, you must create one or more queues before
+ running the test. This is necessary because you can't use Qpid's API to create
+ queues on the broker. The queues are:
+ <itemizedlist>
+ <listitem>
+ The controller queue. You can specify the physical name of this in
+ <filename>perftests/etc/perftests-jndi.properties</filename>.
+ This queue is used by the clients to register with the Controller and to send results to it.
+ </listitem>
+ <listitem>
+ The queue(s) used by your JSON test configuration
+ (unless you have configured a vendor-specific queue creator).
+ </listitem>
+ </itemizedlist>
+ </para>
+ <para>
+ You must also override the Controller's default queue creator using the system
+ property <envar>qpid.disttest.queue.creator.class</envar>. Provide the class name of an
+ implementation of <classname>org.apache.qpid.disttest.jms.QueueCreator</classname>, or
+ <classname>org.apache.qpid.disttest.jms.NoOpQueueCreator</classname> if you are going to create and
+ delete the queues manually.
+ </para>
+ <para>
+ You can also omit the <envar>qpid.dest_syntax</envar> system property if your JMS provider is
+ not Qpid.
+ </para>
+ </section>
+
+ <section id="when-production-is-slower-than-consumption">
+ <title>When message production is slower than consumption</title>
+ <para>
+ A given test configuration may cause messages to be produced faster than they are consumed.
+ Unless you deal with this, the broker may become overwhelmed (e.g. running out of disk space or memory).
+ The steps you can take to mitigate this are:
+ <itemizedlist>
+ <listitem>
+ Apply a small delay between each message publication, using the Producer JSON property <code>_interval</code>.
+ </listitem>
+ <listitem>
+ Use your JMS provider's built-in features for throttling incoming messages.
+ In the case of Qpid, this is done by configuring the queue to enforce flow control, like so:
+ <screen><![CDATA[
+ {
+ "_tests":[
+ {
+ "_name": "My test",
+ "_queues":[
+ {
+ "_name": "direct://amq.direct//myTestQueue",
+ "_attributes":
+ {
+ "x-qpid-capacity": 10000000,
+ "x-qpid-flow-resume-capacity": 8000000
+ }
+ }
+ ],
+ ...
+ }
+ }]]>
+ </screen>
+ </listitem>
+ </itemizedlist>
+ </para>
+ </section>
+
+ <section id="visualising-test-results">
+ <title>Visualising test results</title>
+ <para>
+ The module visualisation-jfc can be used to generate charts as .png files from the results
+ produced by running the perf tests (or in fact from any CSV file with a header row or database table).
+ At runtime the visualisation module accepts the following input:
+ <itemizedlist>
+ <listitem>The test results (either a CSV file or a database table)</listitem>
+ <listitem>
+ The chart definition, which is a properties file specifying settings such as the query to run against the data,
+ and the type of chart to generate.
+ <para>
+ The quickest way to create a new chart definition is to base it on an existing example in
+ <filename>perftests/etc/chartdefs/</filename>.
+ </para>
+ </listitem>
+ </itemizedlist>
+ </para>
+ <section>
+ <title>Chart types</title>
+ <para>
+ Currently, the available chart types are:
+ <itemizedlist>
+ <listitem>LINE</listitem>
+ <listitem>LINE3D</listitem>
+ <listitem>BAR</listitem>
+ <listitem>BAR3D</listitem>
+ <listitem>XYLINE</listitem>
+ <listitem>TIMELINE</listitem>
+ <listitem>STATISTICAL_BAR</listitem>
+ </itemizedlist>
+ </para>
+ </section>
+ <section id="visualising-test-results-queries">
+ <title>Queries</title>
+ <para>
+ The query in the chart definition file is an ANSI SQL query. If reading from a CSV file the SQL table name
+ is simply the file name and the SQL column names are the CSV header row entries.
+ </para>
+ </section>
+ <section>
+ <title>Running the visualisation tool</title>
+ <para>
+ Here is an example of how to run the visualisation tool.
+ <screen>
+$ cd perftests/etc
+$ BASE_DIR=`pwd`
+$ java -cp "${BASE_DIR}:/path/to/extracted/visualistion-module/lib/*" \
+ -Djava.awt.headless=true -Dlog4j.configuration=file:log4j.properties \
+ -DcsvCurrentDir=/path/to/csv-files-directory/ \ # referenced in chart definition file
+ -DcsvBaselineDir=/path/to/baseline-csv-files-directory/ \ # referenced in chart definition file
+ -DbaselineName="My baseline name" \ # referenced in chart definition file
+ org.apache.qpid.disttest.charting.ChartingUtil \
+ chart-defs=/path/to/chartdefs-files/ \
+ </screen>
+ </para>
+ <para>
+ To fetch the results from a database table instead of a CSV file, modify the visualistion tool
+ parameters like so:
+ <screen>
+ $ java -cp "${BASE_DIR}:/path/to/extracted/visualistion-module/lib/*" \
+ ...
+ org.apache.qpid.disttest.charting.ChartingUtil \
+ jdbcUrl=jdbc:your-jdbc-url
+ jdbcDriverClass=your.jdbc.driver.Classname
+ </screen>
+ </para>
+ </section>
+ </section>
+ </chapter>
+</book>
Copied: qpid/trunk/qpid/doc/book/src/java-perftests/Makefile (from r1507085, qpid/trunk/qpid/doc/book/Makefile)
URL: http://svn.apache.org/viewvc/qpid/trunk/qpid/doc/book/src/java-perftests/Makefile?p2=qpid/trunk/qpid/doc/book/src/java-perftests/Makefile&p1=qpid/trunk/qpid/doc/book/Makefile&r1=1507085&r2=1507122&rev=1507122&view=diff
==============================================================================
--- qpid/trunk/qpid/doc/book/Makefile (original)
+++ qpid/trunk/qpid/doc/book/src/java-perftests/Makefile Thu Jul 25 20:53:50 2013
@@ -17,22 +17,4 @@
# under the License.
#
-DIRS = src/java-broker src/cpp-broker src/programming
-
-
-.PHONY: all $(DIRS)
-
-all: $(DIRS)
-
-clean:
- rm -rf build
-
-html: TARGET = html
-html: all
-
-pdf: TARGET = pdf
-pdf: all
-
-$(DIRS):
- $(MAKE) -C $@ $(TARGET) OUTPUTDIR=../../build/
-
+include ../Makefile.inc
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@qpid.apache.org
For additional commands, e-mail: commits-help@qpid.apache.org