Client+sampler lifecycle in activemq-perf-maven-plugin

[Jakub Korab <ja...@gmail.com>]

Hi all,

After far too long of waiting for performance tests to complete, I
decided to have a go at synchronizing the lifecycle of the various
threads in the activemq-perf-maven-plugin.

Fundamentally there are 2 entities in the plugin managed by different
threads/threadpools:

  * the client (generic term for producer of consumer)
  * the samplers (throughput + cpu)

These entities can all be configured separately, and the plugin does not
complete until the slowest of these is done. Now, the problem is that
these things all have separate flags, which generally aren't overridden
unless you want to use a really long command line. For example, to get a
perf test that runs for 10 seconds, you have to do the following:

|mvn activemq-perf:producer -Dproducer.sendDuration=10000
-DtpSampler.duration=10000 -DtpSampler.rampUpTime=0
-DtpSampler.rampDownTime=0 -DcpuSampler.duration=10000
-DcpuSampler.rampUpTime=0 -DcpuSampler.rampDownTime=0

And all this is before you start adding flags to do anything useful with
it. This is because:
|

 1. |sampler threads sleep for the ramp-up time (default 30s); if this
    is longer than the producer's send duration then the sampler won't
    even start doing anything until |the producer has well and truly
    completed
 2. the test runs until the longest configured sampler (by default both
    throughput + cpu run)

So it's kind of a mess. I'd love to be able to use the principle of
least surprise (and configuration) so that you can just do this:

    |mvn activemq-perf:producer -Dproducer.sendDuration=10000
    |

    |mvn activemq-perf:producer -Dproducer.sendType=count
    -Dproducer.sendCount=10000|
    ||

|And the run (including samplers) completes when the client does.
|

|I have had a go at making the samplers subservient to the client. The
most sensible thing that I could come up with for count-based runs is
that since ||timing here is non-deterministic:
|

||

  * ramp up and ramp down times for the samplers are overridden to 0
  * the samplers are manually shut down when a latch on the client is
    triggered

In effect, the sampler timing config is ignored. I have made these
changes and they work pretty sensibly; they make sense to me as a user.

For the timing-based runs, the duration and ramp up/down times of the
samplers is a bit trickier, and here I'd appreciate some input. Here's
what I'm thinking:

  * sampler duration is a bit of a red herring in general; I can't think
    of any time that I want a sampler to run for less than the client
    (certainly not longer)
  * ramp up/down times should never exceed the client run time; perhaps
    these could be a percentage of the run time?

Any thoughts on this would be appreciated.

Thanks,

Jakub