Best place for stand-alone performance tests?

[Knut Anders Hatlen <Kn...@Sun.COM>]

Hi,

As part of my work on a new buffer manager (DERBY-2911), I'm trying to
merge (and extend) the performance test clients attached to DERBY-1961
and DERBY-2911 into a single suite of tests so that they can share code
for generating load with different distributions (back-to-back
transactions and Poisson-distributed transactions in the first round)
and collecting/reporting results. Since these clients might be useful to
others, I thought it might be a good idea to add them to the source
repository, but I'm not sure what's the appropriate location for such
test clients.

The JUnit performance tests reside in
org.apache.derbyTesting.perf.basic, so I thought perhaps
org.apache.derbyTesting.perf.clients would be OK for stand-alone
clients. Other suggestions?

Someone may wonder why I intend to write these test clients as
stand-alone applications rather than JUnit tests. The main reasons are:

  - I want to make it easy to write scripts which run series of tests
    with different parameters (like number of threads, transaction
    injection rate, length of the runs, page cache size, etc). In our
    JUnit test, we normally have a fixed setup, and need to recompile if
    we want to change it.

  - I would like to have the possibility to run the tests against other
    DBMSs than Derby, which wouldn't be trivial in our current JUnit
    framework.

  - They already exist as stand-alone tests on the JIRAs, so the
    conversion is simpler. (Yes, I'm lazy... :)

It shouldn't be difficult to take a couple of fixed configurations and
wiring them into a JUnit test later, though. But since I think it's
easier to wrap the stand-alone clients in a JUnit test than vice versa,
I'd like to do the stand-alone part first.

Thanks,
-- 
Knut Anders