Annotation support fior the PerformanceRunner

[Francesco Mari <ma...@gmail.com>]

Hi all,

I opened SLING-3762 to provide a patch extending my work already
merged as part of SLING-3756.

This patch adds annotation support to the PerformanceRunner. This
enables an easier way to provide initialization and cleanup methods
for performance tests, instead of writing a listener to perform those
tasks.

I realized that this mechanism was required because when writing
SamplePerformanceTest [1] I had to resort to an anonymous inner class
to act as a listener to provide initialization and cleanup code for my
test. A listener was never supposed to be used for this purpose. The
role of a listener, instead, is to aggregate test results and publish
them somewhere.

Francesco

[1]: https://github.com/apache/sling/blob/trunk/testing/samples/sample-tests/src/main/java/org/apache/sling/testing/samples/sampletests/SamplePerformanceTest.java

Re: Annotation support fior the PerformanceRunner

[Bertrand Delacretaz <bd...@apache.org>]

On Tue, Jul 15, 2014 at 1:39 PM, Francesco Mari
<ma...@gmail.com> wrote:
> Yes, there is a test in the same patch, it is
> PerformanceRunnerDynamicsTest...

Ah sorry, I didn't realize you were talking about an unapplied
patch...I have added some comments at SLING-3762, some tests fail.

-Bertrand

Re: Annotation support fior the PerformanceRunner

[Francesco Mari <ma...@gmail.com>]

Yes, there is a test in the same patch, it is
PerformanceRunnerDynamicsTest. It test the execution of annotated
methods by themselves, how they interact with listeners and how they
work together with standard JUnit annotations.

2014-07-15 11:55 GMT+02:00 Bertrand Delacretaz <bd...@apache.org>:
> Hi,
>
> On Tue, Jul 15, 2014 at 10:03 AM, Francesco Mari
> <ma...@gmail.com> wrote:
>> ...I realized that this mechanism was required because when writing
>> SamplePerformanceTest [1] I had to resort to an anonymous inner class
>> to act as a listener to provide initialization and cleanup code for my
>> test...
>
> [1| looks good to me, thanks for that! Adding it broke the sample
> integration tests but I just fixed that in
> http://svn.apache.org/r1610633
>
> Are the warmup annotations tested elsewhere, to verify that the
> actually do what they claim to do in terms of number of calls, warmup
> time etc? I don't think [2] tests that.
>
> -Bertrand
>
> [1] https://svn.apache.org/repos/asf/sling/trunk/testing/samples/sample-tests/src/main/java/org/apache/sling/testing/samples/sampletests/SamplePerformanceTest.java
>
> [2] https://svn.apache.org/repos/asf/sling/trunk/testing/junit/performance/src/test/java/org/apache/sling/junit/performance/runner/PerformanceRunnerTest.java

Re: Annotation support fior the PerformanceRunner

[Bertrand Delacretaz <bd...@apache.org>]

Hi,

On Tue, Jul 15, 2014 at 10:03 AM, Francesco Mari
<ma...@gmail.com> wrote:
> ...I realized that this mechanism was required because when writing
> SamplePerformanceTest [1] I had to resort to an anonymous inner class
> to act as a listener to provide initialization and cleanup code for my
> test...

[1| looks good to me, thanks for that! Adding it broke the sample
integration tests but I just fixed that in
http://svn.apache.org/r1610633

Are the warmup annotations tested elsewhere, to verify that the
actually do what they claim to do in terms of number of calls, warmup
time etc? I don't think [2] tests that.

-Bertrand

[1] https://svn.apache.org/repos/asf/sling/trunk/testing/samples/sample-tests/src/main/java/org/apache/sling/testing/samples/sampletests/SamplePerformanceTest.java

[2] https://svn.apache.org/repos/asf/sling/trunk/testing/junit/performance/src/test/java/org/apache/sling/junit/performance/runner/PerformanceRunnerTest.java