script to compare maven test run times

[Alex Parvulescu <al...@gmail.com>]

hi,

For OAK-624 I needed to compare the test run times between 2 different mk
implementations so I came up with a script that computes the percentage
difference between the 2 'mvn test' runs.

This allows for output like the one in this comment [0].
The interesting part is it is generic enough to be also used when comparing
unit tests that run on oak and jackrabbit (starting with, but not limited
to the tck ones or the query tests that now run on jr & oak).

Please take a look and let me know if you think the output is useful, or if
there is anything more that we could add to it.
The script is here [1].

thanks,
alex



[0]
https://issues.apache.org/jira/browse/OAK-624?focusedCommentId=13579093&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13579093
[1] https://gist.github.com/alexparvulescu/4959583

Re: script to compare maven test run times

[Jukka Zitting <ju...@gmail.com>]

Hi,

On Fri, Feb 15, 2013 at 1:04 PM, Alex Parvulescu
<al...@gmail.com> wrote:
> Please take a look and let me know if you think the output is useful, or if
> there is anything more that we could add to it.

Very nice! +1 to putting it in svn.

A potential extension could be to look into the XML reports that
Surefire outputs and produce a report that drills down to individual
test methods. Though I'm not sure if the added value from the extra
detail is worth the effort.

BR,

Jukka Zitting

Re: script to compare maven test run times

[Michael Dürig <mi...@gmail.com>]

Good stuff!
Michael

On 15.2.13 11:04, Alex Parvulescu wrote:
> hi,
>
> For OAK-624 I needed to compare the test run times between 2 different mk
> implementations so I came up with a script that computes the percentage
> difference between the 2 'mvn test' runs.
>
> This allows for output like the one in this comment [0].
> The interesting part is it is generic enough to be also used when comparing
> unit tests that run on oak and jackrabbit (starting with, but not limited
> to the tck ones or the query tests that now run on jr & oak).
>
> Please take a look and let me know if you think the output is useful, or if
> there is anything more that we could add to it.
> The script is here [1].
>
> thanks,
> alex
>
>
>
> [0]
> https://issues.apache.org/jira/browse/OAK-624?focusedCommentId=13579093&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13579093
> [1] https://gist.github.com/alexparvulescu/4959583
>

Re: script to compare maven test run times

[Stefan Guggisberg <st...@gmail.com>]

On Fri, Feb 15, 2013 at 12:04 PM, Alex Parvulescu
<al...@gmail.com> wrote:
> hi,
>
> For OAK-624 I needed to compare the test run times between 2 different mk
> implementations so I came up with a script that computes the percentage
> difference between the 2 'mvn test' runs.
>
> This allows for output like the one in this comment [0].
> The interesting part is it is generic enough to be also used when comparing
> unit tests that run on oak and jackrabbit (starting with, but not limited
> to the tck ones or the query tests that now run on jr & oak).
>
> Please take a look and let me know if you think the output is useful, or if
> there is anything more that we could add to it.

nice!

cheers
stefan

> The script is here [1].
>
> thanks,
> alex
>
>
>
> [0]
> https://issues.apache.org/jira/browse/OAK-624?focusedCommentId=13579093&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13579093
> [1] https://gist.github.com/alexparvulescu/4959583