[jira] Commented: (SUREFIRE-321) Run tests in alphabetical order

["Simon van der Sluis (JIRA)" <ji...@codehaus.org>]

    [ http://jira.codehaus.org/browse/SUREFIRE-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=198785#action_198785 ] 

Simon van der Sluis commented on SUREFIRE-321:
----------------------------------------------

Some more motivation to implement this:

In one of our software projects we have a unit (or it might be some) test that contaminates a spring application context but doesn't call setDirty() (or use the annotated equivalent).  We have another unit test that is sensitive to this contamination, so the order they are run in determines if one of the tests passes or fails.

I've being trying to identify the test which contaminates the spring application context.  I've tried using both the surefire includes and excludes mechanisms to run various subsets of the tests.

BUT

Depending on which tests are included (or excluded) affects the order in which the tests are run.

So I have no way of knowing if I've excluded the test(s) at fault or if it's the re-ordering that fixesd (or hides) the problem.

With over 500 units tests spread over more than 100 test classes it's far to time consuming to try all the possible combinations.

If the test classes were run in alphabetical order, I could make the sensitive test run last, and add the other tests package by package (or class by class) to determine which one(s) caused the problem.



> Run tests in alphabetical order
> -------------------------------
>
>                 Key: SUREFIRE-321
>                 URL: http://jira.codehaus.org/browse/SUREFIRE-321
>             Project: Maven Surefire
>          Issue Type: Improvement
>    Affects Versions: 2.3
>            Reporter: Daniel Beland
>            Priority: Minor
>             Fix For: 2.x
>
>         Attachments: SUREFIRE-321.patch
>
>
> It would be nice if the tests were run in alphabetical order (with complete package name).
> So all tests in a package run in order and same things for each packages.
> It just makes it easier to know where we currently are in the tests and makes it easier to estimate how long it will take before the tests finish to run.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira