[jira] Commented: (SUREFIRE-47) JUnit suite() method ends up being called twice when tests are run with Surefire

["Dan Fabulich (JIRA)" <ji...@codehaus.org>]

    [ http://jira.codehaus.org/browse/SUREFIRE-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_114740 ] 

Dan Fabulich commented on SUREFIRE-47:
--------------------------------------

This happens in JUnitTestSet; what's strange is that we don't do this in TestNGTestSet.  "getTestCount()" simply returns 1 in that case, with a TODO "need to get this from TestNG somehow".  This is filed as SUREFIRE-94; it's minor, because "this isn't required for correct operation of the tests, but may be if a reporter relies on the correct number in the runStarting method (currently, it is unused)."

If so, why do we count the tests initially at all?  I just tried ripping out all implementations of getTestCount() and just adding 1 to totalTests whenever we would have called it.  It doesn't seem to have done any harm; the only difference I can see is that in the case where you've got some classes that look like tests, we do "execute" the test run, but when we find no tests, the summary banner says:

  Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

... instead of:

  No tests to run.

(It still says "No tests to run" when there are no JUnit classes at all.)

Therefore, I'm inclined to fix SUREFIRE-47 by ripping out getTestCount() from the SurefireTestSet interface and removing its implementation from JUnitTestSet, PojoTestSet and TestNGTestSet.

Does anyone object to this?

> JUnit suite() method ends up being called twice when tests are run with Surefire
> --------------------------------------------------------------------------------
>
>                 Key: SUREFIRE-47
>                 URL: http://jira.codehaus.org/browse/SUREFIRE-47
>             Project: Maven Surefire
>          Issue Type: Bug
>    Affects Versions: 2.0 (2.2 plugin)
>         Environment: maven 2.0.4, ant 1.6.5, this is on MacOSX 10.4.6
>            Reporter: toli kuznets
>             Fix For: 2.4
>
>         Attachments: surefireBug.tar
>
>
> JUnit defines that the static suite() method is only instantiated once.
> However, the recent release of surefire (2.0) seems to call the suite() method twice. 
> the first time it's called from during the creation of all suites (see the first stacktrace in the attached test.out files) - the JUnitTestSet calls "suite()" to instantiate a test object so that it can call countTestCases() on it. 
> The second time the suite() method is called during the actual test execution, when Surefire.run() iterates over all the collected test suites and eventually calls into JUnitTestSet.execute() method.
> a sample project is attached. I added an exception stacktrace into every occurence of the suite() method.
> there's also a build.xml file to run the 'ant test' command for comparison. 
> There's also a test.out file with the sample output.
> Suggested fixes:
> Is it possible to pass the suite itself to the JUnitTestSet w/out having to create it again? 
> We rely on suite() being called only once, and create one-time operations there (such as creating a few singletons) so this bug is a pain.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira