[jira] [Commented] (SUREFIRE-2039) Skipped test classes are getting into the non-skipped test classes reports

["Semyon Danilov (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SUREFIRE-2039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17509454#comment-17509454 ] 

Semyon Danilov commented on SUREFIRE-2039:
------------------------------------------

I have a naive approach patch for solving this issue: https://github.com/apache/maven-surefire/pull/493 but, as I've written in the description for the PR, I don't really like it and some feedback from the maintainers can help me improve it. 

> Skipped test classes are getting into the non-skipped test classes reports
> --------------------------------------------------------------------------
>
>                 Key: SUREFIRE-2039
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-2039
>             Project: Maven Surefire
>          Issue Type: Bug
>    Affects Versions: 3.0.0-M5
>            Reporter: Semyon Danilov
>            Priority: Major
>
> Consider a simple scenario, two test classes,  *ATest* and *BTest*, one +Disabled+ and one not respectively, each has three test methods. After running _mvn surefire:test_ the report will indicate that 4 tests were run and one of them was skipped:
> {noformat}
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running org.example.test.BTest
> [WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.062 s - in org.example.test.BTest
> [INFO] 
> [INFO] Results:
> [INFO] 
> [WARNING] Tests run: 4, Failures: 0, Errors: 0, Skipped: 1
> {noformat}
> And if we enable *ATest* and disable *BTest*, then the report will only mention running three tests, none skipped:
> {noformat}
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO] Running org.example.test.ATest
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.079 s - in org.example.test.ATest
> [INFO] 
> [INFO] Results:
> [INFO] 
> [INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0
> {noformat}
> This behavior is the result of the way how *org.apache.maven.plugin.surefire.report.TestSetRunListener* handles skipped tests. It adds them to the *detailsForThis* and only prints after the test set is finished which happens when a non-disabled test set is finished thus adding skipped test results to the statistics of a non-skipped test set. And, of course, if the disabled test is ran after the non-disabled test, it won't be reported at all.
> The reproducer can be found here: https://github.com/SammyVimesFiledIssues/SUREFIRE-2039



--
This message was sent by Atlassian Jira
(v8.20.1#820001)