[jira] [Closed] (SUREFIRE-2087) rerunFailingTestsCount incorrectly marks failed parametrized test as pass

["Slawomir Jaranowski (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SUREFIRE-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Slawomir Jaranowski closed SUREFIRE-2087.
-----------------------------------------
    Resolution: Duplicate

> rerunFailingTestsCount incorrectly marks failed parametrized test as pass
> -------------------------------------------------------------------------
>
>                 Key: SUREFIRE-2087
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-2087
>             Project: Maven Surefire
>          Issue Type: Bug
>    Affects Versions: 3.0.0-M4
>            Reporter: Ramanan Rajendran
>            Priority: Major
>
> rerunFailingTestsCount is not working as expected with junit5 `@ParameterizedTest` 
> If 1st iteration of the test passes and the 2nd one fails, it still marks the test as flaky passed. 
> Eg: In the following run, iteration 1 with input A passed but iteration 2 with input B failed. Iteration 2 was rerun 2 times and failed in all the attempts. Instead of failing TestA, it's marked as flaky giving an overall pass for the test suite. I believe it's using the Run 1 PASS as a gate to mark the whole test as a pass. 
>  
> [INFO]  Run 1: PASS
> [ERROR]  Run 2: TestA.iteration2:101 1 expectation failed.
> Expected status code <400> doesn't match actual status code <401>.
> [ERROR]  Run 3: TestA.iteration2:101 1 expectation failed.
> Expected status code <400> doesn't match actual status code <401>.
> [ERROR]  Run 4: TestA.iteration2:101 1 expectation failed.
>  <400> doesn't match actual status code <401>.
>  
> Tests run: 23, Failures: 0, Errors: 0, Skipped: 2, Flakes: 1



--
This message was sent by Atlassian Jira
(v8.20.10#820010)