[jira] (SUREFIRE-1087) New option rerunFailingTestsCount

["Qingzhou Luo (JIRA)" <ji...@codehaus.org>]

Qingzhou Luo created SUREFIRE-1087:
--------------------------------------

             Summary: New option rerunFailingTestsCount
                 Key: SUREFIRE-1087
                 URL: https://jira.codehaus.org/browse/SUREFIRE-1087
             Project: Maven Surefire
          Issue Type: New Feature
            Reporter: Qingzhou Luo


Pull Request 40:
https://github.com/apache/maven-surefire/pull/40
    
Add rerunFailingTestsCount option for maven surefire to rerun failing tests immediately after they fail.
    
    When rerunFailingTestsCount is set to  a value k larger than 0, a failing test will get re-run
    up to k times until it passes. If a test passes in any of its reruns,
    the build will be marked as successful and the test will count as a
    flake (or flaky test). If it fails all those k times then it will
    still be marked as a failed test.
    
    In the console output all the flaky tests will be count as "Flakes:
    N". The generated test report XML file is augmented with additional
    information, while still being compatible with existing consumers
    (such as Jenkins). A flaky test will have <flakyFailure> or/and
    <flakyError> under its <testcase> element, to store all the flaky
    runs' information (such as output, stackTrace). So existing consumers
    will still consider it as a passing test, while potential future
    consumers can parse those flaky runs information. A failing test will
    still have <failure> or <error> under <testcase>, but all the
    subsequent re-run information will be stored under <rerunFailure> or
    <rerunError>. So existing consumers will still be able to see it's a
    failed test and parse its failure information, and potential future
    consumers will be able to get all the flaky runs.
    
    It is implemented by keeping a map between test full class name and a
    map between all its test methods and the list of runs. It also takes
    into account Fork and Parallel and have them covered by integration
    tests.
    
    Currently only supports JUnit4.x



--
This message was sent by Atlassian JIRA
(v6.1.6#6162)