[jira] [Commented] (SUREFIRE-1556) Test XML file is not valid when rerun "fails" with an assumption

["Tibor Digana (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SUREFIRE-1556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486164#comment-17486164 ] 

Tibor Digana commented on SUREFIRE-1556:
----------------------------------------

https://github.com/apache/maven-surefire/commit/d24ac6d1ab5a2584a6d8cacf35eea693b68fc8f4

> Test XML file is not valid when rerun "fails" with an assumption
> ----------------------------------------------------------------
>
>                 Key: SUREFIRE-1556
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1556
>             Project: Maven Surefire
>          Issue Type: Bug
>          Components: Maven Surefire Plugin
>    Affects Versions: 2.21.0, 2.22.0
>            Reporter: Michael Weisgrab
>            Assignee: Tibor Digana
>            Priority: Major
>             Fix For: 2.22.3, 3.0.0-M6
>
>         Attachments: TEST-asdf.SearchTest_assumption_valid.xml, TEST-asdf.SearchTest_rerun_assumption_invalid.xml, TEST-asdf.SearchTest_rerun_fail_valid.xml
>
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> We are running UI tests and are doing one retry if there is a bug.
> When a test fails with an Exception on the first try and the second try is skipped by an assumption violation the resulting test report xml file is not valid (not well formatted xml file).
> When the test is skipped on the first try or both runs fail with an exception everything is ok.
> This is a testcase which is retried and a valid xml is generated.
>  
> {code:java}
> <testcase name="xxx" classname="xxx" time="47.905">
>     <error message="message" type="org.openqa.selenium.TimeoutException">
>         ...
>     </error>
>     <system-out>
>         ...
>     </system-out>
>     <system-err>
>         ...
>     </system-err>
>     <rerunError message="message" type="java.lang.IndexOutOfBoundsException">
>         ...
>         <system-out>
>             ...
>         </system-out>
>         <system-err>
>             ...
>         </system-err>
>     </rerunError>
> </testcase>{code}
>  
>  
> Here is a generated invalid xml. The rerunError tag is missing. The retry "error" is an assumption violation so the test should be marked as skipped with message of the assumptionviolation.
> {code:java}
> <testcase name="xxx" classname="xxx" time="54.153">
>   <error message="message" type="org.openqa.selenium.TimeoutException">
>   </error>
>   <system-out>
>     ...
>   </system-out>
>   <system-err>
>     ...
>   </system-err>
>   < message="There should be a list of ads!">
>     <system-out>
>       ...
>     </system-out>
>     <system-err>
>       ...
>     </system-err>
>   </>
> </testcase>
> {code}
> I have attached the following files:
>  * TEST-asdf.SearchTest_assumption_valid.xml (assumption violation in the first test run)
>  * TEST-asdf.SearchTest_rerun_fail_valid.xml (both runs throw exception)
>  * TEST-asdf.SearchTest_rerun_assumption_invalid.xml (first run exception - second run assumption violation)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)