[jira] [Commented] (SUREFIRE-1821) Broken junit report when parallel and rerunFailingTestsCount configureation is used

["Duo Zhang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SUREFIRE-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17393276#comment-17393276 ] 

Duo Zhang commented on SUREFIRE-1821:
-------------------------------------

We have also faced this problem.

https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3550/4/artifact/yetus-jdk11-hadoop3-check/output/patch-unit-hbase-server.txt/*view*/

There are two flaky tests and the jenkis job is broken when passing the xml test report. I downloaded the xml and it is exactly the same with the description of this issue.

{noformat}
  <testcase name="testMultiplePuts" classname="org.apache.hadoop.hbase.replication.regionserver.TestReplicationValueCompressedWAL" time="14.723">
    <flakyFailure message="Waited too much time for replication" type="java.lang.AssertionError">
      <stackTrace><![CDATA[java.lang.AssertionError: Waited too much time for replication
	at org.apache.hadoop.hbase.replication.regionserver.TestReplicationValueCompressedWAL.testMultiplePuts(TestReplicationValueCompressedWAL.java:56)
]]></stackTrace>
      <system-out><![CDATA[Formatting using clusterid: testClusterID
Formatting using clusterid: testClusterID
]]></system-out>
      <system-err><![CDATA[
{noformat}

{noformat}
  <testcase name="killOneMasterRS" classname="org.apache.hadoop.hbase.replication.TestReplicationKillMasterRSWithSeparateOldWALs" time="10.053">
    <flakyFailure message="Waited too much time for queueFailover replication. Waited 54548ms." type="java.lang.AssertionError">
      <stackTrace><![CDATA[java.lang.AssertionError: Waited too much time for queueFailover replication. Waited 54548ms.
]]></stackTrace>
      <system-out><![CDATA[Formatting using clusterid: testClusterID
Formatting using clusterid: testClusterID
]]></system-out>
      <system-err><![CDATA[
{noformat}

Looking foward on the fix in 3.0.0-M6...

> Broken junit report when parallel and rerunFailingTestsCount configureation is used
> -----------------------------------------------------------------------------------
>
>                 Key: SUREFIRE-1821
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1821
>             Project: Maven Surefire
>          Issue Type: Bug
>          Components: Junit 4.7+ (parallel) support, Maven Failsafe Plugin, Maven Surefire Plugin
>    Affects Versions: 2.20.1, 2.22.2, 3.0.0-M5
>            Reporter: Pavel Pustovoyt
>            Priority: Major
>
> *Description:*
> When using *parallel* configuration with *rerunFailingTestsCount* bad xml report is generated for a failing test.
> *How to reproduce:*
> 1. Configure forkCount, parallel and rerun count:
> {code:java}
> <plugin>
>   <groupId>org.apache.maven.plugins</groupId>
>   <artifactId>maven-surefire-plugin</artifactId>
>   <version>3.0.0-M5</version>
>   <executions>
>     <execution>
>       <goals>
>         <goal>test</goal>
>       </goals>
>     </execution>
>   </executions>
>   <configuration>
>     <rerunFailingTestsCount>1</rerunFailingTestsCount>
>     <forkCount>1</forkCount>
>     <threadCount>1</threadCount>
>     <parallel>all</parallel>
>     <redirectTestOutputToFile>true</redirectTestOutputToFile>
>   </configuration>
> </plugin>
> {code}
> 2. Create a failing test that outputs more than 1m characters:
> {code:java}
> public class AppTest {
>     @Test
>     public void testBug() {
>         for(int i = 0;  i < 100000; i++){
>             System.out.println("Some output longer than 10 character");
>         }
>         throw new NullPointerException();
>     }
> }
> {code}
> 3. Run the test and check the report - you will see unexpected end of xml file:
> {code:java}
>     ...
>     <property name="sun.io.unicode.encoding" value="UnicodeBig"/>
>     <property name="java.class.version" value="52.0"/>
>   </properties>
>   <testcase name="testBug" classname="com.mycompany.app.AppTest" time="0.015">
>     <error type="java.lang.NullPointerException"><![CDATA[java.lang.NullPointerException
>         at com.mycompany.app.AppTest.testBug(AppTest.java:12)
> ]]></error>
>     <system-out><![CDATA[  <-- EOF
> {code}
> *Possible cause:*
> It seems that Junit 4.7 Runner considers each rerun as a separate test set, so sends _testSetCompleted_ event each time, but after the first event _DeferredFileOutputStream_ and associated with it temp file gets freed(deleted), however _testSetCompleted_ event for rerun still tries to write something there. If temp file isn't yet created (still in memory) then there will be no bug and it gets created only after 1m characters were written to the stream. I wasn't able to understand wether multiple _testSetCompleted_ events is a bug here or a "rerun" trying to write to _DeferredFileOutputStream_ of initial test run.
>  
> *Project to reproduce:*
> *[https://github.com/himos/surefire-report-bug]*



--
This message was sent by Atlassian Jira
(v8.3.4#803005)