[jira] [Updated] (SUREFIRE-1945) unit tests with large logging output does not produce surefire report

["Tibor Digana (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SUREFIRE-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tibor Digana updated SUREFIRE-1945:
-----------------------------------
    Fix Version/s: 2.22.3
                   3.0.0-M6

> unit tests with large logging output does not produce surefire report
> ---------------------------------------------------------------------
>
>                 Key: SUREFIRE-1945
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1945
>             Project: Maven Surefire
>          Issue Type: Bug
>          Components: Maven Surefire Plugin
>    Affects Versions: 2.20, 2.20.1, 2.21.0, 2.22.0, 2.22.1, 2.22.2, 3.0.0-M2, 3.0.0-M1, 3.0.0-M3, 3.0.0-M4, 3.0.0-M5
>         Environment: Have reproduced with both Java 8 and 11:
> openjdk version "1.8.0_292"
> openjdk version "11.0.11" 2021-04-20
> and maven 
> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
>            Reporter: Mark Miller
>            Priority: Critical
>             Fix For: 2.22.3, 3.0.0-M6
>
>         Attachments: surefire_bug.zip
>
>
> Upon upgrading our maven-surefire-plugin from version 2.15 to version 2.21.0.  We noticed that the number of tests being run by our jenkins test job changed.  Upon further investigation we found a test class with 7 test cases that were not being reported at all (not as passed, failed, or skipped).  
> When we ran just that test class from the command line it failed with
> {quote}
> [ERROR] Crashed tests:
> [ERROR] com.sensus.dataextraction.reportcomparetool.TestFlowPreHHFReports
> [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
> [ERROR] Command was cmd.exe /X /C "C:\tools\java8\jre\bin\java org.apache.maven.surefire.booter.ForkedBooter C:\tools\cygwin\tmp\surefire7735045605128719483 2021-09-17T10-56-30_089-jvmRun1 surefire4447346130857727490tmp surefire_04489252332082939918tmp"
> [ERROR] Process Exit Code: 0
> [ERROR] Crashed tests:
> {quote}
> but did not produce any output in the target/surefire-reports directory explaining why the failures were not reported in the jobs test report.  Trying different maven-surefire-plugin versions the tests pass successfully with version 2.19.1, but fail with every version tried between 2.20 and 3.0.0-M5.  Note we run our tests with these options:
> <reuseForks>false</reuseForks>
> <forkCount>1</forkCount>
> <reportFormat>xml</reportFormat>
> I also noticed that the tests in question did produce a lot of logging output.  I found that the tests would pass if I added the configuration option:
> <redirectTestOutputToFile>true</redirectTestOutputToFile>
> I have reproduced the problem with a simpler test module that is attached.  Upon playing with the output in this module the issue appears to happen when the output is about 1.2MB or higher.
> Not sure what changed between version 2.19.1 and 2.20 with how output from the forked jvm is collected but it appear to have a problem if there is too much output.  If the tests were being reported as failed then this issue would not be a huge problem, but having tests just not be reported because their output grows is a critical issue for us as we may not notice that the test has effectively been dropped from our test suite, unless we closely audit the ~30000 test cases we run every run to detect missing tests which is hard to do as new tests are constantly being added. 
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.1#820001)