You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Tibor Digana (Jira)" <ji...@apache.org> on 2022/02/08 00:10:00 UTC
[jira] [Comment Edited] (SUREFIRE-1945) unit tests with large logging output does not produce surefire report
[ https://issues.apache.org/jira/browse/SUREFIRE-1945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17488500#comment-17488500 ]
Tibor Digana edited comment on SUREFIRE-1945 at 2/8/22, 12:09 AM:
------------------------------------------------------------------
[~markd.miller]
I can fully reproduce the problem.
We have such integration tests but we use std/out.
In this test we do not have readable logs and dump file, so we have only one possibility how to find out the bug. We have to find the commit in between two versions 2.19.1 and 2.20. I would like to kindly ask you to make bisec of the commits, simply divide the commits logarithmically by two and then you will fast iterate to the commit with the bug. The previous commit will make successful build in your project.
So divide the commits in two sets and build the project with the command {{mvn clean install -DskipTests -DskipITs}} and make sure that the POM does not change the version. Use the verion in your test project and run {{mvn -nsu test}}.
Good luck.
I deleted the other plugins and configurations, those were not necessary:
{code:xml}
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.sensus.mavenbug</groupId>
<artifactId>sample</artifactId>
<version>1.0.0</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
</plugin>
</plugins>
</build>
</project>
{code}
was (Author: tibor17):
[~markd.miller]
I can fully reproduce the problem.
We have such integration tests but we use std/out.
In this test we do not have readable logs and dump file, so we have only one possibility how to find out the bug. We have to find the commit in between two versions 2.19.1 and 2.20. I would like to kindly ask you to make bisec of the commits, simply divide the commits logarithmically by two and then you will fast iterate to the commit with the bug. The previous commit will make successful build in your project.
So divide the commits in two sets and build the project with the command {{mvn clean install -DskipTests -DskipITs}} and make sure that the POM does not change the version. Use the verion in your test project and run {{mvn -nsu test}}.
Good luck.
I deleted the other plugins and configurations, those were not necessary:
{xml}
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.sensus.mavenbug</groupId>
<artifactId>sample</artifactId>
<version>1.0.0</version>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.20</version>
</plugin>
</plugins>
</build>
</project>
{code}
> unit tests with large logging output does not produce surefire report
> ---------------------------------------------------------------------
>
> Key: SUREFIRE-1945
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1945
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Surefire Plugin
> Affects Versions: 2.20, 2.20.1, 2.21.0, 2.22.0, 2.22.1, 2.22.2, 3.0.0-M2, 3.0.0-M1, 3.0.0-M3, 3.0.0-M4, 3.0.0-M5
> Environment: Have reproduced with both Java 8 and 11:
> openjdk version "1.8.0_292"
> openjdk version "11.0.11" 2021-04-20
> and maven
> Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> Reporter: Mark Miller
> Priority: Critical
> Attachments: surefire_bug.zip
>
>
> Upon upgrading our maven-surefire-plugin from version 2.15 to version 2.21.0. We noticed that the number of tests being run by our jenkins test job changed. Upon further investigation we found a test class with 7 test cases that were not being reported at all (not as passed, failed, or skipped).
> When we ran just that test class from the command line it failed with
> {quote}
> [ERROR] Crashed tests:
> [ERROR] com.sensus.dataextraction.reportcomparetool.TestFlowPreHHFReports
> [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: ExecutionException The forked VM terminated without properly saying goodbye. VM crash or System.exit called?
> [ERROR] Command was cmd.exe /X /C "C:\tools\java8\jre\bin\java org.apache.maven.surefire.booter.ForkedBooter C:\tools\cygwin\tmp\surefire7735045605128719483 2021-09-17T10-56-30_089-jvmRun1 surefire4447346130857727490tmp surefire_04489252332082939918tmp"
> [ERROR] Process Exit Code: 0
> [ERROR] Crashed tests:
> {quote}
> but did not produce any output in the target/surefire-reports directory explaining why the failures were not reported in the jobs test report. Trying different maven-surefire-plugin versions the tests pass successfully with version 2.19.1, but fail with every version tried between 2.20 and 3.0.0-M5. Note we run our tests with these options:
> <reuseForks>false</reuseForks>
> <forkCount>1</forkCount>
> <reportFormat>xml</reportFormat>
> I also noticed that the tests in question did produce a lot of logging output. I found that the tests would pass if I added the configuration option:
> <redirectTestOutputToFile>true</redirectTestOutputToFile>
> I have reproduced the problem with a simpler test module that is attached. Upon playing with the output in this module the issue appears to happen when the output is about 1.2MB or higher.
> Not sure what changed between version 2.19.1 and 2.20 with how output from the forked jvm is collected but it appear to have a problem if there is too much output. If the tests were being reported as failed then this issue would not be a huge problem, but having tests just not be reported because their output grows is a critical issue for us as we may not notice that the test has effectively been dropped from our test suite, unless we closely audit the ~30000 test cases we run every run to detect missing tests which is hard to do as new tests are constantly being added.
>
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)