[jira] [Updated] (SUREFIRE-2079) exception with long message removes test from result instead of failing it

["Michael Kroll (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SUREFIRE-2079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Kroll updated SUREFIRE-2079:
------------------------------------
    Description: 
surefire 3.0.0-M6 excludes a test from the test result, if an exception with a long message (~1000) is thrown from an inner class.

{noformat}
    $ mvn test
    [...]
    [INFO] Running org.example.ThrowExceptionWithLongMessageTest
    [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 s - in org.example.ThrowExceptionWithLongMessageTest
    [INFO]
    [INFO] Results:
    [INFO]
    [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
    [INFO]
    [WARNING] Corrupted channel by directly writing to native stream in forked JVM 1. See FAQ web page and the dump file     [INFO]     [...]
    [...]
    [INFO] BUILD SUCCESS
{noformat}

with

{code:java|title=test class}
package org.example;

import org.junit.jupiter.api.Test;

class ThrowExceptionWithLongMessageTest {

    @Test
    public void example() {
        Inner.throwException(1000);
    }
        
    public static class Inner {
        public static void throwException(final int messageLen) {
            final var sb = new StringBuilder();
            for (int i = 0; i < messageLen; i++) {
                sb.append("A");
            }
            throw new IllegalStateException(sb.toString());
        }
    }
}
{code}

Demonstrator: see links

surefire 3.0.0-M5 fails the test like expected:
{noformat}
    [INFO] Running org.example.ThrowExceptionWithLongMessageTest
    [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.036 s <<< FAILURE! - in org.example.ThrowExceptionWithLongMessageTest
{noformat}

  was:
surefire 3.0.0-M6 excludes a test from the test result, if an exception with a long message (~1000) is thrown from an inner class.

{noformat}
    $ mvn test
    [...]
    [INFO] Running org.example.ThrowExceptionWithLongMessageTest
    [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 s - in org.example.ThrowExceptionWithLongMessageTest
    [INFO]
    [INFO] Results:
    [INFO]
    [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
    [INFO]
    [WARNING] Corrupted channel by directly writing to native stream in forked JVM 1. See FAQ web page and the dump file     [INFO]     [...]
    [...]
    [INFO] BUILD SUCCESS
{noformat}

with

{code:java|title=test class}
package org.example;

import org.junit.jupiter.api.Test;

class ThrowExceptionWithLongMessageTest {

    @Test
    public void example() {
        Inner.throwException(1000);
    }
        
    public static class Inner {
        public static void throwException(final int messageLen) {
            final var sb = new StringBuilder();
            for (int i = 0; i < messageLen; i++) {
                sb.append("A");
            }
            throw new IllegalStateException(sb.toString());
        }
    }
}
{code}

Demonstrator:
https://github.com/mickroll/demonstrator/tree/surefire-exception-with-long-message-removes-test-from-result

surefire 3.0.0-M5 fails the test like expected:
{noformat}
    [INFO] Running org.example.ThrowExceptionWithLongMessageTest
    [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.036 s <<< FAILURE! - in org.example.ThrowExceptionWithLongMessageTest
{noformat}


> exception with long message removes test from result instead of failing it
> --------------------------------------------------------------------------
>
>                 Key: SUREFIRE-2079
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-2079
>             Project: Maven Surefire
>          Issue Type: Bug
>          Components: Maven Surefire Plugin
>    Affects Versions: 3.0.0-M6
>         Environment: Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
> Maven home: [...]
> Java version: 17.0.3, vendor: BellSoft, runtime: C:\Users\[...]\.sdkman\candidates\java\17.0.3-librca
> Default locale: de_DE, platform encoding: Cp1252
> OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
>            Reporter: Michael Kroll
>            Priority: Critical
>              Labels: test-stability
>
> surefire 3.0.0-M6 excludes a test from the test result, if an exception with a long message (~1000) is thrown from an inner class.
> {noformat}
>     $ mvn test
>     [...]
>     [INFO] Running org.example.ThrowExceptionWithLongMessageTest
>     [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 s - in org.example.ThrowExceptionWithLongMessageTest
>     [INFO]
>     [INFO] Results:
>     [INFO]
>     [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
>     [INFO]
>     [WARNING] Corrupted channel by directly writing to native stream in forked JVM 1. See FAQ web page and the dump file     [INFO]     [...]
>     [...]
>     [INFO] BUILD SUCCESS
> {noformat}
> with
> {code:java|title=test class}
> package org.example;
> import org.junit.jupiter.api.Test;
> class ThrowExceptionWithLongMessageTest {
>     @Test
>     public void example() {
>         Inner.throwException(1000);
>     }
>         
>     public static class Inner {
>         public static void throwException(final int messageLen) {
>             final var sb = new StringBuilder();
>             for (int i = 0; i < messageLen; i++) {
>                 sb.append("A");
>             }
>             throw new IllegalStateException(sb.toString());
>         }
>     }
> }
> {code}
> Demonstrator: see links
> surefire 3.0.0-M5 fails the test like expected:
> {noformat}
>     [INFO] Running org.example.ThrowExceptionWithLongMessageTest
>     [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.036 s <<< FAILURE! - in org.example.ThrowExceptionWithLongMessageTest
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.7#820007)