You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Albert Johnston (Jira)" <ji...@apache.org> on 2020/02/04 10:51:00 UTC
[jira] [Comment Edited] (SUREFIRE-1748) JUnit 5 Assertions.fail()
breaks reporting
[ https://issues.apache.org/jira/browse/SUREFIRE-1748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029745#comment-17029745 ]
Albert Johnston edited comment on SUREFIRE-1748 at 2/4/20 10:50 AM:
--------------------------------------------------------------------
Third time's the charm, I know what the issue is. It's pretty late, so I'll keep this _relatively_ short (meaning no PR and such, this is a pretty long explanation) and come back to it tomorrow. The issue appears to be that when the AssertionError message is the empty string (not null), then it messes up the encoding/decoding process (specifically ForkedChannelEncoder/ForkedChannelDecoder). This actually goes for both JUnit 5 and JUnit 4, the difference being that JUnit 5 coerces a null message into an empty string, whereas JUnit 4 does not. So to trigger the issue in JUnit 4, you have to explicitly pass in an empty string, it cannot simply be null.
Anyways, when you pass in a null message, the message is translated into "-" during the encoding process. Totally fine. However, when you pass in an empty string message, it stays as an empty string when encoded. This causes the surrounding encoding for the message to look like "::". That seems correct, however this leads to the important part. As a result of some part of this process (I'm thinking it's the way it's tokenized when decoded), the smart stack trace is decoded into the message, the stack trace is decoded into the smart stack trace, and the stack trace is left as null. In other words, the message is "skipped", causing everything to be read into the property that comes before it, leaving the stack trace with nothing to fill it. Which, in turn, leads to the null stack trace writer I mentioned in an earlier comment.
Assuming the issue is something with the decoder's tokenization of the encoded failure, it's potentially a relatively easy fix. I already attempted to fix it by simply having the empty string be encoded as "-" as if it was null, but that breaks other tests, and I suppose is technically incorrect (it isn't actually null, after all).
Relevant code:
- [https://github.com/apache/maven-surefire/blob/master/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/booterclient/output/ForkedChannelDecoder.java#L261-L275]
- [https://github.com/apache/maven-surefire/blob/master/surefire-api/src/main/java/org/apache/maven/surefire/booter/ForkedChannelEncoder.java#L292-L309]
was (Author: ajohnston):
Third time's the charm, I know what the issue is. It's pretty late, so I'll keep this _relatively_ short and come back to it tomorrow. The issue appears to be that when the AssertionError message is the empty string (not null), then it messes up the encoding/decoding process (specifically ForkedChannelEncoder/ForkedChannelDecoder). This actually goes for both JUnit 5 and JUnit 4, the difference being that JUnit 5 coerces a null message into an empty string, whereas JUnit 4 does not. So to trigger the issue in JUnit 4, you have to explicitly pass in an empty string, it cannot simply be null.
Anyways, when you pass in a null message, the message is translated into "-" during the encoding process. Totally fine. However, when you pass in an empty string message, it stays as an empty string when encoded. This causes the surrounding encoding for the message to look like "::". That seems correct, however this leads to the important part. As a result of some part of this process (I'm thinking it's the way it's tokenized when decoded), the smart stack trace is decoded into the message, the stack trace is decoded into the smart stack trace, and the stack trace is left as null. In other words, the message is "skipped", causing everything to be read into the property that comes before it, leaving the stack trace with nothing to fill it. Which, in turn, leads to the null stack trace writer I mentioned in an earlier comment.
Assuming the issue is something with the decoder's tokenization of the encoded failure, it's potentially a relatively easy fix. I already attempted to fix it by simply having the empty string be encoded as "-" as if it was null, but that breaks other tests, and I suppose is technically incorrect (it isn't actually null, after all).
Relevant code:
- [https://github.com/apache/maven-surefire/blob/master/maven-surefire-common/src/main/java/org/apache/maven/plugin/surefire/booterclient/output/ForkedChannelDecoder.java#L261-L275]
- [https://github.com/apache/maven-surefire/blob/master/surefire-api/src/main/java/org/apache/maven/surefire/booter/ForkedChannelEncoder.java#L292-L309]
> JUnit 5 Assertions.fail() breaks reporting
> ------------------------------------------
>
> Key: SUREFIRE-1748
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1748
> Project: Maven Surefire
> Issue Type: Bug
> Components: JUnit 5.x support, Maven Surefire Plugin
> Affects Versions: 3.0.0-M4
> Reporter: Albert Johnston
> Priority: Major
>
> Associated versions on my end at time of testing:
> * JUnit Jupiter 5.3.2
> * JDK 1.8u172
> * Maven 3.6.3
> Demonstration available at [https://github.com/ajohnstonTE/surefire-jira-examples/tree/master/junit-assertions-fail]
> If the following JUnit 5 code is run through Surefire, then the reporting breaks for that class, resulting in no reported errors/failures from that class (though any failures will still be reported overall in the counts):
> {code:java}
> public class JUnit5Test {
> @Test
> void failWithNoParameters() {
> Assertions.fail();
> }
> }
> {code}
> Resulting output:
>
> {noformat}
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.016 s <<< FAILURE! - in JUnit5Test
> [ERROR] JUnit5Test.failWithNoParameters Time elapsed: 0.008 s <<< FAILURE!
> [ERROR] Failures:
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test (default-test) on project junit-assertions-fail: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed.: NullPointerException -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace...{noformat}
> And with the -e flag, trimmed to the relevant part:
>
> {noformat}
> [ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.016 s <<< FAILURE! - in JUnit5Test
> [ERROR] JUnit5Test.failWithNoParameters Time elapsed: 0.007 s <<< FAILURE!
> [ERROR] Failures:
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test (default-test) on project junit-assertions-fail: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed.: NullPointerException -> [Help 1]
> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test (default-test) on project junit-assertions-fail: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed.
> Caused by: org.apache.maven.plugin.PluginExecutionException: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M4:test failed.
> Caused by: java.lang.NullPointerException
> at org.apache.maven.plugin.surefire.report.DefaultReporterFactory.printTestFailures (DefaultReporterFactory.java:398)
> at org.apache.maven.plugin.surefire.report.DefaultReporterFactory.runCompleted (DefaultReporterFactory.java:198)
> at org.apache.maven.plugin.surefire.report.DefaultReporterFactory.close (DefaultReporterFactory.java:171)
> at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run (ForkStarter.java:254)
> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider (AbstractSurefireMojo.java:1217)
> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked (AbstractSurefireMojo.java:1063)
> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute (AbstractSurefireMojo.java:889){noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)