[jira] [Comment Edited] (SUREFIRE-1914) XML report omits method signature / display name of Junit 5 parameterized tests if testset reporter is configured to use phrased naming

["Tibor Digana (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SUREFIRE-1914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17345788#comment-17345788 ] 

Tibor Digana edited comment on SUREFIRE-1914 at 5/16/21, 8:39 PM:
------------------------------------------------------------------

If we could, we would already enable the signature {{"test1(String) [1] a"}}. We are doing it. I believe we have the tests proving this. Maybe in the unit tests or ITs. Pls try to find them. There must be some differences but I am sure the signatures are used in some circumstances.

Additionally, see the implementation in {{RunListenerAdapter}}.

Pls keep the original tests and extend them but do not overwrite the old tests.


was (Author: tibor17):
If we could, we would already enable the signature {{"test1(String) [1] a"}}. We are doing it. I believe we have the tests proving this. Maybe in the unit tests or ITs. Pls try to find them. There must be some differences but I am sure the signatures are used in some circumstances.

Additionally, see the implementation in {{RunListenerAdapter}}.

Pls keep the original tests and extend them but do not override old tests.

> XML report omits method signature / display name of Junit 5 parameterized tests if testset reporter is configured to use phrased naming
> ---------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SUREFIRE-1914
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1914
>             Project: Maven Surefire
>          Issue Type: Bug
>          Components: JUnit 5.x support, xml generation
>    Affects Versions: 3.0.0-M5
>            Reporter: Andreas Pabst
>            Priority: Major
>
> h3. Description of the issue
> Given a test class with two parameterized tests as follows:
> {code:java}
>     @ParameterizedTest
>     @ValueSource(strings = {"a", "b"})
>     void test1(String param) {
>         // test code
>     }
>     @ParameterizedTest
>     @ValueSource(strings = {"a", "b"})
>     void test2(String param) {
>         // test code
>     }
> {code}
> and a surefire configuration that includes the following:
> {code:xml}
> <statelessTestsetReporter implementation="org.apache.maven.plugin.surefire.extensions.junit5.JUnit5Xml30StatelessReporter">
>     <version>3.0</version>
>     <usePhrasedTestSuiteClassName>true</usePhrasedTestSuiteClassName>
>     <usePhrasedTestCaseClassName>true</usePhrasedTestCaseClassName>
>     <usePhrasedTestCaseMethodName>true</usePhrasedTestCaseMethodName>
> </statelessTestsetReporter>
> {code}
> the XML report will look like this:
> {code:xml}
>   <testcase name="[1] a" classname="mypackage.MyClass" time="0.001" />
>   <testcase name="[2] b" classname="mypackage.MyClass" time="0.001" />
>   <testcase name="[1] a" classname="mypackage.MyClass" time="0.001" />
>   <testcase name="[2] b" classname="mypackage.MyClass" time="0.001" />
> {code}
> Note: The test method name/signature is not included in the name attribute.
> I would expect something more like this:
> {code:xml}
>   <testcase name="test1(String) [1] a" classname="mypackage.MyClass" time="0.001" />
>   <testcase name="test1(String) [2] b" classname="mypackage.MyClass" time="0.001" />
>   <testcase name="test2(String) [1] a" classname="mypackage.MyClass" time="0.001" />
>   <testcase name="test2(String) [2] b" classname="mypackage.MyClass" time="0.001" />
> {code}
> h3. Some context on why this is bad
> Omitting the test method name makes it impossible to differentiate individual test cases in the XML report if there are multiple {{@ParameterizedTest}}-annotated test cases in the same class that use the same parameters.
> Any software that parses the surefire XML reports will be misled into thinking these are multiple executions of the same test.
> h3. Variant with {{@DisplayName}} usage
> There is a variant of this issue: If the {{@ParameterizedTest}} has a {{@DisplayName}} annotation, whatever was chosen as a display name is not included in the XML report either - unless it is explicitly referenced in the name attribute of the {{@ParameterizedTest}} like so: {{@ParameterizedTest(name = "\{displayName} ...")}}.
> h3. Solution ideas
> This issue has already been brought up during the discussion of SUREFIRE-1546.
> In that discussion [~srdo] and [~marcphilipp] have described an approach how CONTAINER-type TestIdentifiers could be handled properly, but that particular problem seems to have been deferred and has seemingly not been picked up again since.
> h3. Workaround
> A workaround is to explicitly reference the displayName or include some other unique string in the name attribute of each {{@ParameterizedTest}} as mentioned above. In projects with a large legacy testbase that might not be feasible though, so this issue might prevent the adoption of the feature introduced in SUREFIRE-1546.
> I'm submitting this as a bug because to me the current behaviour of surefire is at least unexpected, but it could also be seen as a request for improvement.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)