surefire - compatibility with ant/junit-batchtest XML - system-out and system-err

[Matthias Grunwalde <mg...@googlemail.com>]

Hi,
Is surefire 2.4.3's XML output intended to be consumable by Ant 1.7.1's
junitreport task?
It seems to be mostly compatible, however, there seems to be a discrepancy -
could surefire experts comment on this?

Should the <system-out>...</system-out> and <system-err>...</system-err>
elements be part of the <testsuite>, or attached to the <testcase> in which
they were generated?

The reason I ask is that our build-everything-from-scratch Ant script is not
producing links to the System output/error in the HTML Test Results for
artifacts built by Maven, but it is producing these links for those built by
Ant (we're currently migrating individual Ant-based projects to Maven, but
the overall build is controlled by Ant).

For example, please consider the following Ant output and Surefire output..
the test code:

import org.junit.Assert;
import org.junit.Test;
public class DieTest
{
   @Test
   public void alwaysFail()
   {
      System.out.println("here is some standard output");
      System.err.println("and here is some standard error text");
      Assert.fail("This test will always fail"); // line 21
   }
}

The Surefire 2.4.3 output:
<?xml version="1.0" encoding="UTF-8" ?>
<testsuite errors="0" skipped="0" tests="1" time="0.031" failures="1"
name="DieTest">
  <properties>
     ... yadda yadda yadda
  </properties>
  <testcase classname="DieTest" time="0" name="alwaysFail">
    <failure type="java.lang.AssertionError" message="This test will always
fail">java.lang.AssertionError: This test will always fail
    at org.junit.Assert.fail(Assert.java:69)
    at DieTest.alwaysFail(DieTest.java:21)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at
org.junit.internal.runners.TestMethodRunner.executeMethodBody(TestMethodRunner.java:99)
    at
org.junit.internal.runners.TestMethodRunner.runUnprotected(TestMethodRunner.java:81)
    at
org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)
    at
org.junit.internal.runners.TestMethodRunner.runMethod(TestMethodRunner.java:75)
    at
org.junit.internal.runners.TestMethodRunner.run(TestMethodRunner.java:45)
    at
org.junit.internal.runners.TestClassMethodsRunner.invokeTestMethod(TestClassMethodsRunner.java:75)
    at
org.junit.internal.runners.TestClassMethodsRunner.run(TestClassMethodsRunner.java:36)
    at
org.junit.internal.runners.TestClassRunner$1.runUnprotected(TestClassRunner.java:42)
    at
org.junit.internal.runners.BeforeAndAfterRunner.runProtected(BeforeAndAfterRunner.java:34)
    at
org.junit.internal.runners.TestClassRunner.run(TestClassRunner.java:52)
    at
org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:62)
    at
org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:140)
    at
org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:127)
    at org.apache.maven.surefire.Surefire.run(Surefire.java:177)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at
org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:338)
    at
org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:997)
</failure>
    <system-out>here is some standard output
</system-out>
    <system-err>and here is some standard error text
</system-err>
  </testcase>
</testsuite>

versus Ant 1.7.1's batchtest (formatter="xml"):

<?xml version="1.0" encoding="UTF-8" ?>
<testsuite errors="0" failures="1" hostname="TOAST" name="DieTest" tests="1"
time="0.078" timestamp="2008-12-10T15:55:58">
  <properties>
     ... yadda yadda yadda
  </properties>
<testcase classname="DieTest" name="alwaysFail" time="0.015">
    <failure message="This test will always fail"
type="junit.framework.AssertionFailedError">junit.framework.AssertionFailedError:
This test will always fail
    at DieTest.alwaysFail(Unknown Source)
</failure>
  </testcase>
  <system-out><![CDATA[here is some standard output
]]></system-out>
  <system-err><![CDATA[and here is some standard error text
]]></system-err>
</testsuite>


Thanks in advance.
-- 
Matthias Grünwalde