Re: [testng-users] Re: Test Suites, Ant, Surefire, and JunitReport

[Dan Fabulich <da...@fabulich.com>]

Steve Loughran wrote:

>> What I propose is that, in order to avoid destroying information, Surefire
>> should generate XML that looks like Example 7 (all-in-one-file), and not
>> try to fake it to look like Example 2 (one-file-per-class).  (TestNG's
>> junit-like output also generates files like Example 7.)  However, when it
>> comes time to generate an HTML report, surefire-reports will discard
>> suite-level information, and treat large suites like Example 7 as if they
>> had been presented in separate files like Example 2.  I'd argue that all
>> of the other JunitReport-like tools (including Ant) should probably follow
>> the same lead.
>
> I think that if everyone else has a bug, its hard to call it a bug. More 
> a presentation choice :)

I'd thought somebody might say that! :-)  Still, do you think TestNG and 
other tools should therefore generate multiple XML files, to be compatible 
with the other (arguably broken) reporting tools?

-Dan

Re: [testng-users] Re: Test Suites, Ant, Surefire, and JunitReport

[Steve Loughran <st...@gmail.com>]

On Jan 25, 2008 5:04 PM, Dan Fabulich <da...@fabulich.com> wrote:
>
> Steve Loughran wrote:
>
> >> What I propose is that, in order to avoid destroying information, Surefire
> >> should generate XML that looks like Example 7 (all-in-one-file), and not
> >> try to fake it to look like Example 2 (one-file-per-class).  (TestNG's
> >> junit-like output also generates files like Example 7.)  However, when it
> >> comes time to generate an HTML report, surefire-reports will discard
> >> suite-level information, and treat large suites like Example 7 as if they
> >> had been presented in separate files like Example 2.  I'd argue that all
> >> of the other JunitReport-like tools (including Ant) should probably follow
> >> the same lead.
> >
> > I think that if everyone else has a bug, its hard to call it a bug. More
> > a presentation choice :)
>
> I'd thought somebody might say that! :-)  Still, do you think TestNG and
> other tools should therefore generate multiple XML files, to be compatible
> with the other (arguably broken) reporting tools?

No,

* junit report will take other sets of XSL sheets than those built in
to Ant's own JAR. This allows anyone to fix the XSLs without waiting
for ant's (fairly mature) release cycle.
* CI tools are a separate issue. Don't know there

Looking at the reports -and trying to imagine a time where I ever get
to stop making meta-RPM-RPMs (don't ask), I do want to push the test
formats forward in a way everyone is happy. I really dont like the way
the current stuff sticks summary info as toplevel attributes (time,
results), as it stops us streaming out the XML as we go. I'm starting
to wonder if we couldnt move to a new format, and have the option of
XSL generation of the legacy stuff, purely for those tools that havent
been upgraded in sync.

-steve