You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ant.apache.org by Steve Loughran <st...@apache.org> on 2006/03/01 14:05:42 UTC
Junut4 (was Re: svn commit: r381780 - in /ant/core/trunk: WHATSNEW
docs/manual/OptionalTasks/junit.html src/main/org/apache/tools/ant/taskdefs/optional/junit/JUnitTestRunner.java
src/main/org/apache/tools/ant/taskdefs/optional/junit/JUnitVersionHelper.java)
Martijn Kruithof wrote:
> Isn't this a bit premature, junit 4 isn't even "out" yet.
shipped last week. Gump is still recovering.
>Furthermore I
> thought junit 4 would get rid of the distinction between failure and
> error, wouldn't we want to remain in-line with this new junit behaviour
> when running junit 4 testcases in an junit 4 environment?
Just because junit4 wants that behaviour does not mean we need to follow.
On this topic, Jon Skeet is setting up the logistics for a conference
call between the junit people and the ant team, primarily on the future
of <junit>
Being a conference call, it would probably overload if everyone joined
in, but at the same time, I dont want to keep it a secret from
interested parties. We'll be discussing where we are with junit, what's
wrong with it and where do we go for junit4 --which may mean the junit
team will take over ownership of a junit4 task.
There was some discussion of junit4 implications in the ant-user mail
list, but since then Jesse has just fixed two main failings of <junit>
-classpath setup and (I believe) direct support for junit4.
We need to look at where we stand with this, whether it is enough, and
what else should be done. Here is my opinion
What does Ant's junit task does well:
-integrates testing with the build
-runs well in a new JVM
-segues into reporting with junitreport (a bit fiddly; could be integrated)
-provides some minimal output as you go along, one that lets you infer
state from the logs
-distinguish (possibly needlessly) errors and failures
What it does badly
-installation. <junit> is the first time most people encounter ant lib
setup grief.
-forked JVM classpath setup. you need junit.jar, xerces on the CP.
What it doesnt do
-drop abstract classes from the fileset (would be very nice)
-only try and run classes that extend TestCase or are annotated as
test methods (would be nice, no?)
what third party stuff adds
-choreograph deploy+liveness+test+undeploy (cactus does that)
-distribute. What if I want to test on a different machine? (I think
JBoss' JRunner does this, and my distributed JUnit component for
smartfrog is aimed at this problem)
I dont think the Ant task should try and do the third party things, but
it must enable things like cactus. Furthermore, the stuff in "does
badly" and "doesnt do" should be addressed, while retaining the things
that work today.
There are some other wish-list items I'd like for junit, that I haven't
covered. a medium priority spare time activity for me at work is
distributed junit: deploying tests on hosts 1-10, collecting results on
host 11, trying to present them in a way that makes sense. The GridUnit
project does some of this; they share my code for serializable test
results. I'd really like junit's results to be serializable out the box,
and junit4 results to be compatible, but I can work around this, after
all, exceptions don't serialize and I have to strip-mine meaningful data
from them.
So: open discussion. What would people want from a <junit4> task?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Junut4
Posted by Stefan Bodewig <bo...@apache.org>.
On Wed, 01 Mar 2006, Steve Loughran <st...@apache.org> wrote:
> Martijn Kruithof wrote:
>> Furthermore I thought junit 4 would get rid of the distinction
>> between failure and error, wouldn't we want to remain in-line with
>> this new junit behaviour when running junit 4 testcases in an junit
>> 4 environment?
>
> Just because junit4 wants that behaviour does not mean we need to
> follow.
Agreed. To me there is a difference between a failed assertion and a
NullPointerException.
What our formatters should support in some way is a number of ignored
tests.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Junut4 (was Re: svn commit: r381780 - in /ant/core/trunk: WHATSNEW
docs/manual/OptionalTasks/junit.html src/main/org/apache/tools/ant/taskdefs/optional/junit/JUnitTestRunner.java
src/main/org/apache/tools/ant/taskdefs/optional/junit/JUnitVersionHelper.java)
Posted by Steve Loughran <st...@apache.org>.
Jesse Glick wrote:
> Steve Loughran wrote:
>> There was some discussion of junit4 implications in the ant-user mail
>> list,
>
> Ah, thanks for the pointer, missed that before. Some things I can
> extract from that thread:
>
> - ignored tests ought to be reported (currently they are just silently
> skipped)
...this would impact the style sheets, of course; maybe we'd need a
junit4 set.
>
> - <junit> should support specifying categories of tests
>
> Not really clear to me at this point whether the significant features of
> JUnit 4 can be accommodated comfortably inside <junit> or whether a
> separate task would be better. Making a separate task may however
> discourage migration. With the current patch you can at least put
> junit-4.0.jar on your classpath and migrate a couple of tests to it,
> without touching other tests and without touching your build script.
Before your last patch, I was in favour of a new antlib. now I'm not so
sure. After all, junit is as important as javac.
>> What does Ant's junit task [...] does badly
>> -installation. <junit> is the first time most people encounter ant
>> lib setup grief.
>
> I guess this was fixed by my other <junit> patch a couple days ago?
yes indeed!
>
>> exceptions don't serialize
>
> Throwable implements Serializable, BTW.
yes, but some of the things that extend it dont deserialize at the far
end unless you have a shared classloader, jini style. My
throwable-for-the-log class has to extract the common info from the
entire chain of exceptions before sending it down the wire:
http://cvs.sourceforge.net/viewcvs.py/smartfrog/core/components/junit/src/org/smartfrog/services/junit/ThrowableTraceInfo.java?view=markup
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Junut4
Posted by Steve Loughran <st...@apache.org>.
Kev Jackson wrote:
>
>> +1 for a separate task, packaged in an Ant library. This one could be
>> release before Ant 1.7.0 which would give people using JUnit4 more
>> support sooner.
>>
>> Personally I'd even prefer the JUnit team to provide that Ant library
>> so they have to keep up with their API changes themselves 8-)
>>
>>
> That would certainly be optimal. (+1 for seperate task)
>
> With this addition to Ant 1.7 code base, we now have code that relies on
> jdk1.5 to build correctly (or will it build on <1.4?).
We already do, in the <isReachable> component, but I think it reflects.
I just got "inconvenienced" at work by compareTo suddenly becoming
generic, there is a risk that code compiled on java1.5 that uses, say
Long.compareTo() wont work on java1.4, where the argument is an Object
not a Long. There was a fair bit of casting needed to keep
Long.compareTo() to take an object without a complaint.
> If this is the
> case, is there perhaps a case for moving some of the codebase forward
> from pre 1.2? I know that these changes are only in one of the tasks
> (and an optional one at that), but I think it's time to look at
> deprecating 1.1.8 support and at least move the core up to 1.2 standard
> (with Collections etc).
There is no need to run on Java1.1, as long as we can cross compile;
nothing to stop you using collections. Its just the effort of patching
things and of increasing the difference between SVN_HEAD and the
shipping branch that has made people reluctant.
-steve
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Junut4
Posted by Stefan Bodewig <bo...@apache.org>.
On Fri, 03 Mar 2006, Jesse Glick <je...@sun.com> wrote:
> Stefan Bodewig wrote:
>> putting even more modifications into the current task (adding a new
>> JUnitResultFormatter subclass for ignored tests, for example) will
>> make it even more convoluted than it currently is.
>
> Of course. The question is how this additional complexity balances
> against the duplication of code and resulting maintenance problems
> if we have two tasks.
I here you, and I agree to a certain extent.
> I could try to draft a standalone <junit4> if there is general
> agreement that that is the way to go. I guess that would be in
>
> https://svn.apache.org/repos/asf/ant/antlibs/junit4/trunk/
>
> or something?
https://svn.apache.org/repos/asf/ant/sandbox/antlibs/junit4/trunk/
> The twist in this case is that we are dealing with an upgrade to a
> library for which we already provide support in the Ant distro; no
> clear precedent for that, beyond the customary support for tools in
> new JDK releases.
"no clear precedent" is very true. There are tasks like javacc that
try to adapt to three different versions and there is jspc which only
supports Tomcat 4 since Tomcat 5 now provides a task of its own. I
vaguely recall the ANTLR project wanted to ship an improved antlr task
as well.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
RE: Junut4
Posted by Stephen McConnell <mc...@dpml.net>.
> -----Original Message-----
> From: news [mailto:news@sea.gmane.org] On Behalf Of Jesse Glick
> Sent: Saturday, 4 March 2006 5:50 AM
> To: dev@ant.apache.org
> Subject: Re: Junut4
>
> Stefan Bodewig wrote:
> > putting even more modifications into the current task (adding a new
> > JUnitResultFormatter subclass for ignored tests, for example) will
> > make it even more convoluted than it currently is.
>
> Of course. The question is how this additional complexity
> balances against the duplication of code and resulting
> maintenance problems if we have two tasks. Although there are
> clearly semantic differences between
> 3 and 4, there are also many similarities that can be
> implemented in one place in one task vs. two places in two
> tasks; forking isn't free.
>
> Any other takes on this? I could try to draft a standalone
> <junit4> if there is general agreement that that is the way
> to go.
Jesse:
For what it's worth I would be *very* much in favour of a move to separate
task. In the process you may want to consider constraints that such an
external tasks could (hypothetically) declare within an antlib that would
ensure meaningful error messages from ant if the new task is referenced
without the prerequisites resources.
Cheers, Steve.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Junut4
Posted by Jesse Glick <je...@sun.com>.
Stefan Bodewig wrote:
> putting even more modifications into the current task (adding a new
> JUnitResultFormatter subclass for ignored tests, for example) will
> make it even more convoluted than it currently is.
Of course. The question is how this additional complexity balances
against the duplication of code and resulting maintenance problems if we
have two tasks. Although there are clearly semantic differences between
3 and 4, there are also many similarities that can be implemented in one
place in one task vs. two places in two tasks; forking isn't free.
Any other takes on this? I could try to draft a standalone <junit4> if
there is general agreement that that is the way to go. I guess that
would be in
https://svn.apache.org/repos/asf/ant/antlibs/junit4/trunk/
or something?
>> I do think that whatever form the support takes, it should be in the
>> standard Ant distro.
>
> The drawback here is that it is tied to Ant's release schedule. It
> will take months for your latest cahnges to become available to a
> wider audience. A separate Ant library would allow a much shorter
> release cycle.
True indeed, although you lose the benefit of ubiquity. Means people
effectively have to ship the <junit4> antlib with their projects in
order to ensure that it is available. I guess this is a more general
trade-off. The twist in this case is that we are dealing with an upgrade
to a library for which we already provide support in the Ant distro; no
clear precedent for that, beyond the customary support for tools in new
JDK releases.
>> 6. Sorting of tests (e.g. by last fail time) - not clear to me if
>> this is something an Ant task should address.
>
> Is this something JUnit4 offers or something that users would like to
> see?
Looking more closely, I think it is just something offered by the new
API, in Request.sortWith. (In 3 this is not available since new
TestSuite(Class) just enumerates all test methods in declaration order.)
The task does not need to make use of it to be compliant.
-J.
--
jesse.glick@sun.com x22801 netbeans.org ant.apache.org
http://google.com/search?q=e%5E(pi*i)%2B1
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Junut4
Posted by Stefan Bodewig <bo...@apache.org>.
On Thu, 02 Mar 2006, Jesse Glick <je...@sun.com> wrote:
> Kev Jackson wrote:
>> That would certainly be optimal. (+1 for seperate task)
>
> If others agree, should the existing 4 support in <junit> be
> reverted?
I feel that the changes between JUnit3 and JUnit4 warrant a new task,
putting even more modifications into the current task (adding a new
JUnitResultFormatter subclass for ignored tests, for example) will
make it even more convoluted than it currently is.
> My main concern is that making it a separate task would require
> people to change their build scripts gratuitously.
True.
> I do think that whatever form the support takes, it should be in the
> standard Ant distro.
The drawback here is that it is tied to Ant's release schedule. It
will take months for your latest cahnges to become available to a
wider audience. A separate Ant library would allow a much shorter
release cycle.
> 2. Timeout failures; similar to #1, though this could probably be
> skipped (just treat as a general failure with appropriate
> message).
I'd expect this to be an error like any other error.
> 3. Test failed to throw expected exception; similar to #2.
A failure.
> 4. Display only a "failure" count and no "error" count (if this is
> in fact desirable - does not appear to be consensus)
8-)
> 6. Sorting of tests (e.g. by last fail time) - not clear to me if
> this is something an Ant task should address.
Is this something JUnit4 offers or something that users would like to
see?
> Don't we already presume at least 1.2 to compile (and run) for Ant
> 1.6+ generally?
We do.
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Junut4
Posted by Jesse Glick <je...@sun.com>.
Kev Jackson wrote:
> That would certainly be optimal. (+1 for seperate task)
If others agree, should the existing 4 support in <junit> be reverted?
It can be considered a proof of concept in its current state. My main
concern is that making it a separate task would require people to change
their build scripts gratuitously. I do think that whatever form the
support takes, it should be in the standard Ant distro.
I believe the changes which might require a change to the <junit> task
signature, associated interfaces, or result XML format are:
1. Reporting of ignored tests, which would require an optional extension
to, or alternative to, the JUnitResultFormatter interface for custom
formatters, as well as impl of that extension / alternative in the
shipped formatters. Assuming most people use the standard formatters,
they don't need to touch their script. The XML format for test results
(and its default stylesheet) might also need extensions.
2. Timeout failures; similar to #1, though this could probably be
skipped (just treat as a general failure with appropriate message).
Seems that 4's RunListener anyway doesn't report it specially?
3. Test failed to throw expected exception; similar to #2.
4. Display only a "failure" count and no "error" count (if this is in
fact desirable - does not appear to be consensus); similar to #1.
5. I thought there was support for named "categories" of tests in 4, but
I can't now find it - maybe it was dropped for 4.0, or maybe I was
thinking of TestNG?
6. Sorting of tests (e.g. by last fail time) - not clear to me if this
is something an Ant task should address.
7. Use of Filter, e.g. to run one test method only. But this is
desirable for JUnit 3 anyway (see Bugzilla).
Other changes which probably don't imply any signature change:
1. Handling of StoppedByUserException (not sure when this is thrown).
> With this addition to Ant 1.7 code base, we now have code that relies on
> jdk1.5 to build correctly (or will it build on <1.4?).
It shouldn't require anything above 1.2 to build; everything in
junit-4.0.jar (not much) is accessed via reflection. JDK 1.5 is only
required to run if you actually have junit-4.0.jar in your test's CP
(and not junit-3.8.1.jar first); or you want to run 4-based tests.
> If this is the
> case, is there perhaps a case for moving some of the codebase forward
> from pre 1.2? I know that these changes are only in one of the tasks
> (and an optional one at that), but I think it's time to look at
> deprecating 1.1.8 support and at least move the core up to 1.2 standard
> (with Collections etc).
Don't we already presume at least 1.2 to compile (and run) for Ant 1.6+
generally? There's no reason not to use Collections in Ant today, except
for places where we have a historical API mentioning Vector or Hashtable
that can't be changed compatibly.
-J.
--
jesse.glick@sun.com x22801 netbeans.org ant.apache.org
http://google.com/search?q=e%5E(pi*i)%2B1
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Junut4
Posted by Kev Jackson <ke...@it.fts-vn.com>.
>+1 for a separate task, packaged in an Ant library. This one could be
>release before Ant 1.7.0 which would give people using JUnit4 more
>support sooner.
>
>Personally I'd even prefer the JUnit team to provide that Ant library
>so they have to keep up with their API changes themselves 8-)
>
>
That would certainly be optimal. (+1 for seperate task)
With this addition to Ant 1.7 code base, we now have code that relies on
jdk1.5 to build correctly (or will it build on <1.4?). If this is the
case, is there perhaps a case for moving some of the codebase forward
from pre 1.2? I know that these changes are only in one of the tasks
(and an optional one at that), but I think it's time to look at
deprecating 1.1.8 support and at least move the core up to 1.2 standard
(with Collections etc).
Thoughts?
Also - it'd be good to have <junit> as the old task and <junut> as the
new task ;) [see subject...]
Kev
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Junut4
Posted by Stefan Bodewig <bo...@apache.org>.
On Wed, 01 Mar 2006, Jesse Glick <je...@sun.com> wrote:
> Not really clear to me at this point whether the significant
> features of JUnit 4 can be accommodated comfortably inside <junit>
> or whether a separate task would be better.
+1 for a separate task, packaged in an Ant library. This one could be
release before Ant 1.7.0 which would give people using JUnit4 more
support sooner.
Personally I'd even prefer the JUnit team to provide that Ant library
so they have to keep up with their API changes themselves 8-)
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org
Re: Junut4 (was Re: svn commit: r381780 - in /ant/core/trunk: WHATSNEW
docs/manual/OptionalTasks/junit.html src/main/org/apache/tools/ant/taskdefs/optional/junit/JUnitTestRunner.java
src/main/org/apache/tools/ant/taskdefs/optional/junit/JUnitVersionHelper.java)
Posted by Jesse Glick <je...@sun.com>.
Steve Loughran wrote:
> There was some discussion of junit4 implications in the ant-user mail
> list,
Ah, thanks for the pointer, missed that before. Some things I can
extract from that thread:
- ignored tests ought to be reported (currently they are just silently
skipped)
- <junit> should support specifying categories of tests
Not really clear to me at this point whether the significant features of
JUnit 4 can be accommodated comfortably inside <junit> or whether a
separate task would be better. Making a separate task may however
discourage migration. With the current patch you can at least put
junit-4.0.jar on your classpath and migrate a couple of tests to it,
without touching other tests and without touching your build script.
> What does Ant's junit task [...] does badly
> -installation. <junit> is the first time most people encounter ant lib
> setup grief.
I guess this was fixed by my other <junit> patch a couple days ago?
> exceptions don't serialize
Throwable implements Serializable, BTW.
-J.
--
jesse.glick@sun.com x22801 netbeans.org ant.apache.org
http://google.com/search?q=e%5E(pi*i)%2B1
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org