Re: [jira] Commented: (DERBY-2667) Create more robust junit TestRunner for running derby tests

[Myrna van Lunteren <m....@gmail.com>]

To add my 2 cents: I like using whichever tool suits my needs at a
particular point, so I wouldn't want this to be a mandatory thing, but
it would be nice to see the nightly reports that currently run the
textui.TestRunner be a bit more helpful, especially in the case of
hangs, or failures in tests that are run multiple times.
The recent trouble with upgradeTests/encryptionTests is a case in point.

Also, I know that on zOS Manjula's not been able to use a GUI tool,
but textui.TestRunner hung, so there's another example.
We could all write our own, of course, but it seems a little
uncoordinated and inefficient.

Knut Anders put a simple TestRunner out a while ago (see:
http://mail-archives.apache.org/mod_mbox/db-derby-dev/200610.mbox/%3Cwjo8mz7joyx3.fsf@sun.com%3E),
maybe he wouldn't mind to attach it to this issue & grant licence to
apache and then we can use it if desired instead of the
textui.TestRunner for situations where a gui tool is not necessary,
available or usefull?

Myrna

Re: [jira] Commented: (DERBY-2667) Create more robust junit TestRunner for running derby tests

[Daniel John Debrunner <dj...@apache.org>]

Myrna van Lunteren wrote:
> To add my 2 cents: I like using whichever tool suits my needs at a
> particular point, so I wouldn't want this to be a mandatory thing, 

+1

A verbose text based runner might be great for a human looking into a 
failure, but may not be the best option for a nightly or continuous test 
run.

> but
> it would be nice to see the nightly reports that currently run the
> textui.TestRunner be a bit more helpful,

Some of the issue here is that the nightly test runs have started with 
the test runner that provides the least amount of information.

> especially in the case of
> hangs, or failures in tests that are run multiple times.
> The recent trouble with upgradeTests/encryptionTests is a case in point.

I'm not sure any test runner could have been more helpful than those 
that exist already for this *specific* issue.

The stack trace reported by the various runners was correct.
Running in the swing runner showed which tests failed.

The real problem was that it was a tricky problem and the actual errors 
didn't appear to make any sense.

Dan.