[jira] Updated: (DERBY-2667) Create more robust junit TestRunner for running derby tests

["Mike Matrigali (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mike Matrigali updated DERBY-2667:
----------------------------------


What I am looking for is a runner that produces text based output that is appropriate for offline nightly test suites.  I like to run whatever test runner the systems that are running regression tests against the code line are.  Currently I believe this is sun jvm nightly, sun jvm tinderbox and ibm jvm nightly.  This was if I match my environment it is likely the tool will work.    While it is great
that people can pick/choose their runner - it would be nice to have at least one runner recommended as the default runner that
developers could use and nightly regression tests would also use.

When I started I first tried the ant option, but immediately ran into memory issues, but noticed nightly runs using the other tools
just worked. 

What I am looking for is:
o a tool that will run all the tests without failing, and is tested to continue to do so every night.
o complete set of stack traces, including any nested exceptions.  Dan points out this is likely a derby issue
o the ability to run all the tests and then come back and have a shot at figuring out what went wrong and when.  Things
    that would help are:
    o printing each test as it runs.  Sometimes when you get errors in teardown the stack gives you no idea what test the teardown
        actually is failing for.
    o preserve the persistent information when a failure occurs.  Most important is the derby.log as it may have more information. 
        But a copy of the database is also interesting, especially in cases of non reproducible errors where this may have been 
       our one chance to gather info, or to allow someone to debug a system specific problem when they don't have access to that
       type of system. 
        This may be a test issue rather than the runner.  I wonder if we just need to write some generic "tearDown" that  on error or
         failure copies current state somewhere

> Create more robust junit  TestRunner for running derby tests
> ------------------------------------------------------------
>
>                 Key: DERBY-2667
>                 URL: https://issues.apache.org/jira/browse/DERBY-2667
>             Project: Derby
>          Issue Type: Improvement
>          Components: Test
>    Affects Versions: 10.3.0.0
>            Reporter: Kathey Marsden
>            Priority: Minor
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures.   It would be great to have a standardized TestRunner that everyone uses.  Perhaps someone already has one that they would like to contribute as a starter.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.