You are viewing a plain text version of this content. The canonical link for it is here.
Posted to derby-dev@db.apache.org by "Kathey Marsden (JIRA)" <ji...@apache.org> on 2007/05/18 07:06:16 UTC
[jira] Created: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Create more robust junit TestRunner for running derby tests
------------------------------------------------------------
Key: DERBY-2667
URL: https://issues.apache.org/jira/browse/DERBY-2667
Project: Derby
Issue Type: Improvement
Components: Test
Affects Versions: 10.3.0.0
Reporter: Kathey Marsden
Priority: Minor
Provide a more full featured TestRunner for Derby testing.
junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
Re: [jira] Issue Comment Edited: (DERBY-2667) Create more robust junit TestRunner for running derby tests
Posted by Manjula Kutty <ma...@gmail.com>.
Thank you Myrna,
-Manjula
On 2/22/08, Myrna van Lunteren (JIRA) <ji...@apache.org> wrote:
>
>
> [
> https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571270#action_12571270]
>
> myrna edited comment on DERBY-2667 at 2/22/08 3:36 PM:
> --------------------------------------------------------------------
>
> Thanks Manjula, I like the latest patch.
>
> I committed patch DERBY-2667_diff_02_21.txt with revision 630077.
>
> Probably superfluous, but, I do want to point out that the
> TestConfiguration.getFailureFolder creates a 'fail' folder, and all
> directories underneath
> (fail/<configuration>/<testName>/<failedFixtureName>).
> This means, that if you run a subsequent failing fixture test in the same
> top level directory, previous derby.logs will be overwritten, and also,
> old run's fail directories won't get cleaned out.
>
>
> was (Author: myrna):
> Thanks Manjula, I like the latest patch.
>
> I committed patch DERBY-2667_diff_02_21.txt with revision 63007.
>
> Probably superfluous, but, I do want to point out that the
> TestConfiguration.getFailureFolder creates a 'fail' folder, and all
> directories underneath
> (fail/<configuration>/<testName>/<failedFixtureName>).
> This means, that if you run a subsequent failing fixture test in the same
> top level directory, previous derby.logs will be overwritten, and also,
> old run's fail directories won't get cleaned out.
>
>
> > Create more robust junit TestRunner for running derby tests
> > ------------------------------------------------------------
> >
> > Key: DERBY-2667
> > URL: https://issues.apache.org/jira/browse/DERBY-2667
> > Project: Derby
> > Issue Type: Improvement
> > Components: Test
> > Affects Versions: 10.3.1.4
> > Reporter: Kathey Marsden
> > Priority: Minor
> > Attachments: DERBY-2667_diff_02_06.txt,
> DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt,
> DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt,
> DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt,
> DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt,
> JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
> >
> >
> > Provide a more full featured TestRunner for Derby testing.
> > junit.textui.TestRunner is not very robust. It does not for example
> print the tests as they run or print chained exceptions, create separate
> files for the full report and just failures. It would be great to have a
> standardized TestRunner that everyone uses. Perhaps someone already has one
> that they would like to contribute as a starter.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
--
Thanks,
Manjula.
Re: [jira] Commented: (DERBY-2667) Create more robust junit TestRunner for running derby tests
Posted by Manjula Kutty <ma...@gmail.com>.
I created the patch using Eclipse and did click the source/correct
indentation. Is there a better way to fix this problem? Please let me know,
then I will do the same and will update the patch.
Thanks
Manjula
On 2/22/08, Knut Anders Hatlen (JIRA) <ji...@apache.org> wrote:
>
>
> [
> https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571303#action_12571303]
>
> Knut Anders Hatlen commented on DERBY-2667:
> -------------------------------------------
>
> Thanks for addressing my comments. Please double check the indentation
> settings in your IDE, though. The latest patch reintroduced the strange
> indentation from the earlier patches (each line starts with eight space
> characters, but for some reason tabs are appended if more indentation is
> needed).
>
> > Create more robust junit TestRunner for running derby tests
> > ------------------------------------------------------------
> >
> > Key: DERBY-2667
> > URL: https://issues.apache.org/jira/browse/DERBY-2667
> > Project: Derby
> > Issue Type: Improvement
> > Components: Test
> > Affects Versions: 10.3.1.4
> > Reporter: Kathey Marsden
> > Priority: Minor
> > Attachments: DERBY-2667_diff_02_06.txt,
> DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt,
> DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt,
> DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt,
> DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt,
> JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
> >
> >
> > Provide a more full featured TestRunner for Derby testing.
> > junit.textui.TestRunner is not very robust. It does not for example
> print the tests as they run or print chained exceptions, create separate
> files for the full report and just failures. It would be great to have a
> standardized TestRunner that everyone uses. Perhaps someone already has one
> that they would like to contribute as a starter.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
--
Thanks,
Manjula.
Re: [jira] Commented: (DERBY-2667) Create more robust junit TestRunner for running derby tests
Posted by Manjula Kutty <ma...@gmail.com>.
Hi, the two finally blocks are intentional. As per Knut's suggestion I added
it so that if some how copying the derby.log fails , we will not loose the
original exception.
On 2/22/08, Myrna van Lunteren (JIRA) <ji...@apache.org> wrote:
>
>
> [
> https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571549#action_12571549]
>
> Myrna van Lunteren commented on DERBY-2667:
> -------------------------------------------
>
> I did not notice the weird spacing before committing, sorry...
> I wanted to fix it and noticed I had missed another issue - we now have
> two finally blocks, one of which will never be reached...
>
> > Create more robust junit TestRunner for running derby tests
> > ------------------------------------------------------------
> >
> > Key: DERBY-2667
> > URL: https://issues.apache.org/jira/browse/DERBY-2667
> > Project: Derby
> > Issue Type: Improvement
> > Components: Test
> > Affects Versions: 10.3.1.4
> > Reporter: Kathey Marsden
> > Priority: Minor
> > Attachments: DERBY-2667_diff_02_06.txt,
> DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt,
> DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt,
> DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt,
> DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt,
> JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
> >
> >
> > Provide a more full featured TestRunner for Derby testing.
> > junit.textui.TestRunner is not very robust. It does not for example
> print the tests as they run or print chained exceptions, create separate
> files for the full report and just failures. It would be great to have a
> standardized TestRunner that everyone uses. Perhaps someone already has one
> that they would like to contribute as a starter.
>
> --
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>
>
--
Thanks,
Manjula.
Re: [jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by Daniel John Debrunner <dj...@apache.org>.
Myrna van Lunteren wrote:
> To add my 2 cents: I like using whichever tool suits my needs at a
> particular point, so I wouldn't want this to be a mandatory thing,
+1
A verbose text based runner might be great for a human looking into a
failure, but may not be the best option for a nightly or continuous test
run.
> but
> it would be nice to see the nightly reports that currently run the
> textui.TestRunner be a bit more helpful,
Some of the issue here is that the nightly test runs have started with
the test runner that provides the least amount of information.
> especially in the case of
> hangs, or failures in tests that are run multiple times.
> The recent trouble with upgradeTests/encryptionTests is a case in point.
I'm not sure any test runner could have been more helpful than those
that exist already for this *specific* issue.
The stack trace reported by the various runners was correct.
Running in the swing runner showed which tests failed.
The real problem was that it was a tricky problem and the actual errors
didn't appear to make any sense.
Dan.
Re: [jira] Commented: (DERBY-2667) Create more robust junit TestRunner for running derby tests
Posted by Myrna van Lunteren <m....@gmail.com>.
To add my 2 cents: I like using whichever tool suits my needs at a
particular point, so I wouldn't want this to be a mandatory thing, but
it would be nice to see the nightly reports that currently run the
textui.TestRunner be a bit more helpful, especially in the case of
hangs, or failures in tests that are run multiple times.
The recent trouble with upgradeTests/encryptionTests is a case in point.
Also, I know that on zOS Manjula's not been able to use a GUI tool,
but textui.TestRunner hung, so there's another example.
We could all write our own, of course, but it seems a little
uncoordinated and inefficient.
Knut Anders put a simple TestRunner out a while ago (see:
http://mail-archives.apache.org/mod_mbox/db-derby-dev/200610.mbox/%3Cwjo8mz7joyx3.fsf@sun.com%3E),
maybe he wouldn't mind to attach it to this issue & grant licence to
apache and then we can use it if desired instead of the
textui.TestRunner for situations where a gui tool is not necessary,
available or usefull?
Myrna
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Manjula Kutty (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manjula Kutty updated DERBY-2667:
---------------------------------
Attachment: DERBY-2667_diff_02_15.txt
DERBY-2667_stat_02_15.txt
Thanks for the comments Dan. Please review the new patch attached. I hope I answered all your comments in that patch. Please review and if everything looks good, please commit
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Closed: (DERBY-2667) Improve diagnostic information for
failed derby JUnit tests
Posted by "Kathey Marsden (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kathey Marsden closed DERBY-2667.
---------------------------------
Resolution: Fixed
Fix Version/s: 10.5.0.0
Closing this issue. I will open up another one for the pending patch.
> Improve diagnostic information for failed derby JUnit tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Fix For: 10.5.0.0
>
> Attachments: derby-2667-WriteExceptionsToFileAsTheyHappen.diff, DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Mike Matrigali (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Mike Matrigali updated DERBY-2667:
----------------------------------
What I am looking for is a runner that produces text based output that is appropriate for offline nightly test suites. I like to run whatever test runner the systems that are running regression tests against the code line are. Currently I believe this is sun jvm nightly, sun jvm tinderbox and ibm jvm nightly. This was if I match my environment it is likely the tool will work. While it is great
that people can pick/choose their runner - it would be nice to have at least one runner recommended as the default runner that
developers could use and nightly regression tests would also use.
When I started I first tried the ant option, but immediately ran into memory issues, but noticed nightly runs using the other tools
just worked.
What I am looking for is:
o a tool that will run all the tests without failing, and is tested to continue to do so every night.
o complete set of stack traces, including any nested exceptions. Dan points out this is likely a derby issue
o the ability to run all the tests and then come back and have a shot at figuring out what went wrong and when. Things
that would help are:
o printing each test as it runs. Sometimes when you get errors in teardown the stack gives you no idea what test the teardown
actually is failing for.
o preserve the persistent information when a failure occurs. Most important is the derby.log as it may have more information.
But a copy of the database is also interesting, especially in cases of non reproducible errors where this may have been
our one chance to gather info, or to allow someone to debug a system specific problem when they don't have access to that
type of system.
This may be a test issue rather than the runner. I wonder if we just need to write some generic "tearDown" that on error or
failure copies current state somewhere
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.0.0
> Reporter: Kathey Marsden
> Priority: Minor
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Daniel John Debrunner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12496921 ]
Daniel John Debrunner commented on DERBY-2667:
----------------------------------------------
junitreport has an existing issue DERBY-2234
I also see some permission problems running junit-all (in cruise control), I was planning on looking into it today to see if it was my environment or not.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.0.0
> Reporter: Kathey Marsden
> Priority: Minor
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12509215 ]
Knut Anders Hatlen commented on DERBY-2667:
-------------------------------------------
Thanks Ole! This looks useful to me.
Committed JUnitMethodTrace.diff.txt with revision 552046 after resolving a conflict with another commit in TestConfiguration. I also removed the two new instance variables in BaseTestCase since testCaseName could be accessed through getName(), and the call to TestConfiguration.getCurrent().doTrace() should happen in runBare() (calling it from the constructor could cause the wrong TestConfiguration object to be used).
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.0.0
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571303#action_12571303 ]
Knut Anders Hatlen commented on DERBY-2667:
-------------------------------------------
Thanks for addressing my comments. Please double check the indentation settings in your IDE, though. The latest patch reintroduced the strange indentation from the earlier patches (each line starts with eight space characters, but for some reason tabs are appended if more indentation is needed).
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12509216 ]
Knut Anders Hatlen commented on DERBY-2667:
-------------------------------------------
Committed JUnitMethodTrace_Extra.diff.txt with revision 552047.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.0.0
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Myrna van Lunteren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Myrna van Lunteren updated DERBY-2667:
--------------------------------------
Comment: was deleted
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Kathey Marsden (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12496914 ]
Kathey Marsden commented on DERBY-2667:
---------------------------------------
I am having trouble running junit-all and junitreport. (Permissions errors of course). Any idea how to resolve this issue?
http://www.nabble.com/problem-with-junitreport-target-tf3753226.html#a10606491
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.0.0
> Reporter: Kathey Marsden
> Priority: Minor
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Manjula Kutty (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manjula Kutty updated DERBY-2667:
---------------------------------
Derby Info: [Patch Available]
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Improve diagnostic information for
failed derby JUnit tests
Posted by "Kathey Marsden (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kathey Marsden updated DERBY-2667:
----------------------------------
Summary: Improve diagnostic information for failed derby JUnit tests (was: Create more robust junit TestRunner for running derby tests)
Changing title to more accurately reflect what was done under this issue.
> Improve diagnostic information for failed derby JUnit tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: derby-2667-WriteExceptionsToFileAsTheyHappen.diff, DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12572496#action_12572496 ]
Kristian Waagan commented on DERBY-2667:
----------------------------------------
Dag, it is a valid concern and I have not actually verified that no files are overwritten in all configurations.
People should feel free to do this, or we could just do the easy thing and add a timestamp and append to the file.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: derby-2667-WriteExceptionsToFileAsTheyHappen.diff, DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Manjula Kutty (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manjula Kutty updated DERBY-2667:
---------------------------------
Attachment: DERBY-2667_diff_2_19.txt
DERBY-2667_stat_02_19.txt
Thanks for the comments. Please find the modified patch and I hope I tried to address all your comments in that patch. Please review
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Dyre Tjeldvoll (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dyre Tjeldvoll updated DERBY-2667:
----------------------------------
Derby Info: (was: [Patch Available])
Removing the patch flag since the latest review comments indicate that another rev of the patch is required.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_stat_02_06.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (DERBY-2667) Create more robust junit
TestRunner for running derby tests
Posted by "Myrna van Lunteren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571270#action_12571270 ]
myrna edited comment on DERBY-2667 at 2/22/08 3:36 PM:
--------------------------------------------------------------------
Thanks Manjula, I like the latest patch.
I committed patch DERBY-2667_diff_02_21.txt with revision 630077.
Probably superfluous, but, I do want to point out that the TestConfiguration.getFailureFolder creates a 'fail' folder, and all directories underneath (fail/<configuration>/<testName>/<failedFixtureName>).
This means, that if you run a subsequent failing fixture test in the same top level directory, previous derby.logs will be overwritten, and also, old run's fail directories won't get cleaned out.
was (Author: myrna):
Thanks Manjula, I like the latest patch.
I committed patch DERBY-2667_diff_02_21.txt with revision 63007.
Probably superfluous, but, I do want to point out that the TestConfiguration.getFailureFolder creates a 'fail' folder, and all directories underneath (fail/<configuration>/<testName>/<failedFixtureName>).
This means, that if you run a subsequent failing fixture test in the same top level directory, previous derby.logs will be overwritten, and also, old run's fail directories won't get cleaned out.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Myrna van Lunteren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12570367#action_12570367 ]
Myrna van Lunteren commented on DERBY-2667:
-------------------------------------------
I am wondering why you choose to only catch Exceptions?
For instance, a 'normal' failure would be likely a Throwable, e.g. a junit.framework.AssertionFailedError...
Maybe it makes sense to save derby.log when a Throwable is encountered?
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Manjula Kutty (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manjula Kutty updated DERBY-2667:
---------------------------------
Attachment: DERBY-2667_diff_02_21.txt
DERBY-2667_stat_02_21.txt
Thanks for your comments. Here is the new patch. Please ereview
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kristian Waagan updated DERBY-2667:
-----------------------------------
Attachment: derby-2667-WriteExceptionsToFileAsTheyHappen.diff
'derby-2667-WriteExceptionsToFileAsTheyHappen.diff' is a patch that makes BaseTestCase also write the stack trace to file when an error occurs. The file will be written to the same location as derby.log, and currently it will overwrite any existing file.
I have only tested it lightly, but it seems to work fine for me. Is it something we could consider adding?
The reason why I wrote it, is that it is sometimes very annoying to wait for suites.All to finish when you get an error/failure on the first line of dots...
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: derby-2667-WriteExceptionsToFileAsTheyHappen.diff, DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Kathey Marsden (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12703914#action_12703914 ]
Kathey Marsden commented on DERBY-2667:
---------------------------------------
Several improvements have been made under this issue over several releases, actually none to the TestRunner itself. I am thinking resolving it now so more specific issues can be opened and resolved in the release where they were fixed. There is one outstanding patch from Kristian that would be very useful, but the patch no longer applies to trunk.
Should I leave this open until that makes it in, or just close this one out and open a new one to print the failure trace to the fail directory as they happen?
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: derby-2667-WriteExceptionsToFileAsTheyHappen.diff, DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Kathey Marsden (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571603#action_12571603 ]
Kathey Marsden commented on DERBY-2667:
---------------------------------------
In eclipse I see a warning for this finally block
finally{
throw running;
}
}
finally block does not complete normally.
I think this is because we lose any exception that occurs in saving off the derby.log file.
I assume though that that was intentional because it is more important to preserve the
exception that occurred running the test, than the one that occurred saving the file. Is
that correct. I found this thread about the warning:
http://dev.eclipse.org/newslists/news.eclipse.tools.jdt/msg10787.html
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Daniel John Debrunner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12566407#action_12566407 ]
Daniel John Debrunner commented on DERBY-2667:
----------------------------------------------
It would be good to add comments to the code so that it's clear to readers what the purpose of the code is.
The streams for the files don't seem to get closed.
The caught exception is never re-thrown, so this will hide any failures.
Fetching the property user.dir is likely to result in a security exception since it's not in a priv block,
the class has a utility method for reading system properties, but it's not needed since one does
not need to read user.dir to access files in the current directory. Something like new File("system", "derby.log") should work.
Also it would be good to test that the file derby.log exists, it's not guaranteed that a test will create it.
Also probably better not to construct file paths using File.separator, instead use File objects.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_stat_02_06.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Kathey Marsden (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571630#action_12571630 ]
Kathey Marsden commented on DERBY-2667:
---------------------------------------
No suggestion. I just hadn't seen the warning before so wanted to make sure I understood what it meant.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Knut Anders Hatlen updated DERBY-2667:
--------------------------------------
Attachment: MemRunner.java
TimeRunner.java
I'm attaching the two test runners I have written and granting licence to ASF in case someone wants to play with them. MemRunner runs all the tests in suites.All and reports the increase/decrease in memory footprint for each fixture. TimeRunner runs all the tests in suites.All and reports how many milliseconds each fixture took.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.0.0
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Daniel John Debrunner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12496976 ]
Daniel John Debrunner commented on DERBY-2667:
----------------------------------------------
I think most of those requirements are fulfilled by the ant junit task, except for the last one, which as you say is a test issue not a test runner issue.
I don't see any memory issues running junit-all, I run it as a continuous integration test against Derby. I am getting a few tests failing but have only just started looking into that.
Note that with the first one I'm looking at the lack of information in trying to solve the failure is not due to a test runner issue, but Derby not providing any information, e.g. I just entered DERBY-2671 for the bug where early failures in starting the network server are lost.
I think that dependance on a single junit test runner is not a good approach, Derby and the tests themselves should be providing information that can be useful in any test runner. Diversity in test runners may also expose bugs in Derby, e.g. I would say that I only entered DERBY-2671 because I was running in a different test runner to my usual use of the swing one.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.0.0
> Reporter: Kathey Marsden
> Priority: Minor
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Myrna van Lunteren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Myrna van Lunteren updated DERBY-2667:
--------------------------------------
Derby Info: (was: [Patch Available])
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Myrna van Lunteren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571549#action_12571549 ]
Myrna van Lunteren commented on DERBY-2667:
-------------------------------------------
I did not notice the weird spacing before committing, sorry...
I wanted to fix it and noticed I had missed another issue - we now have two finally blocks, one of which will never be reached...
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Myrna van Lunteren (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571270#action_12571270 ]
Myrna van Lunteren commented on DERBY-2667:
-------------------------------------------
Thanks Manjula, I like the latest patch.
I committed patch DERBY-2667_diff_02_21.txt with revision 63007.
Probably superfluous, but, I do want to point out that the TestConfiguration.getFailureFolder creates a 'fail' folder, and all directories underneath (fail/<configuration>/<testName>/<failedFixtureName>).
This means, that if you run a subsequent failing fixture test in the same top level directory, previous derby.logs will be overwritten, and also, old run's fail directories won't get cleaned out.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Manjula Kutty (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manjula Kutty updated DERBY-2667:
---------------------------------
Attachment: DERBY-2667_diff_02_06.txt
DERBY-2667_stat_02_06.txt
Attaching a patch which will copy the derby.log file of the failed test to the dir user.dir/fail/clientname/testclass/testname. Please review and if looks useful please commit
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_stat_02_06.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Manjula Kutty (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571607#action_12571607 ]
Manjula Kutty commented on DERBY-2667:
--------------------------------------
Yes, that was my intention. But no harm if I can catch the exception from saving the file too. Do you have any good suggestion for that?
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12572461#action_12572461 ]
Kristian Waagan commented on DERBY-2667:
----------------------------------------
Well, maybe I misunderstood, but if you overwrite, it will be because you run the test / suite again, won't it?
The naming logic of the failure folder seems to consist of something related to connection / JDBCClient, the test class and the test method.
So as an example:
./Embedded_40/BlobClob4BlobTest/testBlobAfterClosingConnection/stacktrace.out
I just assumed there could be only one exception thrown for a single run (in runBare), but maybe my comment above regarding overwriting was a bit confusing.
I haven't actually studied the logic for the failure folder naming, but maybe someone else has some info?
But of course, appending to the file might still be a good thing to do, or maybe a timestamp or something in the filename?
(if you tend to run the tests from the same location over and over again when testing/debugging)
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: derby-2667-WriteExceptionsToFileAsTheyHappen.diff, DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Daniel John Debrunner (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12496888 ]
Daniel John Debrunner commented on DERBY-2667:
----------------------------------------------
Not sure if "robust" is the right tem here, do you really mean a "more informative" test runner?
junit.textui.TestRunner does in fact print chained exceptions, it's Derby that is not chaining the exceptions. I had a fix that did chain the exceptions but it broke on jdk16 so it was backed out and put into a Jira entry which I think has yet to be fixed. DERBY-2472
For the separate file issue, have you looked at the way ant runs junit tests? We have the top-level junit-all target and that provides a whole lot of information in XML that can help in diagnosing issues. It's also standard so that others tools know how to handle the information, e.g. CruiseControl presents the ant junit task output as web-pages complete with sections for failed tests etc.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.0.0
> Reporter: Kathey Marsden
> Priority: Minor
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Dag H. Wanvik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12572458#action_12572458 ]
Dag H. Wanvik commented on DERBY-2667:
--------------------------------------
Re derby-2667-WriteExceptionsToFileAsTheyHappen.diff: Why not append to the fle instead of overwriting? I tend to find the first error/failure the most interesting...
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: derby-2667-WriteExceptionsToFileAsTheyHappen.diff, DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12570179#action_12570179 ]
Knut Anders Hatlen commented on DERBY-2667:
-------------------------------------------
The check of testDir.exists() looks misplaced. If it returns false, the construction of a FileInputStream would have failed with FileNotFoundException two lines above it.
Also, I think it would be best only to create the FileOutputStream if testDir exists. Otherwise, we'll create an empty file for each failing test that doesn't have a derby.log file.
Many of the lines use a mix of tabs and spaces for indentation.
Should new File("System", "derby.log"); use "system" instead? I don't think it will work on case-sensitive file systems, since the directory created by the framework is called "system".
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Manjula Kutty (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Manjula Kutty updated DERBY-2667:
---------------------------------
Derby Info: [Patch Available]
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_stat_02_06.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Ole Solberg (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ole Solberg updated DERBY-2667:
-------------------------------
Attachment: JUnitMethodTrace_Extra.diff.txt
JUnitMethodTrace.diff.txt
In the regression testing and also in more experimental testing I have felt a need for more details in the JUnit test reports - e.g. to convince myself that a test was really executed.
I have done this by modifying 'java/testing/org/apache/derbyTesting/junit/BaseTestCase.java' to print the test method name, and have also included a printout of the time (in ms.) used by the test method.
I use "-Dderby.tests.trace=true" to turn this on.
Attachement JUnitMethodTrace.diff.txt.
'java/testing/org/apache/derbyTesting/unitTests/junit/FormatableBitSetTest.java' currently uses JUnit TestCase and would thus not give the desired extra information: The JUnitMethodTrace_Extra.diff.txt patch changes that by using 'BaseTestCase'.
Attachement JUnitMethodTrace_Extra.diff.txt
Example output:
.
testSetCharacterStream used 200ms .
testGetAsciiStream used 664ms .
testGetCharacterStream used 638ms .
testGetCharacterStreamWithUnicode used 817ms .
testTriggersWithClobColumn used 2453ms .
testGetSubString used 431ms E.
testGetSubStringWithUnicode used 855ms .
testPositionString used 713ms .
testPositionStringWithUnicode used 739ms .
testPositionClob used 4204ms .
testPositionClobWithUnicode used 4832ms .
testSmallClobFields used 82ms .
testGetClobFromIntColumn used 302ms F.
testSetClobToIntColumn used 412ms F.
testRaisingOfExceptionsClob used 523ms F.
testSetClob used 1116ms .
testPositionAgressive used 9073ms .
testClobAfterClose used 587ms .
testLockingClob used 60695ms .
testLockingWithLongRowClob used 60178ms .
.
.
.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.0.0
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Kristian Waagan (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12704013#action_12704013 ]
Kristian Waagan commented on DERBY-2667:
----------------------------------------
I'd say close this one and create a new one for the second feature.
Once it is logged, I'll refresh the patch and ready it for review.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: derby-2667-WriteExceptionsToFileAsTheyHappen.diff, DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Knut Anders Hatlen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12570621#action_12570621 ]
Knut Anders Hatlen commented on DERBY-2667:
-------------------------------------------
Is the check for (running != null) necessary? I don't see how it could be null.
First two lines use spaces for indentation, the rest use tabs. Would be good to use one of them consistently. Since the surrounding code uses spaces, I think that's the best option.
Should the call to getFailureFolder() also be moved inside the if? Then we don't create a folder unless we have to.
Would it make sense to put a try/finally block inside the catch block and re-throw the exception in the finally clause? This way we make sure that the original exception is not hidden if saving derby.log fails for some reason.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (DERBY-2667) Create more robust junit TestRunner
for running derby tests
Posted by "Dag H. Wanvik (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/DERBY-2667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12572468#action_12572468 ]
Dag H. Wanvik commented on DERBY-2667:
--------------------------------------
Thanks, Kristian, I was not thinking of the multiple runs case. As long as the first failure/error
in one run is not clobbered by a subsequent one in the same run, I am OK with this solution.
> Create more robust junit TestRunner for running derby tests
> ------------------------------------------------------------
>
> Key: DERBY-2667
> URL: https://issues.apache.org/jira/browse/DERBY-2667
> Project: Derby
> Issue Type: Improvement
> Components: Test
> Affects Versions: 10.3.1.4
> Reporter: Kathey Marsden
> Priority: Minor
> Attachments: derby-2667-WriteExceptionsToFileAsTheyHappen.diff, DERBY-2667_diff_02_06.txt, DERBY-2667_diff_02_15.txt, DERBY-2667_diff_02_21.txt, DERBY-2667_diff_2_19.txt, DERBY-2667_stat_02_06.txt, DERBY-2667_stat_02_15.txt, DERBY-2667_stat_02_19.txt, DERBY-2667_stat_02_21.txt, JUnitMethodTrace.diff.txt, JUnitMethodTrace_Extra.diff.txt, MemRunner.java, TimeRunner.java
>
>
> Provide a more full featured TestRunner for Derby testing.
> junit.textui.TestRunner is not very robust. It does not for example print the tests as they run or print chained exceptions, create separate files for the full report and just failures. It would be great to have a standardized TestRunner that everyone uses. Perhaps someone already has one that they would like to contribute as a starter.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.