JUnit

[Jeremy Boynes <jb...@apache.org>]

Daniel John Debrunner wrote:
 > I have been experimenting with Junit, but that needs its own thread.
So here we go :-)

What are your opinions so far?

I would be in favour of a gradual transition to JUnit based testing due 
to the familiarity many developers have with it. Although migrating the 
current ".java" tests may be fairly easy, migrating the ".sql" tests 
would seem a much larger task.

It also may not be appropriate for other testbeds such as the NIST 
tests. I'm entirely sure what the process is for running all those anyway.

--
Jeremy

Re: JUnit

[Daniel John Debrunner <dj...@debrunners.com>]

Daniel John Debrunner wrote:

> I have to leave for a flight back to the West coast now, once I'm back
> on-line I'll see if I can find my test code.


Attached.

ijRunner.java

Dan.

Re: JUnit

[Thomas Hawtin <ta...@claranet.com>]

Daniel John Debrunner wrote:
> 
> My concerns with JUnit are its platform coverage, and I just haven't
> looked into this. Will it run on JDK 1.3, J2ME/CDC/Foundation, OSGi
> ee.minimum, Z/OS etc.  Would that community be willing to support those
> platforms?

A problem with JUnit on many J2ME platforms is that it uses reflection. 
J2MEUnit is a related framework, but requires extra work to name and 
execute individual test methods.

A possible approach is to insert a DerbyTestCase class into the 
hierarchy. For J2SE builds it would just extends 
junit.framework.TestCase. For J2ME it would extend the J2MEUnit 
equivalent. To run the tests on J2ME some reflection/APT type approach 
would need to generate the simple code to enumerate test methods and the 
switch statement to execute them.

Sometime ago I had a personal project among other things generated the 
required code using BCEL, but I got bored...

Tom Hawtin

Re: JUnit

[Daniel John Debrunner <dj...@debrunners.com>]

myrna wrote:


> 
> I'm interested in being involved in this, not necessarily in doing
> this... :-)
> Those passing tests were .java tests, and it seems our trouble is still
> the .sql tests...

No, I was only running .sql tests. But really there is no difference in
the approach for handling the existing .java and .sql tests. They are
both executions of java code whose output needs to be compared with a
known file.

Dan.

Re: JUnit

[myrna <my...@Golux.Com>]

Daniel John Debrunner wrote:

>Daniel John Debrunner wrote:
>
>  
>
>>      -  Simple model should lead to significantly faster testing times
>>as the JUnit model is to run the tests within the same JVM. The nist
>>tests run in the exising harness like this and they are significantly
>>faster than they used to be.
>>    
>>
>
>20 passing tests with the Derby test harness - 8m03s
>20 passing tests with JUnit (my quick hack)  - 1m36s
>
>even if the actual JUnit solution was twice as slow it would still be a
>significant time saving.
>
>Anyone who stepped up to take this on would have the gratitude of anyone
>running tests against Derby.
>
>Dan.
>
>  
>
Dan,

I'm interested in being involved in this, not necessarily in doing 
this... :-)
Those passing tests were .java tests, and it seems our trouble is still 
the .sql tests...

Also, does JUnit handle client-server situations like we'd like for 
networkserver?

Jeremy,
As I'm not hindered by knowledge of JUnit, I can say that based on what 
Dan's been saying the recently contributed unit tests are good ones to 
try out with JUnit...They hang only loosely in the current functionTests 
framework and their output is only a simple start-stop success/fail.

Myrna

Re: JUnit

[Daniel John Debrunner <dj...@debrunners.com>]

Daniel John Debrunner wrote:

>       -  Simple model should lead to significantly faster testing times
> as the JUnit model is to run the tests within the same JVM. The nist
> tests run in the exising harness like this and they are significantly
> faster than they used to be.

20 passing tests with the Derby test harness - 8m03s
20 passing tests with JUnit (my quick hack)  - 1m36s

even if the actual JUnit solution was twice as slow it would still be a
significant time saving.

Anyone who stepped up to take this on would have the gratitude of anyone
running tests against Derby.

Dan.

Re: JUnit

[Daniel John Debrunner <dj...@debrunners.com>]

Jeremy Boynes wrote:

> Daniel John Debrunner wrote:
>> I have been experimenting with Junit, but that needs its own thread.
> So here we go :-)
> 
> What are your opinions so far?
> 
> I would be in favour of a gradual transition to JUnit based testing due
> to the familiarity many developers have with it. 

 Here are some additional reasons I think JUnit would be good:

      -  Get rid of exising harness code

      -  Simple model should lead to significantly faster testing times
as the JUnit model is to run the tests within the same JVM. The nist
tests run in the exising harness like this and they are significantly
faster than they used to be.

      - move away from "master file" based testing to assert based
testing, though it may require up front writing of utilites that handle
validating result set output. Ie the test provides the expected output
of the result set in some array form and a utility handles comparing
that to the result from Derby, including factoring out any ordering
differences.


My concerns with JUnit are its platform coverage, and I just haven't
looked into this. Will it run on JDK 1.3, J2ME/CDC/Foundation, OSGi
ee.minimum, Z/OS etc.  Would that community be willing to support those
platforms?

> Although migrating the
> current ".java" tests may be fairly easy, migrating the ".sql" tests
> would seem a much larger task.

All the exisitng Derby tests compare the output from the test (on
System.out) to known correct output. Within the Java tests there may be
additional assert type checks, but they are tested by printing different
output.

I actually wrote a really simple JUnit aimed at the .sql tests, I
ignored several items the Derby harness does, just to get an idea of how
it would work. Each .sql test is a JUnit test and a ran as a suite of a
handful of tests. The main differences between this and the existing
Derby harness are:

    - tests run within the same JVM
    - tests run against the same database, I was too lazy to write code
to drop the old database or clean it up, but that's simple coding work.
    - test output is not written to disk, except when the test fails
    - compare with master file works on line by line and fails at the
first difference. Thus on failure a developer would most likely use diff
to see how the test really failed.

It is example (quick hack) code, it could be generalized (probably
through multiple test classes) to handle .java tests and others.

Examples of items the harness does that I didn't address were setting up
the properties files associated with some tests, sed'ing the output to
remove variance, copying across additional files.

I looked at about 10 language .sql tests and about 6 passed within this
environment. Some failed due to the lack of the sed operation, though
that in itself was interesting. I'm not a great fan of the sed approach,
too much potential for failures to be sed'ed out. The insert test only
failed because the existing harness was sed'ing out a timestamp, but
that timestamp was constant, so there was no need for a sed. Some failed
due to the lack of a clean database to start with (again, because I was
to lazy to write the cleanup code).

I have to leave for a flight back to the West coast now, once I'm back
on-line I'll see if I can find my test code.

Getting JUnit to run would be a good project, that does not require
database internal knowledge, I'm unlikely to spend any more time on
coding this.


Dan.