running JUnit tests

[Andreas Korneliussen <An...@Sun.COM>]

It seems to me that for including a new JUnit test into i.e derby-all we 
need to make a new java class with a main() method, which parses a 
command line and set up the testsuite and run it, just like any java 
program. Basically we are running the junit tests as test type "java".

Instead of having to do this for every junit test going into a derby 
test suite, I would propose a different strategy.

I propose to introduce a new test type called "junit" (current test 
types are: sql,sql2,unit,java,multi,demo - unit is not junit)

Then you can use:

java org.apache.derbyTesting.functionTests.harness.RunTest 
<TestCaseClassName>.junit

to run a Junit test - instead of:

java org.apache.derbyTesting.functionTests.harness.RunTest 
<AnotherClassWithMainMethod>.java

When starting a test of type junit, the RunTest class may simply use the
junit.textui.TestRunner class, which has a main method which takes a 
TestCase class name as parameter.  The junit.textui.TestRunner  runs the 
tests defined by the suite() method of the TestCase class.

I think this strategy will make it easier to integrate new JUnit tests 
into the current test suites, since it save you the trouble of creating 
a java class with a main method for every test.


-- Andreas

Re: running JUnit tests

[Andreas Korneliussen <An...@Sun.COM>]

> 
> Is there any write up of the correct way to write new Junit tests?

I am trying to follow the junit cookbook when writing new junit tests.
http://junit.sourceforge.net/doc/cookbook/cookbook.htm

I do have some thoughts on how to write new JUnit Tests.

When writing tests for Derby (using JDBC), I think most tests need to 
have access to some configuration data, like jdbc url, jdbc classname 
etc.  This could be put/gathered  into a immutable singleton object
(i.e DerbyTestConfig) which can be accessed by the tests.
Alternatively it could be gathered in a common testcase class, like 
DerbyJUnitTestCase, however then you may need to duplicate this code if 
you make i.e TestDecorators which are not subclasses of DerbyJUnitTestCase.

Since many tests are using JDBC connections, it could be useful to have 
a common TestCase base class which sets up a connection in the setUp() 
and closes it in the tearDown().  I.e:

class DerbyConnectionTestCase extends TestCase {

   protected Connection con;

   final void setUp() throws Exception
   {
      con =  DriverManager.connect(config.getJdbcUrl(), ..);
      doSetup();
   }

   final void tearDown() throws Exception
   {
      doTearDown();
      con.close(); <- of course with some better exception handling..
   }
..
}

> I think a big mistake we made with the old harness was no formal way to
> add Java tests, which meant multiple ways to start the engine and obtain
> the connection and multiple utility methods (almost) performing the same
> action.
>

When it comes to setting up additional fixture before running a set of 
testcases, I think a powerful and reusable mechanism is the 
junit.extensions.TestSetup. I.e one could make a DerbyNetworkServerSetup 
class. In its setUp() method it starts the network server, and in the 
tearDown() in stops it.  So if you have a suite of tests which requires 
a running network server, they can be wrapped like this:

TestSuite suite = new TestSuite(MyDerbyTestCase.class);
Test t = new DerbyNetworkServerSetup(suite);

When running Test t, it will first call setUp() in 
DerbyNetworkServerSetup, then it will run all testcases in 
MyDerbyTestCase, and finally call tearDown() in DerbyNetworkServerSetup.

Of course, one should consider if DerbyNetworkServerSetup should simply 
do nothing if the test is only testing in embedded mode (i.e make it 
dependent on the test configuration).


> Would be nice if there were guidelines provided by whoever set up the
> initial Junit framework.
> 

I think Rick H /David set it up, so maybe they have some more guidelines 
or ideas.

--Andreas

Re: running JUnit tests

[Daniel John Debrunner <dj...@apache.org>]

Andreas Korneliussen wrote:

> It seems to me that for including a new JUnit test into i.e derby-all we
> need to make a new java class with a main() method, which parses a
> command line and set up the testsuite and run it, just like any java
> program. Basically we are running the junit tests as test type "java".
> 
> Instead of having to do this for every junit test going into a derby
> test suite, I would propose a different strategy.
> 
> I propose to introduce a new test type called "junit" (current test
> types are: sql,sql2,unit,java,multi,demo - unit is not junit)
<snip>

Sounds good.

Is there any write up of the correct way to write new Junit tests?
I think a big mistake we made with the old harness was no formal way to
add Java tests, which meant multiple ways to start the engine and obtain
the connection and multiple utility methods (almost) performing the same
action.

Would be nice if there were guidelines provided by whoever set up the
initial Junit framework.

It would also be great if there was some documentation on how the tests
are expected to run. Looking at DerbyJUnitTest I can't understand why
there is static state, such as _databaseName, I though Junit tests were
objects?

Dan.

Re: running JUnit tests

[Andreas Korneliussen <An...@Sun.COM>]

John Embretsen wrote:
> Andreas Korneliussen wrote:
> 
>> I propose to introduce a new test type called "junit" (current test 
>> types are: sql,sql2,unit,java,multi,demo - unit is not junit)
>>
>> Then you can use:
>>
>> java org.apache.derbyTesting.functionTests.harness.RunTest 
>> <TestCaseClassName>.junit
>>
>> to run a Junit test - instead of:
>>
>> java org.apache.derbyTesting.functionTests.harness.RunTest 
>> <AnotherClassWithMainMethod>.java
> 
> 
> If I understand this proposal correctly, there will be a one-to-one 
> mapping between so-called ".junit" tests and ".java" files, right? And 
> there will be no actual files ending with ".junit", but the harness will 
>  map the test names ending with ".junit" to actual java/class files 
> containing the actual JUnit test code?

Yes, the class name.

The harness uses the extension of the file submitted to figure out the 
test type.

I.e if it is .sql it starts ij, if it is .java it starts java with the 
class as argument, if it is unit it starts 
org.apache.derbyTesting.unitTests.harness.UnitTestMain, if it is multi 
it starts org.apache.derbyTesting.functionTests.harness.MultiTest

Which java program it starts is defined in the
buildTestCommand(..) method in 
org.apache.derbyTesting.functionTests.harness.RunTest

The proposal is that if it is junit, it starts "java 
junit.textui.TestRunner" application with the class as argument.

Andreas

Re: running JUnit tests

[John Embretsen <Jo...@Sun.COM>]

Andreas Korneliussen wrote:

> I propose to introduce a new test type called "junit" (current test 
> types are: sql,sql2,unit,java,multi,demo - unit is not junit)
> 
> Then you can use:
> 
> java org.apache.derbyTesting.functionTests.harness.RunTest 
> <TestCaseClassName>.junit
> 
> to run a Junit test - instead of:
> 
> java org.apache.derbyTesting.functionTests.harness.RunTest 
> <AnotherClassWithMainMethod>.java

If I understand this proposal correctly, there will be a one-to-one 
mapping between so-called ".junit" tests and ".java" files, right? And 
there will be no actual files ending with ".junit", but the harness will 
  map the test names ending with ".junit" to actual java/class files 
containing the actual JUnit test code?

I guess I am asking because at the moment I don't fully understand how 
the different types in the current harness work, i.e. how to run/invoke 
the different types of tests, and how the harness handles these cases. 
Is this documented somewhere? The java/testing/README.htm file mentions 
briefly that the .java, .sql, .sql2, .multi and .unit test types exist, 
but not much more. The "demo" type is not mentioned at all...

By the way, there is apparently a bug in this README.html file, which 
states that for multi tests, "the actual test files live under 
org/apache/derbyTesting/functionTests/multiTests" (a directory which 
does not exist in the repository). I guess this directory should be 
"org/apache/derbyTesting/functionTests/multi".


-- 
John