testScript.xml and DerbyJUnitTest refactoring

[Andreas Korneliussen <An...@Sun.COM>]

I have considered refactoring DerbyJUnitTest because of the following:

The way DerbyJUnitTest supports configuring the testcase may be error 
phrone, since it requires the calling of a number of static methods in a 
  specific order for the static members to initialize.  I.e it is not 
possible to run any of the exisiting JUnit tests using a standard Junit 
  testrunner without getting a NullPointerException. The static members 
are also non-final, and this allows testcases to call modifiers, 
potentially causing side-effects in other testcase objects.

Example:DerbyJUnitTest has a public static method called 
setDatabaseName(String dbName). If I do not call it, the _databaseName 
variable is null.  If one of my testcases call the setDatabaseName(..) 
method, it has side-effects which will affect all other testcases (they 
may start using another database).

The way DerbyJUnitTest supports configuring the correct JDBC client, is 
by having a String[] of client settings for a framework. This can be 
initialized by calling setClient(..) or findClientFromProperties()

         private static String[] _defaultClientSettings;

	public	static	void		setClient( String[] client ) { 
_defaultClientSettings = client; }

	public	boolean	usingDB2Client() { return ( _defaultClientSettings == 
DB2JCC_CLIENT ); }
	
	public	static	final	String[][]	LEGAL_CLIENTS =
	{
		DB2JCC_CLIENT,
		DERBY_CLIENT,
		EMBEDDED_CLIENT
	};

The problem with this approach is that:
1. I may call setClient(..) with any array of strings not being part of 
LEGAL_CLIENTS. This will cause the methods usingDB2Client(), 
using...Client() to all return false, even if my String[] array contains 
a valid set of strings for JDBC client settings.

2. Calling setClient(..) has side-effects on other testcases, and it 
allows "non-standard" initialization of the client settings

3. If I do not call setClient(..) or findClientFromProperties().. 
methods like getConnection() fails with NullPointerException

4. The problem of configuring the client settings has been solved 
before, i.e in TestUtil, where there is a integer enum representing the 
framework.

One thing preventing me from refactoring DerbyJUnitTest is that I 
noticed the testScript.xml in CompatibilitySuite.  This seems to be a 
well-documented test harness for running the CompatibilitySuite against 
different versions of jdbc clients and derby servers. It consists of 
multiple "for loops" implemented in ant, calling 
CompatibiltySuite.main() with different arguments.

CompatibilitySuite.main takes advantage of the public setClient(..) 
method in DerbyJUnitTest, and the client settings are initialized using 
another mechanism than the one in i.e findClientFromProperties(), or any 
other mechanism found in the Derby test harness.

The main() method in the CompatibilitySuite is called from this script 
with the drivername, and by using stringmatching against drivernames in 
the LEGAL_CLIENTS[i][DRIVER_NAME], the correct client is loaded, and set 
using setClient(..).

My goal is to be able to run Derby junit tests from the junit 
testrunner, and to allow easy integration of JUnit tests into the 
current harness in Derby. Maintaining another test harness written as an 
ant.xml script is not really my itch. My question is therefore: could I 
leave it to someone else to maintain testScript.xml, or could it 
alternatively be removed ?

(Script 
org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/testScript.xml)

--Sincerely

Andreas

Re: testScript.xml and DerbyJUnitTest refactoring

[Andreas Korneliussen <An...@Sun.COM>]

Rick Hillegas wrote:

>> ..
>
> When you have a harness that runs JUnit tests, then I can write a new 
> script for running the compatibility combinations. But please don't 
> throw this suite away. Instead, before you change DerbyJUnitTest, 
> maybe you can clone it to some other class just for use by the 
> compatiblity suite. Once you have revamped DerbyJUnitTest and built 
> out the harness, I can convert the compatibility test and suite.
>
Great, thanks for the feedback - I think your suggestion will work fine

-- Andreas

Re: testScript.xml and DerbyJUnitTest refactoring

[Rick Hillegas <Ri...@Sun.COM>]

Andreas Korneliussen wrote:

> I have considered refactoring DerbyJUnitTest because of the following:
>
>
Lots of good stuff here. Thanks for volunteering to turn this piece of 
scaffolding into something robust.

> My goal is to be able to run Derby junit tests from the junit 
> testrunner, and to allow easy integration of JUnit tests into the 
> current harness in Derby. Maintaining another test harness written as 
> an ant.xml script is not really my itch. My question is therefore: 
> could I leave it to someone else to maintain testScript.xml, or could 
> it alternatively be removed ?
>
> (Script 
> org/apache/derbyTesting/functionTests/tests/junitTests/compatibility/testScript.xml) 
>
>
When you have a harness that runs JUnit tests, then I can write a new 
script for running the compatibility combinations. But please don't 
throw this suite away. Instead, before you change DerbyJUnitTest, maybe 
you can clone it to some other class just for use by the compatiblity 
suite. Once you have revamped DerbyJUnitTest and built out the harness, 
I can convert the compatibility test and suite.

Thanks,
-Rick