[Db-derby Wiki] Update of "DerbyJUnitTesting" by DanDebrunner

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Db-derby Wiki" for change notification.

The following page has been changed by DanDebrunner:
http://wiki.apache.org/db-derby/DerbyJUnitTesting

------------------------------------------------------------------------------
  === The suite() method ===
  The `suite()` method is a static method returning the tests to be run. We use it to gain more flexibility in which test methods are executed and the environment they are executed in. For instance, the use of the `suite()` method allows us to use test decorators (the most common case is setting up and populating a common database used by all tests in the suite).
  
- By default, all test methods whose name starts with `test` are added to the suite, typically with `new TestSuite(MyDerbyTests.class, "My suite name")`. If there are tests that should only be run in certain scenarioes, logic for doing this should be placed in `suite()`. The prominent example is a test method that is only to be run in the !DerbyNetClient framework. To fix this, name the method for instance `clientSomeTest` and add it manually to the suite if `BaseJDBCTestCase.usingDerbyNetClient()` returns true with `suite.addTest(new MyDerbyTests("clientSomeTest"))`. In general, all tests should be written to be able to run under any framework, and tests written for a specific framework should be placed in a separate test class. When it is run should be controlled by the mechanisms provided by the test harness (exclude/skip files).
+ By default, all test methods whose name starts with `test` are added to the suite, typically with `new TestSuite(MyDerbyTests.class, "My suite name")`. If there are tests that should only be run in certain scenarioes, logic for doing this should be placed in `suite()`. The prominent example is a test method that is only to be run in the !DerbyNetClient framework. To fix this, name the method for instance `clientSomeTest` and add it manually to the suite if `BaseJDBCTestCase.usingDerbyNetClient()` returns true with `suite.addTest(new MyDerbyTests("clientSomeTest"))`. In general, all tests should be written to be able to run under any framework. Tests that need to be written for a specific framework should be placed in a separate test class and have their `suite()` method return an empty suite when running in a different framework.
+ 
+ Thus a test is self-contained about which frameworks and it can be run in. This avoids the situation with the [:KillDerbyTestHarness:old Derby harness] where discovering if a test is being run in a framework and/or environment is very painful. WHile JUnit tests continue to be run under the [:KillDerbyTestHarness:old Derby harness] when it is run should be controlled by the mechanisms provided (exclude/skip files).
+ 
+ === Package Level _Suite ===
+ 
+ A single suite defined by a class `_Suite` in a package should exist for all the JUnit tests in that package.
+ That is it has a `suite()` method that returns all the tests in the package by invoking their `suite()` method and adding
+ the result to an instance of TestSuite or adding them by using `classname>.class`.
+ E.g. for the package `org.apache.derbyTesting.functionTests.tests.jdbcapi` a class `org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite` would exist that runs all the tests in that package.
+ 
+ While the the [:KillDerbyTestHarness:old Derby harness] continues to exist the _Suite class should only include JUnit tests that successfully run
+ using JUnit test runners directly. As `_Suite` classes evolve they may replace individual test entries in the old suite runall files, so the old
+ harness runs the _Suite instead of the JUnit tests individually. If a test requires existing functionality of the old harness (such as skip in J2ME mode)
+ that does not yet have a corresponding mechansim with JUnit then it should not be in the package suite.
+ 
+ Some issues about _Suite:
+  * For a suite such as jdbcapi how should running in different frameworks be handled?
+    * Should the _Suite automatically return a suite that runs all the tests in all the frameworks (individual tests can opt out of a framework)?
+      * then how to run only embedded jdbcapi tests ?
+    * Or should a higher level suite add the ability to run in different frameworks?
+      * Then how to handle tests that only run in a specific framework, maybe a separate package?
  
  === Usage of SQL states in JUnit tests ===
  In your tests, you might want to check for a specific SQL state when an SQLException is thrown. The recommended way to handle this, is to use the `BaseJDBCTestCase.assertSQLState(String,String,SQLException)` and either pass in a constant from `org.apache.derby.functionTesting.util.SQLStateConstants`, or simply hardcode the value or use your own test-local constant. Doing this will assure tests break if someone changes the SQL state for an exception in the internal class, and raises the awareness of SQL state changes in the community.