[Apache JDO Wiki] Updated: TestRunner

[jd...@db.apache.org]

   Date: 2005-01-21T15:28:40
   Editor: MichelleCaisse
   Wiki: Apache JDO Wiki
   Page: TestRunner
   URL: http://wiki.apache.org/jdo/TestRunner

   no comment

Change Log:

------------------------------------------------------------------------------
@@ -2,10 +2,20 @@
 This project is about extending the TCK extensions of the JUnit test runners so that the same TCK tests can be run in multiple configurations. For example, the same TCK test program needs to be run with and without security turned on, and with Application Identity and Datastore Identity. When we add different mappings for Chapter 18 (ORM) tests, the same test will need to be run with security, identity, and different mappings.
 
 == Configuration Parameters ==
-This is the list of configuration parameters that the test runner should be able to handle:
+This is the list of configuration parameters that the test runner should be able to handle, read from an xml file:
  *Test description string
  *Test class
  *Security on/off
  *Application identity / Datastore identity
  *Mapping file (.orm)
  *Test data
+
+== Open Questions ==
+ *Should the test runner execute a single test (comprised of a single combination of description, class, security setting, identity setting, mapping, and data)?  Should it be able to run all test classes, or a specific list of test classes, with both security settings? with both identity types? What is the mechanism for running a batch of tests?
+
+== Requirements ==
+
+ *Each test execution must report the full set of parameters, as listed above. Perhaps this report is the test description and no additional description need be specified by the test writer.
+ *All TCK tests must be run both with security on and security off (true or false?)
+ *All TCK tests must be run both with application identity and datastore identity (true or false?)
+ *Only a small subset of tests for chapter 18 must be run with multiple mapping files and multiple test data sets (true?)