Re: Running all Jackrabbit tests with multiple configurations

[Marcel Reutegger <ma...@gmx.net>]

Hi Thomas,

I would rather parametrize the current test setup and invoke the test suites 
multiple times.

without having looked at it in detail, that requires the following steps:

- create multiple repository.xml and workspace.xml files in src/test/repository/**

- for each repository.xml file the default workspace will be different -> 
different workspace.xml file (e.g. different persistence manager)

- change JackrabbitRepositoryStub to also take into account system properties, 
like it already does for the jaas config.

 > mvn install

will run just like today, but

 > mvn 
-Dorg.apache.jackrabbit.repository.config=target/repository/repository-with-data-store.xml 
install

will now run the tests with a datastore configured.

if needed we can specify different repository homes similarly.

regards
  marcel

Thomas Mueller wrote:
> Hi,
> 
> Currently the Jackrabbit core tests are only run with the default
> configuration (with the default persistence manager, without data
> store). The tests should be run with different configurations as well
> (for example with the BundleFsPersistenceManager, with data store). To
> do that, we need to do two things: the test repository.xml must be
> 'dynamic', that is, generated / modified / replaced while the tests
> run. Second, the tests must be invoked multiple times. It would be bad
> to loop in each test case (run the first test with all persistence
> managers, then the second, and so on), because it would be very slow.
> Better is to run all tests with the default configuration, and then a
> second time with different settings, and so on. To do that, we could
> use a 'super test suite' (uber test suite). I found a way to do that.
> If somebody has a better idea please help (I'm not a JUnit
> specialist)!
> 
> // new "super test suite"
> package unit;
> import junit.framework.*;
> public class TestEverything extends TestCase {
>     public static int config = 0;
>     public static Test suite() {
>         TestSuite suite = new TestSuite("All tests with all configs");
>         suite.addTest(unit.sub.TestAll.suite());
>         suite.addTestSuite(TestEverything.class);
>         suite.addTest(unit.sub.TestAll.suite());
>         suite.addTestSuite(TestEverything.class);
>         suite.addTest(unit.sub.TestAll.suite());
>         return suite;
>     }
>     public void testNext() {
>         config++;
>     }
> }
> 
> // current test suite
> package unit.sub;
> import junit.framework.*;
> public class TestAll extends TestCase {
>     public static Test suite() {
>         TestSuite suite = new TestSuite("Subproject Tests");
>         suite.addTestSuite(TestOne.class);
>         return suite;
>     }
> }
> 
> // single test
> package unit.sub;
> import junit.framework.TestCase;
> public class TestOne extends TestCase {
>     public void testIt() {
>         System.out.println("config: " + unit.TestEverything.config);
>     }
> }
> 
> Regards,
> Thomas
> 

Re: Running all Jackrabbit tests with multiple configurations

[Thomas Mueller <th...@gmail.com>]

Hi,


> - create multiple repository.xml and workspace.xml files in
> src/test/repository/**


If we hard code everything, we would need to maintain a large number of such
files.

- no data store
- FileDataStore
- DbDataStore

If you want to be able to test those with each persistence manager
(currently we have about 12 PMs), we would have 36 configurations (3 data
store * 12 pm). Also, you would need to have fixed settings for the
persistence managers (fixed JDBC URLs). Then, let's say we test multiple
search index configurations, you would get another dimension. For each
binary setting, you would double the number of such files. I think
dynamically creating the repository.xml and workspace.xml is better.

 > mvn
> -
> Dorg.apache.jackrabbit.repository.config=target/repository/repository-with-data-store.xml
> install


Then you would need to call mvn for each configuration. But using a system
property is a good idea, what about:

mvn install
  runs only some tests with some combinations (the most popular)
mvn -Dorg.apache.jackrabbit.test=all install
  runs all possible test combinations (could take hours, run by Continuum)
mvn -Dorg.apache.jackrabbit.testPm=derby,h2 install
  runs the tests with the Derby PM and the H2 PM
mvn -Dorg.apache.jackrabbit.testDataStore=none,db install
  runs the tests with the no data store, and the db data store

Another idea is to define the test combinations in a properties file:

pmDerby.class=org.apache.jackrabbit.core.persistence.bundle.DerbyPersistenceManager
pmDerby.url=jdbc:derby:...
pmDerby.minBlobSize=1000

pmPostgreSQL.class=...
...

testPM=pmDerby,pmPostgreSQL

You could then have multiple of those properties files, and you could set
which one to use using:

mvn -Dorg.apache.jackrabbit.test=myconfigs.properties install
  runs the tests defined in myconfigs.properties

Regards,
Thomas