[jira] [Commented] (CONTINUUM-2751) Build fails due to test interactions: data not being cleaned consistently

["Brent N Atkinson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CONTINUUM-2751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14493139#comment-14493139 ] 

Brent N Atkinson commented on CONTINUUM-2751:
---------------------------------------------

This one looked very strange, since the test didn't change and suddenly the database had multiple values magically. After looking at how the test data was loaded (an in-memory database), the only thing I could think of was leaked test data between the tests. However, it did not occur locally and it appeared that each test method gets its own database, from {{AbstractContinuumTest}}:

{code}
        String url = "jdbc:hsqldb:mem:" + getClass().getName() + "." + getName();
        jdoFactory.setUrl( url );
        jdoFactory.reconfigure();
{code}

It turns out that:
  * the foundation of the data access layer uses {{AbstractDao.getPersistenceManager()}}
  * {{AbstractDao.getPersistenceManager()}} uses {{StoreUtilities.getContinuumPersistenceManagerFactory()}} to get the factory to create persistence managers
  * {{StoreUtilities.getContinuumPersistenceManagerFactory()}} caches and always returns the first value returned by its {{JdoFactory}} (jdoFactory#continuum)

While many tests may (and do) reconfigure the {{JdoFactory}} to use an isolated data source, the first one configured always wins. Therefor the order the spring and plexus definitions are loaded in will determine what datasource the DAOs will be configured with, since the reconfigured factory is never used (remember StoreUtilities caches the first value). Because of this, every test ends up sharing the same database, which elevates the importance of leaving it in a clean state.

NOTE: During conversion of the tests to JUnit 4 in CONTINUUM-2745 the new plexus-spring test case implementation was put in continuum-test. Tests that previously only depended on the implementation provided by the plexus-spring library were updated to depend on continuum-test instead. Unfortunately, continuum-test defines a test database that satisfies jdoFactory#continuum, which may have changed the semantics depending on the order the configurations are loaded in by {{PlexusSpringTestCase}}. Ideally, we should only be loading just enough spring and plexus configuration for the given tests. However, due to the way that the XML files are named (all 'spring-context.xml' or 'plexus/components.xml') selectively loading only certain modules is not possible.

> Build fails due to test interactions: data not being cleaned consistently
> -------------------------------------------------------------------------
>
>                 Key: CONTINUUM-2751
>                 URL: https://issues.apache.org/jira/browse/CONTINUUM-2751
>             Project: Continuum
>          Issue Type: Bug
>    Affects Versions: 1.5.0
>            Reporter: Brent N Atkinson
>            Assignee: Brent N Atkinson
>             Fix For: 1.5.0
>
>
> Very similar in spirit to CONTINUUM-2736, tests are not well-behaved which is leading to build instability. On continuum-ci.a.o, I noticed the builds were consistently failing despite the full test suite comprehensively passing locally.
> The problem is visible in the build results as the test failure: 
> {noformat}
> Results :
> Failed tests:   testRemoveRepository(org.apache.continuum.repository.DefaultRepositoryServiceTest): check # repositories expected:<1> but was:<2>
> Tests run: 150, Failures: 1, Errors: 0, Skipped: 0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)