Rerunning tests and additional rest tests with mocked persistence?

[Christian Schneider <ch...@die-schneider.net>]

Hi all,

while testing how adding the CXF libs influences the syncope tests I
found a problem with the current rest tests.

The problem is that at least some of the tests can not simply be rerun.
For example org.apache.syncope.core.rest.RoleTestITCase.create() only
seems to succeed once.
When I start the server and rerun the test from eclipse it fails as the
role is already there.

So one way to solve this would be to make each test method clean up
after itself. So at least after a successful test the db should be ready
for another run.

One other way would be to create some additional tests that work with a
mocked persistence layer. These tests would then always be able to rerun
as they do not need or change the db.

I think it makes sense to follow both of these approaches. So what do
you think?

Christian

RE: Rerunning tests and additional rest tests with mocked persistence?

[Andrei Shakirin <as...@talend.com>]

Hi Christian,

I find tests with mocked persistence layer really good, but for me they are fast Unit tests.
We need additionally (independent from Unit) integration/systests.
For integration tests I see two possibilities to make them successfully running more than once:
A) Generate unique ID for DB keys;
B) Clean up after each test.

Every approach has pros and cons, personally I would accept both. 
Sure, we should be able to run also integration tests more than once without restart and reinitialize DB.

Cheers,
Andrei.

> -----Original Message-----
> From: Christian Schneider [mailto:cschneider111@gmail.com] On Behalf Of
> Christian Schneider
> Sent: Montag, 10. Dezember 2012 13:26
> To: dev@syncope.apache.org
> Subject: Rerunning tests and additional rest tests with mocked persistence?
> 
> Hi all,
> 
> while testing how adding the CXF libs influences the syncope tests I found a
> problem with the current rest tests.
> 
> The problem is that at least some of the tests can not simply be rerun.
> For example org.apache.syncope.core.rest.RoleTestITCase.create() only
> seems to succeed once.
> When I start the server and rerun the test from eclipse it fails as the role is
> already there.
> 
> So one way to solve this would be to make each test method clean up after
> itself. So at least after a successful test the db should be ready for another
> run.
> 
> One other way would be to create some additional tests that work with a
> mocked persistence layer. These tests would then always be able to rerun as
> they do not need or change the db.
> 
> I think it makes sense to follow both of these approaches. So what do you
> think?
> 
> Christian

Re: Rerunning tests and additional rest tests with mocked persistence?

[Francesco Chicchiriccò <il...@apache.org>]

On 10/12/2012 13:26, Christian Schneider wrote:
> Hi all,
>
> while testing how adding the CXF libs influences the syncope tests I
> found a problem with the current rest tests.
>
> The problem is that at least some of the tests can not simply be rerun.
> For example org.apache.syncope.core.rest.RoleTestITCase.create() only
> seems to succeed once.
> When I start the server and rerun the test from eclipse it fails as the
> role is already there.
>
> So one way to solve this would be to make each test method clean up
> after itself. So at least after a successful test the db should be ready
> for another run.
>
> One other way would be to create some additional tests that work with a
> mocked persistence layer. These tests would then always be able to rerun
> as they do not need or change the db.
>
> I think it makes sense to follow both of these approaches. So what do
> you think?

Hi Christian,
as you have reported above, I wouldn't expect that REST tests are able 
to run more than once, mainly because of different data constraints 
enforced in the underlying database.

Moreover, considering the amount of relationships and different entities 
accessing the persistence layer (Quartz, Activiti, Logback and... 
Syncope via OpenJPA), I would rather prefer some kind of cleanup.

Since the core test environment relies upon an in-memory H2 instance, we 
could find a way to re-initialize (possibly upon request) the Spring web 
context, more or less as already happening with JUnit for unit tests 
(managed via @RunWith(SpringJUnit4ClassRunner.class)). This should fit 
the job.

Regards.

-- 
Francesco Chicchiriccò

ASF Member, Apache Syncope PMC chair, Apache Cocoon PMC Member
http://people.apache.org/~ilgrosso/