You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@syncope.apache.org by Andrei Shakirin <as...@talend.com> on 2013/02/02 10:10:50 UTC
[DISCUSSION] Integration tests instability causes by Ids collision
(AUTO generation strategy)
Hi,
Analyses shows, that real reason of failed/unstable tests is not AUTO generation strategy itself as reported in [1], but collision between newly created and pre-configured entries.
Actually following entities have AUTO generation strategy:
- AbstractDerAttr
- AbstractVirAttr
- Notification
- UserRequest
- UAttr
I propose tree possible ways to fix the problem:
A) Increase Id of corresponded pre-configured entities to large number (>1000) to make collision non probable.
B) Avoid pre-configured objects for all entities with AUTO generation strategy. Redesign tests to create necessary data on the fly
C) Change AUTO generation strategy to TABLE one.
(A) is easiest, but looks like workaround for me. (B) sounds as the most reasonable. (C) is possible, but requires change in persistence strategy just because of tests (not the best practice)
WDYT?
Regards,
Andrei.
[1] https://issues.apache.org/jira/browse/SYNCOPE-298
Re: [DISCUSSION] Integration tests instability causes by Ids collision
(AUTO generation strategy)
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 02/02/2013 11:58, Andrei Shakirin wrote:
> Hi Francesco,
>
>> I agree with you that (A) is easy and sounds like a workaround but...
>> we're talking about test data, hence workarounds don't smell like as usual :-)
>> IMO, relying upon some default test data is actually very comfortable, so I
>> wouldn't opt for (B).
> Hmm ... for sure quality of tests data should not be on the same level as product/project data.
> But to be honest I am a little bit scared about side effects in integration tests - sometimes problems were caused by patches that not related with problem at all - very difficult to analyse and debug.
> I generally will vote to make integration tests more isolated, independent and reliable. Of course it will not happens in one day.
> So my +1 still for option (B) if it is doesn't require huge efforts (we should do it only for 5 entities, not for all).
Andrei, there is an additional point I haven't reported so far to
justify my preference for (A), e.g. embedded mode / standalone distribution.
These are based on test data, so generating some of these only during
actual test execution would result in some poorer user experience for
users approaching Syncope for the first times. In this direction it is
also to consider SYNCOPE-265 that will improve the quality of test data
for such purpose.
On the other side, maintaining two separate data sets (for tests and for
embedded / standalone) would be a considerable effort that we can save.
Regards.
>> I agree with you about (C) : would this mean that SYNCOPE-298 could be
>> marked as invalid at the end of the current discussion?
> Yep. I will close it.
>
> Cheers,
> Andrei.
>
>> -----Original Message-----
>> From: Francesco Chicchiriccò [mailto:ilgrosso@apache.org]
>> Sent: Samstag, 2. Februar 2013 11:26
>> To: dev@syncope.apache.org
>> Subject: Re: [DISCUSSION] Integration tests instability causes by Ids collision
>> (AUTO generation strategy)
>>
>> On 02/02/2013 10:10, Andrei Shakirin wrote:
>>> Hi,
>>>
>>> Analyses shows, that real reason of failed/unstable tests is not AUTO
>> generation strategy itself as reported in [1], but collision between newly
>> created and pre-configured entries.
>>> Actually following entities have AUTO generation strategy:
>>> - AbstractDerAttr
>>> - AbstractVirAttr
>>> - Notification
>>> - UserRequest
>>> - UAttr
>>>
>>> I propose tree possible ways to fix the problem:
>>> A) Increase Id of corresponded pre-configured entities to large number
>> (>1000) to make collision non probable.
>>> B) Avoid pre-configured objects for all entities with AUTO generation
>>> strategy. Redesign tests to create necessary data on the fly
>>> C) Change AUTO generation strategy to TABLE one.
>>>
>>> (A) is easiest, but looks like workaround for me. (B) sounds as the
>>> most reasonable. (C) is possible, but requires change in persistence
>>> strategy just because of tests (not the best practice)
>> Hi Andrei,
>> first of all, thanks for caring about this!
>>
>> I agree with you that (A) is easy and sounds like a workaround but...
>> we're talking about test data, hence workarounds don't smell like as usual :-)
>>
>> IMO, relying upon some default test data is actually very comfortable, so I
>> wouldn't opt for (B).
>>
>> I agree with you about (C) : would this mean that SYNCOPE-298 could be
>> marked as invalid at the end of the current discussion?
>>
>> Regards.
>>
>>> [1] https://issues.apache.org/jira/browse/SYNCOPE-298
--
Francesco Chicchiriccò
ASF Member, Apache Syncope PMC chair, Apache Cocoon PMC Member
http://people.apache.org/~ilgrosso/
RE: [DISCUSSION] Integration tests instability causes by Ids
collision (AUTO generation strategy)
Posted by Andrei Shakirin <as...@talend.com>.
Hi Francesco,
> I agree with you that (A) is easy and sounds like a workaround but...
> we're talking about test data, hence workarounds don't smell like as usual :-)
> IMO, relying upon some default test data is actually very comfortable, so I
> wouldn't opt for (B).
Hmm ... for sure quality of tests data should not be on the same level as product/project data.
But to be honest I am a little bit scared about side effects in integration tests - sometimes problems were caused by patches that not related with problem at all - very difficult to analyse and debug.
I generally will vote to make integration tests more isolated, independent and reliable. Of course it will not happens in one day.
So my +1 still for option (B) if it is doesn't require huge efforts (we should do it only for 5 entities, not for all).
> I agree with you about (C) : would this mean that SYNCOPE-298 could be
> marked as invalid at the end of the current discussion?
Yep. I will close it.
Cheers,
Andrei.
> -----Original Message-----
> From: Francesco Chicchiriccò [mailto:ilgrosso@apache.org]
> Sent: Samstag, 2. Februar 2013 11:26
> To: dev@syncope.apache.org
> Subject: Re: [DISCUSSION] Integration tests instability causes by Ids collision
> (AUTO generation strategy)
>
> On 02/02/2013 10:10, Andrei Shakirin wrote:
> > Hi,
> >
> > Analyses shows, that real reason of failed/unstable tests is not AUTO
> generation strategy itself as reported in [1], but collision between newly
> created and pre-configured entries.
> > Actually following entities have AUTO generation strategy:
> > - AbstractDerAttr
> > - AbstractVirAttr
> > - Notification
> > - UserRequest
> > - UAttr
> >
> > I propose tree possible ways to fix the problem:
> > A) Increase Id of corresponded pre-configured entities to large number
> (>1000) to make collision non probable.
> > B) Avoid pre-configured objects for all entities with AUTO generation
> > strategy. Redesign tests to create necessary data on the fly
> > C) Change AUTO generation strategy to TABLE one.
> >
> > (A) is easiest, but looks like workaround for me. (B) sounds as the
> > most reasonable. (C) is possible, but requires change in persistence
> > strategy just because of tests (not the best practice)
>
> Hi Andrei,
> first of all, thanks for caring about this!
>
> I agree with you that (A) is easy and sounds like a workaround but...
> we're talking about test data, hence workarounds don't smell like as usual :-)
>
> IMO, relying upon some default test data is actually very comfortable, so I
> wouldn't opt for (B).
>
> I agree with you about (C) : would this mean that SYNCOPE-298 could be
> marked as invalid at the end of the current discussion?
>
> Regards.
>
> > [1] https://issues.apache.org/jira/browse/SYNCOPE-298
>
> --
> Francesco Chicchiriccò
>
> ASF Member, Apache Syncope PMC chair, Apache Cocoon PMC Member
> http://people.apache.org/~ilgrosso/
Re: [DISCUSSION] Integration tests instability causes by Ids collision
(AUTO generation strategy)
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 02/02/2013 10:10, Andrei Shakirin wrote:
> Hi,
>
> Analyses shows, that real reason of failed/unstable tests is not AUTO generation strategy itself as reported in [1], but collision between newly created and pre-configured entries.
> Actually following entities have AUTO generation strategy:
> - AbstractDerAttr
> - AbstractVirAttr
> - Notification
> - UserRequest
> - UAttr
>
> I propose tree possible ways to fix the problem:
> A) Increase Id of corresponded pre-configured entities to large number (>1000) to make collision non probable.
> B) Avoid pre-configured objects for all entities with AUTO generation strategy. Redesign tests to create necessary data on the fly
> C) Change AUTO generation strategy to TABLE one.
>
> (A) is easiest, but looks like workaround for me. (B) sounds as the most reasonable. (C) is possible, but requires change in persistence strategy just because of tests (not the best practice)
Hi Andrei,
first of all, thanks for caring about this!
I agree with you that (A) is easy and sounds like a workaround but...
we're talking about test data, hence workarounds don't smell like as
usual :-)
IMO, relying upon some default test data is actually very comfortable,
so I wouldn't opt for (B).
I agree with you about (C) : would this mean that SYNCOPE-298 could be
marked as invalid at the end of the current discussion?
Regards.
> [1] https://issues.apache.org/jira/browse/SYNCOPE-298
--
Francesco Chicchiriccò
ASF Member, Apache Syncope PMC chair, Apache Cocoon PMC Member
http://people.apache.org/~ilgrosso/