You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@syncope.apache.org by Andrei Shakirin <as...@talend.com> on 2013/01/11 11:00:14 UTC
Persistence id generation strategy: TABLE vs AUTO
Hi,
Working on integration tests improvement (1), I discovered that almost all persistence beans have TABLE generated Id strategy defined in orm.xml: user, role, membership, mapping, connInstance, attr.
Anyway AUTO generated Id strategy is defined for following of the persistence beans: AbstractDerAttr, AbstractVirAttr, Notification, UserRequest.
Is there good reason to keep it differently?
Actually I am struggling with strange effect in NotificationTestITCase: create() and issueSYNCOPE83() work successfully ONLY if delete() removes Notification with id=101.
If delete() is marked with @Ignore - create() and issueSYNCOPE83() throw DataIntegrityViolation exception.
Even more strange: if previously running tests: AuthenticationTestITCase, ConfigurationtestITCase, ConnInstanceTestITCase, DerivedSchemaTestITCase are deactivated, problem is not reproducible anymore - create() and issueSYNCOPE83() work successfully without delete().
I guess the problem is caused Id generation, will investigate it a little bit more.
Have anyone other ideas what could be the reason?
Cheers,
Andrei.
(1) https://issues.apache.org/jira/browse/SYNCOPE-268
Re: Persistence id generation strategy: TABLE vs AUTO
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 14/01/2013 09:10, Andrei Shakirin wrote:
> I have digged into the NotificationTestITCase problem a little bit more and discovered the following:
>
> Initial load (context.xml) creates two notifications with id= 100 and 101.
> It seems that AUTO generation strategy, used in Notification now, starts with ID=101 (not sure how to influence on it).
> Therefore, if 101 Notification is not deleted, every create attempt fails with error:
> "Unique index or primary key violation: "PRIMARY_KEY_A ON PUBLIC.NOTIFICATION(ID)"; SQL statement:
> INSERT INTO PUBLIC.Notification (ID, RECIPIENTATTRNAME, RECIPIENTATTRTYPE, SELFASRECIPIENT, SENDER, SUBJECT, TEMPLATE, TRACELEVEL, XMLABOUT, XMLRECIPIENTS) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) [23505-170] {prepstmnt 1643545237 INSERT INTO PUBLIC.Notification (ID, RECIPIENTATTRNAME, RECIPIENTATTRTYPE, SELFASRECIPIENT, SENDER, SUBJECT, TEMPLATE, TRACELEVEL, XMLABOUT, XMLRECIPIENTS) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)} [code=23505, state=23505]"
>
> It makes create() and issueSYNCOPE83() tests depending on delete() test, that deletes Notification with id=101.
Nice catch!
> Suggestion: leave it as it is for CXF migration step (re-runnable delete() will check for notification id=101 and delete it only if it is exists, if not - create and delete other notification).
> After migration change Notification id generation strategy to Table (defined in orm.xml) and start index from 1000. It makes NotificationTestITCase tests more stable and helps to avoid check for 101 in delete(). Fix must be verified for all supported DBs. Corresponded JIRA issue will be created for it.
>
> Does it sounds reasonable?
Table generators have been defined so far when the corresponding table
entity is expected to be of considerable size. This in opposite to AUTO,
i.e. the system sequence generator in OpenJPA (according to [2]).
Since Notification entities are expected to be a small number in a
running Syncope environment, I am actually not particularly enthusiast
of adding a dedicated table generator only for testing purpose.
If there is no better option, your proposed plan sound fine to me.
Alternatively, one can play with openjpa.Sequence [3] (the default
generator mapped to AUTO) and tweak its configuration only for tests,
but I've never tried this before, personally.
Regards.
[2]
http://openjpa.apache.org/builds/2.2.1/apache-openjpa/docs/ref_guide_sequence.html
[3]
http://openjpa.apache.org/builds/2.2.1/apache-openjpa/docs/ref_guide_conf_openjpa.html#openjpa.Sequence
>> -----Original Message-----
>> From: Andrei Shakirin [mailto:ashakirin@talend.com]
>> Sent: Freitag, 11. Januar 2013 12:01
>> To: dev@syncope.apache.org
>> Subject: RE: Persistence id generation strategy: TABLE vs AUTO
>>
>> Hi,
>>
>> Fabio and Francesco: thanks for the fast feedback.
>>
>>>> I cannot remember the good reason for this differences but it there was
>> ...
>>>> In our experience with Apache Syncope (especially at the beginning)
>>>> there
>>> are troubles with AUTO generated id in case of high concurrence.
>>>
>>> Not only: some specific table generator associated to a given entity
>>> (or set of
>>> entities) were defined, and defined in orm.xml, not by annotations -
>>> in order to give more flexibility when dealing with specific
>>> requirements in id generation.
>> I also have not nice experience with AUTO strategy under high concurrency.
>> The question was more in following direction: does it make sense potentially
>> to use TABLE (in orm.xml) also for AbstractDerAttr, AbstractVirAttr,
>> Notification, UserRequest or there is good reason to keep them with AUTO
>> annotation?
>>
>>>> I cannot be sure about the reason but, please, consider that
>>>> integration
>>> tests are interdependent: the order and the existence of a certain
>>> test are often fundamental.
>>>
>>> Exactly, here's why text execution order is fixed.
>>> Consider that integration tests were added over time and that their
>>> number is starting to be quite considerable...
>> Yep, already see it from @FixMethodOrder annotation and from tests logic.
>> I am trying to found more or less reliable solution to keep
>> NotificationITTestCase re-runnable (SYNCOPE-268) - will investigate the
>> problem additionally and inform about result.
>>
>> Cheers,
>> Andrei.
>>
>>> Regards.
>>>
>>>>> Cheers,
>>>>> Andrei.
>>>>>
>>>>>
>>>>> (1) https://issues.apache.org/jira/browse/SYNCOPE-268
--
Francesco Chicchiriccò
ASF Member, Apache Syncope PMC chair, Apache Cocoon PMC Member
http://people.apache.org/~ilgrosso/
RE: Persistence id generation strategy: TABLE vs AUTO
Posted by Andrei Shakirin <as...@talend.com>.
I have digged into the NotificationTestITCase problem a little bit more and discovered the following:
Initial load (context.xml) creates two notifications with id= 100 and 101.
It seems that AUTO generation strategy, used in Notification now, starts with ID=101 (not sure how to influence on it).
Therefore, if 101 Notification is not deleted, every create attempt fails with error:
"Unique index or primary key violation: "PRIMARY_KEY_A ON PUBLIC.NOTIFICATION(ID)"; SQL statement:
INSERT INTO PUBLIC.Notification (ID, RECIPIENTATTRNAME, RECIPIENTATTRTYPE, SELFASRECIPIENT, SENDER, SUBJECT, TEMPLATE, TRACELEVEL, XMLABOUT, XMLRECIPIENTS) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) [23505-170] {prepstmnt 1643545237 INSERT INTO PUBLIC.Notification (ID, RECIPIENTATTRNAME, RECIPIENTATTRTYPE, SELFASRECIPIENT, SENDER, SUBJECT, TEMPLATE, TRACELEVEL, XMLABOUT, XMLRECIPIENTS) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)} [code=23505, state=23505]"
It makes create() and issueSYNCOPE83() tests depending on delete() test, that deletes Notification with id=101.
Suggestion: leave it as it is for CXF migration step (re-runnable delete() will check for notification id=101 and delete it only if it is exists, if not - create and delete other notification).
After migration change Notification id generation strategy to Table (defined in orm.xml) and start index from 1000. It makes NotificationTestITCase tests more stable and helps to avoid check for 101 in delete(). Fix must be verified for all supported DBs. Corresponded JIRA issue will be created for it.
Does it sounds reasonable?
Cheers,
Andrei.
> -----Original Message-----
> From: Andrei Shakirin [mailto:ashakirin@talend.com]
> Sent: Freitag, 11. Januar 2013 12:01
> To: dev@syncope.apache.org
> Subject: RE: Persistence id generation strategy: TABLE vs AUTO
>
> Hi,
>
> Fabio and Francesco: thanks for the fast feedback.
>
> > > I cannot remember the good reason for this differences but it there was
> ...
> > > In our experience with Apache Syncope (especially at the beginning)
> > > there
> > are troubles with AUTO generated id in case of high concurrence.
> >
> > Not only: some specific table generator associated to a given entity
> > (or set of
> > entities) were defined, and defined in orm.xml, not by annotations -
> > in order to give more flexibility when dealing with specific
> > requirements in id generation.
>
> I also have not nice experience with AUTO strategy under high concurrency.
> The question was more in following direction: does it make sense potentially
> to use TABLE (in orm.xml) also for AbstractDerAttr, AbstractVirAttr,
> Notification, UserRequest or there is good reason to keep them with AUTO
> annotation?
>
> > > I cannot be sure about the reason but, please, consider that
> > > integration
> > tests are interdependent: the order and the existence of a certain
> > test are often fundamental.
> >
> > Exactly, here's why text execution order is fixed.
> > Consider that integration tests were added over time and that their
> > number is starting to be quite considerable...
>
> Yep, already see it from @FixMethodOrder annotation and from tests logic.
> I am trying to found more or less reliable solution to keep
> NotificationITTestCase re-runnable (SYNCOPE-268) - will investigate the
> problem additionally and inform about result.
>
> Cheers,
> Andrei.
>
> >
> > Regards.
> >
> > >> Cheers,
> > >> Andrei.
> > >>
> > >>
> > >> (1) https://issues.apache.org/jira/browse/SYNCOPE-268
> >
> > --
> > Francesco Chicchiriccò
> >
> > ASF Member, Apache Syncope PMC chair, Apache Cocoon PMC Member
> > http://people.apache.org/~ilgrosso/
RE: Persistence id generation strategy: TABLE vs AUTO
Posted by Andrei Shakirin <as...@talend.com>.
Hi,
Fabio and Francesco: thanks for the fast feedback.
> > I cannot remember the good reason for this differences but it there was ...
> > In our experience with Apache Syncope (especially at the beginning) there
> are troubles with AUTO generated id in case of high concurrence.
>
> Not only: some specific table generator associated to a given entity (or set of
> entities) were defined, and defined in orm.xml, not by annotations - in order
> to give more flexibility when dealing with specific requirements in id
> generation.
I also have not nice experience with AUTO strategy under high concurrency. The question was more in following direction: does it make sense potentially to use TABLE (in orm.xml) also for AbstractDerAttr, AbstractVirAttr, Notification, UserRequest or there is good reason to keep them with AUTO annotation?
> > I cannot be sure about the reason but, please, consider that integration
> tests are interdependent: the order and the existence of a certain test are
> often fundamental.
>
> Exactly, here's why text execution order is fixed.
> Consider that integration tests were added over time and that their number
> is starting to be quite considerable...
Yep, already see it from @FixMethodOrder annotation and from tests logic.
I am trying to found more or less reliable solution to keep NotificationITTestCase re-runnable (SYNCOPE-268) - will investigate the problem additionally and inform about result.
Cheers,
Andrei.
>
> Regards.
>
> >> Cheers,
> >> Andrei.
> >>
> >>
> >> (1) https://issues.apache.org/jira/browse/SYNCOPE-268
>
> --
> Francesco Chicchiriccò
>
> ASF Member, Apache Syncope PMC chair, Apache Cocoon PMC Member
> http://people.apache.org/~ilgrosso/
Re: Persistence id generation strategy: TABLE vs AUTO
Posted by Francesco Chicchiriccò <il...@apache.org>.
On 11/01/2013 11:21, Fabio Martelli wrote:
> Il giorno 11/gen/2013, alle ore 11.00, Andrei Shakirin ha scritto:
>
>> Hi,
>>
>> Working on integration tests improvement (1), I discovered that almost all persistence beans have TABLE generated Id strategy defined in orm.xml: user, role, membership, mapping, connInstance, attr.
>> Anyway AUTO generated Id strategy is defined for following of the persistence beans: AbstractDerAttr, AbstractVirAttr, Notification, UserRequest.
>> Is there good reason to keep it differently?
> Hi Andrei,
> I cannot remember the good reason for this differences but it there was ...
> In our experience with Apache Syncope (especially at the beginning) there are troubles with AUTO generated id in case of high concurrence.
Not only: some specific table generator associated to a given entity (or
set of entities) were defined, and defined in orm.xml, not by
annotations - in order to give more flexibility when dealing with
specific requirements in id generation.
> From my PPOV you can try to change this implementation but you have to verify your fix with all the supported databases generating a lot of concurrent accesses to the entities interested by your changes.
>> Actually I am struggling with strange effect in NotificationTestITCase: create() and issueSYNCOPE83() work successfully ONLY if delete() removes Notification with id=101.
>> If delete() is marked with @Ignore - create() and issueSYNCOPE83() throw DataIntegrityViolation exception.
>> Even more strange: if previously running tests: AuthenticationTestITCase, ConfigurationtestITCase, ConnInstanceTestITCase, DerivedSchemaTestITCase are deactivated, problem is not reproducible anymore - create() and issueSYNCOPE83() work successfully without delete().
>> I guess the problem is caused Id generation, will investigate it a little bit more.
>>
>> Have anyone other ideas what could be the reason?
> I cannot be sure about the reason but, please, consider that integration tests are interdependent: the order and the existence of a certain test are often fundamental.
Exactly, here's why text execution order is fixed.
Consider that integration tests were added over time and that their
number is starting to be quite considerable...
Regards.
>> Cheers,
>> Andrei.
>>
>>
>> (1) https://issues.apache.org/jira/browse/SYNCOPE-268
--
Francesco Chicchiriccò
ASF Member, Apache Syncope PMC chair, Apache Cocoon PMC Member
http://people.apache.org/~ilgrosso/
Re: Persistence id generation strategy: TABLE vs AUTO
Posted by Fabio Martelli <fa...@gmail.com>.
Il giorno 11/gen/2013, alle ore 11.00, Andrei Shakirin ha scritto:
> Hi,
>
> Working on integration tests improvement (1), I discovered that almost all persistence beans have TABLE generated Id strategy defined in orm.xml: user, role, membership, mapping, connInstance, attr.
> Anyway AUTO generated Id strategy is defined for following of the persistence beans: AbstractDerAttr, AbstractVirAttr, Notification, UserRequest.
> Is there good reason to keep it differently?
Hi Andrei,
I cannot remember the good reason for this differences but it there was ...
In our experience with Apache Syncope (especially at the beginning) there are troubles with AUTO generated id in case of high concurrence.
From my PPOV you can try to change this implementation but you have to verify your fix with all the supported databases generating a lot of concurrent accesses to the entities interested by your changes.
> Actually I am struggling with strange effect in NotificationTestITCase: create() and issueSYNCOPE83() work successfully ONLY if delete() removes Notification with id=101.
> If delete() is marked with @Ignore - create() and issueSYNCOPE83() throw DataIntegrityViolation exception.
> Even more strange: if previously running tests: AuthenticationTestITCase, ConfigurationtestITCase, ConnInstanceTestITCase, DerivedSchemaTestITCase are deactivated, problem is not reproducible anymore - create() and issueSYNCOPE83() work successfully without delete().
> I guess the problem is caused Id generation, will investigate it a little bit more.
>
> Have anyone other ideas what could be the reason?
I cannot be sure about the reason but, please, consider that integration tests are interdependent: the order and the existence of a certain test are often fundamental.
Cheers,
F.
>
> Cheers,
> Andrei.
>
>
> (1) https://issues.apache.org/jira/browse/SYNCOPE-268