You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openjpa.apache.org by pankajpriyadarshi <pa...@gmail.com> on 2012/10/30 03:19:38 UTC
Savepoint issue with OpenJPA
Hello, I am using the Savepoint feature of OpenJPA (setSavepoint and
rollbackToSavepoint). When I do a rollbackToSavepoint the order of the
TransactionalObjects becomes random. E.g. If the rollbackToSavepoint call is
not made, the insertions are made in the order of the operation: 1) Insert
Record 1 into ParentTable 2) Insert Record 1.1 into ChildTable 3) Insert
Record 1.2 into ChildTable 4) Insert Record 1.3 into ChildTable 5) Insert
Record 2 into ParentTable 6) Insert Record 2.1 into ChildTable 7) Insert
Record 2.2 into ChildTable 8) Insert Record 2.3 into ChildTable If the
rollbackToSavepoint call is made, the insertions order becomes random and I
am getting foreign key constraint violations. I have tried to set
openjpa.jdbc.UpdateManager to operation order but it has no affect. Is there
any way I can preserve the original order of the operation. Why would
OpenJPA try to reorder the transactional objects when doing a
rollbackToSavepoint . Has someone faced this issue before. OpenJPA version
info:revision.number=422266:1141200 openjpa.version=2.1.1-SNAPSHOT
openjpa.enhancer.revision=1055128 Thank You, Pankaj Priyadarshi
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
Thank You. This works.
However don't you think that the default behavior for OpenJPA should be to
honour the foreign key constraints even if the openjpa.jdbc.MappingDefaults
is not overriden
I thought OpenJPA did a topological sort to reorder the SQLs so that the
database foreign key constraints are not violated.
Again thank you for the solution. However I think that this should be the
default behavior without the application having to override the
openjpa.jdbc.MappingDefaults
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582897.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by Pinaki Poddar <pp...@apache.org>.
Hi,
We have looked at the test case you have submitted with this issue. Few
comments
1. The test verifies that SQLs are issued in a particular order. As I had
mentioned before, OpenJPA does not guarantee that. What it ensures that
foreign key constraints are honored when inserting or updating database
records.
2. The test case will function (in terms of intended behavior, not in an
intended SQL execution order) if in persistence.xml, the following mapping
defaults are set
<property name="openjpa.jdbc.MappingDefaults"
value="ForeignKeyDeleteAction=restrict,
JoinForeignKeyDeleteAction=none"/>
This will add foreign key constraints that are commensurate with the
extra DDL script that you have supplied.
-----
Pinaki Poddar
Chair, Apache OpenJPA Project
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582881.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
Is there someone working on this issue? I am not sure what the timeline is
for the JIRA issues to be resolved but an acknowledgement that this is an
issue would be nice.
I have already attached the test cases to illustrate the issue. In case more
information is required, I would be more than glad to provide that
information.
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582857.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
I tried it but it did not work. These test cases are specific to using the
rollbackToSavepoint feature. Thanks.
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582656.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by Michael Dick <mi...@gmail.com>.
Have you tried setting openjpa.jdbc.SchemaFactory=native in your
persistence.xml or as a system property? Using the native schema factory
will tell OpenJPA to read the table definitions from the database.
I haven't had a chance to try the testcases, but if it is an SQL ordering
problem, the schema factory will usually help.
-mike
On Tue, Jan 29, 2013 at 2:14 PM, pankajpriyadarshi <pa...@gmail.com>wrote:
> Any updates yet. Do the test cases correctly depict the problem? Thank
> You.
>
>
>
> --
> View this message in context:
> http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582652.html
> Sent from the OpenJPA Developers mailing list archive at Nabble.com.
>
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
Any updates yet. Do the test cases correctly depict the problem? Thank You.
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582652.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
Any updates on this issue? Thanks.
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582461.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
Hello Pinaki,
I have 2 new files to the JIRA issue: a) the latest jar file
along with b) a small SQL script to be run on
openjpa-persistence-jdbc\target\database\openjpa-derby-database Derby
database. This script will add foreign constraints to the tables.
With this latest jar you can see the testInsertWithoutSavepoint passing but
testInsertWithSavepoint failing when doing a commit. I also tried the
<property name="openjpa.jdbc.SynchronizeMappings"
value="buildSchema(ForeignKeys=true)"/> but it makes no difference to this
issue.
Please let me know if you need anything else from me. This is the URL to the
JIRA issue where I have added the 2 new files:
https://issues.apache.org/jira/browse/OPENJPA-2291?page=com.atlassian.jirafisheyeplugin:fisheye-issuepanel#comment-13538436.
The exception that I got was:
Caused by: <openjpa-2.3.0-SNAPSHOT-runknown fatal general error>
org.apache.openjpa.persistence.PersistenceException: INSERT on table
'PER_JDBC_KERN_STORY' caused a violation of foreign key constraint
'SQL121227131511400' for key (25). The statement has been rolled back.
{prepstmnt 1898869038 INSERT INTO PER_JDBC_KERN_STORY (ID, TASK_ID, VERSION)
VALUES (?, ?, ?) [params=?, ?, ?]} [code=20000, state=23503]
FailedObject: org.apache.openjpa.jdbc.kernel.Story1@73607360
at org.apache.openjpa.jdbc.sql.DBDictionary.narrow(DBDictionary.java:4940)
at
org.apache.openjpa.jdbc.sql.DBDictionary.newStoreException(DBDictionary.java:4906)
at
org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:137)
at
org.apache.openjpa.jdbc.sql.SQLExceptions.getStore(SQLExceptions.java:78)
at
org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:144)
at
org.apache.openjpa.jdbc.kernel.BatchingPreparedStatementManagerImpl.flushAndUpdate(BatchingPreparedStatementManagerImpl.java:79)
at
com.ibm.ws.persistence.jdbc.kernel.PreparedStatementManagerImpl.flushAndUpdate(PreparedStatementManagerImpl.java:63)
at
org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flushInternal(PreparedStatementManagerImpl.java:100)
at
org.apache.openjpa.jdbc.kernel.PreparedStatementManagerImpl.flush(PreparedStatementManagerImpl.java:88)
at
org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:550)
at
org.apache.openjpa.jdbc.kernel.ConstraintUpdateManager.flush(ConstraintUpdateManager.java:106)
at
org.apache.openjpa.jdbc.kernel.BatchingConstraintUpdateManager.flush(BatchingConstraintUpdateManager.java:59)
at
org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:105)
at
org.apache.openjpa.jdbc.kernel.AbstractUpdateManager.flush(AbstractUpdateManager.java:78)
at
org.apache.openjpa.jdbc.kernel.JDBCStoreManager.flush(JDBCStoreManager.java:735)
at
org.apache.openjpa.kernel.DelegatingStoreManager.flush(DelegatingStoreManager.java:131)
Thank You,
Pankaj Priyadarshi
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582368.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
I am not sure if <property name="openjpa.jdbc.SynchronizeMappings"
value="buildSchema(ForeignKeys=true)"/> is a viable option for me. I would
need to understand its implications before using it within my application
> the topological sort is not being done based on foreign key constraints as
> you would normally expect.
> what is the basis of this statement? The savepoint feature is sparingly
> used, so I am not certain, but I would presume that whether OpenJPA
> reorders the SQL statements during flush/commit has nothing to do with
> whether savepoint feature has been used or not.
This is based on the commit failure we are getting with tables which have
foreign key constraints enforced in our application. In any case when
savepoint rollback is used, the broker copies transactional objects from one
map to another while trying to exclude the rolledback savepoint
transactional objects. This is not the normal behavior when not using
savepoints. I am not sure whether this is the issue ,but differences like
these might be making it work when not using savepoints.
I guess at this point, the only way to move forward with this is to identify
entities in OpenJPA source project that have foreign key constraints
defined. This might take a few days but I will modify the test case after
identifying them.
The reason I am so certain that this is a bug is based on the fact that it
is not working in a simple test case within our application.
As always thank you for your quick response. I will identify the entities
when I am back in office on Wednesday and respond with further updates.
Thank You,
Pankaj Priyadarshi
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582359.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by Pinaki Poddar <pp...@apache.org>.
> of course the commit would have failed. The reason would be that the child
table insert is fired first before the parent table insert.
Not necessarily correct.
As I had mentioned, OpenJPA is capable of ordering the SQL statements such
that the existing foreign key constraints are honored. This facility
requires that OpenJPA is instructed to read existing foreign key constraints
from the database schema. A common way is to specify
<property name="openjpa.jdbc.SynchronizeMappings"
value="buildSchema(ForeignKeys=true)"/>
If you observe that OpenJPA fails to recorder the SQL despite such a
setting, then we will consider that to be a defect.
> If you can let me know entities in the OpenJPA source code that have
> foreign constraints enforced
OpenJPA has 4000+ test cases and some of them, I am sure, uses foreign keys.
> the topological sort is not being done based on foreign key constraints as
> you would normally expect.
what is the basis of this statement? The savepoint feature is sparingly
used, so I am not certain, but I would presume that whether OpenJPA reorders
the SQL statements during flush/commit has nothing to do with whether
savepoint feature has been used or not.
-----
Pinaki Poddar
Chair, Apache OpenJPA Project
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582358.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
Pinaki,
Our application transaction does fail and a commit would
not be succesful with savepoint. In the test case I wrote, the entities used
do not have a foreign key constraint defined in the database. If there was a
foreign key constraint between the entities, off course the commit would
have failed. The reason would be that the child table insert is fired first
before the parent table insert.
I am not aware of entities in the OpenJPA source code that have foreign key
constraints defined, that is the reason I chose Employee, Task and Story
tables. As I mentioned if there were foreign key constraints between the
Employee, Task and Story tables, the commit would have failed. Also as I
mentioned in my original post, our tables are well defined relational tables
and have foreign constraints enforced (as are most mature table designs), so
this bug would cause commit failures.
My idea of writing the test cases was to show that the insert order has
changed and hence would cause foreign constraint violations. If you can let
me know entities in the OpenJPA source code that have foreign constraints
enforced, I could change the test case to show that the commit would fail.
Thank You.
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582357.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by Pinaki Poddar <pp...@apache.org>.
Hi,
The test case as presented tries to verify a more stringent condition than
the original report. According to the original report, while using savepoint
rollback, a commit failed, whereas the commit would succeed without a
savepoint rollback.
However, the attached test case is verifying whether the SQLs are issued in
order, not whether the commit succeeds.
I had verified that the commit does succeed in both cases, though the order
of the SQL gets modified by savepoint rollback.
While a commit failure can be termed as a real bug, I am not sure whether
the same can be said if the SQL ordering is not maintained.
In any case, OpenJPA uses several internal collections for the inflight
managed/transactional objects and not all of them warrant a stable iteration
order. Of course, with savepoint rollback, maintaining the order becomes
more difficult. So, with a brief analysis of the issue and given the
sporadic usage of savepoint as a feature, we do not foresee at this point to
force these internal collections to maintain a consistent
insertion/iteration order.
So as long as your application transaction does not fail under savepoint
rollback, I will advise not to expect that OpenJPA will issue the SQLs in
the same order. As you are aware, OpenJPA runs a topological sort to reorder
the SQLs so that the database foreign key constraints are not violated. That
algorithm so far has been proven robust within reason, but if the SQL
reordering by OpenJPA breaks your commit when you introduce a savepoint
rollback, then that is a concern and we will consider fixing it (but the
resolution will be difficult as far as I can see).
-----
Pinaki Poddar
Chair, Apache OpenJPA Project
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582356.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
I have added the test case as a jar file to the JIRA issue I created. This is
the link to the JIRA issue:
https://issues.apache.org/jira/browse/OPENJPA-2291?page=com.atlassian.jirafisheyeplugin:fisheye-issuepanel#comment-13538436
Please let me know if you need anything else from my side.
Thank You,
Pankaj Priyadarshi
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582355.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by Pinaki Poddar <pp...@apache.org>.
We do not have any immediate plan to investigate the issue. If we see a test
case, it perhaps will help its relative importance.
-----
Pinaki Poddar
Chair, Apache OpenJPA Project
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582352.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
Sorry, not yet. We will have it soon. Do you want to go ahead with the issue
in the meantime ?
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582347.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by Pinaki Poddar <pp...@apache.org>.
did you submit a test case?
-----
Pinaki Poddar
Chair, Apache OpenJPA Project
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582346.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
Any updates on this issue. Do we have a patch available?
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7582345.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
Thanks Pinaki. I will send you the test cases as per the OpenJPA guidelines
as soon as possible.
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7581600.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by Pinaki Poddar <pp...@apache.org>.
Hi,
> Do you want me to share those test cases or is it a simple test on your
side to investigate?
We prefer Junit test cases from the reporter i.e. you.
Most (but not all) test cases in OpenJPA are written using the following
guideline [1]. But you are free to write a standalone case deriving from
base Junit TestCase.
But please use assert*() rather than println() for us to know what
exactly the test is expecting.
[1] http://openjpa.apache.org/writing-test-cases-for-openjpa.html
-----
Pinaki Poddar
Chair, Apache OpenJPA Project
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7581582.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
Pinaki, Also with respect to your comments, I wanted to clarify that my issue
is with the transactional objects and not managed objects. In case I have
mutiple tx being opened and closed on the same EM, my transactional objects
would be a subset of my managed objects in the EM (which is the case in my
application where we are using a batch framework with checkpoint restart
capability)
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7581563.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by pankajpriyadarshi <pa...@gmail.com>.
Thanks Pinaki. Do you want me to open a JIRA for this? I have some test cases
written that show this issue. Do you want me to share those test cases or is
it a simple test on your side to investigate?
Thank You,
Pankaj Priyadarshi
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7581559.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by Pinaki Poddar <pp...@apache.org>.
Savepoint is not a commonly used feature but it may be relevant to mention
that OpenJPA Audit facility is based on Savepoint support.
The reported case where using savepoint changes the ordering of the managed
objects needs to be investigated.
-----
Pinaki Poddar
Chair, Apache OpenJPA Project
--
View this message in context: http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548p7581557.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by Kevin Sutter <kw...@gmail.com>.
Hi,
I'm going to cross-post to the Users forum since you may find some users of
the Savepoint feature out there... Savepoints are not what I would call a
"mainline" feature. Yes, they have been part of the OpenJPA project since
day one, but I have not heard of too many actual customer scenarios using
this feature. Hopefully, you'll get some hits from the Users forum...
Good luck!
Kevin
On Mon, Oct 29, 2012 at 9:19 PM, pankajpriyadarshi <pa...@gmail.com>wrote:
> Hello, I am using the Savepoint feature of OpenJPA (setSavepoint and
> rollbackToSavepoint). When I do a rollbackToSavepoint the order of the
> TransactionalObjects becomes random. E.g. If the rollbackToSavepoint call
> is
> not made, the insertions are made in the order of the operation: 1) Insert
> Record 1 into ParentTable 2) Insert Record 1.1 into ChildTable 3) Insert
> Record 1.2 into ChildTable 4) Insert Record 1.3 into ChildTable 5) Insert
> Record 2 into ParentTable 6) Insert Record 2.1 into ChildTable 7) Insert
> Record 2.2 into ChildTable 8) Insert Record 2.3 into ChildTable If the
> rollbackToSavepoint call is made, the insertions order becomes random and I
> am getting foreign key constraint violations. I have tried to set
> openjpa.jdbc.UpdateManager to operation order but it has no affect. Is
> there
> any way I can preserve the original order of the operation. Why would
> OpenJPA try to reorder the transactional objects when doing a
> rollbackToSavepoint . Has someone faced this issue before. OpenJPA version
> info:revision.number=422266:1141200 openjpa.version=2.1.1-SNAPSHOT
> openjpa.enhancer.revision=1055128 Thank You, Pankaj Priyadarshi
>
>
>
> --
> View this message in context:
> http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548.html
> Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Savepoint issue with OpenJPA
Posted by Kevin Sutter <kw...@gmail.com>.
Hi,
I'm going to cross-post to the Users forum since you may find some users of
the Savepoint feature out there... Savepoints are not what I would call a
"mainline" feature. Yes, they have been part of the OpenJPA project since
day one, but I have not heard of too many actual customer scenarios using
this feature. Hopefully, you'll get some hits from the Users forum...
Good luck!
Kevin
On Mon, Oct 29, 2012 at 9:19 PM, pankajpriyadarshi <pa...@gmail.com>wrote:
> Hello, I am using the Savepoint feature of OpenJPA (setSavepoint and
> rollbackToSavepoint). When I do a rollbackToSavepoint the order of the
> TransactionalObjects becomes random. E.g. If the rollbackToSavepoint call
> is
> not made, the insertions are made in the order of the operation: 1) Insert
> Record 1 into ParentTable 2) Insert Record 1.1 into ChildTable 3) Insert
> Record 1.2 into ChildTable 4) Insert Record 1.3 into ChildTable 5) Insert
> Record 2 into ParentTable 6) Insert Record 2.1 into ChildTable 7) Insert
> Record 2.2 into ChildTable 8) Insert Record 2.3 into ChildTable If the
> rollbackToSavepoint call is made, the insertions order becomes random and I
> am getting foreign key constraint violations. I have tried to set
> openjpa.jdbc.UpdateManager to operation order but it has no affect. Is
> there
> any way I can preserve the original order of the operation. Why would
> OpenJPA try to reorder the transactional objects when doing a
> rollbackToSavepoint . Has someone faced this issue before. OpenJPA version
> info:revision.number=422266:1141200 openjpa.version=2.1.1-SNAPSHOT
> openjpa.enhancer.revision=1055128 Thank You, Pankaj Priyadarshi
>
>
>
> --
> View this message in context:
> http://openjpa.208410.n2.nabble.com/Savepoint-issue-with-OpenJPA-tp7581548.html
> Sent from the OpenJPA Developers mailing list archive at Nabble.com.