You are viewing a plain text version of this content. The canonical link for it is here.
Posted to jdo-dev@db.apache.org by "Marco (JIRA)" <ji...@apache.org> on 2008/11/27 13:40:44 UTC
[jira] Created: (JDO-620) datastore identifier needed for improving
replication of objects
datastore identifier needed for improving replication of objects
----------------------------------------------------------------
Key: JDO-620
URL: https://issues.apache.org/jira/browse/JDO-620
Project: JDO
Issue Type: New Feature
Reporter: Marco
Fix For: JDO 2 maintenance release 3
We are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
1) Detach an object graph from datastore A (using pm.detachCopy(...)).
2) Send the object graph to the destination (using Java native serialisation).
3) Mark the complete object graph dirty (recursively).
4) Attach the object graph to datastore B (using pm.makePersistent(...)).
Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
* Throw an exception and do not startup the PMF.
* Copy the value from the persistence.xml into the datastore.
* Use the value from the persistence.xml without writing it into the datastore.
* Use the value from the datastore and ignore the setting in the persistence.xml.
Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
String getDatastoreIdentifier()
Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
String getDatastoreIdentifier(Object persistenceCapableObject)
When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (JDO-620) datastore identifier needed
for improving replication of objects
Posted by "Jörg von Frantzius (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651553#action_12651553 ]
jfrantzius edited comment on JDO-620 at 11/28/08 5:19 AM:
------------------------------------------------------------------
If the object got modified once on the target DB before replicating, then the object already may have the version number that will be replicated from remote (following your approach).
Now if the object gets modified concurrently on the target DB while you attach your replicated version, the modifying transaction won't detect that the object got modified by replication in the meantime.
I think that's exactly only happening in the case of a conflict, i.e. when the object got modified independently on both systems.
was (Author: jfrantzius):
In the general case, the object might get modified concurrently while you attach your replicated version. You may per chance end up with the same version number for those two distinct modifications while you shouldn't.
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JDO-620) datastore identifier needed for
improving replication of objects
Posted by "Marco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651547#action_12651547 ]
Marco commented on JDO-620:
---------------------------
Sorry, I don't understand why the proposed change would affect optimistic transactions. When an object is modified (i.e. a field becomes dirty), the version is incremented - this is now the case and would still be the case after our proposed change. The only thing that changes is that an object's version is not incremented by mere replication anymore.
So could you please explain your worries about optimistic transactions? Maybe I'm overlooking sth.
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JDO-620) datastore identifier needed for
improving replication of objects
Posted by "Marco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651544#action_12651544 ]
Marco commented on JDO-620:
---------------------------
> If it is not increased during attachment, you may break optimistic verification on the target DB.
Just a note: The version *is* incremented, because the object was modified on the primary datastore, before it is replicated. Otherwise there would be no replication event happening.
In our current scenario, all objects belong to an organisation and are only modified there - the replicated objects are not modifed directly.
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JDO-620) datastore identifier needed for
improving replication of objects
Posted by "Jörg von Frantzius (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651523#action_12651523 ]
Jörg von Frantzius commented on JDO-620:
----------------------------------------
Hi Marko,
the first thing I wonder about is: why do you want to have the same version values after replication? AFAIK, the version number is not a user-accessible value, and its only purpose is for optimistic verification.
If it is not increased during attachment, you may break optimistic verification on the target DB.
Out of curiosity, do you use databases with deferred constraints checking?
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JDO-620) datastore identifier needed for
improving replication of objects
Posted by "Marco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656179#action_12656179 ]
Marco commented on JDO-620:
---------------------------
Jörg, I thought quite a while about it and - unfortunately - was busy with other projects, thus the delay. Well, I think it is indeed necessary to serve both use cases. Especially, I think it's very likely that we will earlier or later face the situation that multiple organisations (=datastores) can modify the same object. That's why I think in combination with the datastore-identifier, we need a new versioning strategy: A version that combines the datastore idenfier of the last change AND the (long or timestamp - i.e. maybe 2 versioning strategies?!) version assigned by this datastore.
This way, a change being replicated preserves the version number (no increment during replication) and at the same time changes in multiple datastores are supported.
However, I still see a problems in this: Let's assume that an object has been replicated from datastore A to datastore B and has changed in datastore B. Thus, the versions are now: A.111 and B.2. Now, a new replication happens from A to B thus changing the version in B to be A.111, too. If we now modify the object in B again, it should be B.3, but the current version is A.111 - how do we know that the version was B.2 the last time that we modified it in B?
Hmmm... This seems to require some more thoughts. Anyone having any useful ideas?
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JDO-620) datastore identifier needed for
improving replication of objects
Posted by "Marco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12656182#action_12656182 ]
Marco commented on JDO-620:
---------------------------
...just to clarify: Even though we don't use optimistic transactions, we use the version in the client-sided cache and thus the problem you described for optimistic transactions might affect our cache invalidation strategy, too. Some way to incorporate the datastore identifier into the version would at least solve our cache issues (don't know for sure whether it would help with the optimistic transactions - but I think so).
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JDO-620) datastore identifier needed for improving
replication of objects
Posted by "Andy Jefferson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Jefferson updated JDO-620:
-------------------------------
Fix Version/s: (was: JDO 2 maintenance release 3)
Moving out of scope for 2.3 since not enough background scoping done on this change and would be nice to get JDO2.3 out. Add more thoughts when you have time so then it could be early JDO2.4
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JDO-620) datastore identifier needed for
improving replication of objects
Posted by "Marco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651545#action_12651545 ]
Marco commented on JDO-620:
---------------------------
Why shouldn't the object's version be used? And why should it be incremented when there was in fact no change? Replicating an object without modifying it is IMHO no reason for incrementing a version.
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JDO-620) datastore identifier needed for
improving replication of objects
Posted by "Jörg von Frantzius (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651553#action_12651553 ]
Jörg von Frantzius commented on JDO-620:
----------------------------------------
In the general case, the object might get modified concurrently while you attach your replicated version. You may per chance end up with the same version number for those two distinct modifications while you shouldn't.
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JDO-620) datastore identifier needed for
improving replication of objects
Posted by "Marco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651541#action_12651541 ]
Marco commented on JDO-620:
---------------------------
...uuups - was doing too many things in parallel and forgot to finish that sentence ;-)
...all transaction boundaries are within EJB methods. We do use XA transactions, thus nested EJB methods (especially when involving multiple organisations spread over multiple JavaEE servers) might be integrated into an already running transaction. But still these are pessimistic transactions (they're short-running, thus it's OK).
We use transaction isolation read committed with SELECT ... FOR UPDATE for objects that are about to be modified in the transaction. If it's not sure whether the objects were already locked (in a method deeper down in the call stack), we use pm.refreshAll(...) to ensure that (SELECT ... FOR UPDATE is executed on all of the objects in the collection passed to pm.refreshAll(...)).
Best regards, Marco :-)
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (JDO-620) datastore identifier needed
for improving replication of objects
Posted by "Jörg von Frantzius (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651553#action_12651553 ]
jfrantzius edited comment on JDO-620 at 11/28/08 5:20 AM:
------------------------------------------------------------------
If the object got modified once on the target DB before replicating, then the object already may have the version number that will be replicated from remote (following your approach).
Now if the object gets modified concurrently once more on the target DB while you attach your replicated version, the modifying transaction won't detect that the object got modified by replication in the meantime.
I think that's exactly only happening in the case of a conflict, i.e. when the object got modified independently on both systems.
was (Author: jfrantzius):
If the object got modified once on the target DB before replicating, then the object already may have the version number that will be replicated from remote (following your approach).
Now if the object gets modified concurrently on the target DB while you attach your replicated version, the modifying transaction won't detect that the object got modified by replication in the meantime.
I think that's exactly only happening in the case of a conflict, i.e. when the object got modified independently on both systems.
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (JDO-620) datastore identifier needed for improving
replication of objects
Posted by "Marco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marco updated JDO-620:
----------------------
Description:
We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
1) Detach an object graph from datastore A (using pm.detachCopy(...)).
2) Send the object graph to the destination (using Java native serialisation).
3) Mark the complete object graph dirty (recursively).
4) Attach the object graph to datastore B (using pm.makePersistent(...)).
Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
* Throw an exception and do not startup the PMF.
* Copy the value from the persistence.xml into the datastore.
* Use the value from the persistence.xml without writing it into the datastore.
* Use the value from the datastore and ignore the setting in the persistence.xml.
Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
String getDatastoreIdentifier()
Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
String getDatastoreIdentifier(Object persistenceCapableObject)
When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
was:
We are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
1) Detach an object graph from datastore A (using pm.detachCopy(...)).
2) Send the object graph to the destination (using Java native serialisation).
3) Mark the complete object graph dirty (recursively).
4) Attach the object graph to datastore B (using pm.makePersistent(...)).
Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
* Throw an exception and do not startup the PMF.
* Copy the value from the persistence.xml into the datastore.
* Use the value from the persistence.xml without writing it into the datastore.
* Use the value from the datastore and ignore the setting in the persistence.xml.
Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
String getDatastoreIdentifier()
Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
String getDatastoreIdentifier(Object persistenceCapableObject)
When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JDO-620) datastore identifier needed for
improving replication of objects
Posted by "Guido Anzuoni (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12657593#action_12657593 ]
Guido Anzuoni commented on JDO-620:
-----------------------------------
IMHO I think that attach/detach feature is primarily intended to be used within the same datastore.
Don't know exactly what the original spec says on the topic but I think that version (that is there to support - local - optimistic transactions)
should not be replicated literally.
I mean that if an object O is detached from a PM1 of datastore D1 and attached to PM2 of datastore D2, JDO should behave exactly as
the object with the same identity of O is loaded in PM2 and "manually" updated by the application.
In this sense, maybe yes, JDO should know the PMF from where O was detached in order to apply correct management of
version.
Coming back to the topic, I think that version management should not be stretched to implement multi-way replication.
Version is there to support optimistic transactions.
Support for objects replication is a whole different job and I think it should be designed from scratch (there should be consolidated literature on this topic, i.e. algorithms - Arjuna had this feature, if I remember well).
And it should be an optional feature.
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JDO-620) datastore identifier needed for
improving replication of objects
Posted by "Jörg von Frantzius (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651543#action_12651543 ]
Jörg von Frantzius commented on JDO-620:
----------------------------------------
Of course, other people might be using optimistic transactions...
How about maintaining your own field for ordering? Maybe you already have some kind of lastModification timestamp (provided that only a single datastore is used for generating those timestamps by means of PM.getServerDate()). Or you could maintain your own int version field.
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (JDO-620) datastore identifier needed for
improving replication of objects
Posted by "Marco (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/JDO-620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12651536#action_12651536 ]
Marco commented on JDO-620:
---------------------------
Hello Jörg,
the version of a JDO object is accessible: JDOHelper.getVersion(Object persistenceCapableObject)
Because my replication is done asynchronously and two events might be processed for the same object in reverse order, I use the version to find out whether I need to persist the object or whether I'm processing an out-dated event (version of detached object is older than the one in the datastore).
We currently support Derby and MySQL. AFAIK both of them do not defer the constraint-checks.
We do not use optimistic transactions - all transaction boundaries are within
Marco :-)
> datastore identifier needed for improving replication of objects
> ----------------------------------------------------------------
>
> Key: JDO-620
> URL: https://issues.apache.org/jira/browse/JDO-620
> Project: JDO
> Issue Type: New Feature
> Reporter: Marco
> Fix For: JDO 2 maintenance release 3
>
>
> We - http://www.jfire.org - are using multiple datastores (spread over multiple servers accross the world) and have to replicate objects between them. Following the documentation in http://www.datanucleus.org/products/accessplatform/guides/jdo/replication/ an object replication is done this way:
> 1) Detach an object graph from datastore A (using pm.detachCopy(...)).
> 2) Send the object graph to the destination (using Java native serialisation).
> 3) Mark the complete object graph dirty (recursively).
> 4) Attach the object graph to datastore B (using pm.makePersistent(...)).
> Unfortunately, this imposes the following problems: The JDO implementation optimizes write operations and thus normally only writes dirty fields to the datastore. This is a great feature, but it requires step (3) - making all fields dirty manually - to ensure that existing objects are updated completely. The negative side effect of this is that the version of the object is incremented. That means, the object has a higher version in datastore B as in datastore A, even though the object was simply copied and thus should have the same version number.
> We therefore kindly request the following extension of the JDO standard - if possible for the next release (2.3):
> Every datastore should have an (optional) identifier. This datastore identifier should be specified in the persistence.xml properties of the PersistenceManagerFactory. The first time such a PersistenceManagerFactory is started up, the datastore identifier should be persisted into the datastore (maybe a configurable, optional operation). At every following startup, the persistent datastore identifier should be compared with the one in the persistence.xml. If they do not match, there should be different strategies configurable in the persistence.xml:
> * Throw an exception and do not startup the PMF.
> * Copy the value from the persistence.xml into the datastore.
> * Use the value from the persistence.xml without writing it into the datastore.
> * Use the value from the datastore and ignore the setting in the persistence.xml.
> Please add an API method to the javax.jdo.PersistenceManagerFactory for getting the datastore identifier:
> String getDatastoreIdentifier()
> Every detached object should know the datastore identifier of the datastore where it was detached. It should be possible to access this value via the JDOHelper. Thus, please add the following method to javax.jdo.JDOHelper:
> String getDatastoreIdentifier(Object persistenceCapableObject)
> When calling this method on an object which is currently connected to a PersistenceManager, the method returns the same as would this code:
> JDOHelper.getPersistenceManager(myObject).getPersistenceManagerFactory().getDatastoreIdentifier();
> When the object has not yet been persisted (i.e. not detached, no PersistenceManager assigned), the method JDOHelper.getDatastoreIdentifier(...) should return null (just like JDOHelper.getVersion(...) does).
> When there is no datastore identifier configured for the PersistenceManagerFactory, the result should be an empty String rather than null.
> When attaching an object to a datastore, the JDO implementation should check, whether this is the same datastore where the object was detached or whether it is a different one - i.e. a replication is happening. If the object is replicated, all fields should be written - no matter whether they're dirty or not. If the object is attached to the same datastore where it was detached before, only dirty fields should be written.
> If there was no change (i.e. there is no dirty field), attaching should not modify the object's version. This way, a replicated object should always have the same version as the original.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.