You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ojb-user@db.apache.org by "Nase, Paul R." <Na...@mayo.edu> on 2004/09/28 01:49:41 UTC
OJB Cache and materialization.
Hi,
We are encountering a problem with our cache materializing half-baked objects
when under a heavy multi-user load.
We are currently using OJB RC5 and are not in a position to move to 1.0.1 yet,
although in the future we would like to. We also have a JCS cache configured.
Our problem occurs when you have an object that contains references to 2 or more
collections of other objects. The code in question is in the
RsIterator.getObjectFromResultSet() method. A snippet of the code is shown below.
synchronized (result)
{
getCache().enableMaterializationCache();
getCache().cache(oid, result);
/*
* arminw: move LoadedObjectsRegistry to odmg-layer
*/
// LoadedObjectsRegistry.register(result);
/**
* MBAIRD if you have multiple classes mapped to a
* table, and you query on the base class you could get
* back NON base class objects, so we shouldn't pass
* m_cld, but rather the class descriptor for the
* actual class.
*/
// fill reference and collection attributes
ClassDescriptor cld =
getQueryObject().getClassDescriptor().getRepository().getDescriptorFor(result.getClass());
// don't force loading of reference
final boolean unforced = false;
// Maps ReferenceDescriptors to HashSets of owners
getBroker().getReferenceBroker().retrieveReferences(result, cld, unforced);
getBroker().getReferenceBroker().retrieveCollections(result, cld, unforced);
getCache().disableMaterializationCache();
}
When we have our object being materialized, it has 2 collections of persistent
objects. When getCache().cache(oid, result); is called, the object is placed
into a 'local' cache. When retrieveCollections() is called, it is quite possible
that this same method (getObjectFromResultSet) will be called during the collection
object's materialization. The materialization of the collection object then
calls getCache().disableMaterializationCache(), which at this point the 'local'
cache contains objects from only one of the collections (the one currently being
processed) plus the original object being materialized. The
disableMaterializationCache() causes any of the objects in the 'local' cache to
be 'pushed' to the 'real' cache. But the problem comes because the original
object we are materializing has two collections, one collection has been
retrieved, we are about to retrieve the other, but the original object and
only one of its collections has been pushed to the real cache. Another thread
may come and steal this invalid data out of the real cache and do nasty things.
Is there a handy solution to this problem?
I am very very sorry for the long post...
Paul Nase
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-user-help@db.apache.org
Re: OJB Cache and materialization.
Posted by Armin Waibel <ar...@apache.org>.
Hi Paul,
it's fixed in CVS branch "OJB_1_0_RELEASE".
>> We are currently using OJB RC5 and are not in a position to move to
>> 1.0.1 yet,
Think you are using rc6, because the "materialization buffer"
(InternalCache class) was introduced after rc5 - AFAIK
It shouldn't be a problem to adapt the patch for earlier releases (after
rc5) of OJB. I only introduce an 'counter' which counts the number of
invocations of method InternalCache#enableMaterializationCache.
Modified classes are:
InternalCache
PersistenceBrokerImpl
RsIterator
Diff see below.
regards,
Armin
Revision Changes Path
No revision
No revision
1.63.2.3 +27 -21
db-ojb/src/java/org/apache/ojb/broker/accesslayer/RsIterator.java
Index: RsIterator.java
===================================================================
RCS file:
/home/cvs/db-ojb/src/java/org/apache/ojb/broker/accesslayer/RsIterator.java,v
retrieving revision 1.63.2.2
retrieving revision 1.63.2.3
diff -u -r1.63.2.2 -r1.63.2.3
--- RsIterator.java 9 Aug 2004 07:51:26 -0000 1.63.2.2
+++ RsIterator.java 30 Sep 2004 14:02:03 -0000 1.63.2.3
@@ -452,26 +452,32 @@
synchronized (result)
{
getCache().enableMaterializationCache();
- getCache().cache(oid, result);
- /*
- * arminw: move LoadedObjectsRegistry to odmg-layer
- */
- // LoadedObjectsRegistry.register(result);
- /**
- * MBAIRD if you have multiple classes
mapped to a
- * table, and you query on the base class
you could get
- * back NON base class objects, so we
shouldn't pass
- * m_cld, but rather the class descriptor
for the
- * actual class.
- */
- // fill reference and collection attributes
- ClassDescriptor cld =
getBroker().getClassDescriptor(result.getClass());
- // don't force loading of reference
- final boolean unforced = false;
- // Maps ReferenceDescriptors to HashSets of
owners
-
getBroker().getReferenceBroker().retrieveReferences(result, cld, unforced);
-
getBroker().getReferenceBroker().retrieveCollections(result, cld, unforced);
- getCache().disableMaterializationCache();
+ try
+ {
+ getCache().cache(oid, result);
+ /*
+ * arminw: move LoadedObjectsRegistry to
odmg-layer
+ */
+ // LoadedObjectsRegistry.register(result);
+ /**
+ * MBAIRD if you have multiple classes
mapped to a
+ * table, and you query on the base
class you could get
+ * back NON base class objects, so we
shouldn't pass
+ * m_cld, but rather the class
descriptor for the
+ * actual class.
+ */
+ // fill reference and collection attributes
+ ClassDescriptor cld =
getBroker().getClassDescriptor(result.getClass());
+ // don't force loading of reference
+ final boolean unforced = false;
+ // Maps ReferenceDescriptors to HashSets
of owners
+
getBroker().getReferenceBroker().retrieveReferences(result, cld, unforced);
+
getBroker().getReferenceBroker().retrieveCollections(result, cld, unforced);
+ }
+ finally
+ {
+ getCache().disableMaterializationCache();
+ }
}
}
}
No revision
No revision
1.5.2.1 +18 -3
db-ojb/src/java/org/apache/ojb/broker/cache/Attic/InternalCache.java
Index: InternalCache.java
===================================================================
RCS file:
/home/cvs/db-ojb/src/java/org/apache/ojb/broker/cache/Attic/InternalCache.java,v
retrieving revision 1.5
retrieving revision 1.5.2.1
diff -u -r1.5 -r1.5.2.1
--- InternalCache.java 25 Jun 2004 16:09:50 -0000 1.5
+++ InternalCache.java 30 Sep 2004 14:02:03 -0000 1.5.2.1
@@ -40,6 +40,7 @@
private ObjectCache realCache;
private HashMap localCache;
private boolean enabledReadCache;
+ private int invokeCounter;
public InternalCache(ObjectCache realCache)
{
@@ -50,13 +51,25 @@
public void enableMaterializationCache()
{
+ ++invokeCounter;
enabledReadCache = true;
}
public void disableMaterializationCache()
{
- enabledReadCache = false;
- pushToRealCache();
+ if(enabledReadCache)
+ {
+ --invokeCounter;
+ /*
+ if materialization of the requested object was
completed, the
+ counter represents '0'
+ */
+ if (invokeCounter == 0)
+ {
+ enabledReadCache = false;
+ pushToRealCache();
+ }
+ }
}
private void pushToRealCache()
@@ -114,6 +127,8 @@
" push to real ObjectCache");
}
localCache.clear();
+ invokeCounter = 0;
+ enabledReadCache = false;
}
public void clear()
No revision
No revision
1.83.2.4 +40 -3
db-ojb/src/java/org/apache/ojb/broker/core/PersistenceBrokerImpl.java
Index: PersistenceBrokerImpl.java
===================================================================
RCS file:
/home/cvs/db-ojb/src/java/org/apache/ojb/broker/core/PersistenceBrokerImpl.java,v
retrieving revision 1.83.2.3
retrieving revision 1.83.2.4
diff -u -r1.83.2.3 -r1.83.2.4
--- PersistenceBrokerImpl.java 21 Sep 2004 00:29:01 -0000 1.83.2.3
+++ PersistenceBrokerImpl.java 30 Sep 2004 14:02:03 -0000 1.83.2.4
@@ -446,6 +446,21 @@
{
logger.warn("No running tx found, please only delete
objects in context of an PB-transaction" +
", to avoid side-effects - e.g. when rollback of
complex objects");
+ /*
+ arminw:
+ this could help user to find missing tx declaration
+ */
+ if(logger.isEnabledFor(Logger.INFO))
+ {
+ try
+ {
+ throw new Exception("** Try to delete object
without active PersistenceBroker transaction **");
+ }
+ catch(Exception e)
+ {
+ e.printStackTrace();
+ }
+ }
}
try
{
@@ -1336,8 +1351,15 @@
public Object getObjectByIdentity(Identity id) throws
PersistenceBrokerException
{
objectCache.enableMaterializationCache();
- Object result = doGetObjectByIdentity(id);
- objectCache.disableMaterializationCache();
+ Object result;
+ try
+ {
+ result = doGetObjectByIdentity(id);
+ }
+ finally
+ {
+ objectCache.disableMaterializationCache();
+ }
return result;
}
@@ -1527,6 +1549,21 @@
{
logger.warn("No running tx found, please only store in
context of an PB-transaction" +
", to avoid side-effects - e.g. when rollback of
complex objects");
+ /*
+ arminw:
+ this could help user to find missing tx declaration
+ */
+ if(logger.isEnabledFor(Logger.INFO))
+ {
+ try
+ {
+ throw new Exception("** Try to store object
without active PersistenceBroker transaction **");
+ }
+ catch(Exception e)
+ {
+ e.printStackTrace();
+ }
+ }
}
// Invoke events on PersistenceBrokerAware instances and
listeners
if (insert)
Armin Waibel wrote:
> Hi Paul,
>
> I will have a look at this.
> Stay tuned!
>
> regards,
> Armin
>
> Nase, Paul R. wrote:
>
>> Hi,
>>
>> We are encountering a problem with our cache materializing half-baked
>> objects
>> when under a heavy multi-user load.
>>
>> We are currently using OJB RC5 and are not in a position to move to
>> 1.0.1 yet,
>> although in the future we would like to. We also have a JCS cache
>> configured.
>>
>> Our problem occurs when you have an object that contains references to
>> 2 or more
>> collections of other objects. The code in question is in the
>> RsIterator.getObjectFromResultSet() method. A snippet of the code is
>> shown below.
>> synchronized (result)
>> {
>> getCache().enableMaterializationCache();
>> getCache().cache(oid, result);
>> /*
>> * arminw: move LoadedObjectsRegistry to odmg-layer
>> */
>> // LoadedObjectsRegistry.register(result);
>> /**
>> * MBAIRD if you have multiple classes mapped to a
>> * table, and you query on the base class you could get
>> * back NON base class objects, so we shouldn't pass
>> * m_cld, but rather the class descriptor for the
>> * actual class.
>> */
>> // fill reference and collection attributes
>> ClassDescriptor cld =
>> getQueryObject().getClassDescriptor().getRepository().getDescriptorFor(result.getClass());
>>
>> // don't force loading of reference
>> final boolean unforced = false;
>> // Maps ReferenceDescriptors to HashSets of owners
>> getBroker().getReferenceBroker().retrieveReferences(result, cld,
>> unforced);
>> getBroker().getReferenceBroker().retrieveCollections(result,
>> cld, unforced);
>> getCache().disableMaterializationCache();
>> }
>> When we have our object being materialized, it has 2 collections of
>> persistent
>> objects. When getCache().cache(oid, result); is called, the object is
>> placed
>> into a 'local' cache. When retrieveCollections() is called, it is
>> quite possible
>> that this same method (getObjectFromResultSet) will be called during
>> the collection
>> object's materialization. The materialization of the collection
>> object then
>> calls getCache().disableMaterializationCache(), which at this point
>> the 'local'
>> cache contains objects from only one of the collections (the one
>> currently being
>> processed) plus the original object being materialized. The
>> disableMaterializationCache() causes any of the objects in the 'local'
>> cache to
>> be 'pushed' to the 'real' cache. But the problem comes because the
>> original
>> object we are materializing has two collections, one collection has been
>> retrieved, we are about to retrieve the other, but the original object
>> and
>> only one of its collections has been pushed to the real cache.
>> Another thread
>> may come and steal this invalid data out of the real cache and do
>> nasty things.
>>
>> Is there a handy solution to this problem?
>>
>> I am very very sorry for the long post...
>>
>> Paul Nase
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
>> For additional commands, e-mail: ojb-user-help@db.apache.org
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-user-help@db.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-user-help@db.apache.org
Re: OJB Cache and materialization.
Posted by Armin Waibel <ar...@apache.org>.
Hi Paul,
I will have a look at this.
Stay tuned!
regards,
Armin
Nase, Paul R. wrote:
> Hi,
>
> We are encountering a problem with our cache materializing half-baked objects
> when under a heavy multi-user load.
>
> We are currently using OJB RC5 and are not in a position to move to 1.0.1 yet,
> although in the future we would like to. We also have a JCS cache configured.
>
> Our problem occurs when you have an object that contains references to 2 or more
> collections of other objects. The code in question is in the
> RsIterator.getObjectFromResultSet() method. A snippet of the code is shown below.
> synchronized (result)
> {
> getCache().enableMaterializationCache();
> getCache().cache(oid, result);
> /*
> * arminw: move LoadedObjectsRegistry to odmg-layer
> */
> // LoadedObjectsRegistry.register(result);
> /**
> * MBAIRD if you have multiple classes mapped to a
> * table, and you query on the base class you could get
> * back NON base class objects, so we shouldn't pass
> * m_cld, but rather the class descriptor for the
> * actual class.
> */
> // fill reference and collection attributes
> ClassDescriptor cld =
> getQueryObject().getClassDescriptor().getRepository().getDescriptorFor(result.getClass());
> // don't force loading of reference
> final boolean unforced = false;
> // Maps ReferenceDescriptors to HashSets of owners
> getBroker().getReferenceBroker().retrieveReferences(result, cld, unforced);
> getBroker().getReferenceBroker().retrieveCollections(result, cld, unforced);
> getCache().disableMaterializationCache();
> }
> When we have our object being materialized, it has 2 collections of persistent
> objects. When getCache().cache(oid, result); is called, the object is placed
> into a 'local' cache. When retrieveCollections() is called, it is quite possible
> that this same method (getObjectFromResultSet) will be called during the collection
> object's materialization. The materialization of the collection object then
> calls getCache().disableMaterializationCache(), which at this point the 'local'
> cache contains objects from only one of the collections (the one currently being
> processed) plus the original object being materialized. The
> disableMaterializationCache() causes any of the objects in the 'local' cache to
> be 'pushed' to the 'real' cache. But the problem comes because the original
> object we are materializing has two collections, one collection has been
> retrieved, we are about to retrieve the other, but the original object and
> only one of its collections has been pushed to the real cache. Another thread
> may come and steal this invalid data out of the real cache and do nasty things.
>
> Is there a handy solution to this problem?
>
> I am very very sorry for the long post...
>
> Paul Nase
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-user-help@db.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-user-help@db.apache.org