You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cayenne.apache.org by Andrus Adamchik <an...@objectstyle.org> on 2006/11/14 16:02:53 UTC
Leak-free context - requesting feedback
As we all know, DataContext doesn't clean up unused registered
objects on an assumption that the entire DataContext has a finite
lifespan and will be garbage-collected eventually.
In my session-less web application I am using an app-scoped read-only
DC shared by all requests. Sure enough it leaks memory, given a big
enough database. In the past we solved this problem by periodically
replacing a shared instance of DC with a new one. Since this approach
intrerfered with the caching mechanism and generally seemed dirty,
the other day I implemented a self-cleaning ObjectStore based on
commons-collections ReferenceMap:
class LeakFreeObjectStore extends ObjectStore {
public LeakFreeObjectStore(DataRowStore dataRowCache) {
super(dataRowCache);
objectMap = new ReferenceMap(AbstractReferenceMap.HARD,
AbstractReferenceMap.WEAK);
}
}
Note that I couldn't use LRUMap, as we can't unregister objects that
are referenced by other application objects, so instead I used weak
references to cleanup otherwise unreferenced instances. Been watching
this in production for a day and it works perfectly.
Now the question is whether we want this behavior as a default (or as
an option) for the DataContext? In other words should I move this fix
to Cayenne? (I will have to modify it to create hard references to
the dirty objects as we can't deallocate those even if they are not
referenced in the app).
I'd say yes, but I was wondering if premature garbage collection of
registered objects is bad for any reason? Thoughts?
Andrus
Re: Leak-free context - requesting feedback
Posted by Andrus Adamchik <an...@objectstyle.org>.
FYI: just documented leak-free DataContext behavior on Wiki (there
are a few scenarios where it results in extra db queries) -
http://cwiki.apache.org/CAY/objectcontext-memory-management.html
Andrus
On Nov 21, 2006, at 10:53 PM, Andrus Adamchik wrote:
> I just checked in this code. Nice side effect is that now we can
> deprecate ObjectStore.startTrackingNewObjects() and
> ObjectStore.unregisterNewObjects() - obscure methods used for
> manual memory management.
>
> Since this is a big paradigm shift in Cayenne in general, I
> appreciate feedback. Also for those who want the old behavior, I
> added an extra constructor to the ObjectStore that takes a map, so
> that current default could be overridden (e.g. via
> DataContextFactory).
>
> The change currently only covers only traditional DataContext. I
> figured that with ROP, since there is no local cache of object
> snapshots, deallocating prematurely may result in extra remote
> calls for relationships... Still somehow I feel that adding this
> change as a default strategy across the board is cleaner than
> leaving things as is.
>
> Andrus
>
>
> On Nov 14, 2006, at 11:46 AM, Tore Halset wrote:
>
>> Hello.
>>
>> +1 (move to cayenne and make it default behaviour)
>>
>> This is a realy good idea. It should improve situations with a
>> context bound to a session as well. We need to have a pretty long
>> session timeout so this improvement should be significant.
>>
>> I see that you have created a pluggable QueryCache. As discussed
>> before we should also create a DataRowStore equivalent with a
>> pluggable caching solution. I think you mentioned that this store
>> should cache DataObjects instead of DataRows, so I do not know
>> where to start :)
>>
>> - Tore.
>>
>> On Nov 14, 2006, at 16:02, Andrus Adamchik wrote:
>>
>>> As we all know, DataContext doesn't clean up unused registered
>>> objects on an assumption that the entire DataContext has a finite
>>> lifespan and will be garbage-collected eventually.
>>>
>>> In my session-less web application I am using an app-scoped read-
>>> only DC shared by all requests. Sure enough it leaks memory,
>>> given a big enough database. In the past we solved this problem
>>> by periodically replacing a shared instance of DC with a new one.
>>> Since this approach intrerfered with the caching mechanism and
>>> generally seemed dirty, the other day I implemented a self-
>>> cleaning ObjectStore based on commons-collections ReferenceMap:
>>>
>>> class LeakFreeObjectStore extends ObjectStore {
>>>
>>> public LeakFreeObjectStore(DataRowStore dataRowCache) {
>>> super(dataRowCache);
>>> objectMap = new ReferenceMap(AbstractReferenceMap.HARD,
>>> AbstractReferenceMap.WEAK);
>>> }
>>> }
>>>
>>> Note that I couldn't use LRUMap, as we can't unregister objects
>>> that are referenced by other application objects, so instead I
>>> used weak references to cleanup otherwise unreferenced instances.
>>> Been watching this in production for a day and it works perfectly.
>>>
>>> Now the question is whether we want this behavior as a default
>>> (or as an option) for the DataContext? In other words should I
>>> move this fix to Cayenne? (I will have to modify it to create
>>> hard references to the dirty objects as we can't deallocate those
>>> even if they are not referenced in the app).
>>>
>>> I'd say yes, but I was wondering if premature garbage collection
>>> of registered objects is bad for any reason? Thoughts?
>>>
>>> Andrus
>>>
>>>
>>>
>>
>>
>
>
Re: Leak-free context - requesting feedback
Posted by Andrus Adamchik <an...@objectstyle.org>.
I just checked in this code. Nice side effect is that now we can
deprecate ObjectStore.startTrackingNewObjects() and
ObjectStore.unregisterNewObjects() - obscure methods used for manual
memory management.
Since this is a big paradigm shift in Cayenne in general, I
appreciate feedback. Also for those who want the old behavior, I
added an extra constructor to the ObjectStore that takes a map, so
that current default could be overridden (e.g. via DataContextFactory).
The change currently only covers only traditional DataContext. I
figured that with ROP, since there is no local cache of object
snapshots, deallocating prematurely may result in extra remote calls
for relationships... Still somehow I feel that adding this change as
a default strategy across the board is cleaner than leaving things as
is.
Andrus
On Nov 14, 2006, at 11:46 AM, Tore Halset wrote:
> Hello.
>
> +1 (move to cayenne and make it default behaviour)
>
> This is a realy good idea. It should improve situations with a
> context bound to a session as well. We need to have a pretty long
> session timeout so this improvement should be significant.
>
> I see that you have created a pluggable QueryCache. As discussed
> before we should also create a DataRowStore equivalent with a
> pluggable caching solution. I think you mentioned that this store
> should cache DataObjects instead of DataRows, so I do not know
> where to start :)
>
> - Tore.
>
> On Nov 14, 2006, at 16:02, Andrus Adamchik wrote:
>
>> As we all know, DataContext doesn't clean up unused registered
>> objects on an assumption that the entire DataContext has a finite
>> lifespan and will be garbage-collected eventually.
>>
>> In my session-less web application I am using an app-scoped read-
>> only DC shared by all requests. Sure enough it leaks memory, given
>> a big enough database. In the past we solved this problem by
>> periodically replacing a shared instance of DC with a new one.
>> Since this approach intrerfered with the caching mechanism and
>> generally seemed dirty, the other day I implemented a self-
>> cleaning ObjectStore based on commons-collections ReferenceMap:
>>
>> class LeakFreeObjectStore extends ObjectStore {
>>
>> public LeakFreeObjectStore(DataRowStore dataRowCache) {
>> super(dataRowCache);
>> objectMap = new ReferenceMap(AbstractReferenceMap.HARD,
>> AbstractReferenceMap.WEAK);
>> }
>> }
>>
>> Note that I couldn't use LRUMap, as we can't unregister objects
>> that are referenced by other application objects, so instead I
>> used weak references to cleanup otherwise unreferenced instances.
>> Been watching this in production for a day and it works perfectly.
>>
>> Now the question is whether we want this behavior as a default (or
>> as an option) for the DataContext? In other words should I move
>> this fix to Cayenne? (I will have to modify it to create hard
>> references to the dirty objects as we can't deallocate those even
>> if they are not referenced in the app).
>>
>> I'd say yes, but I was wondering if premature garbage collection
>> of registered objects is bad for any reason? Thoughts?
>>
>> Andrus
>>
>>
>>
>
>
Re: Leak-free context - requesting feedback
Posted by Tore Halset <ha...@pvv.ntnu.no>.
Hello.
+1 (move to cayenne and make it default behaviour)
This is a realy good idea. It should improve situations with a
context bound to a session as well. We need to have a pretty long
session timeout so this improvement should be significant.
I see that you have created a pluggable QueryCache. As discussed
before we should also create a DataRowStore equivalent with a
pluggable caching solution. I think you mentioned that this store
should cache DataObjects instead of DataRows, so I do not know where
to start :)
- Tore.
On Nov 14, 2006, at 16:02, Andrus Adamchik wrote:
> As we all know, DataContext doesn't clean up unused registered
> objects on an assumption that the entire DataContext has a finite
> lifespan and will be garbage-collected eventually.
>
> In my session-less web application I am using an app-scoped read-
> only DC shared by all requests. Sure enough it leaks memory, given
> a big enough database. In the past we solved this problem by
> periodically replacing a shared instance of DC with a new one.
> Since this approach intrerfered with the caching mechanism and
> generally seemed dirty, the other day I implemented a self-cleaning
> ObjectStore based on commons-collections ReferenceMap:
>
> class LeakFreeObjectStore extends ObjectStore {
>
> public LeakFreeObjectStore(DataRowStore dataRowCache) {
> super(dataRowCache);
> objectMap = new ReferenceMap(AbstractReferenceMap.HARD,
> AbstractReferenceMap.WEAK);
> }
> }
>
> Note that I couldn't use LRUMap, as we can't unregister objects
> that are referenced by other application objects, so instead I used
> weak references to cleanup otherwise unreferenced instances. Been
> watching this in production for a day and it works perfectly.
>
> Now the question is whether we want this behavior as a default (or
> as an option) for the DataContext? In other words should I move
> this fix to Cayenne? (I will have to modify it to create hard
> references to the dirty objects as we can't deallocate those even
> if they are not referenced in the app).
>
> I'd say yes, but I was wondering if premature garbage collection of
> registered objects is bad for any reason? Thoughts?
>
> Andrus
>
>
>