You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by robert burrell donkin <ro...@blueyonder.co.uk> on 2003/04/30 12:28:38 UTC
[PATCH][collections] purging reference map entries when key is garbage collected
<background>
i'm looking into per context classloader BeanUtilsBean instances. the idea
is that each web app can have it's own isolated instances (rather than a
singleton) return on the basis on the thread's contest classloader.
</background>
i need a map which holds it's keys with a weak reference and when the key
is garbage collected, the entry and the value needs to be purged from the
map. i tested WeakHashMap and it doesn't seem to fit the bill. i tried
ReferenceMap but this does not purge (hard referenced) values when the key
is collected.
i've patched ReferenceMap so that it can be setup to do what i need (see
attachment below).
- robert
Re: [PATCH][collections] purging reference map entries when key is garbage collected
Posted by robert burrell donkin <ro...@blueyonder.co.uk>.
On Wednesday, April 30, 2003, at 05:04 PM, Craig R. McClanahan wrote:
>
>
> On Wed, 30 Apr 2003, robert burrell donkin wrote:
>
>> Date: Wed, 30 Apr 2003 13:54:01 +0100
>> From: robert burrell donkin <ro...@blueyonder.co.uk>
>> Reply-To: Jakarta Commons Developers List <commons-
>> dev@jakarta.apache.org>
>> To: Jakarta Commons Developers List <co...@jakarta.apache.org>
>> Subject: Re: [PATCH][collections] purging reference map entries when key
>> is garbage collected
>>
>> On Wednesday, April 30, 2003, at 02:20 PM, Juozas Baliuka wrote:
>>
>>>> <background>
>>>> i'm looking into per context classloader BeanUtilsBean instances. the
>>>> idea
>>>> is that each web app can have it's own isolated instances (rather
>>>> than a
>>>> singleton) return on the basis on the thread's contest classloader.
>>>> </background>
>>>
>>> Possible I do not understan a use case, but
>>> each web will have it's own isolated instances, if beanutils.jar is
>>> cloned
>>> per each WEB-INF/lib .
>>> Do you want to have "global" jar loaded by single class loader, but
>>> singletons per classLoader ?
>>
>> (i'll explain as well as i can: craig's the expert)
>>
>> beanutils used to use static utility classes. we've been moving the
>> functionality into beans (this gives greater flexibility for users). as
>> a
>> preliminary step, the static utility classes used a proper singleton.
>> this
>> means that if beanutils are loaded by the same classloader in different
>> web apps, it is possible that changes made to the beanutils settings in
>> one web app may effect another.
>>
>> threads servicing different web applications should have different
>> ContextClassLoaders so (as i understand it) by returning an instance
>> based
>> on the ContextClassLoaders of the thread making the call we should make
>> sure that different web apps get different instances. (of course, some
>> badly behaved servlet containers might confound this scheme by using
>> different classloader instances without providing a good equals method.)
>>
>>> You can use ThreadLocal, it is internaly implemented as weak map without
>>> dependancy on collections.
>>
>> AFAIK ThreadLocal can only be used to store variables associated with a
>> particular thread. if so, then this isn't really what we're looking for.
>> (i'd be glad to be corrected if i've missed something.)
>>
>
> Robert has the basics exactly right.
>
> The use case we've got is a servlet container (like Tomcat) that provides
> a class loader for each web application, plus a shared class loader that
> is the parent of all the webapp class loaders. For details on the
> architecture, see:
>
> http://jakarta.apache.org/tomcat/tomcat-4.1-doc/class-loader-howto.html
>
> With previous beanutils code (using only the static singleton), things
> worked fine if you put commons-beanutils.jar in each webapp's /WEB-INF/lib
> directory -- each webapp got their own copy of the singleton, with no
> conflicts. However, if commons-beanutils.jar was put in a parent class
> loader (in Tomcat, by putting commons-beanutils.jar into shared/lib or
> common/lib), we ran into conflicts.
>
> The first step, as Robert points out, was to isolate the actual processing
> logic of each class into a separate class that used instance methods
> instead of the static ones (BeanUtils --> BeanUtilsBean and so on). Next,
> we need to work out the strategy of ensuring that each caller receives the
> BeanUtilsBean instance that is appropriate for their environment. Here is
> where we need to do some interesting planning.
>
> For a servlet container, using ThreadLocal is not the right answer,
> because the container will recycle each request processing thread across
> multiple webapps -- what we really want to do is allocate a BeanUtilsBean
> instance per web application. The easiest way to do that is to use the
> Thread's context class loader -- this works because the container is
> required to set the thread context class loader to point at the webapp
> class loader before invoking the servlet to process a request. Therefore,
> we should be able to use the class loader as the key to a map to keep the
> right instances.
>
> However, we probably want to provide a strategy plugin for creating
> instances -- I would advocate for the behavior described in the previous
> paragraph as the default, but other strategies certainly seem feasible as
> well for non-servlet-container environments.
i've now completed a basic prototype of a suitable system that returns an
instance on the basis of the Thread's context class loader. in order to
prevent problems with memory leaks, i need a map which holds keys with
weak references and purges entries when then key is garbage collected. the
patch i submitted adds this functionality to ReferenceMap in collections.
i'd prefer to see some more feedback from the collections team rather than
just adding my name and committing this change.
i support the idea of a strategy plugin but i think that we need to think
clearly about the design. it's going to have to act at the class level and
so is going to have to be discovery-esque (but i'd prefer not to depend on
discovery - at least until there's a stable release).
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [PATCH][collections] purging reference map entries when key is
garbage collected
Posted by "Craig R. McClanahan" <cr...@apache.org>.
On Wed, 30 Apr 2003, robert burrell donkin wrote:
> Date: Wed, 30 Apr 2003 13:54:01 +0100
> From: robert burrell donkin <ro...@blueyonder.co.uk>
> Reply-To: Jakarta Commons Developers List <co...@jakarta.apache.org>
> To: Jakarta Commons Developers List <co...@jakarta.apache.org>
> Subject: Re: [PATCH][collections] purging reference map entries when key
> is garbage collected
>
> On Wednesday, April 30, 2003, at 02:20 PM, Juozas Baliuka wrote:
>
> >> <background>
> >> i'm looking into per context classloader BeanUtilsBean instances. the
> >> idea
> >> is that each web app can have it's own isolated instances (rather than a
> >> singleton) return on the basis on the thread's contest classloader.
> >> </background>
> >
> > Possible I do not understan a use case, but
> > each web will have it's own isolated instances, if beanutils.jar is
> > cloned
> > per each WEB-INF/lib .
> > Do you want to have "global" jar loaded by single class loader, but
> > singletons per classLoader ?
>
> (i'll explain as well as i can: craig's the expert)
>
> beanutils used to use static utility classes. we've been moving the
> functionality into beans (this gives greater flexibility for users). as a
> preliminary step, the static utility classes used a proper singleton. this
> means that if beanutils are loaded by the same classloader in different
> web apps, it is possible that changes made to the beanutils settings in
> one web app may effect another.
>
> threads servicing different web applications should have different
> ContextClassLoaders so (as i understand it) by returning an instance based
> on the ContextClassLoaders of the thread making the call we should make
> sure that different web apps get different instances. (of course, some
> badly behaved servlet containers might confound this scheme by using
> different classloader instances without providing a good equals method.)
>
> > You can use ThreadLocal, it is internaly implemented as weak map without
> > dependancy on collections.
>
> AFAIK ThreadLocal can only be used to store variables associated with a
> particular thread. if so, then this isn't really what we're looking for.
> (i'd be glad to be corrected if i've missed something.)
>
Robert has the basics exactly right.
The use case we've got is a servlet container (like Tomcat) that provides
a class loader for each web application, plus a shared class loader that
is the parent of all the webapp class loaders. For details on the
architecture, see:
http://jakarta.apache.org/tomcat/tomcat-4.1-doc/class-loader-howto.html
With previous beanutils code (using only the static singleton), things
worked fine if you put commons-beanutils.jar in each webapp's /WEB-INF/lib
directory -- each webapp got their own copy of the singleton, with no
conflicts. However, if commons-beanutils.jar was put in a parent class
loader (in Tomcat, by putting commons-beanutils.jar into shared/lib or
common/lib), we ran into conflicts.
The first step, as Robert points out, was to isolate the actual processing
logic of each class into a separate class that used instance methods
instead of the static ones (BeanUtils --> BeanUtilsBean and so on). Next,
we need to work out the strategy of ensuring that each caller receives the
BeanUtilsBean instance that is appropriate for their environment. Here is
where we need to do some interesting planning.
For a servlet container, using ThreadLocal is not the right answer,
because the container will recycle each request processing thread across
multiple webapps -- what we really want to do is allocate a BeanUtilsBean
instance per web application. The easiest way to do that is to use the
Thread's context class loader -- this works because the container is
required to set the thread context class loader to point at the webapp
class loader before invoking the servlet to process a request. Therefore,
we should be able to use the class loader as the key to a map to keep the
right instances.
However, we probably want to provide a strategy plugin for creating
instances -- I would advocate for the behavior described in the previous
paragraph as the default, but other strategies certainly seem feasible as
well for non-servlet-container environments.
> - robert
>
Craig
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [PATCH][collections] purging reference map entries when key is garbage collected
Posted by Juozas Baliuka <ba...@mwm.lt>.
----- Original Message -----
From: "robert burrell donkin" <ro...@blueyonder.co.uk>
To: "Jakarta Commons Developers List" <co...@jakarta.apache.org>
Sent: Wednesday, April 30, 2003 2:54 PM
Subject: Re: [PATCH][collections] purging reference map entries when key is
garbage collected
> On Wednesday, April 30, 2003, at 02:20 PM, Juozas Baliuka wrote:
>
> >> <background>
> >> i'm looking into per context classloader BeanUtilsBean instances. the
> >> idea
> >> is that each web app can have it's own isolated instances (rather than
a
> >> singleton) return on the basis on the thread's contest classloader.
> >> </background>
> >
> > Possible I do not understan a use case, but
> > each web will have it's own isolated instances, if beanutils.jar is
> > cloned
> > per each WEB-INF/lib .
> > Do you want to have "global" jar loaded by single class loader, but
> > singletons per classLoader ?
>
> (i'll explain as well as i can: craig's the expert)
>
> beanutils used to use static utility classes. we've been moving the
> functionality into beans (this gives greater flexibility for users). as a
> preliminary step, the static utility classes used a proper singleton. this
> means that if beanutils are loaded by the same classloader in different
> web apps, it is possible that changes made to the beanutils settings in
> one web app may effect another.
>
> threads servicing different web applications should have different
> ContextClassLoaders so (as i understand it) by returning an instance based
> on the ContextClassLoaders of the thread making the call we should make
> sure that different web apps get different instances. (of course, some
> badly behaved servlet containers might confound this scheme by using
> different classloader instances without providing a good equals method.)
I do not think it is a very good workaround, I prefer not to asume
"ContextClassLoaders" is set in reusable component,
and I think WEB-INF/lib and class loading delegation workaround in servlet
spec. is better, it isolates applications, solves version conflicts,
simlifies deployment . Do you need this stuff to save momory on server ?
>
> > You can use ThreadLocal, it is internaly implemented as weak map without
> > dependancy on collections.
>
> AFAIK ThreadLocal can only be used to store variables associated with a
> particular thread. if so, then this isn't really what we're looking for.
> (i'd be glad to be corrected if i've missed something.)
"ThreadLocal" instances will be created per thread not per web app, it is
not good choise for this use case.
>
> - robert
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [PATCH][collections] purging reference map entries when key is garbage collected
Posted by robert burrell donkin <ro...@blueyonder.co.uk>.
On Wednesday, April 30, 2003, at 02:20 PM, Juozas Baliuka wrote:
>> <background>
>> i'm looking into per context classloader BeanUtilsBean instances. the
>> idea
>> is that each web app can have it's own isolated instances (rather than a
>> singleton) return on the basis on the thread's contest classloader.
>> </background>
>
> Possible I do not understan a use case, but
> each web will have it's own isolated instances, if beanutils.jar is
> cloned
> per each WEB-INF/lib .
> Do you want to have "global" jar loaded by single class loader, but
> singletons per classLoader ?
(i'll explain as well as i can: craig's the expert)
beanutils used to use static utility classes. we've been moving the
functionality into beans (this gives greater flexibility for users). as a
preliminary step, the static utility classes used a proper singleton. this
means that if beanutils are loaded by the same classloader in different
web apps, it is possible that changes made to the beanutils settings in
one web app may effect another.
threads servicing different web applications should have different
ContextClassLoaders so (as i understand it) by returning an instance based
on the ContextClassLoaders of the thread making the call we should make
sure that different web apps get different instances. (of course, some
badly behaved servlet containers might confound this scheme by using
different classloader instances without providing a good equals method.)
> You can use ThreadLocal, it is internaly implemented as weak map without
> dependancy on collections.
AFAIK ThreadLocal can only be used to store variables associated with a
particular thread. if so, then this isn't really what we're looking for.
(i'd be glad to be corrected if i've missed something.)
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org
Re: [PATCH][collections] purging reference map entries when key is garbage collected
Posted by Juozas Baliuka <ba...@mwm.lt>.
> <background>
> i'm looking into per context classloader BeanUtilsBean instances. the idea
> is that each web app can have it's own isolated instances (rather than a
> singleton) return on the basis on the thread's contest classloader.
> </background>
Possible I do not understan a use case, but
each web will have it's own isolated instances, if beanutils.jar is cloned
per each WEB-INF/lib .
Do you want to have "global" jar loaded by single class loader, but
singletons per classLoader ?
You can use ThreadLocal, it is internaly implemented as weak map without
dependancy on collections.
>
> i need a map which holds it's keys with a weak reference and when the key
> is garbage collected, the entry and the value needs to be purged from the
> map. i tested WeakHashMap and it doesn't seem to fit the bill. i tried
> ReferenceMap but this does not purge (hard referenced) values when the key
> is collected.
>
> i've patched ReferenceMap so that it can be setup to do what i need (see
> attachment below).
>
> - robert
>
>
>
----------------------------------------------------------------------------
----
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: commons-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commons-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: commons-dev-help@jakarta.apache.org