You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hivemind.apache.org by Jean-Francois Poilpret <jf...@hcm.vnn.vn> on 2004/12/03 17:42:33 UTC
Potential memory leak with Threaded model and Events listeners
Hello,
I was wondering about one potential problem (I did not test it but I believe
it happens and I wonder about whether it could be fixed in HiveMind should
be worked around in applications):
- let's suppose I have a service A that supplies events
- let's suppose I have a service B that listens to those events, and that
uses the threaded model
- when a new thread is executed and first time service B is used by this
thread, a new instance of B is created for the thread, and it is added as a
listener to events produced by service A: hence A retains a reference to the
actual implementation of B for the thread
- when the thread is cleaned up, then that service B instance ceases to
exist (it has been "discarded") and nothing refers to it anymore...
- ...nothing except service A that still holds a reference to that instance
(in its list of listeners)
This means that in this situation if I have a huge amount of threads during
my application's lifecycle, then a huge amount of service B instances will
be created and never garbage collected.
In addition, whenever service A produces an event, it will dispatch it to
this huge amount of "zombie" instances, reducing performance a lot.
Is my reasoning correct? If necessary, I can try to write a simple test case
to produce this problem.
If this is correct, what should be the correct way to fix the problem?
- in HiveMind? Actually, I do not see really where (the problem in HM1.0 is
that the link of the Event listener to the producer is done by the Factory,
but only the ServiceModel manages the full lifecycle of the service
instance, and it knows nothing about what the factory did).
I remember some time ago, Howard discussed the idea to change the
responsibilities of ServiceModels and Factories to make it clearer. Will
this be done in HM1.1? Will this possibly solve that problem?
- in the application itself? Yes it is possible to implement the Discardable
interface in the service B, and when the service is discarded then we can
possibly call serviceA.removeEventListener(this). However, this means that
the link that was configured dynamically through the module configuration
has to be undone by some kind of hard-code, reducing the interest of using
the BuilderFactory to link the event listener to the event supplier.
Has somebody already met such kind of problem and could come out with some
satisfactory solution?
Some more info about why I ask: actually I am working on the "user" service
model that I described a few days ago (ie, one different instance for each
connected user, discarded when user logs out), it works correctly until now,
but in one "real-life" application that I am currently working on and that
needs to use it, I am faced with this kind of problem, so I wonder how I can
circumvent this.
Thank you for any feedback
Cheers
Jean-Francois
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
RE: Potential memory leak with Threaded model and Events listeners
Posted by Albert Kwong <ma...@yahoo.com>.
We already have an <event-listener/> declaration for
BuilderFactory, so, how about insert a
Discardable/PoolManageable aspect between the proxy
and the underlying service? This service would call
the event publisher's remove* method whenever they are
discarded.
The aspect need not be another level of proxy. It can
be a dynamic extension of the service implementation
class.
Albert
--- Jean-Francois Poilpret <jf...@hcm.vnn.vn>
內容：
> No, I don't want each __created__ instance to
> receive notifications, I just
> want each __active__ instance to receive them, and
> any __inactive___
> instance not to receive them.
> Currently with either threaded or pooled model (as
> far as I can see), all
> created instances actually receive notifications,
> even when inactive. Worse,
> for threaded model, inactive instances never get
> garbage-collected because
> they are strongly referenced by the service which
> fire events.
>
> For me, the expected behavior of HiveMind would be:
> - when a service instance gets created (threaded) or
> activated (pooled),
> then it should be added as a listener to the service
> firing events
> - when a service instance gets discarded (threaded)
> or deactivated (pooled),
> then it should be removed as a listener from the
> service supplying events.
>
> NB: this description applies to the case where the
> event-firing service is a
> singleton (my situation right now), but the behavior
> would/should probably
> be different if that service used a different model.
>
> Currently, the best work-around is:
> - for threaded model, the listening service must
> implement Discardable and
> remove itself from the listeners list of the event
> firing service, when its
> threadDidDiscardService() method is called. The
> registration as a listener
> can either be done by configuration (BuilderFactory)
> or in the constructor.
>
> - for pooled model, the listening service must
> implement PoolManageable and
> it must register itself as a listener in its XXXX
> methods, and unregister
> itself when its XXXX() method is called. Here
> automatic Event registration
> through BuilderFactory is useless.
>
> These work-arounds work well, but they are not very
> "clean" in the sense
> that this is code the end-developer has to write
> that HiveMind could
> directly handle for him.
> In addition, in the threaded situation, if we use
> the BuilderFactory to wire
> the listener to the event supplier, then the
> work-around introduces some
> asymmetry (registration in config, unregistration in
> the code), making the
> use of BuilderFactory for this aspect useless.
>
> I think before changing anything in HiveMind, we
> (all concerned developers)
> should discuss the various situations (models for
> supplier/consumer:
> singleton/threaded, singleton/pooled,
> threaded/singleton,
> threaded/threaded...) and specify for each what the
> behavior should be, then
> the actual coding can start. Should we start a new
> page on the WIKI?
>
> Cheers
>
> -----Original Message-----
> From: Howard Lewis Ship [mailto:hlship@gmail.com]
> Sent: Sunday, December 05, 2004 4:09 AM
> To: hivemind-user@jakarta.apache.org
> Subject: Re: Potential memory leak with Threaded
> model and Events listeners
>
> If you want each created instance to receive the
> notification, use the
> pooled, not threaded, service model. Pooled
> instances exist
> indefinately once created. The only down side is
> that code related to
> the event needs to be thread safe, and that
> deactivated instances in
> the pool will also receive notifications.
>
>
> On Sun, 5 Dec 2004 00:18:27 +0700, Jean-Francois
> Poilpret
> <jf...@hcm.vnn.vn> wrote:
> > Hi Luke,
> >
> > Thank you for your remark.
> > Although your remark is correct in many cases of
> Listeners, I think it
> would
> > generally not apply in HiveMind context, I mean
> (if people follow the
> "right
> > path") that normally, only a service will
> implement a Listener interface,
> > thus at least HiveMind itself (through one of the
> generated proxies) will
> > hold a strong reference to that service/listener.
> But of course, if
> someone
> > decides to create a Listener on-the-fly and add it
> "by hand" to the event
> > supplier service, then the problem you described
> would occur.
> >
> > Anyway, I just did some tests (with strong and
> with weak references) and
> in
> > fact, the result is not "as good" as I expected
> initially. Indeed, when
> > suggesting that idea, I simply forgot that the
> handling of weak references
> > is totally non-deterministic, ie the weak ref
> could still be valid even
> > though there is no more existing strong ref. The
> time at which the weak
> ref
> > would be cleaned is totally up to the GC...
> > So as my test have shown me, using weak refs
> improve the issue, but they
> do
> > not solve it really:-(
> >
> > In my tests, I did something like that:
> > - create EventSupplier service (singleton)
> > - create EventConsumer service (threaded)
> > - instantiate EventConsumer in 1000 "threads"
> (they are real threads but
> > they do not run concurrently, but I force
> consecutive runs, with a call to
> > Registry.cleanupThread() at the end of each
> thread)
> > - each thread call the EventSupplier to generate a
> new event
> > - all calls are traced
> > - each EventConsumer instance allocates 10KB on
> the heap and never release
> > them.
> >
> > With strong refs:
> > - the 1000th thread sends an event that is
> received by 1000 EventConsumer
> > instances
> >
> > With weak refs:
> > - the 1000th thread sends an event that is
> received by about 30
> > EventConsumer instances.
> >
> > Of course that is much better, but the ideal, here
> in my testcase, would
> be
> > always just one EventConsumer instance receiving
> the event.
> >
> > So finally, maybe I will stick with the previous
> -not very clean from
> design
> > viewpoint- workaround:
> > - have EventConsumer implement Discardable and
> remove itself from the
> > EventSupplier list of listeners, when
> threadDidDiscardService() is called.
> >
> > And then I will eagerly wait for HM1.1 release
> (hope it will fix this
> > issue).
> >
> > Cheers
> >
> >
> >
> > -----Original Message-----
> > From: Luke Blanshard [mailto:luke@blanshard.us]
> > Sent: Saturday, December 04, 2004 8:13 PM
> > To: hivemind-user@jakarta.apache.org
> > Subject: Re: Potential memory leak with Threaded
> model and Events
> listeners
> >
> > Jean-Francois,
> >
> > The problem with holding listeners weakly is that
> people commonly create
> > listener objects on the fly and then retain no
> other references to them
> > once they've added them. So these listeners are
> thrown away on the next
> > GC cycle, and they stop working.
> >
>
=== message truncated ===
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
RE: Potential memory leak with Threaded model and Events listeners
Posted by James Carman <ja...@carmanconsulting.com>.
I personally don't even like the automatic event listener registration
facility provided by BuilderFactory, but maybe that's just me. Anyway, what
if people actually do want each created instance (in a pool) to receive the
event notification? Do we force those folks to do it manually (as you
explained in your workaround)? Which case do you think would be the
majority (I would have no idea, really)?
If I had time to write it, I'd try to come up with an event dispatching
facility which was tightly integrated with HiveMind's lifecycle management
(knows about discard, passivate, and activate). First, I'd create a
mechanism where you could register for lifecycle events in HiveMind...
// A new service in HiveMind...
public interface LifecycleManager
{
public void addLifecycleListener( LifecycleListener l );
public void removeLifecycleListener( LifecycleListener l );
}
public interface LifecycleListener
{
public void objectPassivated( Object o );
public void objectActivated( Object o );
public void objectDiscarded( Object o );
// public void objectCreated( Object o ); ?
}
Maybe anything implementing LifecycleListener would automatically be
registered, but then again, maybe not. Anyway, then I'd create an event
dispatcher service...
public interface EventDispatcher
{
public void registerEventListener( String topicId, Object listener );
public void deregisterEventListener( String topicId, Object listener );
public Object createListenerProxy( String topicId, Class listenerInterface
);
public Object createListenerProxy( String topicId, Class
listenerInterface, boolean disregardPassivatedInstances );
}
I chose to call them "topics" because I am loosely modeling this idea on the
notion of JMS topics. The implementation of MessageDispatcher would
implement the LifecycleListener interface and be registered with the
LifecycleManager. Here's how you would use it manually...
public class MyServiceImpl implements MyService
{
private MyEventListener listenerProxy;
private EventDispatcher dispatcher; // To be injected!
public void init()
{
listenerProxy = ( MyEventListener )dispatcher.createListenerProxy(
"MyService", MyEventListener.class, true );
}
public void someMethod()
{
listenerProxy.someMethodInvoked( new MyEvent( this ) );
}
}
public class MyOtherServiceImpl implements MyOtherService, MyEventListener
{
private EventDispatcher dispatcher;
public void init()
{
dispatcher.registerEventListener( "MyService", this );
}
public void someMethodInvoked( MyEvent e )
{
}
}
Here, the implementation of MyService has decided that it wants to be able
to "publish" events to the "MyService" topic via the MyEventListener
interface (note, multiple listener interfaces could be used on the same
topic). So, it asks the EventDispatcher to create a proxy object which
implements the listener interface (and also skips over passivated objects).
When it wants to "publish" an event, all it has to do is call a method on
the proxy object and all registered listeners will receive the event
notification! On the other side of things, the implementation of
MyOtherService has decided that it would like to listen for events via the
MyEventListener interface on the "MyService" topic. Of course, I can see
easily how we could extend the hivemodule.xml file to facilitate this
automagically...
<set-listener-proxy property="listenerProxy" topicId="MyService"
listenerInterface="com.myco.MyEventListener" skipPassivated="true" />
<register-listener topicId="MyService" />
These would be new additions of BuilderFactory's schema for creating
listener proxies. One cool thing about this is that you don't have to
register to receive events on each and every source of the event. As a
matter of fact, you don't even necessarily have to know where they're coming
from. All you need to know is the topicId and implement the appropriate
listener interface to receive events! I might want to add in a new method
for registering an event listener where you can specify exactly which
listener interfaces you wish to use to receive events on the specified
topic, but that may be overkill for most situations. We would also have to
agree on what the default behavior is with respect to skipping passivated
listeners or always make the user specify what they want. Anyway, so much
for checking my email quickly and heading out the door! Hope I didn't use
up too much of your allotted email space!
-----Original Message-----
From: Jean-Francois Poilpret [mailto:jfpoilpret@hcm.vnn.vn]
Sent: Saturday, December 04, 2004 7:34 PM
To: hivemind-user@jakarta.apache.org
Subject: RE: Potential memory leak with Threaded model and Events listeners
No, I don't want each __created__ instance to receive notifications, I just
want each __active__ instance to receive them, and any __inactive___
instance not to receive them.
Currently with either threaded or pooled model (as far as I can see), all
created instances actually receive notifications, even when inactive. Worse,
for threaded model, inactive instances never get garbage-collected because
they are strongly referenced by the service which fire events.
For me, the expected behavior of HiveMind would be:
- when a service instance gets created (threaded) or activated (pooled),
then it should be added as a listener to the service firing events
- when a service instance gets discarded (threaded) or deactivated (pooled),
then it should be removed as a listener from the service supplying events.
NB: this description applies to the case where the event-firing service is a
singleton (my situation right now), but the behavior would/should probably
be different if that service used a different model.
Currently, the best work-around is:
- for threaded model, the listening service must implement Discardable and
remove itself from the listeners list of the event firing service, when its
threadDidDiscardService() method is called. The registration as a listener
can either be done by configuration (BuilderFactory) or in the constructor.
- for pooled model, the listening service must implement PoolManageable and
it must register itself as a listener in its XXXX methods, and unregister
itself when its XXXX() method is called. Here automatic Event registration
through BuilderFactory is useless.
These work-arounds work well, but they are not very "clean" in the sense
that this is code the end-developer has to write that HiveMind could
directly handle for him.
In addition, in the threaded situation, if we use the BuilderFactory to wire
the listener to the event supplier, then the work-around introduces some
asymmetry (registration in config, unregistration in the code), making the
use of BuilderFactory for this aspect useless.
I think before changing anything in HiveMind, we (all concerned developers)
should discuss the various situations (models for supplier/consumer:
singleton/threaded, singleton/pooled, threaded/singleton,
threaded/threaded...) and specify for each what the behavior should be, then
the actual coding can start. Should we start a new page on the WIKI?
Cheers
-----Original Message-----
From: Howard Lewis Ship [mailto:hlship@gmail.com]
Sent: Sunday, December 05, 2004 4:09 AM
To: hivemind-user@jakarta.apache.org
Subject: Re: Potential memory leak with Threaded model and Events listeners
If you want each created instance to receive the notification, use the
pooled, not threaded, service model. Pooled instances exist
indefinately once created. The only down side is that code related to
the event needs to be thread safe, and that deactivated instances in
the pool will also receive notifications.
On Sun, 5 Dec 2004 00:18:27 +0700, Jean-Francois Poilpret
<jf...@hcm.vnn.vn> wrote:
> Hi Luke,
>
> Thank you for your remark.
> Although your remark is correct in many cases of Listeners, I think it
would
> generally not apply in HiveMind context, I mean (if people follow the
"right
> path") that normally, only a service will implement a Listener interface,
> thus at least HiveMind itself (through one of the generated proxies) will
> hold a strong reference to that service/listener. But of course, if
someone
> decides to create a Listener on-the-fly and add it "by hand" to the event
> supplier service, then the problem you described would occur.
>
> Anyway, I just did some tests (with strong and with weak references) and
in
> fact, the result is not "as good" as I expected initially. Indeed, when
> suggesting that idea, I simply forgot that the handling of weak references
> is totally non-deterministic, ie the weak ref could still be valid even
> though there is no more existing strong ref. The time at which the weak
ref
> would be cleaned is totally up to the GC...
> So as my test have shown me, using weak refs improve the issue, but they
do
> not solve it really:-(
>
> In my tests, I did something like that:
> - create EventSupplier service (singleton)
> - create EventConsumer service (threaded)
> - instantiate EventConsumer in 1000 "threads" (they are real threads but
> they do not run concurrently, but I force consecutive runs, with a call to
> Registry.cleanupThread() at the end of each thread)
> - each thread call the EventSupplier to generate a new event
> - all calls are traced
> - each EventConsumer instance allocates 10KB on the heap and never release
> them.
>
> With strong refs:
> - the 1000th thread sends an event that is received by 1000 EventConsumer
> instances
>
> With weak refs:
> - the 1000th thread sends an event that is received by about 30
> EventConsumer instances.
>
> Of course that is much better, but the ideal, here in my testcase, would
be
> always just one EventConsumer instance receiving the event.
>
> So finally, maybe I will stick with the previous -not very clean from
design
> viewpoint- workaround:
> - have EventConsumer implement Discardable and remove itself from the
> EventSupplier list of listeners, when threadDidDiscardService() is called.
>
> And then I will eagerly wait for HM1.1 release (hope it will fix this
> issue).
>
> Cheers
>
>
>
> -----Original Message-----
> From: Luke Blanshard [mailto:luke@blanshard.us]
> Sent: Saturday, December 04, 2004 8:13 PM
> To: hivemind-user@jakarta.apache.org
> Subject: Re: Potential memory leak with Threaded model and Events
listeners
>
> Jean-Francois,
>
> The problem with holding listeners weakly is that people commonly create
> listener objects on the fly and then retain no other references to them
> once they've added them. So these listeners are thrown away on the next
> GC cycle, and they stop working.
>
> One solution to this is to have a tagging interface (we call ours
> WeakListener) that tells the event source that it should hold the
> listener weakly. All listeners that don't implement this interface are
> held strongly. This way, you as the implementer of the listener are
> telling the event source whether to hold the listener weakly (meaning
> you better have a strong reference to it) or strongly (meaning you don't
> have to). As you say, it's pretty easy to write a listeners collection
> that honors this WeakListener interface and throws away the expired
> listeners when you ask for the current list of listeners. (We don't use
> an iterator, but rather a method that returns an array of listeners, for
> thread safety reasons.)
>
> Luke
>
> Jean-Francois Poilpret wrote:
>
> >...
> >To Howard:
> >During the night (French people say that the "night gives good
advice";-)),
> >I thought of a quite easy workaround, that can be done at the event
> >supplying service level: just use a weak reference to the listeners
> (instead
> >of a strong one), I think that should do the trick.
> >Maybe it would even be possible to write a general "WeakEventListeners"
> >class to manage all that for the event supplier:
> >- when addListener, is called generate a weak reference to the added
> >listener and store it in the list
> >- create a specific Iterator for WeakEventListeners that would skip the
> refs
> >that have become weak-reachable (and remove these from its list at the
same
> >time).
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
>
>
--
Howard M. Lewis Ship
Independent J2EE / Open-Source Java Consultant
Creator, Jakarta Tapestry
Creator, Jakarta HiveMind
http://howardlewisship.com
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
RE: Erratum: Potential memory leak with Threaded model and Events listeners
Posted by Jean-Francois Poilpret <jf...@hcm.vnn.vn>.
Whoops,
In my previous mail, replace both appearances of XXXX with
activateService/deactivateService respectively. I pushed the "send" button
too eagerly;-)
-----Original Message-----
From: Jean-Francois Poilpret [mailto:jfpoilpret@hcm.vnn.vn]
Sent: Sunday, December 05, 2004 7:34 AM
To: hivemind-user@jakarta.apache.org
Subject: RE: Potential memory leak with Threaded model and Events listeners
No, I don't want each __created__ instance to receive notifications, I just
want each __active__ instance to receive them, and any __inactive___
instance not to receive them.
Currently with either threaded or pooled model (as far as I can see), all
created instances actually receive notifications, even when inactive. Worse,
for threaded model, inactive instances never get garbage-collected because
they are strongly referenced by the service which fire events.
For me, the expected behavior of HiveMind would be:
- when a service instance gets created (threaded) or activated (pooled),
then it should be added as a listener to the service firing events
- when a service instance gets discarded (threaded) or deactivated (pooled),
then it should be removed as a listener from the service supplying events.
NB: this description applies to the case where the event-firing service is a
singleton (my situation right now), but the behavior would/should probably
be different if that service used a different model.
Currently, the best work-around is:
- for threaded model, the listening service must implement Discardable and
remove itself from the listeners list of the event firing service, when its
threadDidDiscardService() method is called. The registration as a listener
can either be done by configuration (BuilderFactory) or in the constructor.
- for pooled model, the listening service must implement PoolManageable and
it must register itself as a listener in its XXXX methods, and unregister
itself when its XXXX() method is called. Here automatic Event registration
through BuilderFactory is useless.
These work-arounds work well, but they are not very "clean" in the sense
that this is code the end-developer has to write that HiveMind could
directly handle for him.
In addition, in the threaded situation, if we use the BuilderFactory to wire
the listener to the event supplier, then the work-around introduces some
asymmetry (registration in config, unregistration in the code), making the
use of BuilderFactory for this aspect useless.
I think before changing anything in HiveMind, we (all concerned developers)
should discuss the various situations (models for supplier/consumer:
singleton/threaded, singleton/pooled, threaded/singleton,
threaded/threaded...) and specify for each what the behavior should be, then
the actual coding can start. Should we start a new page on the WIKI?
Cheers
-----Original Message-----
From: Howard Lewis Ship [mailto:hlship@gmail.com]
Sent: Sunday, December 05, 2004 4:09 AM
To: hivemind-user@jakarta.apache.org
Subject: Re: Potential memory leak with Threaded model and Events listeners
If you want each created instance to receive the notification, use the
pooled, not threaded, service model. Pooled instances exist
indefinately once created. The only down side is that code related to
the event needs to be thread safe, and that deactivated instances in
the pool will also receive notifications.
On Sun, 5 Dec 2004 00:18:27 +0700, Jean-Francois Poilpret
<jf...@hcm.vnn.vn> wrote:
> Hi Luke,
>
> Thank you for your remark.
> Although your remark is correct in many cases of Listeners, I think it
would
> generally not apply in HiveMind context, I mean (if people follow the
"right
> path") that normally, only a service will implement a Listener interface,
> thus at least HiveMind itself (through one of the generated proxies) will
> hold a strong reference to that service/listener. But of course, if
someone
> decides to create a Listener on-the-fly and add it "by hand" to the event
> supplier service, then the problem you described would occur.
>
> Anyway, I just did some tests (with strong and with weak references) and
in
> fact, the result is not "as good" as I expected initially. Indeed, when
> suggesting that idea, I simply forgot that the handling of weak references
> is totally non-deterministic, ie the weak ref could still be valid even
> though there is no more existing strong ref. The time at which the weak
ref
> would be cleaned is totally up to the GC...
> So as my test have shown me, using weak refs improve the issue, but they
do
> not solve it really:-(
>
> In my tests, I did something like that:
> - create EventSupplier service (singleton)
> - create EventConsumer service (threaded)
> - instantiate EventConsumer in 1000 "threads" (they are real threads but
> they do not run concurrently, but I force consecutive runs, with a call to
> Registry.cleanupThread() at the end of each thread)
> - each thread call the EventSupplier to generate a new event
> - all calls are traced
> - each EventConsumer instance allocates 10KB on the heap and never release
> them.
>
> With strong refs:
> - the 1000th thread sends an event that is received by 1000 EventConsumer
> instances
>
> With weak refs:
> - the 1000th thread sends an event that is received by about 30
> EventConsumer instances.
>
> Of course that is much better, but the ideal, here in my testcase, would
be
> always just one EventConsumer instance receiving the event.
>
> So finally, maybe I will stick with the previous -not very clean from
design
> viewpoint- workaround:
> - have EventConsumer implement Discardable and remove itself from the
> EventSupplier list of listeners, when threadDidDiscardService() is called.
>
> And then I will eagerly wait for HM1.1 release (hope it will fix this
> issue).
>
> Cheers
>
>
>
> -----Original Message-----
> From: Luke Blanshard [mailto:luke@blanshard.us]
> Sent: Saturday, December 04, 2004 8:13 PM
> To: hivemind-user@jakarta.apache.org
> Subject: Re: Potential memory leak with Threaded model and Events
listeners
>
> Jean-Francois,
>
> The problem with holding listeners weakly is that people commonly create
> listener objects on the fly and then retain no other references to them
> once they've added them. So these listeners are thrown away on the next
> GC cycle, and they stop working.
>
> One solution to this is to have a tagging interface (we call ours
> WeakListener) that tells the event source that it should hold the
> listener weakly. All listeners that don't implement this interface are
> held strongly. This way, you as the implementer of the listener are
> telling the event source whether to hold the listener weakly (meaning
> you better have a strong reference to it) or strongly (meaning you don't
> have to). As you say, it's pretty easy to write a listeners collection
> that honors this WeakListener interface and throws away the expired
> listeners when you ask for the current list of listeners. (We don't use
> an iterator, but rather a method that returns an array of listeners, for
> thread safety reasons.)
>
> Luke
>
> Jean-Francois Poilpret wrote:
>
> >...
> >To Howard:
> >During the night (French people say that the "night gives good
advice";-)),
> >I thought of a quite easy workaround, that can be done at the event
> >supplying service level: just use a weak reference to the listeners
> (instead
> >of a strong one), I think that should do the trick.
> >Maybe it would even be possible to write a general "WeakEventListeners"
> >class to manage all that for the event supplier:
> >- when addListener, is called generate a weak reference to the added
> >listener and store it in the list
> >- create a specific Iterator for WeakEventListeners that would skip the
> refs
> >that have become weak-reachable (and remove these from its list at the
same
> >time).
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
>
>
--
Howard M. Lewis Ship
Independent J2EE / Open-Source Java Consultant
Creator, Jakarta Tapestry
Creator, Jakarta HiveMind
http://howardlewisship.com
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
RE: Potential memory leak with Threaded model and Events listeners
Posted by Jean-Francois Poilpret <jf...@hcm.vnn.vn>.
No, I don't want each __created__ instance to receive notifications, I just
want each __active__ instance to receive them, and any __inactive___
instance not to receive them.
Currently with either threaded or pooled model (as far as I can see), all
created instances actually receive notifications, even when inactive. Worse,
for threaded model, inactive instances never get garbage-collected because
they are strongly referenced by the service which fire events.
For me, the expected behavior of HiveMind would be:
- when a service instance gets created (threaded) or activated (pooled),
then it should be added as a listener to the service firing events
- when a service instance gets discarded (threaded) or deactivated (pooled),
then it should be removed as a listener from the service supplying events.
NB: this description applies to the case where the event-firing service is a
singleton (my situation right now), but the behavior would/should probably
be different if that service used a different model.
Currently, the best work-around is:
- for threaded model, the listening service must implement Discardable and
remove itself from the listeners list of the event firing service, when its
threadDidDiscardService() method is called. The registration as a listener
can either be done by configuration (BuilderFactory) or in the constructor.
- for pooled model, the listening service must implement PoolManageable and
it must register itself as a listener in its XXXX methods, and unregister
itself when its XXXX() method is called. Here automatic Event registration
through BuilderFactory is useless.
These work-arounds work well, but they are not very "clean" in the sense
that this is code the end-developer has to write that HiveMind could
directly handle for him.
In addition, in the threaded situation, if we use the BuilderFactory to wire
the listener to the event supplier, then the work-around introduces some
asymmetry (registration in config, unregistration in the code), making the
use of BuilderFactory for this aspect useless.
I think before changing anything in HiveMind, we (all concerned developers)
should discuss the various situations (models for supplier/consumer:
singleton/threaded, singleton/pooled, threaded/singleton,
threaded/threaded...) and specify for each what the behavior should be, then
the actual coding can start. Should we start a new page on the WIKI?
Cheers
-----Original Message-----
From: Howard Lewis Ship [mailto:hlship@gmail.com]
Sent: Sunday, December 05, 2004 4:09 AM
To: hivemind-user@jakarta.apache.org
Subject: Re: Potential memory leak with Threaded model and Events listeners
If you want each created instance to receive the notification, use the
pooled, not threaded, service model. Pooled instances exist
indefinately once created. The only down side is that code related to
the event needs to be thread safe, and that deactivated instances in
the pool will also receive notifications.
On Sun, 5 Dec 2004 00:18:27 +0700, Jean-Francois Poilpret
<jf...@hcm.vnn.vn> wrote:
> Hi Luke,
>
> Thank you for your remark.
> Although your remark is correct in many cases of Listeners, I think it
would
> generally not apply in HiveMind context, I mean (if people follow the
"right
> path") that normally, only a service will implement a Listener interface,
> thus at least HiveMind itself (through one of the generated proxies) will
> hold a strong reference to that service/listener. But of course, if
someone
> decides to create a Listener on-the-fly and add it "by hand" to the event
> supplier service, then the problem you described would occur.
>
> Anyway, I just did some tests (with strong and with weak references) and
in
> fact, the result is not "as good" as I expected initially. Indeed, when
> suggesting that idea, I simply forgot that the handling of weak references
> is totally non-deterministic, ie the weak ref could still be valid even
> though there is no more existing strong ref. The time at which the weak
ref
> would be cleaned is totally up to the GC...
> So as my test have shown me, using weak refs improve the issue, but they
do
> not solve it really:-(
>
> In my tests, I did something like that:
> - create EventSupplier service (singleton)
> - create EventConsumer service (threaded)
> - instantiate EventConsumer in 1000 "threads" (they are real threads but
> they do not run concurrently, but I force consecutive runs, with a call to
> Registry.cleanupThread() at the end of each thread)
> - each thread call the EventSupplier to generate a new event
> - all calls are traced
> - each EventConsumer instance allocates 10KB on the heap and never release
> them.
>
> With strong refs:
> - the 1000th thread sends an event that is received by 1000 EventConsumer
> instances
>
> With weak refs:
> - the 1000th thread sends an event that is received by about 30
> EventConsumer instances.
>
> Of course that is much better, but the ideal, here in my testcase, would
be
> always just one EventConsumer instance receiving the event.
>
> So finally, maybe I will stick with the previous -not very clean from
design
> viewpoint- workaround:
> - have EventConsumer implement Discardable and remove itself from the
> EventSupplier list of listeners, when threadDidDiscardService() is called.
>
> And then I will eagerly wait for HM1.1 release (hope it will fix this
> issue).
>
> Cheers
>
>
>
> -----Original Message-----
> From: Luke Blanshard [mailto:luke@blanshard.us]
> Sent: Saturday, December 04, 2004 8:13 PM
> To: hivemind-user@jakarta.apache.org
> Subject: Re: Potential memory leak with Threaded model and Events
listeners
>
> Jean-Francois,
>
> The problem with holding listeners weakly is that people commonly create
> listener objects on the fly and then retain no other references to them
> once they've added them. So these listeners are thrown away on the next
> GC cycle, and they stop working.
>
> One solution to this is to have a tagging interface (we call ours
> WeakListener) that tells the event source that it should hold the
> listener weakly. All listeners that don't implement this interface are
> held strongly. This way, you as the implementer of the listener are
> telling the event source whether to hold the listener weakly (meaning
> you better have a strong reference to it) or strongly (meaning you don't
> have to). As you say, it's pretty easy to write a listeners collection
> that honors this WeakListener interface and throws away the expired
> listeners when you ask for the current list of listeners. (We don't use
> an iterator, but rather a method that returns an array of listeners, for
> thread safety reasons.)
>
> Luke
>
> Jean-Francois Poilpret wrote:
>
> >...
> >To Howard:
> >During the night (French people say that the "night gives good
advice";-)),
> >I thought of a quite easy workaround, that can be done at the event
> >supplying service level: just use a weak reference to the listeners
> (instead
> >of a strong one), I think that should do the trick.
> >Maybe it would even be possible to write a general "WeakEventListeners"
> >class to manage all that for the event supplier:
> >- when addListener, is called generate a weak reference to the added
> >listener and store it in the list
> >- create a specific Iterator for WeakEventListeners that would skip the
> refs
> >that have become weak-reachable (and remove these from its list at the
same
> >time).
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
>
>
--
Howard M. Lewis Ship
Independent J2EE / Open-Source Java Consultant
Creator, Jakarta Tapestry
Creator, Jakarta HiveMind
http://howardlewisship.com
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
Re: Potential memory leak with Threaded model and Events listeners
Posted by Howard Lewis Ship <hl...@gmail.com>.
If you want each created instance to receive the notification, use the
pooled, not threaded, service model. Pooled instances exist
indefinately once created. The only down side is that code related to
the event needs to be thread safe, and that deactivated instances in
the pool will also receive notifications.
On Sun, 5 Dec 2004 00:18:27 +0700, Jean-Francois Poilpret
<jf...@hcm.vnn.vn> wrote:
> Hi Luke,
>
> Thank you for your remark.
> Although your remark is correct in many cases of Listeners, I think it would
> generally not apply in HiveMind context, I mean (if people follow the "right
> path") that normally, only a service will implement a Listener interface,
> thus at least HiveMind itself (through one of the generated proxies) will
> hold a strong reference to that service/listener. But of course, if someone
> decides to create a Listener on-the-fly and add it "by hand" to the event
> supplier service, then the problem you described would occur.
>
> Anyway, I just did some tests (with strong and with weak references) and in
> fact, the result is not "as good" as I expected initially. Indeed, when
> suggesting that idea, I simply forgot that the handling of weak references
> is totally non-deterministic, ie the weak ref could still be valid even
> though there is no more existing strong ref. The time at which the weak ref
> would be cleaned is totally up to the GC...
> So as my test have shown me, using weak refs improve the issue, but they do
> not solve it really:-(
>
> In my tests, I did something like that:
> - create EventSupplier service (singleton)
> - create EventConsumer service (threaded)
> - instantiate EventConsumer in 1000 "threads" (they are real threads but
> they do not run concurrently, but I force consecutive runs, with a call to
> Registry.cleanupThread() at the end of each thread)
> - each thread call the EventSupplier to generate a new event
> - all calls are traced
> - each EventConsumer instance allocates 10KB on the heap and never release
> them.
>
> With strong refs:
> - the 1000th thread sends an event that is received by 1000 EventConsumer
> instances
>
> With weak refs:
> - the 1000th thread sends an event that is received by about 30
> EventConsumer instances.
>
> Of course that is much better, but the ideal, here in my testcase, would be
> always just one EventConsumer instance receiving the event.
>
> So finally, maybe I will stick with the previous -not very clean from design
> viewpoint- workaround:
> - have EventConsumer implement Discardable and remove itself from the
> EventSupplier list of listeners, when threadDidDiscardService() is called.
>
> And then I will eagerly wait for HM1.1 release (hope it will fix this
> issue).
>
> Cheers
>
>
>
> -----Original Message-----
> From: Luke Blanshard [mailto:luke@blanshard.us]
> Sent: Saturday, December 04, 2004 8:13 PM
> To: hivemind-user@jakarta.apache.org
> Subject: Re: Potential memory leak with Threaded model and Events listeners
>
> Jean-Francois,
>
> The problem with holding listeners weakly is that people commonly create
> listener objects on the fly and then retain no other references to them
> once they've added them. So these listeners are thrown away on the next
> GC cycle, and they stop working.
>
> One solution to this is to have a tagging interface (we call ours
> WeakListener) that tells the event source that it should hold the
> listener weakly. All listeners that don't implement this interface are
> held strongly. This way, you as the implementer of the listener are
> telling the event source whether to hold the listener weakly (meaning
> you better have a strong reference to it) or strongly (meaning you don't
> have to). As you say, it's pretty easy to write a listeners collection
> that honors this WeakListener interface and throws away the expired
> listeners when you ask for the current list of listeners. (We don't use
> an iterator, but rather a method that returns an array of listeners, for
> thread safety reasons.)
>
> Luke
>
> Jean-Francois Poilpret wrote:
>
> >...
> >To Howard:
> >During the night (French people say that the "night gives good advice";-)),
> >I thought of a quite easy workaround, that can be done at the event
> >supplying service level: just use a weak reference to the listeners
> (instead
> >of a strong one), I think that should do the trick.
> >Maybe it would even be possible to write a general "WeakEventListeners"
> >class to manage all that for the event supplier:
> >- when addListener, is called generate a weak reference to the added
> >listener and store it in the list
> >- create a specific Iterator for WeakEventListeners that would skip the
> refs
> >that have become weak-reachable (and remove these from its list at the same
> >time).
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
>
>
--
Howard M. Lewis Ship
Independent J2EE / Open-Source Java Consultant
Creator, Jakarta Tapestry
Creator, Jakarta HiveMind
http://howardlewisship.com
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
RE: Potential memory leak with Threaded model and Events listeners
Posted by Jean-Francois Poilpret <jf...@hcm.vnn.vn>.
Hi Luke,
Thank you for your remark.
Although your remark is correct in many cases of Listeners, I think it would
generally not apply in HiveMind context, I mean (if people follow the "right
path") that normally, only a service will implement a Listener interface,
thus at least HiveMind itself (through one of the generated proxies) will
hold a strong reference to that service/listener. But of course, if someone
decides to create a Listener on-the-fly and add it "by hand" to the event
supplier service, then the problem you described would occur.
Anyway, I just did some tests (with strong and with weak references) and in
fact, the result is not "as good" as I expected initially. Indeed, when
suggesting that idea, I simply forgot that the handling of weak references
is totally non-deterministic, ie the weak ref could still be valid even
though there is no more existing strong ref. The time at which the weak ref
would be cleaned is totally up to the GC...
So as my test have shown me, using weak refs improve the issue, but they do
not solve it really:-(
In my tests, I did something like that:
- create EventSupplier service (singleton)
- create EventConsumer service (threaded)
- instantiate EventConsumer in 1000 "threads" (they are real threads but
they do not run concurrently, but I force consecutive runs, with a call to
Registry.cleanupThread() at the end of each thread)
- each thread call the EventSupplier to generate a new event
- all calls are traced
- each EventConsumer instance allocates 10KB on the heap and never release
them.
With strong refs:
- the 1000th thread sends an event that is received by 1000 EventConsumer
instances
With weak refs:
- the 1000th thread sends an event that is received by about 30
EventConsumer instances.
Of course that is much better, but the ideal, here in my testcase, would be
always just one EventConsumer instance receiving the event.
So finally, maybe I will stick with the previous -not very clean from design
viewpoint- workaround:
- have EventConsumer implement Discardable and remove itself from the
EventSupplier list of listeners, when threadDidDiscardService() is called.
And then I will eagerly wait for HM1.1 release (hope it will fix this
issue).
Cheers
-----Original Message-----
From: Luke Blanshard [mailto:luke@blanshard.us]
Sent: Saturday, December 04, 2004 8:13 PM
To: hivemind-user@jakarta.apache.org
Subject: Re: Potential memory leak with Threaded model and Events listeners
Jean-Francois,
The problem with holding listeners weakly is that people commonly create
listener objects on the fly and then retain no other references to them
once they've added them. So these listeners are thrown away on the next
GC cycle, and they stop working.
One solution to this is to have a tagging interface (we call ours
WeakListener) that tells the event source that it should hold the
listener weakly. All listeners that don't implement this interface are
held strongly. This way, you as the implementer of the listener are
telling the event source whether to hold the listener weakly (meaning
you better have a strong reference to it) or strongly (meaning you don't
have to). As you say, it's pretty easy to write a listeners collection
that honors this WeakListener interface and throws away the expired
listeners when you ask for the current list of listeners. (We don't use
an iterator, but rather a method that returns an array of listeners, for
thread safety reasons.)
Luke
Jean-Francois Poilpret wrote:
>...
>To Howard:
>During the night (French people say that the "night gives good advice";-)),
>I thought of a quite easy workaround, that can be done at the event
>supplying service level: just use a weak reference to the listeners
(instead
>of a strong one), I think that should do the trick.
>Maybe it would even be possible to write a general "WeakEventListeners"
>class to manage all that for the event supplier:
>- when addListener, is called generate a weak reference to the added
>listener and store it in the list
>- create a specific Iterator for WeakEventListeners that would skip the
refs
>that have become weak-reachable (and remove these from its list at the same
>time).
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
Re: Potential memory leak with Threaded model and Events listeners
Posted by Luke Blanshard <lu...@blanshard.us>.
Jean-Francois,
The problem with holding listeners weakly is that people commonly create
listener objects on the fly and then retain no other references to them
once they've added them. So these listeners are thrown away on the next
GC cycle, and they stop working.
One solution to this is to have a tagging interface (we call ours
WeakListener) that tells the event source that it should hold the
listener weakly. All listeners that don't implement this interface are
held strongly. This way, you as the implementer of the listener are
telling the event source whether to hold the listener weakly (meaning
you better have a strong reference to it) or strongly (meaning you don't
have to). As you say, it's pretty easy to write a listeners collection
that honors this WeakListener interface and throws away the expired
listeners when you ask for the current list of listeners. (We don't use
an iterator, but rather a method that returns an array of listeners, for
thread safety reasons.)
Luke
Jean-Francois Poilpret wrote:
>...
>To Howard:
>During the night (French people say that the "night gives good advice";-)),
>I thought of a quite easy workaround, that can be done at the event
>supplying service level: just use a weak reference to the listeners (instead
>of a strong one), I think that should do the trick.
>Maybe it would even be possible to write a general "WeakEventListeners"
>class to manage all that for the event supplier:
>- when addListener, is called generate a weak reference to the added
>listener and store it in the list
>- create a specific Iterator for WeakEventListeners that would skip the refs
>that have become weak-reachable (and remove these from its list at the same
>time).
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
RE: Potential memory leak with Threaded model and Events listeners
Posted by Jean-Francois Poilpret <jf...@hcm.vnn.vn>.
To James:
- I am using the BuilderFactory to do it for me through hivemind.xml
configuration.
To Howard:
During the night (French people say that the "night gives good advice";-)),
I thought of a quite easy workaround, that can be done at the event
supplying service level: just use a weak reference to the listeners (instead
of a strong one), I think that should do the trick.
Maybe it would even be possible to write a general "WeakEventListeners"
class to manage all that for the event supplier:
- when addListener, is called generate a weak reference to the added
listener and store it in the list
- create a specific Iterator for WeakEventListeners that would skip the refs
that have become weak-reachable (and remove these from its list at the same
time).
Don't you think it would be a better solution than to make the current
proxies more complex?
I will try to give it a try this week (but only after writing a simple test
case that exhibits the leakage problem).
Regarding the solution you have in mind, there is something that puzzles me:
You say that the proxy would delegate the EventListener methods calls to the
__thread-local__ service implementation. I am not sure this is the way we
want it (or at least that I need it in my current application):
- when I register to an Event supplier service, I want __all__ my
implementations to be notified when an event occurs, independently of which
thread originates the event, and not only the one implementation that is in
the same thread as the event supplier.
Cheers
Jean-Francois
-----Original Message-----
From: Howard Lewis Ship [mailto:hlship@gmail.com]
Sent: Saturday, December 04, 2004 12:08 AM
To: hivemind-user@jakarta.apache.org
Subject: Re: Potential memory leak with Threaded model and Events listeners
This is something I am aware of. I have a solution in mind, but
haven't had time to implement it.
Basically, for the threaded and pooled service models, a new type of
proxy must be created, one that implements the necessary listener
interfaces and delegates to the (thread-local) service implementation.
This will require a bit of refactoring, a tighter collaboration
between BuilderFactory and ServiceModel.
As a workaround: have the event source injected into the listener
service implementation, and have the listener service impl manage
itself: registrer for events inside activateService() or
initializeService(), unregister inside passivateService() or
threadDidDiscardService().
On Fri, 3 Dec 2004 23:42:33 +0700, Jean-Francois Poilpret
<jf...@hcm.vnn.vn> wrote:
> Hello,
>
> I was wondering about one potential problem (I did not test it but I
believe
> it happens and I wonder about whether it could be fixed in HiveMind should
> be worked around in applications):
>
> - let's suppose I have a service A that supplies events
> - let's suppose I have a service B that listens to those events, and that
> uses the threaded model
> - when a new thread is executed and first time service B is used by this
> thread, a new instance of B is created for the thread, and it is added as
a
> listener to events produced by service A: hence A retains a reference to
the
> actual implementation of B for the thread
> - when the thread is cleaned up, then that service B instance ceases to
> exist (it has been "discarded") and nothing refers to it anymore...
> - ...nothing except service A that still holds a reference to that
instance
> (in its list of listeners)
>
> This means that in this situation if I have a huge amount of threads
during
> my application's lifecycle, then a huge amount of service B instances will
> be created and never garbage collected.
> In addition, whenever service A produces an event, it will dispatch it to
> this huge amount of "zombie" instances, reducing performance a lot.
>
> Is my reasoning correct? If necessary, I can try to write a simple test
case
> to produce this problem.
> If this is correct, what should be the correct way to fix the problem?
> - in HiveMind? Actually, I do not see really where (the problem in HM1.0
is
> that the link of the Event listener to the producer is done by the
Factory,
> but only the ServiceModel manages the full lifecycle of the service
> instance, and it knows nothing about what the factory did).
> I remember some time ago, Howard discussed the idea to change the
> responsibilities of ServiceModels and Factories to make it clearer. Will
> this be done in HM1.1? Will this possibly solve that problem?
> - in the application itself? Yes it is possible to implement the
Discardable
> interface in the service B, and when the service is discarded then we can
> possibly call serviceA.removeEventListener(this). However, this means that
> the link that was configured dynamically through the module configuration
> has to be undone by some kind of hard-code, reducing the interest of using
> the BuilderFactory to link the event listener to the event supplier.
>
> Has somebody already met such kind of problem and could come out with some
> satisfactory solution?
>
> Some more info about why I ask: actually I am working on the "user"
service
> model that I described a few days ago (ie, one different instance for each
> connected user, discarded when user logs out), it works correctly until
now,
> but in one "real-life" application that I am currently working on and that
> needs to use it, I am faced with this kind of problem, so I wonder how I
can
> circumvent this.
>
> Thank you for any feedback
>
> Cheers
>
> Jean-Francois
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
>
>
--
Howard M. Lewis Ship
Independent J2EE / Open-Source Java Consultant
Creator, Jakarta Tapestry
Creator, Jakarta HiveMind
http://howardlewisship.com
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
Re: Potential memory leak with Threaded model and Events listeners
Posted by Howard Lewis Ship <hl...@gmail.com>.
This is something I am aware of. I have a solution in mind, but
haven't had time to implement it.
Basically, for the threaded and pooled service models, a new type of
proxy must be created, one that implements the necessary listener
interfaces and delegates to the (thread-local) service implementation.
This will require a bit of refactoring, a tighter collaboration
between BuilderFactory and ServiceModel.
As a workaround: have the event source injected into the listener
service implementation, and have the listener service impl manage
itself: registrer for events inside activateService() or
initializeService(), unregister inside passivateService() or
threadDidDiscardService().
On Fri, 3 Dec 2004 23:42:33 +0700, Jean-Francois Poilpret
<jf...@hcm.vnn.vn> wrote:
> Hello,
>
> I was wondering about one potential problem (I did not test it but I believe
> it happens and I wonder about whether it could be fixed in HiveMind should
> be worked around in applications):
>
> - let's suppose I have a service A that supplies events
> - let's suppose I have a service B that listens to those events, and that
> uses the threaded model
> - when a new thread is executed and first time service B is used by this
> thread, a new instance of B is created for the thread, and it is added as a
> listener to events produced by service A: hence A retains a reference to the
> actual implementation of B for the thread
> - when the thread is cleaned up, then that service B instance ceases to
> exist (it has been "discarded") and nothing refers to it anymore...
> - ...nothing except service A that still holds a reference to that instance
> (in its list of listeners)
>
> This means that in this situation if I have a huge amount of threads during
> my application's lifecycle, then a huge amount of service B instances will
> be created and never garbage collected.
> In addition, whenever service A produces an event, it will dispatch it to
> this huge amount of "zombie" instances, reducing performance a lot.
>
> Is my reasoning correct? If necessary, I can try to write a simple test case
> to produce this problem.
> If this is correct, what should be the correct way to fix the problem?
> - in HiveMind? Actually, I do not see really where (the problem in HM1.0 is
> that the link of the Event listener to the producer is done by the Factory,
> but only the ServiceModel manages the full lifecycle of the service
> instance, and it knows nothing about what the factory did).
> I remember some time ago, Howard discussed the idea to change the
> responsibilities of ServiceModels and Factories to make it clearer. Will
> this be done in HM1.1? Will this possibly solve that problem?
> - in the application itself? Yes it is possible to implement the Discardable
> interface in the service B, and when the service is discarded then we can
> possibly call serviceA.removeEventListener(this). However, this means that
> the link that was configured dynamically through the module configuration
> has to be undone by some kind of hard-code, reducing the interest of using
> the BuilderFactory to link the event listener to the event supplier.
>
> Has somebody already met such kind of problem and could come out with some
> satisfactory solution?
>
> Some more info about why I ask: actually I am working on the "user" service
> model that I described a few days ago (ie, one different instance for each
> connected user, discarded when user logs out), it works correctly until now,
> but in one "real-life" application that I am currently working on and that
> needs to use it, I am faced with this kind of problem, so I wonder how I can
> circumvent this.
>
> Thank you for any feedback
>
> Cheers
>
> Jean-Francois
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
>
>
--
Howard M. Lewis Ship
Independent J2EE / Open-Source Java Consultant
Creator, Jakarta Tapestry
Creator, Jakarta HiveMind
http://howardlewisship.com
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
RE: Potential memory leak with Threaded model and Events listeners
Posted by James Carman <ja...@carmanconsulting.com>.
How are you registering for service A's events?
-----Original Message-----
From: Jean-Francois Poilpret [mailto:jfpoilpret@hcm.vnn.vn]
Sent: Friday, December 03, 2004 11:43 AM
To: hivemind-user@jakarta.apache.org
Subject: Potential memory leak with Threaded model and Events listeners
Hello,
I was wondering about one potential problem (I did not test it but I believe
it happens and I wonder about whether it could be fixed in HiveMind should
be worked around in applications):
- let's suppose I have a service A that supplies events
- let's suppose I have a service B that listens to those events, and that
uses the threaded model
- when a new thread is executed and first time service B is used by this
thread, a new instance of B is created for the thread, and it is added as a
listener to events produced by service A: hence A retains a reference to the
actual implementation of B for the thread
- when the thread is cleaned up, then that service B instance ceases to
exist (it has been "discarded") and nothing refers to it anymore...
- ...nothing except service A that still holds a reference to that instance
(in its list of listeners)
This means that in this situation if I have a huge amount of threads during
my application's lifecycle, then a huge amount of service B instances will
be created and never garbage collected.
In addition, whenever service A produces an event, it will dispatch it to
this huge amount of "zombie" instances, reducing performance a lot.
Is my reasoning correct? If necessary, I can try to write a simple test case
to produce this problem.
If this is correct, what should be the correct way to fix the problem?
- in HiveMind? Actually, I do not see really where (the problem in HM1.0 is
that the link of the Event listener to the producer is done by the Factory,
but only the ServiceModel manages the full lifecycle of the service
instance, and it knows nothing about what the factory did).
I remember some time ago, Howard discussed the idea to change the
responsibilities of ServiceModels and Factories to make it clearer. Will
this be done in HM1.1? Will this possibly solve that problem?
- in the application itself? Yes it is possible to implement the Discardable
interface in the service B, and when the service is discarded then we can
possibly call serviceA.removeEventListener(this). However, this means that
the link that was configured dynamically through the module configuration
has to be undone by some kind of hard-code, reducing the interest of using
the BuilderFactory to link the event listener to the event supplier.
Has somebody already met such kind of problem and could come out with some
satisfactory solution?
Some more info about why I ask: actually I am working on the "user" service
model that I described a few days ago (ie, one different instance for each
connected user, discarded when user logs out), it works correctly until now,
but in one "real-life" application that I am currently working on and that
needs to use it, I am faced with this kind of problem, so I wonder how I can
circumvent this.
Thank you for any feedback
Cheers
Jean-Francois
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hivemind-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: hivemind-user-help@jakarta.apache.org