You are viewing a plain text version of this content. The canonical link for it is here.
Posted to ojb-user@db.apache.org by Pieter <pi...@gamesquare.nl> on 2005/02/09 10:46:51 UTC
Memory problem: PBStateListener[] keeps growing
Hello list readers,
I'm running into a memory problem with OJB on a long running server. After
a couple of weeks the heap has grown in size from +- 150 MB to over 500
MB. Today I used the "jmap" tool that comes with je JDK to actually dump a
heap histogram:
189072640 107 org.apache.ojb.broker.PBStateListener[]
109750848 4572952 java.lang.ref.WeakReference
73035968 4564748
org.apache.ojb.broker.accesslayer.RsIterator$ResourceWrapper
27267184 22219 byte[]
25734504 302777 char[]
7466688 311112 java.lang.String
7442584 57688 * ConstMethodKlass
4159032 57688 * MethodKlass
189 MB is taken up by just 107 PBStateListener arrays! If we add up the
majority of WeakReference and RsIterator$ResourceWrapper instances we end
up with a leak of more than 300 MB, just about the size of the unexpected
heap growth we have been experiencing.
After analysing the OJB source code, I found that the PBStateListener
management in PersistenceBrokerAbstractImpl might be suspect. The arrays
that store the listeners (permanent and temporary) only seem to be able to
grow. The "removeListener" method is never actually called, and
considering the inefficiency of the algorithms used to add and remove
listeners I suspect that these methods where never meant to manage large
amounts of active listeners. The "removeAllListeners" method only seem te
be called when closing and destroying the PersistenceBrokerImpl instance.
The "addListener" method is called on every query though, so the size of
the arrays after a couple of weeks does not surprise me.
WeakReferences are used to make sure no references to the actual
PBStateListener implementors are leaked, but the week references
themselves (and the wrappers around them) are never removed, thus causing
the PBStateListener arrays to keep growing.
If this analysis is correct, I don't understand why nobody else is having
this problem. I must be doing something radically different (more stupid?)
than most other OJB users. Is there some other way these arrays are
supposed to get freed that I'm not using because of a bug in my code? If
so could you point me in the right direction where I might begin looking
for problems?
As you can imagine this is a serious performance problem for us, so I
would very much appreciate any help or advice.
Regards,
Pieter.
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-user-help@db.apache.org
Re: Memory problem: PBStateListener[] keeps growing
Posted by Pieter <pi...@gamesquare.nl>.
Armin wrote:
> I think your analysis is correct and the listener show the expected
> behavior, see javadoc
>
> http://db.apache.org/ojb/api/org/apache/ojb/broker/PersistenceBroker.html#addListener(org.apache.ojb.broker.PBListener)
>
> the temporary listener will only be removed on the PB.close() call. It's
> strongly recommended to close each used PB instance immediately after
> use (or guarantee that it is happen sometime), because only in this case
> OJB is able to do all needed cleanup. All PB instances are pooled, so
> this procedure doesn't affect performance.
> Here is a usage example
> http://db.apache.org/ojb/docu/tutorials/pb-tutorial.html#A+First+Look+-+Persisting+New+Objects
>
> Did you use the PB-api in the same way?
Apparantly not ;-)
Seriously, I've studied the example you quote before I started coding, but
I interpreted it wrongly. I though I would be save if I requested a
PersistenceBroker only once and then reused it throughout my application
(as long as I used one instance per thread as I learned the hard way ;-).
> Why is it inefficient? Because we use System.arraycopy calls? How many
> active listener do you use between PB lookup and close?
There is no problem if I use the PB api the way you describe. I will fix
my application to close the broker after use and reopen an new one when
needed. This will probably even simplify my code because I do not need
ThreadLocal variables to keep track of a single PersistenceBroker object
per thread anymore.
Thanks for the tip and thanks for a great product! OJB has saved me
hundreds of hours of development time and helped me eradicate almost all
traces of SQL from my application ;-)
Regards,
Pieter.
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-user-help@db.apache.org
Re: Memory problem: PBStateListener[] keeps growing
Posted by Armin Waibel <ar...@apache.org>.
Hi Pieter,
> If this analysis is correct, I don't understand why nobody else is having
> this problem. I must be doing something radically different (more
stupid?)
> than most other OJB users. Is there some other way these arrays are
> supposed to get freed that I'm not using because of a bug in my code? If
> so could you point me in the right direction where I might begin looking
> for problems?
I think your analysis is correct and the listener show the expected
behavior, see javadoc
http://db.apache.org/ojb/api/org/apache/ojb/broker/PersistenceBroker.html#addListener(org.apache.ojb.broker.PBListener)
the temporary listener will only be removed on the PB.close() call. It's
strongly recommended to close each used PB instance immediately after
use (or guarantee that it is happen sometime), because only in this case
OJB is able to do all needed cleanup. All PB instances are pooled, so
this procedure doesn't affect performance.
Here is a usage example
http://db.apache.org/ojb/docu/tutorials/pb-tutorial.html#A+First+Look+-+Persisting+New+Objects
Did you use the PB-api in the same way?
> considering the inefficiency of the algorithms used to add and remove
> listeners I suspect that these methods where never meant to manage large
> amounts of active listeners.
Why is it inefficient? Because we use System.arraycopy calls? How many
active listener do you use between PB lookup and close?
regards,
Armin
Pieter wrote:
> Hello list readers,
>
> I'm running into a memory problem with OJB on a long running server. After
> a couple of weeks the heap has grown in size from +- 150 MB to over 500
> MB. Today I used the "jmap" tool that comes with je JDK to actually dump a
> heap histogram:
>
> 189072640 107 org.apache.ojb.broker.PBStateListener[]
> 109750848 4572952 java.lang.ref.WeakReference
> 73035968 4564748
> org.apache.ojb.broker.accesslayer.RsIterator$ResourceWrapper
> 27267184 22219 byte[]
> 25734504 302777 char[]
> 7466688 311112 java.lang.String
> 7442584 57688 * ConstMethodKlass
> 4159032 57688 * MethodKlass
>
> 189 MB is taken up by just 107 PBStateListener arrays! If we add up the
> majority of WeakReference and RsIterator$ResourceWrapper instances we end
> up with a leak of more than 300 MB, just about the size of the unexpected
> heap growth we have been experiencing.
>
> After analysing the OJB source code, I found that the PBStateListener
> management in PersistenceBrokerAbstractImpl might be suspect. The arrays
> that store the listeners (permanent and temporary) only seem to be able to
> grow. The "removeListener" method is never actually called, and
> considering the inefficiency of the algorithms used to add and remove
> listeners I suspect that these methods where never meant to manage large
> amounts of active listeners. The "removeAllListeners" method only seem te
> be called when closing and destroying the PersistenceBrokerImpl instance.
>
> The "addListener" method is called on every query though, so the size of
> the arrays after a couple of weeks does not surprise me.
>
> WeakReferences are used to make sure no references to the actual
> PBStateListener implementors are leaked, but the week references
> themselves (and the wrappers around them) are never removed, thus causing
> the PBStateListener arrays to keep growing.
>
> If this analysis is correct, I don't understand why nobody else is having
> this problem. I must be doing something radically different (more stupid?)
> than most other OJB users. Is there some other way these arrays are
> supposed to get freed that I'm not using because of a bug in my code? If
> so could you point me in the right direction where I might begin looking
> for problems?
>
> As you can imagine this is a serious performance problem for us, so I
> would very much appreciate any help or advice.
>
> Regards,
> Pieter.
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-user-help@db.apache.org
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
For additional commands, e-mail: ojb-user-help@db.apache.org