You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avalon.apache.org by Berin Loritsch <bl...@apache.org> on 2002/02/20 16:39:11 UTC
Memory leak
There was a memory leak introduced in the ECM patches last night.
Yesterday, the test kept a constant memory consumption (somewhere
around 16 MB for the ECM portion of the test). To day, the JVM
memory usage grows constantly with each lookup/release cycle,
starting around 15 MB, cand growing to 27 MB.
Could someone track that down?
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Memory leak
Posted by Berin Loritsch <bl...@apache.org>.
Leif Mortenson wrote:
> Still working on the memoryu, but here are the results of running the
> ContainerProfile
> test before and after the patch was added.
Try running the tests with the -server flag set. It makes a world of
difference for this type of test. Not only that, it is more
representative of how ECM/ContainerManager will be used. The time for
the ECM usage test jumps below 100,000 ms (~93 seconds). This narrows
the gap to within a 15* differential.
>
> While startup time slowed down a bit, usage time got slightly better.
> The new ContainerManager
> is a big improvement in speed though. (kudos)
>
> You might want to try running a modified version of the
> ExcaliburComponentManagerTestCase
> on it and see how it handles the dispose order problems that the patch
> fixed.
>
> I was able to reproduce memory leak. Still looking into how to fix it.
It is due to the fact that somewhere in this reference counting code you
are creating an object/objects and maintaining the reference for those
objects so it is never garbage collected. The memory leak was
introduced with the reference counting code.
We may find that (since we *are* shutting down) all this work will be
for little real value.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Memory leak
Posted by Berin Loritsch <bl...@apache.org>.
Leif Mortenson wrote:
> Still working on the memoryu, but here are the results of running the
> ContainerProfile
> test before and after the patch was added.
>
> Before:
> ---
> INFO 10142 [test ] (): ExcaliburComponentManager time =
> 259493ms. to use 50000 calls on 3 components.
> ---
>
> After
> ---
> INFO 10142 [test ] (): ExcaliburComponentManager time =
> 213668ms. to use 50000 calls on 3 components.
> ---
Keep in mind that this is within the margin of error so to speak. It
all depends on the timing of the threads. Most time is spent in the
java.lang.Thread.yield() method which is acceptible. I have seen the
ECM take anywhere between 193 seconds to ~300 seconds. It is a bear
to run this test again and again, but the only way to test trends and
really determine if you are making headway is to run it multiple times.
> While startup time slowed down a bit, usage time got slightly better.
This is still within the margin of error.
> The new ContainerManager
> is a big improvement in speed though. (kudos)
Thanks. The big difference between the two is that the ContainerManager
moves management off the critical path and handles it asynchronously.
The ECM handles all management tasks in the critical path (aka
synchronously). As a result, each lookup() is doing alot more work than
the ContainerManager.
> You might want to try running a modified version of the
> ExcaliburComponentManagerTestCase
> on it and see how it handles the dispose order problems that the patch
> fixed.
That is something for the ECM testing. The ContainerProfile test is
strictly to compare how the systems stack up to each other objectively.
> I was able to reproduce memory leak. Still looking into how to fix it.
I have a bunch of hprof text dumps, if you want me to send them to you
off line, I would be happy to. It is the results of running the
ContainerProfile testcase with the -Xrunhprof:.... option set (the
reason for the new test.sh class).
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Memory leak
Posted by Leif Mortenson <le...@silveregg.co.jp>.
Still working on the memoryu, but here are the results of running the
ContainerProfile
test before and after the patch was added.
Before:
---
Running org.apache.avalon.excalibur.system.test.ContainerProfile
DEBUG 10142 [ ] (): added new LogTargetFactory of type file
DEBUG 10142 [ ] (): added new LogTarget of id root
DEBUG 10142 [ ] (): added logger for category test.test
INFO 10142 [test ] (): Test Case: ECM_ContainerManager_StartTime
INFO 10142 [test ] (): ECM time = 1522ms.
INFO 10142 [test ] (): ContainerManager time = 1412ms.
INFO 10142 [test ] (): => ContainerManager is 1.07 X as fast
as ExcaliburComponentManager on init.
INFO 10142 [test ] (): => ExcaliburComponentManager is 0.92 X
as fast as ContainerManager on init.
INFO 10142 [test ] (): Test Case: Test Case:
ECM_ContainerManager_UseageTime
INFO 10142 [test ] ():
AbstractContainer$ContainerComponentManager time = 4647ms. to use 50000
calls on 3 components.
INFO 10142 [test ] (): ExcaliburComponentManager time =
259493ms. to use 50000 calls on 3 components.
INFO 10142 [test ] (): =>
AbstractContainer$ContainerComponentManager is 55.84 X as fast as
ExcaliburComponentManager.
INFO 10142 [test ] (): => ExcaliburComponentManager is 0.01 X
as fast as AbstractContainer$ContainerComponentManager.
INFO 10142 [test ] (): Test Case: ECM_ContainerManager_KillTime
INFO 10142 [test ] (): ECM time = 10ms.
INFO 10142 [test ] (): ContainerManager time = 671ms.
INFO 10142 [test ] (): => ContainerManager is 0.01 X as fast
as ExcaliburComponentManager on dispose.
INFO 10142 [test ] (): => ExcaliburComponentManager is 67.1 X
as fast as ContainerManager on dispose.
Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 269.979 sec
Testsuite: org.apache.avalon.excalibur.system.test.ContainerProfile
Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 269.979 sec
Testcase: testCompare_ECM_ContainerManager_UseageTime took 269.979 sec
---
After
---
Running org.apache.avalon.excalibur.system.test.ContainerProfile
DEBUG 10142 [ ] (): added new LogTargetFactory of type file
DEBUG 10142 [ ] (): added new LogTarget of id root
DEBUG 10142 [ ] (): added logger for category test.test
INFO 10142 [test ] (): Test Case: ECM_ContainerManager_StartTime
INFO 10142 [test ] (): ECM time = 2353ms.
INFO 10142 [test ] (): ContainerManager time = 1402ms.
INFO 10142 [test ] (): => ContainerManager is 1.67 X as fast
as ExcaliburComponentManager on init.
INFO 10142 [test ] (): => ExcaliburComponentManager is 0.59 X
as fast as ContainerManager on init.
INFO 10142 [test ] (): Test Case: Test Case:
ECM_ContainerManager_UseageTime
INFO 10142 [test ] ():
AbstractContainer$ContainerComponentManager time = 4046ms. to use 50000
calls on 3 components.
INFO 10142 [test ] (): ExcaliburComponentManager time =
213668ms. to use 50000 calls on 3 components.
INFO 10142 [test ] (): =>
AbstractContainer$ContainerComponentManager is 52.8 X as fast as
ExcaliburComponentManager.
INFO 10142 [test ] (): => ExcaliburComponentManager is 0.01 X
as fast as AbstractContainer$ContainerComponentManager.
INFO 10142 [test ] (): Test Case: ECM_ContainerManager_KillTime
INFO 10142 [test ] (): ECM time = 10ms.
INFO 10142 [test ] (): ContainerManager time = 691ms.
INFO 10142 [test ] (): => ContainerManager is 0.01 X as fast
as ExcaliburComponentManager on dispose.
INFO 10142 [test ] (): => ExcaliburComponentManager is 69.1 X
as fast as ContainerManager on dispose.
Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 229.64 sec
Testsuite: org.apache.avalon.excalibur.system.test.ContainerProfile
Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 229.64 sec
Testcase: testCompare_ECM_ContainerManager_UseageTime took 229.64 sec
---
While startup time slowed down a bit, usage time got slightly better.
The new ContainerManager
is a big improvement in speed though. (kudos)
You might want to try running a modified version of the
ExcaliburComponentManagerTestCase
on it and see how it handles the dispose order problems that the patch
fixed.
I was able to reproduce memory leak. Still looking into how to fix it.
Leif
Berin Loritsch wrote:
> There was a memory leak introduced in the ECM patches last night.
> Yesterday, the test kept a constant memory consumption (somewhere
> around 16 MB for the ECM portion of the test). To day, the JVM
> memory usage grows constantly with each lookup/release cycle,
> starting around 15 MB, cand growing to 27 MB.
>
> Could someone track that down?
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Memory leak
Posted by Berin Loritsch <bl...@apache.org>.
Leif Mortenson wrote:
> Ok, I think that I got this fixed. The ComponentHandler - Component
> maps were not being
> cleaned up correctly for SingleThreaded and Poolable components. Can
> you take a look and
> make sure that it passes the tests that you used to find this.
Confirmed fixed. Thank you.
>
> Cheers,
> Leif
>
>
> Berin Loritsch wrote:
>
>> There was a memory leak introduced in the ECM patches last night.
>> Yesterday, the test kept a constant memory consumption (somewhere
>> around 16 MB for the ECM portion of the test). To day, the JVM
>> memory usage grows constantly with each lookup/release cycle,
>> starting around 15 MB, cand growing to 27 MB.
>>
>> Could someone track that down?
>>
>
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
> .
>
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Memory leak
Posted by Leif Mortenson <le...@silveregg.co.jp>.
Ok, I think that I got this fixed. The ComponentHandler - Component
maps were not being
cleaned up correctly for SingleThreaded and Poolable components. Can
you take a look and
make sure that it passes the tests that you used to find this.
Cheers,
Leif
Berin Loritsch wrote:
> There was a memory leak introduced in the ECM patches last night.
> Yesterday, the test kept a constant memory consumption (somewhere
> around 16 MB for the ECM portion of the test). To day, the JVM
> memory usage grows constantly with each lookup/release cycle,
> starting around 15 MB, cand growing to 27 MB.
>
> Could someone track that down?
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Memory leak
Posted by Leif Mortenson <le...@silveregg.co.jp>.
<:-/ I'll look into this tomorrow... I was planning to write some tests
to pound on the ECM in a multithreaded capacity anyway. I'll keep an eye
out for memory problems as well.
Leif
Berin Loritsch wrote:
> There was a memory leak introduced in the ECM patches last night.
> Yesterday, the test kept a constant memory consumption (somewhere
> around 16 MB for the ECM portion of the test). To day, the JVM
> memory usage grows constantly with each lookup/release cycle,
> starting around 15 MB, cand growing to 27 MB.
>
> Could someone track that down?
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>