You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@openjpa.apache.org by "michael.couck" <mi...@hotmail.com> on 2010/08/24 10:07:21 UTC
Memory leak
Hello,
I am an avid fan of OpenJpa, been using it for years, no problems at all,
good work! Nice error messages, had a look at the code too, also good and
the documentation has no drivel, just to the point. However I have to
mention that I have found a memory leak, reproducable of course.
Environment:
I am using no caching. Windows, Java 1.6, Spring, Bitronix, OpenJpa,
BirtonixPoolingDataSource, DB2. I am inserting several million records in a
table, at about 100 per second, but every second one of the objects that get
inserted doesn't get collected, the real object, in this case it is an
Event, and the enhanced object something like xxx*xxx*xxx*Event.
I saw the leak and it is immediately apparent where it is, but I didn't
believe it, so I tried for days to find the meak in my own code. Couldn't
because the objects are allocated on the stack, so once the method exists
they get collected. So I reluctently switched to Hibernate, took about an
hour to do the changes. And the meak disappeared completely, so I can only
conclude that the leak is in OpenJpa.
I profiled with VisualVM and CodePro, with the same results. I could switch
back to OpenJpa (have the configuration already of course) and profile again
and save the snapshots, in both OpenJpa and Hibernate for comparison if
necessary.
Anyone else seen this?
Regards,
Michael
--
View this message in context: http://openjpa.208410.n2.nabble.com/Memory-leak-tp5455918p5455918.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Memory leak
Posted by "michael.couck" <mi...@hotmail.com>.
Hi Kevin,
Thanks for the quick response! You are right, I am using the runtime
enhancement, and I know that it is not recommended, but I just find it
cleaner.
We are in a performance testing phase at the moment so I can't fiddle with
the configuration right now, but I will get round to setting the JavaAgent
on the command line and re-testing for a leak. Adding enhancement at build
time is not an option as we do all sorts of other stuff and I am pretty sure
this will create it's own issues.
In any case I don't think anyone will hit this any time soon so no problem
there.
Cheers and thanks,
Michael
--
View this message in context: http://openjpa.208410.n2.nabble.com/Memory-leak-tp5455918p5464778.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Memory leak
Posted by Kevin Sutter <kw...@gmail.com>.
Hi Michael,
I sent this reply once already, but the dev@openjpa.apache.org site rejected
it for some reason. I'll try again by doing a simple reply (with no
history)...
Hi Michael,
I didn't see the screen shots attached... But, I looked at your
configuration. One thing that jumped out at me is this property:
<entry key="openjpa.RuntimeUnenhancedClasses" value="supported" />
This indicates that you are running with the "not ready for prime time"
subclassing support. For production environments, we do not recommend
running with this option. WebSphere has even turned this off in their
environment to prevent users from accidentally running with this option.
And, most recently, OpenJPA 2.0 turned this off as the default to prevent
accidental usage. Bottom line is that the enhancement processing is
preferred, both from a functional and performance standpoint.
I'm not positive that this is where the memory leak is coming from, but it's
an excellent first guess. :-) Especially since we do not perform any
memory leak tests with this option turned on.
My suggestion would be to turn this off. I can't tell what version of
OpenJPA you are running with, so the safest way to turn it off is to set the
value to "warn" or "unsupported". And, then use one of the enhancement
mechanisms [1] that fit your environment. Since I don't know how well the
-javaagent meshes with the Spring environment, the best mechanism might be
to put the enhancement processing into your build. If you want another read
on the enhancement processing, here's a blog post [2] that might help.
If you can try this and let us know if the memory leak still exists, that
would be great. If it does exist only with the subclassing support, then it
probably won't get much attention. If it does exist with enhancement, then
we'll be much more interested.
Thanks for your help,
Kevin
[1]
http://openjpa.apache.org/builds/2.0.0/apache-openjpa-2.0.0/docs/manual/manual.html#ref_guide_pc_enhance
[2]
http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
RE: Memory leak
Posted by "michael.couck" <mi...@hotmail.com>.
Hi Kevin,
Of course I understand, I have attached screen shots of two snapshots of the memory. The events are building up slowly you can see, although they are being inserted at a rate of 100 per second, only one or two per second sticks around. I can send you the heap dumps from CodePro if you wish (you can get a trial to have a look at the dumps if necessary), but they are 30 meg each, if you have a server somewhere I can load them up to that would be easier.
Here is the configuration:
<?xml version="1.0" encoding="UTF-8"?>
<beans
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://www.springframework.org/schema/beans"
xmlns:property="http://www.springframework.org/schema/p"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
<bean id="jpaAdapter"
class="org.springframework.orm.jpa.vendor.OpenJpaVendorAdapter" />
<bean id="transactionManager"
class="org.springframework.orm.jpa.JpaTransactionManager"
property:entityManagerFactory-ref="entityManagerFactoryJpa" />
<bean id="entityManagerFactoryJpa"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean"
property:persistenceXmlLocation="${persistence-jpa-openjpa.xml}"
property:dataSource-ref="searchDataSource"
property:jpaVendorAdapter-ref="jpaAdapter"
property:jpaPropertyMap-ref="jpaPropertyMap" />
<util:map id="jpaPropertyMap">
<entry key="openjpa.TransactionMode" value="managed" />
<entry key="openjpa.ConnectionFactoryMode" value="managed" />
<entry key="openjpa.ManagedRuntime" value="invocation(TransactionManagerMethod=bitronix.tm.TransactionManagerServices.getTransactionManager)" />
<entry key="openjpa.ConnectionFactoryProperties" value="PrettyPrint=true,PrettyPrintLineLength=150" />
<entry key="openjpa.Log" value="DefaultLevel=INFO,SQL=INFO,File=openjpa.log,Runtime=INFO,Tool=INFO" />
<entry key="openjpa.DataCache" value="false" />
<entry key="openjpa.QueryCache" value="false" />
<!--<entry key="openjpa.QueryCache" value="${jpa.cache}" />
<entry key="openjpa.DataCacheManager" value="${jpa.cache}" />-->
<entry key="openjpa.LockTimeout" value="6000000" />
<entry key="openjpa.ReadLockLevel" value="read" />
<entry key="openjpa.WriteLockLevel" value="write" />
<entry key="openjpa.Optimistic" value="false" />
<entry key="openjpa.Multithreaded" value="true" />
<entry key="openjpa.NontransactionalRead" value="false" />
<entry key="openjpa.AutoDetach" value="commit" />
<entry key="openjpa.FlushBeforeQueries" value="false" />
<entry key="openjpa.RuntimeUnenhancedClasses" value="supported" />
<entry key="openjpa.jdbc.SynchronizeMappings" value="buildSchema" />
<entry key="openjpa.Connection2UserName" value="${jdbc.user}" />
<entry key="openjpa.Connection2Password" value="${jdbc.password}" />
<entry key="openjpa.Connection2URL" value="${jdbc.url}" />
<entry key="openjpa.Connection2DriverName" value="${jdbc.driver}" />
</util:map>
</beans>
C3po datasource, DB2. With Hibernate there is at most 10 events on the heap at any time, I tested to 5 000 000 events, so I am pretty sure.
Not to worry, no one else is likely to hit this problem. I find the most unusual things and hit every bug in everything, found a memory leak in an Oracle driver once, sent them a mail asking what to do, they said upgrade. We had a contract for support with them as well for €50 000 per year. Go figure.
Cheers,
Michael
Date: Tue, 24 Aug 2010 07:10:16 -0700
From: ml-node+5457029-19047376-287055@n2.nabble.com
To: michaelcouck@hotmail.com
Subject: Re: Memory leak
Hi Michael,
No, we are not aware of any memory leaks. We do extensive testing looking
for memory leaks. Especially before cutting a new release (or point
release). What version of OpenJPA are you running with? If you are running
with a nightly from trunk, then I suppose there could be a memory leak that
we haven't discovered yet.
I think we would need more "proof" of the memory leak before anybody would
dive into this exercise. Can you provide more data that would pinpoint the
source of the leak? The description below is just a little too high level.
Thanks for your help,
Kevin
On Tue, Aug 24, 2010 at 3:07 AM, michael.couck <[hidden email]>wrote:
>
> Hello,
>
> I am an avid fan of OpenJpa, been using it for years, no problems at all,
> good work! Nice error messages, had a look at the code too, also good and
> the documentation has no drivel, just to the point. However I have to
> mention that I have found a memory leak, reproducable of course.
>
> Environment:
> I am using no caching. Windows, Java 1.6, Spring, Bitronix, OpenJpa,
> BirtonixPoolingDataSource, DB2. I am inserting several million records in a
> table, at about 100 per second, but every second one of the objects that
> get
> inserted doesn't get collected, the real object, in this case it is an
> Event, and the enhanced object something like xxx*xxx*xxx*Event.
>
> I saw the leak and it is immediately apparent where it is, but I didn't
> believe it, so I tried for days to find the meak in my own code. Couldn't
> because the objects are allocated on the stack, so once the method exists
> they get collected. So I reluctently switched to Hibernate, took about an
> hour to do the changes. And the meak disappeared completely, so I can only
> conclude that the leak is in OpenJpa.
>
> I profiled with VisualVM and CodePro, with the same results. I could switch
> back to OpenJpa (have the configuration already of course) and profile
> again
> and save the snapshots, in both OpenJpa and Hibernate for comparison if
> necessary.
>
> Anyone else seen this?
>
> Regards,
> Michael
> --
> View this message in context:
> http://openjpa.208410.n2.nabble.com/Memory-leak-tp5455918p5455918.html
> Sent from the OpenJPA Developers mailing list archive at Nabble.com.
>
View message @ http://openjpa.208410.n2.nabble.com/Memory-leak-tp5455918p5457029.html
To unsubscribe from Memory leak, click here.
--
View this message in context: http://openjpa.208410.n2.nabble.com/Memory-leak-tp5455918p5460615.html
Sent from the OpenJPA Developers mailing list archive at Nabble.com.
Re: Memory leak
Posted by Kevin Sutter <kw...@gmail.com>.
Hi Michael,
No, we are not aware of any memory leaks. We do extensive testing looking
for memory leaks. Especially before cutting a new release (or point
release). What version of OpenJPA are you running with? If you are running
with a nightly from trunk, then I suppose there could be a memory leak that
we haven't discovered yet.
I think we would need more "proof" of the memory leak before anybody would
dive into this exercise. Can you provide more data that would pinpoint the
source of the leak? The description below is just a little too high level.
Thanks for your help,
Kevin
On Tue, Aug 24, 2010 at 3:07 AM, michael.couck <mi...@hotmail.com>wrote:
>
> Hello,
>
> I am an avid fan of OpenJpa, been using it for years, no problems at all,
> good work! Nice error messages, had a look at the code too, also good and
> the documentation has no drivel, just to the point. However I have to
> mention that I have found a memory leak, reproducable of course.
>
> Environment:
> I am using no caching. Windows, Java 1.6, Spring, Bitronix, OpenJpa,
> BirtonixPoolingDataSource, DB2. I am inserting several million records in a
> table, at about 100 per second, but every second one of the objects that
> get
> inserted doesn't get collected, the real object, in this case it is an
> Event, and the enhanced object something like xxx*xxx*xxx*Event.
>
> I saw the leak and it is immediately apparent where it is, but I didn't
> believe it, so I tried for days to find the meak in my own code. Couldn't
> because the objects are allocated on the stack, so once the method exists
> they get collected. So I reluctently switched to Hibernate, took about an
> hour to do the changes. And the meak disappeared completely, so I can only
> conclude that the leak is in OpenJpa.
>
> I profiled with VisualVM and CodePro, with the same results. I could switch
> back to OpenJpa (have the configuration already of course) and profile
> again
> and save the snapshots, in both OpenJpa and Hibernate for comparison if
> necessary.
>
> Anyone else seen this?
>
> Regards,
> Michael
> --
> View this message in context:
> http://openjpa.208410.n2.nabble.com/Memory-leak-tp5455918p5455918.html
> Sent from the OpenJPA Developers mailing list archive at Nabble.com.
>