You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@openjpa.apache.org by Kurt T Stam <ku...@gmail.com> on 2010/01/13 19:17:43 UTC

memory leak? - simple question

Hi guys,

[DESCRIPTION] The code below inserts a 1000 records in the database.

for (int i=1; i<1000; i++) {
            EntityManager em = PersistenceManager.getEntityManager();
            EntityTransaction tx = em.getTransaction();
            try {
                tx.begin();
   
   
                // Generate auth token and store it!
                String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
                org.apache.juddi.model.AuthToken modelAuthToken = new 
org.apache.juddi.model.AuthToken();
                if (authInfo != null) {
                    modelAuthToken.setAuthToken(authInfo);
                    modelAuthToken.setCreated(new Date());
                    modelAuthToken.setLastUsed(new Date());
                    modelAuthToken.setAuthorizedName(publisherId);
                    modelAuthToken.setNumberOfUses(0);
                    modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
   
                    em.persist(modelAuthToken);
                }
   
                apiAuthToken = new org.uddi.api_v3.AuthToken();
   
                MappingModelToApi.mapAuthToken(modelAuthToken, 
apiAuthToken);
   
                tx.commit();
               
            } finally {
                if (tx.isActive()) {
                    tx.rollback();
                }
                em.clear();
                em.close();
            }
        }


[ISSUE]
After it leaving this code I end up with a 1000 
org.apache.juddi.model.AuthToken objects in memory. I've been using the 
profiler, and these objects cannot be garbage collected.

This seems to be pretty the most common use case of using an OR-mapping 
tool, so I find it hard to believe openjpa has a memory leak here. Does 
anyone see what I'm doing wrong? Or can someone point me to an example 
that does not exhibit this behavior? BTW same code using hibernate does 
not accumulate these objects.

We're using openjpa 1.2.1.


Thx,


Kurt

Apache jUDDI.

Re: memory leak? - simple question

Posted by Kurt T Stam <ku...@gmail.com>.
Hi Rick, thank you for you quick response. My comments inline.

Rick Curtis wrote:
> If you change the 1000 to something like 1000000... does your application go
> OOM? 
Yep it does.
> Are you running in a JSE environment?
running in tomcat.
>  What is PersistenceManager?
>   

http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/src/main/java/org/apache/juddi/config/PersistenceManager.java

The code referenced below is based on

http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/src/main/java/org/apache/juddi/api/impl/UDDISecurityImpl.java

I just added the loop to make the problem more obvious. This method is 
called from a web service client.
> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com> wrote:
>
>   
>> BTW I'm running with the cache off
>>
>> <property name="openjpa.DataCache" value="false"/>
>>
>> (that turns it off right?)
>>
>> --Kurt
>>
>>
>>
>> Kurt T Stam wrote:
>>
>>     
>>> Hi guys,
>>>
>>> [DESCRIPTION] The code below inserts a 1000 records in the database.
>>>
>>> for (int i=1; i<1000; i++) {
>>>           EntityManager em = PersistenceManager.getEntityManager();
>>>           EntityTransaction tx = em.getTransaction();
>>>           try {
>>>               tx.begin();
>>>                   // Generate auth token and store it!
>>>               String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
>>>               org.apache.juddi.model.AuthToken modelAuthToken = new
>>> org.apache.juddi.model.AuthToken();
>>>               if (authInfo != null) {
>>>                   modelAuthToken.setAuthToken(authInfo);
>>>                   modelAuthToken.setCreated(new Date());
>>>                   modelAuthToken.setLastUsed(new Date());
>>>                   modelAuthToken.setAuthorizedName(publisherId);
>>>                   modelAuthToken.setNumberOfUses(0);
>>>                   modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>                     em.persist(modelAuthToken);
>>>               }
>>>                 apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>                 MappingModelToApi.mapAuthToken(modelAuthToken,
>>> apiAuthToken);
>>>                 tx.commit();
>>>                         } finally {
>>>               if (tx.isActive()) {
>>>                   tx.rollback();
>>>               }
>>>               em.clear();
>>>               em.close();
>>>           }
>>>       }
>>>
>>>
>>> [ISSUE]
>>> After it leaving this code I end up with a 1000
>>> org.apache.juddi.model.AuthToken objects in memory. I've been using the
>>> profiler, and these objects cannot be garbage collected.
>>>
>>> This seems to be pretty the most common use case of using an OR-mapping
>>> tool, so I find it hard to believe openjpa has a memory leak here. Does
>>> anyone see what I'm doing wrong? Or can someone point me to an example that
>>> does not exhibit this behavior? BTW same code using hibernate does not
>>> accumulate these objects.
>>>
>>> We're using openjpa 1.2.1.
>>>
>>>
>>> Thx,
>>>
>>>
>>> Kurt
>>>
>>> Apache jUDDI.
>>>
>>>       
>>     
>
>
>   


Re: memory leak? - simple question

Posted by Kurt T Stam <ku...@gmail.com>.
Thanks Mark :). That looks much better.  --Kurt

Mark Struberg wrote:
> Btw (slightly OT)
>
> You could move addDefaultConstructor and enforcePropertyRestrictions in your pom.xml to the configuration section of the openjpa-maven-plugin without, having to use the <toolProperties>. This should do exactly the same, but is easier to read ;)
>
> <configuration>
>   <addDefaultConstructor>true</addDefaultConstructor>
>   <enforcePropertyRestrictions>true</enforcePropertyRestrictions>
> </configuration>
>
> LieGrue,
> strub
>
> --- On Thu, 1/14/10, Kurt T Stam <ku...@gmail.com> wrote:
>
>   
>> From: Kurt T Stam <ku...@gmail.com>
>> Subject: Re: memory leak? - simple question
>> To: users@openjpa.apache.org
>> Date: Thursday, January 14, 2010, 4:27 AM
>> Thanks Kevin,
>>
>> We're enhancing at build time:
>>
>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>
>> Yeah we've been running load tests and things are nice and
>> stable with 
>> Hibernate but with Openjpa we see increasing memory use,
>> blocking 
>> threads and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. 
>> Our preference would be to ship with openjpa by default;
>> but our build 
>> supports both hibernate and openjpa.
>>
>> And yes we use openjpa 1.2.1 (latest stable version).
>>
>> --Kurt
>>
>> Kevin Sutter wrote:
>>     
>>> Interesting detective work, Kurt.  Thanks.
>>>
>>> Why the WebService version of the app would behave
>>>       
>> differently as far as GC
>>     
>>> is concerned is a mystery.  And, you said that
>>>       
>> plugging in Hibernate into
>>     
>>> this scenario, everything works okay?  Very
>>>       
>> confusing.
>>     
>>> How are you performing the Entity enhancement
>>>       
>> processing?  Are you
>>     
>>> pre-enhancing via your build process?  Or, are
>>>       
>> you using the -javaagent
>>     
>>> mechanism?  Or, are you falling back to the
>>>       
>> subclassing support within
>>     
>>> OpenJPA?  (See [1] for more information on these
>>>       
>> questions in case they
>>     
>>> don't make sense.)
>>>
>>> This would be one area that is different between
>>>       
>> Hibernate and OpenJPA --
>>     
>>> enhancement processing.
>>>
>>> In the Tomcat environment, you may be falling back to
>>>       
>> the subclassing
>>     
>>> support (which we do not recommend) and hitting a
>>>       
>> memory leak with that.
>>     
>>> You said OpenJPA 1.2.x, right?
>>>
>>> Just a couple of thoughts on the subject...
>>> Kevin
>>>
>>> [1]
>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>
>>>
>>>
>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com>
>>>       
>> wrote:
>>     
>>>    
>>>       
>>>> The same code executed straight from a java client
>>>>         
>> (inVM) shows no memory
>>     
>>>> leak.
>>>>
>>>> So is the fact that it is WebService significant
>>>>         
>> then? What else can be
>>     
>>>> different? I think one thread remains up, and
>>>>         
>> somehow this causes openjpa
>>     
>>>> not being able to clean up after itself. What can
>>>>         
>> I do to debug this more? I
>>     
>>>> can actually see in the profiler that the objects
>>>>         
>> are allocated by the
>>     
>>>> WebService, but why aren't they cleaned up?
>>>>
>>>> Thx,
>>>>
>>>>
>>>> --Kurt
>>>>
>>>>
>>>> Kurt T Stam wrote:
>>>>
>>>>      
>>>>         
>>>>> Thanks Kevin, thanks for your response.
>>>>>
>>>>> I just replaced the static call by:
>>>>>
>>>>>            
>>>>>           
>>    apiAuthToken = new
>> org.uddi.api_v3.AuthToken();
>>     
>>>>>            
>>>>>           
>>    apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>     
>>>>>            
>>>>>           
>>    //MappingModelToApi.mapAuthToken(modelAuthToken,
>>     
>>>>> apiAuthToken);
>>>>>
>>>>> which did not make a difference.
>>>>>
>>>>> I'm wondering if the fact that my class is a
>>>>>           
>> webservice makes a
>>     
>>>>> difference. I'll try extracting it into
>>>>> a regular class with a main method and profile
>>>>>           
>> that. At least I know that
>>     
>>>>> I didn't forget something
>>>>> completely obvious..
>>>>>
>>>>> --Kurt
>>>>>
>>>>> Kevin Sutter wrote:
>>>>>
>>>>>        
>>>>>           
>>>>>> Kurt,
>>>>>> I agree that this is very common usage of
>>>>>>             
>> the JPA programming model.
>>     
>>>>>>   And,
>>>>>> we are not aware of any memory
>>>>>>             
>> leaks.  About the only thing that jumps
>>     
>>>>>> out
>>>>>> at me is the following two lines:
>>>>>>
>>>>>>            
>>>>>>             
>>     apiAuthToken = new
>> org.uddi.api_v3.AuthToken();
>>     
>>>>>>            
>>>>>>             
>>    
>> MappingModelToApi.mapAuthToken(modelAuthToken,
>>     
>>>>>> apiAuthToken);
>>>>>>
>>>>>> What do these do?  Can you comment
>>>>>>             
>> these out and see if the memory leak
>>     
>>>>>> still exists?  Since you are passing
>>>>>>             
>> the modelAuthToken into this method,
>>     
>>>>>> I
>>>>>> don't know what it's doing with the
>>>>>>             
>> reference and could it be holding
>>     
>>>>>> onto
>>>>>> something to prevent the GC from cleaning
>>>>>>             
>> up?
>>     
>>>>>> The rest of your example seems very
>>>>>>             
>> straight forward with creating and
>>     
>>>>>> persisting objects.
>>>>>>
>>>>>> Kevin
>>>>>>
>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick
>>>>>>             
>> Curtis <cu...@gmail.com>
>> wrote:
>>     
>>>>>>
>>>>>>          
>>>>>>             
>>>>>>> If you change the 1000 to something
>>>>>>>               
>> like 1000000... does your
>>     
>>>>>>> application
>>>>>>> go
>>>>>>> OOM? Are you running in a JSE
>>>>>>>               
>> environment? What is PersistenceManager?
>>     
>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt
>>>>>>>               
>> T Stam <ku...@gmail.com>
>>     
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>        
>>>>>>>               
>>    
>>     
>>>>>>>> BTW I'm running with the cache
>>>>>>>>                 
>> off
>>     
>>>>>>>> <property
>>>>>>>>                 
>> name="openjpa.DataCache" value="false"/>
>>     
>>>>>>>> (that turns it off right?)
>>>>>>>>
>>>>>>>> --Kurt
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Kurt T Stam wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>          
>>>>>>>>                 
>>    
>>     
>>>>>>>>> Hi guys,
>>>>>>>>>
>>>>>>>>> [DESCRIPTION] The code below
>>>>>>>>>                   
>> inserts a 1000 records in the database.
>>     
>>>>>>>>> for (int i=1; i<1000; i++)
>>>>>>>>>                   
>> {
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>   EntityManager em =
>> PersistenceManager.getEntityManager();
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>   EntityTransaction tx = em.getTransaction();
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>   try {
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>       tx.begin();
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>           // Generate auth token
>> and store it!
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>       String authInfo = AUTH_TOKEN_PREFIX +
>> UUID.randomUUID();
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>       org.apache.juddi.model.AuthToken
>> modelAuthToken = new
>>     
>> org.apache.juddi.model.AuthToken();
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>       if (authInfo != null) {
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>          
>> modelAuthToken.setAuthToken(authInfo);
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>          
>> modelAuthToken.setCreated(new Date());
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>          
>> modelAuthToken.setLastUsed(new Date());
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>          
>> modelAuthToken.setAuthorizedName(publisherId);
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>          
>> modelAuthToken.setNumberOfUses(0);
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>          
>> modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>            
>> em.persist(modelAuthToken);
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>       }
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>         apiAuthToken = new
>> org.uddi.api_v3.AuthToken();
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>        
>> MappingModelToApi.mapAuthToken(modelAuthToken,
>>     
>>>>>>>>> apiAuthToken);
>>>>>>>>>        
>>>>>>>>>                   
>>         tx.commit();
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>                 }
>> finally {
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>       if (tx.isActive()) {
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>           tx.rollback();
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>       }
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>       em.clear();
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>       em.close();
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>   }
>>     
>>>>>>>>>       }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [ISSUE]
>>>>>>>>> After it leaving this code I
>>>>>>>>>                   
>> end up with a 1000
>>     
>> org.apache.juddi.model.AuthToken objects in memory. I've
>> been using
>>     
>>>>>>>>> the
>>>>>>>>> profiler, and these objects
>>>>>>>>>                   
>> cannot be garbage collected.
>>     
>>>>>>>>> This seems to be pretty the
>>>>>>>>>                   
>> most common use case of using an
>>     
>>>>>>>>> OR-mapping
>>>>>>>>> tool, so I find it hard to
>>>>>>>>>                   
>> believe openjpa has a memory leak here.
>>     
>>>>>>>>> Does
>>>>>>>>> anyone see what I'm doing
>>>>>>>>>                   
>> wrong? Or can someone point me to an example
>>     
>>>>>>>>>        
>>>>>>>>>                   
>>        
>>     
>>>>>>>> that
>>>>>>>>          
>>>>>>>>                 
>>    
>>     
>>>>>>>        
>>>>>>>               
>>    
>>     
>>>>>>>> does not exhibit this behavior?
>>>>>>>>                 
>> BTW same code using hibernate does not
>>     
>>>>>>>>          
>>>>>>>>                 
>>    
>>     
>>>>>>>>> accumulate these objects.
>>>>>>>>>
>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thx,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Kurt
>>>>>>>>>
>>>>>>>>> Apache jUDDI.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>        
>>>>>>>>>                   
>>        
>>     
>>>>>>>>          
>>>>>>>>                 
>>    
>>     
>>>>>>> --
>>>>>>> Thanks,
>>>>>>> Rick
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>        
>>>>>>>               
>>    
>>     
>>>>>>          
>>>>>>             
>>>>>        
>>>>>           
>>>    
>>>       
>>     
>
>
>       
>   


Re: memory leak? - simple question

Posted by Mark Struberg <st...@yahoo.de>.
Btw (slightly OT)

You could move addDefaultConstructor and enforcePropertyRestrictions in your pom.xml to the configuration section of the openjpa-maven-plugin without, having to use the <toolProperties>. This should do exactly the same, but is easier to read ;)

<configuration>
  <addDefaultConstructor>true</addDefaultConstructor>
  <enforcePropertyRestrictions>true</enforcePropertyRestrictions>
</configuration>

LieGrue,
strub

--- On Thu, 1/14/10, Kurt T Stam <ku...@gmail.com> wrote:

> From: Kurt T Stam <ku...@gmail.com>
> Subject: Re: memory leak? - simple question
> To: users@openjpa.apache.org
> Date: Thursday, January 14, 2010, 4:27 AM
> Thanks Kevin,
> 
> We're enhancing at build time:
> 
> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
> 
> Yeah we've been running load tests and things are nice and
> stable with 
> Hibernate but with Openjpa we see increasing memory use,
> blocking 
> threads and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. 
> Our preference would be to ship with openjpa by default;
> but our build 
> supports both hibernate and openjpa.
> 
> And yes we use openjpa 1.2.1 (latest stable version).
> 
> --Kurt
> 
> Kevin Sutter wrote:
> > Interesting detective work, Kurt.  Thanks.
> >
> > Why the WebService version of the app would behave
> differently as far as GC
> > is concerned is a mystery.  And, you said that
> plugging in Hibernate into
> > this scenario, everything works okay?  Very
> confusing.
> >
> > How are you performing the Entity enhancement
> processing?  Are you
> > pre-enhancing via your build process?  Or, are
> you using the -javaagent
> > mechanism?  Or, are you falling back to the
> subclassing support within
> > OpenJPA?  (See [1] for more information on these
> questions in case they
> > don't make sense.)
> >
> > This would be one area that is different between
> Hibernate and OpenJPA --
> > enhancement processing.
> >
> > In the Tomcat environment, you may be falling back to
> the subclassing
> > support (which we do not recommend) and hitting a
> memory leak with that.
> >
> > You said OpenJPA 1.2.x, right?
> >
> > Just a couple of thoughts on the subject...
> > Kevin
> >
> > [1]
> > http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
> >
> >
> >
> > On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com>
> wrote:
> >
> >   
> >> The same code executed straight from a java client
> (inVM) shows no memory
> >> leak.
> >>
> >> So is the fact that it is WebService significant
> then? What else can be
> >> different? I think one thread remains up, and
> somehow this causes openjpa
> >> not being able to clean up after itself. What can
> I do to debug this more? I
> >> can actually see in the profiler that the objects
> are allocated by the
> >> WebService, but why aren't they cleaned up?
> >>
> >> Thx,
> >>
> >>
> >> --Kurt
> >>
> >>
> >> Kurt T Stam wrote:
> >>
> >>     
> >>> Thanks Kevin, thanks for your response.
> >>>
> >>> I just replaced the static call by:
> >>>
> >>>           
>    apiAuthToken = new
> org.uddi.api_v3.AuthToken();
> >>>           
>    apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
> >>>           
>    //MappingModelToApi.mapAuthToken(modelAuthToken,
> >>> apiAuthToken);
> >>>
> >>> which did not make a difference.
> >>>
> >>> I'm wondering if the fact that my class is a
> webservice makes a
> >>> difference. I'll try extracting it into
> >>> a regular class with a main method and profile
> that. At least I know that
> >>> I didn't forget something
> >>> completely obvious..
> >>>
> >>> --Kurt
> >>>
> >>> Kevin Sutter wrote:
> >>>
> >>>       
> >>>> Kurt,
> >>>> I agree that this is very common usage of
> the JPA programming model.
> >>>>  And,
> >>>> we are not aware of any memory
> leaks.  About the only thing that jumps
> >>>> out
> >>>> at me is the following two lines:
> >>>>
> >>>>           
>     apiAuthToken = new
> org.uddi.api_v3.AuthToken();
> >>>>           
>    
> MappingModelToApi.mapAuthToken(modelAuthToken,
> >>>> apiAuthToken);
> >>>>
> >>>> What do these do?  Can you comment
> these out and see if the memory leak
> >>>> still exists?  Since you are passing
> the modelAuthToken into this method,
> >>>> I
> >>>> don't know what it's doing with the
> reference and could it be holding
> >>>> onto
> >>>> something to prevent the GC from cleaning
> up?
> >>>>
> >>>> The rest of your example seems very
> straight forward with creating and
> >>>> persisting objects.
> >>>>
> >>>> Kevin
> >>>>
> >>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick
> Curtis <cu...@gmail.com>
> wrote:
> >>>>
> >>>>
> >>>>
> >>>>         
> >>>>> If you change the 1000 to something
> like 1000000... does your
> >>>>> application
> >>>>> go
> >>>>> OOM? Are you running in a JSE
> environment? What is PersistenceManager?
> >>>>>
> >>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt
> T Stam <ku...@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>       
>    
> >>>>>> BTW I'm running with the cache
> off
> >>>>>>
> >>>>>> <property
> name="openjpa.DataCache" value="false"/>
> >>>>>>
> >>>>>> (that turns it off right?)
> >>>>>>
> >>>>>> --Kurt
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Kurt T Stam wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>         
>    
> >>>>>>> Hi guys,
> >>>>>>>
> >>>>>>> [DESCRIPTION] The code below
> inserts a 1000 records in the database.
> >>>>>>>
> >>>>>>> for (int i=1; i<1000; i++)
> {
> >>>>>>>       
>   EntityManager em =
> PersistenceManager.getEntityManager();
> >>>>>>>       
>   EntityTransaction tx = em.getTransaction();
> >>>>>>>       
>   try {
> >>>>>>>       
>       tx.begin();
> >>>>>>>       
>           // Generate auth token
> and store it!
> >>>>>>>       
>       String authInfo = AUTH_TOKEN_PREFIX +
> UUID.randomUUID();
> >>>>>>>       
>       org.apache.juddi.model.AuthToken
> modelAuthToken = new
> >>>>>>>
> org.apache.juddi.model.AuthToken();
> >>>>>>>       
>       if (authInfo != null) {
> >>>>>>>       
>          
> modelAuthToken.setAuthToken(authInfo);
> >>>>>>>       
>          
> modelAuthToken.setCreated(new Date());
> >>>>>>>       
>          
> modelAuthToken.setLastUsed(new Date());
> >>>>>>>       
>          
> modelAuthToken.setAuthorizedName(publisherId);
> >>>>>>>       
>          
> modelAuthToken.setNumberOfUses(0);
> >>>>>>>       
>          
> modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
> >>>>>>>       
>            
> em.persist(modelAuthToken);
> >>>>>>>       
>       }
> >>>>>>>       
>         apiAuthToken = new
> org.uddi.api_v3.AuthToken();
> >>>>>>>       
>        
> MappingModelToApi.mapAuthToken(modelAuthToken,
> >>>>>>> apiAuthToken);
> >>>>>>>       
>         tx.commit();
> >>>>>>>       
>                 }
> finally {
> >>>>>>>       
>       if (tx.isActive()) {
> >>>>>>>       
>           tx.rollback();
> >>>>>>>       
>       }
> >>>>>>>       
>       em.clear();
> >>>>>>>       
>       em.close();
> >>>>>>>       
>   }
> >>>>>>>      }
> >>>>>>>
> >>>>>>>
> >>>>>>> [ISSUE]
> >>>>>>> After it leaving this code I
> end up with a 1000
> >>>>>>>
> org.apache.juddi.model.AuthToken objects in memory. I've
> been using
> >>>>>>> the
> >>>>>>> profiler, and these objects
> cannot be garbage collected.
> >>>>>>>
> >>>>>>> This seems to be pretty the
> most common use case of using an
> >>>>>>> OR-mapping
> >>>>>>> tool, so I find it hard to
> believe openjpa has a memory leak here.
> >>>>>>> Does
> >>>>>>> anyone see what I'm doing
> wrong? Or can someone point me to an example
> >>>>>>>
> >>>>>>>
> >>>>>>>       
>        
> >>>>>> that
> >>>>>>         
>    
> >>>>>       
>    
> >>>>>> does not exhibit this behavior?
> BTW same code using hibernate does not
> >>>>>>         
>    
> >>>>>>> accumulate these objects.
> >>>>>>>
> >>>>>>> We're using openjpa 1.2.1.
> >>>>>>>
> >>>>>>>
> >>>>>>> Thx,
> >>>>>>>
> >>>>>>>
> >>>>>>> Kurt
> >>>>>>>
> >>>>>>> Apache jUDDI.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>       
>        
> >>>>>>         
>    
> >>>>> --
> >>>>> Thanks,
> >>>>> Rick
> >>>>>
> >>>>>
> >>>>>
> >>>>>       
>    
> >>>>
> >>>>         
> >>>       
> >
> >   
> 
> 


      

Re: memory leak? - simple question

Posted by Kevin Sutter <kw...@gmail.com>.
Great.  Glad to hear that you're back in business.  I've created a JIRA [1]
for this issue just in case we resurrect the subclassing support to a
first-class citizen.  :-)

Thanks,
Kevin

[1]  https://issues.apache.org/jira/browse/OPENJPA-1462

On Fri, Jan 15, 2010 at 10:15 AM, Kurt T Stam <ku...@gmail.com> wrote:

> Thanks so much Kevin, that did the trick. Some of the issues you mentioned
> where introduced by me not copying right from the
> jUDDI project, but the main issue is that the openjpa-plugin simply does
> not run for some reason. Anyway it is no longer leaking now! I learned a lot
> from this exercise. And yes my class jumped from 2 to 10 k. BTW I'm pretty
> sure this used to work..
>
> Thanks again! Going to fix it the jUDDI build now :)!
>
> --Kurt
>
> Kevin Sutter wrote:
>
>> Hi Kurt,
>>
>> I have not run a profiler to verify the memory leak, but I think I found
>> the
>> problem...
>>
>> Although you thought you were running the PCEnhancer during build time, it
>> was not being performed.  When I looked at the Tomcat log with the
>> original
>> jar file, the following message was logged which indicates you were
>> falling
>> back to the "not ready for production" subclassing support:
>>
>> 1844  juddiDatabase  INFO   [http-8080-1] openjpa.Enhance - Creating
>> subclass for "[class org.apache.juddi.AuthToken]". This means that your
>> application will be  less efficient and will consume more memory than it
>> would if you ran the OpenJPA enhancer. Additionally, lazy loading will not
>> be available for one-to-one and many-to-one persistent attributes in types
>> using field access; they will be loaded eagerly instead.
>>
>> I made a few changes to your pom.xml and the "core" module to get the
>> enhancement to work and I believe the memory leak has went away.  Like I
>> said, I did not run a profiler, but I added a "finalize" method to your
>> AuthToken class and I now see that these entities are getting GC'd.
>>
>> One thing I would do is add the following property to your
>> persistence.xml.
>>
>> <property name="openjpa.RuntimeUnenhancedClasses" value="warn"/>
>>
>> This property will produce the following warning if you accidentally fall
>> into this subclassing support.  We turn this off by default when running
>> within WebSphere since we really don't want customers to accidentally use
>> this subclassing.  And, we have turned this off for 1.3.x and trunk due to
>> the various problems associated with it.  Unfortunately, the 1.2.x branch
>> is
>> kind of stuck with it...
>>
>> 1688  juddiDatabase  WARN   [http-8080-1] openjpa.Enhance - This
>> configuration disallows runtime optimization, but the following listed
>> types
>> were not enhanced at build time or at class load time with a javaagent:
>> "[class org.apache.juddi.AuthToken]".
>>
>> And, of course, your webservice no longer works either.  You will get an
>> HTTP Status 500 error with an error message and call stack that further
>> explains the error.
>>
>> To get the maven plugin to work properly...  I made several changes, so
>> hopefully I remember everything...
>>
>> o  The number one thing is that the openjpa:enhance goal doesn't seem to
>> get
>> automatically invoked when you compile.  I have no idea why, but that
>> doesn't seem to work.
>>
>> o  You have to make your persistence.xml file available to the "core"
>> module.  For my testing, I just created a new directory in the "core"
>> module
>> called "\openjpa-leak\core\src\main\resources\META-INF\persistence.xml".
>> This will automatically get copied over to "target" and then the
>> openjpa:enhance goal will find it.
>>
>> o  How you want to make your persistence.xml file available to both your
>> "core" module and the "leak-war" module is up to you...  :-)
>>
>> o  I added a dependency in your core\pom.xml file for the 1.2.1 openjpa.
>> The documentation [1] says it will default to 1.2.0, if none is specified.
>>
>>            <dependencies>
>>                <dependency>
>>                    <groupId>org.apache.openjpa</groupId>
>>                    <artifactId>openjpa</artifactId>
>>                    <version>1.2.1</version>
>>                </dependency>
>>            </dependencies>
>>
>> o  Your plugin properties don't seem to be specified correctly, at least
>> according to the documentation, but I didn't touch them and things still
>> seemed to work (once I got around the other problems).
>>
>> o  I also changed where to find the entities to enhance since this pom.xml
>> still specified the "model" directory.  There also seems to be confusion
>> as
>> to whether to specify <classes> or <includes>.  I went with <includes>
>> since
>> that's what the example showed.
>>
>> o  It's very easy to see if the enhancement process worked or not since
>> the
>> size of your entity classes increases by a few thousand bytes.  And, with
>> the extra property above, this will prevent you from accidentally using
>> the
>> unenhanced version.
>>
>> Hope this helps!  Let me know if this gets you around the memory leak.
>>
>> Kevin
>>
>> [1]  http://mojo.codehaus.org/openjpa-maven-plugin/usage.html
>>
>>
>>
>> On Thu, Jan 14, 2010 at 2:12 PM, Kurt T Stam <ku...@gmail.com> wrote:
>>
>>
>>
>>> 1. Cool that you got it running :)
>>>
>>> 2. I just ran with openjpa 1.2.2.SNAPSHOT and still see an accumulation
>>> of
>>> AuthToken objects.
>>>
>>> --K
>>>
>>>
>>> Kevin Sutter wrote:
>>>
>>>
>>>
>>>> Hey, this wasn't so difficult...  :-)  Maybe I can still learn new
>>>> things...  I have Tomcat up and running and I can get your app running.
>>>>
>>>> On Thu, Jan 14, 2010 at 1:46 PM, Kevin Sutter <kw...@gmail.com>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Hi Kurt,
>>>>> These instructions, of course, assume that I know something about
>>>>> Tomcat.
>>>>> This would not be a good assumption...  :-)  If you really want me to
>>>>> see
>>>>> this in action, I will need more information on how to start Tomcat and
>>>>> how
>>>>> to load applications.  In the mean time, a trace log of OpenJPA would
>>>>> be
>>>>> an
>>>>> excellent start.
>>>>>
>>>>> Thanks!
>>>>> Kevin
>>>>>
>>>>>
>>>>> On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <ku...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Hi Kevin,
>>>>>>
>>>>>> I tried adding the javaagent stuff, and it still ends up leaking. I
>>>>>> extracted the code into a tiny
>>>>>> war with just one servlet which persist the authToken. The build
>>>>>> creates
>>>>>> a
>>>>>> Tomcat distro backed by a
>>>>>> derby database. You can download it from:
>>>>>>
>>>>>> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz>
>>>>>> <http://people.apache.org/%7Ekstam/leak.tgz>
>>>>>> <http://people.apache.org/%7Ekstam/leak.tgz>
>>>>>>
>>>>>>
>>>>>> You can build it with: mvn install, you can find the final tomcat in
>>>>>> tomcat/target/tomcat-1.0.SNAPSHOT.zip
>>>>>>
>>>>>> or if you don't want to build it:
>>>>>> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>>>>> <http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>>>>> <http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>>>>>
>>>>>>
>>>>>> After you bring up tomcat it persistent an authToken when you hit
>>>>>> http://localhost:8080/leak/test
>>>>>>
>>>>>> In the profiler I see an org.apache.juddi.AuthToken left behind for
>>>>>> each
>>>>>> request.
>>>>>>
>>>>>> Hopefully this little example will pinpoint what's going on.
>>>>>>
>>>>>> Thx Kev for sticking with me :)
>>>>>>
>>>>>> --Kurt
>>>>>>
>>>>>>
>>>>>>
>>>>>> Kevin Sutter wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hi Kurt,
>>>>>>> Since you have the environment to reproduce this, could you run with
>>>>>>> the
>>>>>>> OpenJPA trace enabled?  I'm not even sure what I'm looking for at
>>>>>>> this
>>>>>>> point.  :-)  But, maybe there's some tracing that would point us in
>>>>>>> the
>>>>>>> right direction.
>>>>>>>
>>>>>>> <property name="openjpa.Log" value="DefaultLevel=TRACE"/>
>>>>>>>
>>>>>>> We have been always been on the watch for memory leaks, so this one
>>>>>>> is
>>>>>>> puzzling.  You did mention blocked threads as well.  At this point,
>>>>>>> you
>>>>>>> are
>>>>>>> probably hitting blocked threads due to the GC and Memory problems.
>>>>>>>  But,
>>>>>>> we
>>>>>>> did recently resolve some locking situations [1] in our
>>>>>>> MetaDataRepository
>>>>>>> that could affect scaling (under extreme load).  This was resolved in
>>>>>>> the
>>>>>>> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2
>>>>>>> release,
>>>>>>> so
>>>>>>> that should be coming soon).
>>>>>>>
>>>>>>> Does your WebService version of the problem use EntityManager
>>>>>>> transactions,
>>>>>>> or is the transaction service coming from Tomcat?  And, does your
>>>>>>> WebService
>>>>>>> version also do the em.clear() and em.close()?  Basically, are you
>>>>>>> using
>>>>>>> application-managed or container-managed persistence while running as
>>>>>>> a
>>>>>>> WebService?
>>>>>>>
>>>>>>> I'm very interested in helping to resolve this issue.  Like I
>>>>>>> mentioned,
>>>>>>> we
>>>>>>> do a lot of memory leakage and scalability testing of the OpenJPA
>>>>>>> solution
>>>>>>> and we're not aware of anything.  Although there is always the
>>>>>>> possibility
>>>>>>> that something was resolved in the 1.2.x branch after the 1.2.1
>>>>>>> release
>>>>>>> that
>>>>>>> I'm not remembering...
>>>>>>>
>>>>>>> Thanks for your help,
>>>>>>> Kevin
>>>>>>>
>>>>>>> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
>>>>>>>
>>>>>>> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <ku...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Thanks Kevin,
>>>>>>>>
>>>>>>>> We're enhancing at build time:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>>>>>>>
>>>>>>>> Yeah we've been running load tests and things are nice and stable
>>>>>>>> with
>>>>>>>> Hibernate but with Openjpa we see increasing memory use, blocking
>>>>>>>> threads
>>>>>>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267.
>>>>>>>> Our
>>>>>>>> preference would be to ship with openjpa by default; but our build
>>>>>>>> supports
>>>>>>>> both hibernate and openjpa.
>>>>>>>>
>>>>>>>> And yes we use openjpa 1.2.1 (latest stable version).
>>>>>>>>
>>>>>>>> --Kurt
>>>>>>>>
>>>>>>>>
>>>>>>>> Kevin Sutter wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Interesting detective work, Kurt.  Thanks.
>>>>>>>>>
>>>>>>>>> Why the WebService version of the app would behave differently as
>>>>>>>>> far
>>>>>>>>> as
>>>>>>>>> GC
>>>>>>>>> is concerned is a mystery.  And, you said that plugging in
>>>>>>>>> Hibernate
>>>>>>>>> into
>>>>>>>>> this scenario, everything works okay?  Very confusing.
>>>>>>>>>
>>>>>>>>> How are you performing the Entity enhancement processing?  Are you
>>>>>>>>> pre-enhancing via your build process?  Or, are you using the
>>>>>>>>> -javaagent
>>>>>>>>> mechanism?  Or, are you falling back to the subclassing support
>>>>>>>>> within
>>>>>>>>> OpenJPA?  (See [1] for more information on these questions in case
>>>>>>>>> they
>>>>>>>>> don't make sense.)
>>>>>>>>>
>>>>>>>>> This would be one area that is different between Hibernate and
>>>>>>>>> OpenJPA
>>>>>>>>> --
>>>>>>>>> enhancement processing.
>>>>>>>>>
>>>>>>>>> In the Tomcat environment, you may be falling back to the
>>>>>>>>> subclassing
>>>>>>>>> support (which we do not recommend) and hitting a memory leak with
>>>>>>>>> that.
>>>>>>>>>
>>>>>>>>> You said OpenJPA 1.2.x, right?
>>>>>>>>>
>>>>>>>>> Just a couple of thoughts on the subject...
>>>>>>>>> Kevin
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> The same code executed straight from a java client (inVM) shows no
>>>>>>>>>> memory
>>>>>>>>>> leak.
>>>>>>>>>>
>>>>>>>>>> So is the fact that it is WebService significant then? What else
>>>>>>>>>> can
>>>>>>>>>> be
>>>>>>>>>> different? I think one thread remains up, and somehow this causes
>>>>>>>>>> openjpa
>>>>>>>>>> not being able to clean up after itself. What can I do to debug
>>>>>>>>>> this
>>>>>>>>>> more? I
>>>>>>>>>> can actually see in the profiler that the objects are allocated by
>>>>>>>>>> the
>>>>>>>>>> WebService, but why aren't they cleaned up?
>>>>>>>>>>
>>>>>>>>>> Thx,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --Kurt
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Thanks Kevin, thanks for your response.
>>>>>>>>>>>
>>>>>>>>>>> I just replaced the static call by:
>>>>>>>>>>>
>>>>>>>>>>>           apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>
>>>>>>>>>>> apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>>>>>>>>           //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>
>>>>>>>>>>> which did not make a difference.
>>>>>>>>>>>
>>>>>>>>>>> I'm wondering if the fact that my class is a webservice makes a
>>>>>>>>>>> difference. I'll try extracting it into
>>>>>>>>>>> a regular class with a main method and profile that. At least I
>>>>>>>>>>> know
>>>>>>>>>>> that
>>>>>>>>>>> I didn't forget something
>>>>>>>>>>> completely obvious..
>>>>>>>>>>>
>>>>>>>>>>> --Kurt
>>>>>>>>>>>
>>>>>>>>>>> Kevin Sutter wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Kurt,
>>>>>>>>>>>> I agree that this is very common usage of the JPA programming
>>>>>>>>>>>> model.
>>>>>>>>>>>>  And,
>>>>>>>>>>>> we are not aware of any memory leaks.  About the only thing that
>>>>>>>>>>>> jumps
>>>>>>>>>>>> out
>>>>>>>>>>>> at me is the following two lines:
>>>>>>>>>>>>
>>>>>>>>>>>>            apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>>            MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>>
>>>>>>>>>>>> What do these do?  Can you comment these out and see if the
>>>>>>>>>>>> memory
>>>>>>>>>>>> leak
>>>>>>>>>>>> still exists?  Since you are passing the modelAuthToken into
>>>>>>>>>>>> this
>>>>>>>>>>>> method,
>>>>>>>>>>>> I
>>>>>>>>>>>> don't know what it's doing with the reference and could it be
>>>>>>>>>>>> holding
>>>>>>>>>>>> onto
>>>>>>>>>>>> something to prevent the GC from cleaning up?
>>>>>>>>>>>>
>>>>>>>>>>>> The rest of your example seems very straight forward with
>>>>>>>>>>>> creating
>>>>>>>>>>>> and
>>>>>>>>>>>> persisting objects.
>>>>>>>>>>>>
>>>>>>>>>>>> Kevin
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <
>>>>>>>>>>>> curtisr7@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> If you change the 1000 to something like 1000000... does your
>>>>>>>>>>>>> application
>>>>>>>>>>>>> go
>>>>>>>>>>>>> OOM? Are you running in a JSE environment? What is
>>>>>>>>>>>>> PersistenceManager?
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <
>>>>>>>>>>>>> kurt.stam@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> BTW I'm running with the cache off
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (that turns it off right?)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --Kurt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the
>>>>>>>>>>>>>>> database.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>>>>>>>>      EntityManager em =
>>>>>>>>>>>>>>> PersistenceManager.getEntityManager();
>>>>>>>>>>>>>>>      EntityTransaction tx = em.getTransaction();
>>>>>>>>>>>>>>>      try {
>>>>>>>>>>>>>>>          tx.begin();
>>>>>>>>>>>>>>>              // Generate auth token and store it!
>>>>>>>>>>>>>>>          String authInfo = AUTH_TOKEN_PREFIX +
>>>>>>>>>>>>>>> UUID.randomUUID();
>>>>>>>>>>>>>>>          org.apache.juddi.model.AuthToken modelAuthToken =
>>>>>>>>>>>>>>> new
>>>>>>>>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>>>>>>>>          if (authInfo != null) {
>>>>>>>>>>>>>>>              modelAuthToken.setAuthToken(authInfo);
>>>>>>>>>>>>>>>              modelAuthToken.setCreated(new Date());
>>>>>>>>>>>>>>>              modelAuthToken.setLastUsed(new Date());
>>>>>>>>>>>>>>>              modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>>>>>>>>              modelAuthToken.setNumberOfUses(0);
>>>>>>>>>>>>>>>              modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>>>>>>>>                em.persist(modelAuthToken);
>>>>>>>>>>>>>>>          }
>>>>>>>>>>>>>>>            apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>>>>>            MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>>>>>            tx.commit();
>>>>>>>>>>>>>>>                    } finally {
>>>>>>>>>>>>>>>          if (tx.isActive()) {
>>>>>>>>>>>>>>>              tx.rollback();
>>>>>>>>>>>>>>>          }
>>>>>>>>>>>>>>>          em.clear();
>>>>>>>>>>>>>>>          em.close();
>>>>>>>>>>>>>>>      }
>>>>>>>>>>>>>>>  }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [ISSUE]
>>>>>>>>>>>>>>> After it leaving this code I end up with a 1000
>>>>>>>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been
>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This seems to be pretty the most common use case of using an
>>>>>>>>>>>>>>> OR-mapping
>>>>>>>>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak
>>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>>> Does
>>>>>>>>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to
>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>> example
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> does not exhibit this behavior? BTW same code using hibernate
>>>>>>>>>>>>>> does
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> accumulate these objects.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thx,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Kurt
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Apache jUDDI.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Rick
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>

Re: memory leak? - simple question

Posted by Mark Struberg <st...@yahoo.de>.
Kurt, I'll try to look at your maven plugin configuration next week to check if there's any problem. I like to roll a new release of the openjpa-maven-plugin anyway. I'm just not sure if I should still stick with having a specific openjpa version part of the plugin dependencies, or if I should move it to <scope>provided</scope>. This way the users would need to specify the openjpa dependency of the plugin in any way - but that may also make the use less error prone because the user always will use the correct openjpa version. This would also be a benefit for OpenJPA-1 vs OpenJPA2 (because they have different artifactIds).

LieGrue,
strub


--- On Fri, 1/15/10, Kurt T Stam <ku...@gmail.com> wrote:

> From: Kurt T Stam <ku...@gmail.com>
> Subject: Re: memory leak? - simple question
> To: users@openjpa.apache.org
> Date: Friday, January 15, 2010, 5:15 PM
> Thanks so much Kevin, that did the
> trick. Some of the issues you 
> mentioned where introduced by me not copying right from
> the
> jUDDI project, but the main issue is that the
> openjpa-plugin simply does 
> not run for some reason. Anyway it is no longer leaking
> now! I learned a 
> lot from this exercise. And yes my class jumped from 2 to
> 10 k. BTW I'm 
> pretty sure this used to work..
> 
> Thanks again! Going to fix it the jUDDI build now :)!
> 
> --Kurt
> 
> Kevin Sutter wrote:
> > Hi Kurt,
> > I have not run a profiler to verify the memory leak,
> but I think I found the
> > problem...
> >
> > Although you thought you were running the PCEnhancer
> during build time, it
> > was not being performed.  When I looked at the
> Tomcat log with the original
> > jar file, the following message was logged which
> indicates you were falling
> > back to the "not ready for production" subclassing
> support:
> >
> > 1844  juddiDatabase 
> INFO   [http-8080-1] openjpa.Enhance -
> Creating
> > subclass for "[class org.apache.juddi.AuthToken]".
> This means that your
> > application will be  less efficient and will
> consume more memory than it
> > would if you ran the OpenJPA enhancer. Additionally,
> lazy loading will not
> > be available for one-to-one and many-to-one persistent
> attributes in types
> > using field access; they will be loaded eagerly
> instead.
> >
> > I made a few changes to your pom.xml and the "core"
> module to get the
> > enhancement to work and I believe the memory leak has
> went away.  Like I
> > said, I did not run a profiler, but I added a
> "finalize" method to your
> > AuthToken class and I now see that these entities are
> getting GC'd.
> >
> > One thing I would do is add the following property to
> your persistence.xml.
> >
> > <property name="openjpa.RuntimeUnenhancedClasses"
> value="warn"/>
> >
> > This property will produce the following warning if
> you accidentally fall
> > into this subclassing support.  We turn this off
> by default when running
> > within WebSphere since we really don't want customers
> to accidentally use
> > this subclassing.  And, we have turned this off
> for 1.3.x and trunk due to
> > the various problems associated with it. 
> Unfortunately, the 1.2.x branch is
> > kind of stuck with it...
> >
> > 1688  juddiDatabase 
> WARN   [http-8080-1] openjpa.Enhance - This
> > configuration disallows runtime optimization, but the
> following listed types
> > were not enhanced at build time or at class load time
> with a javaagent:
> > "[class org.apache.juddi.AuthToken]".
> >
> > And, of course, your webservice no longer works
> either.  You will get an
> > HTTP Status 500 error with an error message and call
> stack that further
> > explains the error.
> >
> > To get the maven plugin to work properly...  I
> made several changes, so
> > hopefully I remember everything...
> >
> > o  The number one thing is that the
> openjpa:enhance goal doesn't seem to get
> > automatically invoked when you compile.  I have
> no idea why, but that
> > doesn't seem to work.
> >
> > o  You have to make your persistence.xml file
> available to the "core"
> > module.  For my testing, I just created a new
> directory in the "core" module
> > called
> "\openjpa-leak\core\src\main\resources\META-INF\persistence.xml".
> > This will automatically get copied over to "target"
> and then the
> > openjpa:enhance goal will find it.
> >
> > o  How you want to make your persistence.xml file
> available to both your
> > "core" module and the "leak-war" module is up to
> you...  :-)
> >
> > o  I added a dependency in your core\pom.xml file
> for the 1.2.1 openjpa.
> > The documentation [1] says it will default to 1.2.0,
> if none is specified.
> >
> >         
>    <dependencies>
> >             
>    <dependency>
> >               
>  
>    <groupId>org.apache.openjpa</groupId>
> >               
>  
>    <artifactId>openjpa</artifactId>
> >               
>  
>    <version>1.2.1</version>
> >             
>    </dependency>
> >         
>    </dependencies>
> >
> > o  Your plugin properties don't seem to be
> specified correctly, at least
> > according to the documentation, but I didn't touch
> them and things still
> > seemed to work (once I got around the other
> problems).
> >
> > o  I also changed where to find the entities to
> enhance since this pom.xml
> > still specified the "model" directory.  There
> also seems to be confusion as
> > to whether to specify <classes> or
> <includes>.  I went with <includes> since
> > that's what the example showed.
> >
> > o  It's very easy to see if the enhancement
> process worked or not since the
> > size of your entity classes increases by a few
> thousand bytes.  And, with
> > the extra property above, this will prevent you from
> accidentally using the
> > unenhanced version.
> >
> > Hope this helps!  Let me know if this gets you
> around the memory leak.
> >
> > Kevin
> >
> > [1]  http://mojo.codehaus.org/openjpa-maven-plugin/usage.html
> >
> >
> >
> > On Thu, Jan 14, 2010 at 2:12 PM, Kurt T Stam <ku...@gmail.com>
> wrote:
> >
> >   
> >> 1. Cool that you got it running :)
> >>
> >> 2. I just ran with openjpa 1.2.2.SNAPSHOT and
> still see an accumulation of
> >> AuthToken objects.
> >>
> >> --K
> >>
> >>
> >> Kevin Sutter wrote:
> >>
> >>     
> >>> Hey, this wasn't so difficult... 
> :-)  Maybe I can still learn new
> >>> things...  I have Tomcat up and running
> and I can get your app running.
> >>>
> >>> On Thu, Jan 14, 2010 at 1:46 PM, Kevin Sutter
> <kw...@gmail.com>
> wrote:
> >>>
> >>>
> >>>
> >>>       
> >>>> Hi Kurt,
> >>>> These instructions, of course, assume that
> I know something about Tomcat.
> >>>> This would not be a good
> assumption...  :-)  If you really want me to see
> >>>> this in action, I will need more
> information on how to start Tomcat and
> >>>> how
> >>>> to load applications.  In the mean
> time, a trace log of OpenJPA would be
> >>>> an
> >>>> excellent start.
> >>>>
> >>>> Thanks!
> >>>> Kevin
> >>>>
> >>>>
> >>>> On Thu, Jan 14, 2010 at 11:30 AM, Kurt T
> Stam <ku...@gmail.com>
> >>>> wrote:
> >>>>
> >>>>
> >>>>
> >>>>         
> >>>>> Hi Kevin,
> >>>>>
> >>>>> I tried adding the javaagent stuff,
> and it still ends up leaking. I
> >>>>> extracted the code into a tiny
> >>>>> war with just one servlet which
> persist the authToken. The build creates
> >>>>> a
> >>>>> Tomcat distro backed by a
> >>>>> derby database. You can download it
> from:
> >>>>>
> >>>>> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz>
> >>>>> <http://people.apache.org/%7Ekstam/leak.tgz>
> >>>>>
> >>>>>
> >>>>> You can build it with: mvn install,
> you can find the final tomcat in
> >>>>> tomcat/target/tomcat-1.0.SNAPSHOT.zip
> >>>>>
> >>>>> or if you don't want to build it:
> >>>>> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
> >>>>> <http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
> >>>>>
> >>>>>
> >>>>> After you bring up tomcat it
> persistent an authToken when you hit
> >>>>> http://localhost:8080/leak/test
> >>>>>
> >>>>> In the profiler I see an
> org.apache.juddi.AuthToken left behind for each
> >>>>> request.
> >>>>>
> >>>>> Hopefully this little example will
> pinpoint what's going on.
> >>>>>
> >>>>> Thx Kev for sticking with me :)
> >>>>>
> >>>>> --Kurt
> >>>>>
> >>>>>
> >>>>>
> >>>>> Kevin Sutter wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>       
>    
> >>>>>> Hi Kurt,
> >>>>>> Since you have the environment to
> reproduce this, could you run with
> >>>>>> the
> >>>>>> OpenJPA trace enabled?  I'm
> not even sure what I'm looking for at this
> >>>>>> point.  :-)  But, maybe
> there's some tracing that would point us in the
> >>>>>> right direction.
> >>>>>>
> >>>>>> <property name="openjpa.Log"
> value="DefaultLevel=TRACE"/>
> >>>>>>
> >>>>>> We have been always been on the
> watch for memory leaks, so this one is
> >>>>>> puzzling.  You did mention
> blocked threads as well.  At this point, you
> >>>>>> are
> >>>>>> probably hitting blocked threads
> due to the GC and Memory problems.
> >>>>>>  But,
> >>>>>> we
> >>>>>> did recently resolve some locking
> situations [1] in our
> >>>>>> MetaDataRepository
> >>>>>> that could affect scaling (under
> extreme load).  This was resolved in
> >>>>>> the
> >>>>>> 1.2.x SNAPSHOT drivers (Mike is
> currently pushing for a 1.2.2 release,
> >>>>>> so
> >>>>>> that should be coming soon).
> >>>>>>
> >>>>>> Does your WebService version of
> the problem use EntityManager
> >>>>>> transactions,
> >>>>>> or is the transaction service
> coming from Tomcat?  And, does your
> >>>>>> WebService
> >>>>>> version also do the em.clear() and
> em.close()?  Basically, are you
> >>>>>> using
> >>>>>> application-managed or
> container-managed persistence while running as a
> >>>>>> WebService?
> >>>>>>
> >>>>>> I'm very interested in helping to
> resolve this issue.  Like I
> >>>>>> mentioned,
> >>>>>> we
> >>>>>> do a lot of memory leakage and
> scalability testing of the OpenJPA
> >>>>>> solution
> >>>>>> and we're not aware of
> anything.  Although there is always the
> >>>>>> possibility
> >>>>>> that something was resolved in the
> 1.2.x branch after the 1.2.1 release
> >>>>>> that
> >>>>>> I'm not remembering...
> >>>>>>
> >>>>>> Thanks for your help,
> >>>>>> Kevin
> >>>>>>
> >>>>>> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
> >>>>>>
> >>>>>> On Wed, Jan 13, 2010 at 9:27 PM,
> Kurt T Stam <ku...@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>         
>    
> >>>>>>> Thanks Kevin,
> >>>>>>>
> >>>>>>> We're enhancing at build
> time:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
> >>>>>>>
> >>>>>>> Yeah we've been running load
> tests and things are nice and stable with
> >>>>>>> Hibernate but with Openjpa we
> see increasing memory use, blocking
> >>>>>>> threads
> >>>>>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
> >>>>>>> preference would be to ship
> with openjpa by default; but our build
> >>>>>>> supports
> >>>>>>> both hibernate and openjpa.
> >>>>>>>
> >>>>>>> And yes we use openjpa 1.2.1
> (latest stable version).
> >>>>>>>
> >>>>>>> --Kurt
> >>>>>>>
> >>>>>>>
> >>>>>>> Kevin Sutter wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>       
>        
> >>>>>>>> Interesting detective
> work, Kurt.  Thanks.
> >>>>>>>>
> >>>>>>>> Why the WebService version
> of the app would behave differently as far
> >>>>>>>> as
> >>>>>>>> GC
> >>>>>>>> is concerned is a
> mystery.  And, you said that plugging in Hibernate
> >>>>>>>> into
> >>>>>>>> this scenario, everything
> works okay?  Very confusing.
> >>>>>>>>
> >>>>>>>> How are you performing the
> Entity enhancement processing?  Are you
> >>>>>>>> pre-enhancing via your
> build process?  Or, are you using the
> >>>>>>>> -javaagent
> >>>>>>>> mechanism?  Or, are
> you falling back to the subclassing support
> >>>>>>>> within
> >>>>>>>> OpenJPA?  (See [1]
> for more information on these questions in case
> >>>>>>>> they
> >>>>>>>> don't make sense.)
> >>>>>>>>
> >>>>>>>> This would be one area
> that is different between Hibernate and
> >>>>>>>> OpenJPA
> >>>>>>>> --
> >>>>>>>> enhancement processing.
> >>>>>>>>
> >>>>>>>> In the Tomcat environment,
> you may be falling back to the subclassing
> >>>>>>>> support (which we do not
> recommend) and hitting a memory leak with
> >>>>>>>> that.
> >>>>>>>>
> >>>>>>>> You said OpenJPA 1.2.x,
> right?
> >>>>>>>>
> >>>>>>>> Just a couple of thoughts
> on the subject...
> >>>>>>>> Kevin
> >>>>>>>>
> >>>>>>>> [1]
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, Jan 13, 2010 at
> 4:25 PM, Kurt T Stam <ku...@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>       
>          
> >>>>>>>>> The same code executed
> straight from a java client (inVM) shows no
> >>>>>>>>> memory
> >>>>>>>>> leak.
> >>>>>>>>>
> >>>>>>>>> So is the fact that it
> is WebService significant then? What else can
> >>>>>>>>> be
> >>>>>>>>> different? I think one
> thread remains up, and somehow this causes
> >>>>>>>>> openjpa
> >>>>>>>>> not being able to
> clean up after itself. What can I do to debug this
> >>>>>>>>> more? I
> >>>>>>>>> can actually see in
> the profiler that the objects are allocated by
> >>>>>>>>> the
> >>>>>>>>> WebService, but why
> aren't they cleaned up?
> >>>>>>>>>
> >>>>>>>>> Thx,
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --Kurt
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Kurt T Stam wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>     
>              
> >>>>>>>>>> Thanks Kevin,
> thanks for your response.
> >>>>>>>>>>
> >>>>>>>>>> I just replaced
> the static call by:
> >>>>>>>>>>
> >>>>>>>>>>   
>         apiAuthToken = new
> org.uddi.api_v3.AuthToken();
> >>>>>>>>>>   
>        
> apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
> >>>>>>>>>>   
>        
> //MappingModelToApi.mapAuthToken(modelAuthToken,
> >>>>>>>>>> apiAuthToken);
> >>>>>>>>>>
> >>>>>>>>>> which did not make
> a difference.
> >>>>>>>>>>
> >>>>>>>>>> I'm wondering if
> the fact that my class is a webservice makes a
> >>>>>>>>>> difference. I'll
> try extracting it into
> >>>>>>>>>> a regular class
> with a main method and profile that. At least I
> >>>>>>>>>> know
> >>>>>>>>>> that
> >>>>>>>>>> I didn't forget
> something
> >>>>>>>>>> completely
> obvious..
> >>>>>>>>>>
> >>>>>>>>>> --Kurt
> >>>>>>>>>>
> >>>>>>>>>> Kevin Sutter
> wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>   
>              
>    
> >>>>>>>>>>> Kurt,
> >>>>>>>>>>> I agree that
> this is very common usage of the JPA programming
> >>>>>>>>>>> model.
> >>>>>>>>>>>  And,
> >>>>>>>>>>> we are not
> aware of any memory leaks.  About the only thing that
> >>>>>>>>>>> jumps
> >>>>>>>>>>> out
> >>>>>>>>>>> at me is the
> following two lines:
> >>>>>>>>>>>
> >>>>>>>>>>>   
>          apiAuthToken = new
> org.uddi.api_v3.AuthToken();
> >>>>>>>>>>>   
>      
>    MappingModelToApi.mapAuthToken(modelAuthToken,
> >>>>>>>>>>>
> apiAuthToken);
> >>>>>>>>>>>
> >>>>>>>>>>> What do these
> do?  Can you comment these out and see if the memory
> >>>>>>>>>>> leak
> >>>>>>>>>>> still
> exists?  Since you are passing the modelAuthToken into
> this
> >>>>>>>>>>> method,
> >>>>>>>>>>> I
> >>>>>>>>>>> don't know
> what it's doing with the reference and could it be
> >>>>>>>>>>> holding
> >>>>>>>>>>> onto
> >>>>>>>>>>> something to
> prevent the GC from cleaning up?
> >>>>>>>>>>>
> >>>>>>>>>>> The rest of
> your example seems very straight forward with creating
> >>>>>>>>>>> and
> >>>>>>>>>>> persisting
> objects.
> >>>>>>>>>>>
> >>>>>>>>>>> Kevin
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Jan
> 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>   
>                
>    
> >>>>>>>>>>>> If you
> change the 1000 to something like 1000000... does your
> >>>>>>>>>>>>
> application
> >>>>>>>>>>>> go
> >>>>>>>>>>>> OOM? Are
> you running in a JSE environment? What is
> >>>>>>>>>>>>
> PersistenceManager?
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed,
> Jan 13, 2010 at 2:05 PM, Kurt T Stam <
> >>>>>>>>>>>> kurt.stam@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> 
>                
>        
> >>>>>>>>>>>>> BTW
> I'm running with the cache off
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> <property name="openjpa.DataCache" value="false"/>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> (that
> turns it off right?)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> --Kurt
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Kurt T
> Stam wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 
>                
>          
> >>>>>>>>>>>>>> Hi
> guys,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> [DESCRIPTION] The code below inserts a 1000 records in the
> >>>>>>>>>>>>>>
> database.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> for (int i=1; i<1000; i++) {
> >>>>>>>>>>>>>> 
>      EntityManager em =
> PersistenceManager.getEntityManager();
> >>>>>>>>>>>>>> 
>      EntityTransaction tx =
> em.getTransaction();
> >>>>>>>>>>>>>> 
>      try {
> >>>>>>>>>>>>>> 
>          tx.begin();
> >>>>>>>>>>>>>> 
>              //
> Generate auth token and store it!
> >>>>>>>>>>>>>> 
>          String authInfo =
> AUTH_TOKEN_PREFIX +
> >>>>>>>>>>>>>>
> UUID.randomUUID();
> >>>>>>>>>>>>>> 
>      
>    org.apache.juddi.model.AuthToken
> modelAuthToken = new
> >>>>>>>>>>>>>>
> org.apache.juddi.model.AuthToken();
> >>>>>>>>>>>>>> 
>          if (authInfo != null)
> {
> >>>>>>>>>>>>>> 
>          
>    modelAuthToken.setAuthToken(authInfo);
> >>>>>>>>>>>>>> 
>          
>    modelAuthToken.setCreated(new Date());
> >>>>>>>>>>>>>> 
>          
>    modelAuthToken.setLastUsed(new Date());
> >>>>>>>>>>>>>> 
>          
>    modelAuthToken.setAuthorizedName(publisherId);
> >>>>>>>>>>>>>> 
>          
>    modelAuthToken.setNumberOfUses(0);
> >>>>>>>>>>>>>> 
>          
>    modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
> >>>>>>>>>>>>>> 
>            
>    em.persist(modelAuthToken);
> >>>>>>>>>>>>>> 
>          }
> >>>>>>>>>>>>>> 
>            apiAuthToken =
> new org.uddi.api_v3.AuthToken();
> >>>>>>>>>>>>>> 
>        
>    MappingModelToApi.mapAuthToken(modelAuthToken,
> >>>>>>>>>>>>>>
> apiAuthToken);
> >>>>>>>>>>>>>> 
>            tx.commit();
> >>>>>>>>>>>>>> 
>                
>    } finally {
> >>>>>>>>>>>>>> 
>          if (tx.isActive()) {
> >>>>>>>>>>>>>> 
>          
>    tx.rollback();
> >>>>>>>>>>>>>> 
>          }
> >>>>>>>>>>>>>> 
>          em.clear();
> >>>>>>>>>>>>>> 
>          em.close();
> >>>>>>>>>>>>>> 
>      }
> >>>>>>>>>>>>>>   }
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> [ISSUE]
> >>>>>>>>>>>>>>
> After it leaving this code I end up with a 1000
> >>>>>>>>>>>>>>
> org.apache.juddi.model.AuthToken objects in memory. I've
> been
> >>>>>>>>>>>>>>
> using
> >>>>>>>>>>>>>>
> the
> >>>>>>>>>>>>>>
> profiler, and these objects cannot be garbage collected.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> This seems to be pretty the most common use case of using
> an
> >>>>>>>>>>>>>>
> OR-mapping
> >>>>>>>>>>>>>>
> tool, so I find it hard to believe openjpa has a memory
> leak
> >>>>>>>>>>>>>>
> here.
> >>>>>>>>>>>>>>
> Does
> >>>>>>>>>>>>>>
> anyone see what I'm doing wrong? Or can someone point me to
> an
> >>>>>>>>>>>>>>
> example
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 
>                
>            
> >>>>>>>>>>>>> that
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 
>                
>          
> >>>>>>>>>>>> 
>                
>        
> >>>>>>>>>>>>> does
> not exhibit this behavior? BTW same code using hibernate
> >>>>>>>>>>>>> does
> >>>>>>>>>>>>> not
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 
>                
>          
> >>>>>>>>>>>>>>
> accumulate these objects.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> We're using openjpa 1.2.1.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> Thx,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> Kurt
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> Apache jUDDI.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 
>                
>            
> >>>>>>>>>>>>> 
>                
>          
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> Rick
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> 
>                
>        
> >>>>>>>>>>>
> >>>>>>>>>>>   
>                
>    
> >>>>>>>>>>   
>              
>    
> >>>>>>
> >>>>>>         
>    
> >>>>>       
>    
> >>>       
> >>     
> >
> >   
> 
> 


      

Re: memory leak? - simple question

Posted by Kurt T Stam <ku...@gmail.com>.
Thanks so much Kevin, that did the trick. Some of the issues you 
mentioned where introduced by me not copying right from the
jUDDI project, but the main issue is that the openjpa-plugin simply does 
not run for some reason. Anyway it is no longer leaking now! I learned a 
lot from this exercise. And yes my class jumped from 2 to 10 k. BTW I'm 
pretty sure this used to work..

Thanks again! Going to fix it the jUDDI build now :)!

--Kurt

Kevin Sutter wrote:
> Hi Kurt,
> I have not run a profiler to verify the memory leak, but I think I found the
> problem...
>
> Although you thought you were running the PCEnhancer during build time, it
> was not being performed.  When I looked at the Tomcat log with the original
> jar file, the following message was logged which indicates you were falling
> back to the "not ready for production" subclassing support:
>
> 1844  juddiDatabase  INFO   [http-8080-1] openjpa.Enhance - Creating
> subclass for "[class org.apache.juddi.AuthToken]". This means that your
> application will be  less efficient and will consume more memory than it
> would if you ran the OpenJPA enhancer. Additionally, lazy loading will not
> be available for one-to-one and many-to-one persistent attributes in types
> using field access; they will be loaded eagerly instead.
>
> I made a few changes to your pom.xml and the "core" module to get the
> enhancement to work and I believe the memory leak has went away.  Like I
> said, I did not run a profiler, but I added a "finalize" method to your
> AuthToken class and I now see that these entities are getting GC'd.
>
> One thing I would do is add the following property to your persistence.xml.
>
> <property name="openjpa.RuntimeUnenhancedClasses" value="warn"/>
>
> This property will produce the following warning if you accidentally fall
> into this subclassing support.  We turn this off by default when running
> within WebSphere since we really don't want customers to accidentally use
> this subclassing.  And, we have turned this off for 1.3.x and trunk due to
> the various problems associated with it.  Unfortunately, the 1.2.x branch is
> kind of stuck with it...
>
> 1688  juddiDatabase  WARN   [http-8080-1] openjpa.Enhance - This
> configuration disallows runtime optimization, but the following listed types
> were not enhanced at build time or at class load time with a javaagent:
> "[class org.apache.juddi.AuthToken]".
>
> And, of course, your webservice no longer works either.  You will get an
> HTTP Status 500 error with an error message and call stack that further
> explains the error.
>
> To get the maven plugin to work properly...  I made several changes, so
> hopefully I remember everything...
>
> o  The number one thing is that the openjpa:enhance goal doesn't seem to get
> automatically invoked when you compile.  I have no idea why, but that
> doesn't seem to work.
>
> o  You have to make your persistence.xml file available to the "core"
> module.  For my testing, I just created a new directory in the "core" module
> called "\openjpa-leak\core\src\main\resources\META-INF\persistence.xml".
> This will automatically get copied over to "target" and then the
> openjpa:enhance goal will find it.
>
> o  How you want to make your persistence.xml file available to both your
> "core" module and the "leak-war" module is up to you...  :-)
>
> o  I added a dependency in your core\pom.xml file for the 1.2.1 openjpa.
> The documentation [1] says it will default to 1.2.0, if none is specified.
>
>             <dependencies>
>                 <dependency>
>                     <groupId>org.apache.openjpa</groupId>
>                     <artifactId>openjpa</artifactId>
>                     <version>1.2.1</version>
>                 </dependency>
>             </dependencies>
>
> o  Your plugin properties don't seem to be specified correctly, at least
> according to the documentation, but I didn't touch them and things still
> seemed to work (once I got around the other problems).
>
> o  I also changed where to find the entities to enhance since this pom.xml
> still specified the "model" directory.  There also seems to be confusion as
> to whether to specify <classes> or <includes>.  I went with <includes> since
> that's what the example showed.
>
> o  It's very easy to see if the enhancement process worked or not since the
> size of your entity classes increases by a few thousand bytes.  And, with
> the extra property above, this will prevent you from accidentally using the
> unenhanced version.
>
> Hope this helps!  Let me know if this gets you around the memory leak.
>
> Kevin
>
> [1]  http://mojo.codehaus.org/openjpa-maven-plugin/usage.html
>
>
>
> On Thu, Jan 14, 2010 at 2:12 PM, Kurt T Stam <ku...@gmail.com> wrote:
>
>   
>> 1. Cool that you got it running :)
>>
>> 2. I just ran with openjpa 1.2.2.SNAPSHOT and still see an accumulation of
>> AuthToken objects.
>>
>> --K
>>
>>
>> Kevin Sutter wrote:
>>
>>     
>>> Hey, this wasn't so difficult...  :-)  Maybe I can still learn new
>>> things...  I have Tomcat up and running and I can get your app running.
>>>
>>> On Thu, Jan 14, 2010 at 1:46 PM, Kevin Sutter <kw...@gmail.com> wrote:
>>>
>>>
>>>
>>>       
>>>> Hi Kurt,
>>>> These instructions, of course, assume that I know something about Tomcat.
>>>> This would not be a good assumption...  :-)  If you really want me to see
>>>> this in action, I will need more information on how to start Tomcat and
>>>> how
>>>> to load applications.  In the mean time, a trace log of OpenJPA would be
>>>> an
>>>> excellent start.
>>>>
>>>> Thanks!
>>>> Kevin
>>>>
>>>>
>>>> On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <ku...@gmail.com>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> Hi Kevin,
>>>>>
>>>>> I tried adding the javaagent stuff, and it still ends up leaking. I
>>>>> extracted the code into a tiny
>>>>> war with just one servlet which persist the authToken. The build creates
>>>>> a
>>>>> Tomcat distro backed by a
>>>>> derby database. You can download it from:
>>>>>
>>>>> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz>
>>>>> <http://people.apache.org/%7Ekstam/leak.tgz>
>>>>>
>>>>>
>>>>> You can build it with: mvn install, you can find the final tomcat in
>>>>> tomcat/target/tomcat-1.0.SNAPSHOT.zip
>>>>>
>>>>> or if you don't want to build it:
>>>>> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>>>> <http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>>>>
>>>>>
>>>>> After you bring up tomcat it persistent an authToken when you hit
>>>>> http://localhost:8080/leak/test
>>>>>
>>>>> In the profiler I see an org.apache.juddi.AuthToken left behind for each
>>>>> request.
>>>>>
>>>>> Hopefully this little example will pinpoint what's going on.
>>>>>
>>>>> Thx Kev for sticking with me :)
>>>>>
>>>>> --Kurt
>>>>>
>>>>>
>>>>>
>>>>> Kevin Sutter wrote:
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> Hi Kurt,
>>>>>> Since you have the environment to reproduce this, could you run with
>>>>>> the
>>>>>> OpenJPA trace enabled?  I'm not even sure what I'm looking for at this
>>>>>> point.  :-)  But, maybe there's some tracing that would point us in the
>>>>>> right direction.
>>>>>>
>>>>>> <property name="openjpa.Log" value="DefaultLevel=TRACE"/>
>>>>>>
>>>>>> We have been always been on the watch for memory leaks, so this one is
>>>>>> puzzling.  You did mention blocked threads as well.  At this point, you
>>>>>> are
>>>>>> probably hitting blocked threads due to the GC and Memory problems.
>>>>>>  But,
>>>>>> we
>>>>>> did recently resolve some locking situations [1] in our
>>>>>> MetaDataRepository
>>>>>> that could affect scaling (under extreme load).  This was resolved in
>>>>>> the
>>>>>> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release,
>>>>>> so
>>>>>> that should be coming soon).
>>>>>>
>>>>>> Does your WebService version of the problem use EntityManager
>>>>>> transactions,
>>>>>> or is the transaction service coming from Tomcat?  And, does your
>>>>>> WebService
>>>>>> version also do the em.clear() and em.close()?  Basically, are you
>>>>>> using
>>>>>> application-managed or container-managed persistence while running as a
>>>>>> WebService?
>>>>>>
>>>>>> I'm very interested in helping to resolve this issue.  Like I
>>>>>> mentioned,
>>>>>> we
>>>>>> do a lot of memory leakage and scalability testing of the OpenJPA
>>>>>> solution
>>>>>> and we're not aware of anything.  Although there is always the
>>>>>> possibility
>>>>>> that something was resolved in the 1.2.x branch after the 1.2.1 release
>>>>>> that
>>>>>> I'm not remembering...
>>>>>>
>>>>>> Thanks for your help,
>>>>>> Kevin
>>>>>>
>>>>>> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
>>>>>>
>>>>>> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <ku...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> Thanks Kevin,
>>>>>>>
>>>>>>> We're enhancing at build time:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>>>>>>
>>>>>>> Yeah we've been running load tests and things are nice and stable with
>>>>>>> Hibernate but with Openjpa we see increasing memory use, blocking
>>>>>>> threads
>>>>>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
>>>>>>> preference would be to ship with openjpa by default; but our build
>>>>>>> supports
>>>>>>> both hibernate and openjpa.
>>>>>>>
>>>>>>> And yes we use openjpa 1.2.1 (latest stable version).
>>>>>>>
>>>>>>> --Kurt
>>>>>>>
>>>>>>>
>>>>>>> Kevin Sutter wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> Interesting detective work, Kurt.  Thanks.
>>>>>>>>
>>>>>>>> Why the WebService version of the app would behave differently as far
>>>>>>>> as
>>>>>>>> GC
>>>>>>>> is concerned is a mystery.  And, you said that plugging in Hibernate
>>>>>>>> into
>>>>>>>> this scenario, everything works okay?  Very confusing.
>>>>>>>>
>>>>>>>> How are you performing the Entity enhancement processing?  Are you
>>>>>>>> pre-enhancing via your build process?  Or, are you using the
>>>>>>>> -javaagent
>>>>>>>> mechanism?  Or, are you falling back to the subclassing support
>>>>>>>> within
>>>>>>>> OpenJPA?  (See [1] for more information on these questions in case
>>>>>>>> they
>>>>>>>> don't make sense.)
>>>>>>>>
>>>>>>>> This would be one area that is different between Hibernate and
>>>>>>>> OpenJPA
>>>>>>>> --
>>>>>>>> enhancement processing.
>>>>>>>>
>>>>>>>> In the Tomcat environment, you may be falling back to the subclassing
>>>>>>>> support (which we do not recommend) and hitting a memory leak with
>>>>>>>> that.
>>>>>>>>
>>>>>>>> You said OpenJPA 1.2.x, right?
>>>>>>>>
>>>>>>>> Just a couple of thoughts on the subject...
>>>>>>>> Kevin
>>>>>>>>
>>>>>>>> [1]
>>>>>>>>
>>>>>>>>
>>>>>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> The same code executed straight from a java client (inVM) shows no
>>>>>>>>> memory
>>>>>>>>> leak.
>>>>>>>>>
>>>>>>>>> So is the fact that it is WebService significant then? What else can
>>>>>>>>> be
>>>>>>>>> different? I think one thread remains up, and somehow this causes
>>>>>>>>> openjpa
>>>>>>>>> not being able to clean up after itself. What can I do to debug this
>>>>>>>>> more? I
>>>>>>>>> can actually see in the profiler that the objects are allocated by
>>>>>>>>> the
>>>>>>>>> WebService, but why aren't they cleaned up?
>>>>>>>>>
>>>>>>>>> Thx,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --Kurt
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>> Thanks Kevin, thanks for your response.
>>>>>>>>>>
>>>>>>>>>> I just replaced the static call by:
>>>>>>>>>>
>>>>>>>>>>            apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>            apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>>>>>>>            //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>> apiAuthToken);
>>>>>>>>>>
>>>>>>>>>> which did not make a difference.
>>>>>>>>>>
>>>>>>>>>> I'm wondering if the fact that my class is a webservice makes a
>>>>>>>>>> difference. I'll try extracting it into
>>>>>>>>>> a regular class with a main method and profile that. At least I
>>>>>>>>>> know
>>>>>>>>>> that
>>>>>>>>>> I didn't forget something
>>>>>>>>>> completely obvious..
>>>>>>>>>>
>>>>>>>>>> --Kurt
>>>>>>>>>>
>>>>>>>>>> Kevin Sutter wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>>> Kurt,
>>>>>>>>>>> I agree that this is very common usage of the JPA programming
>>>>>>>>>>> model.
>>>>>>>>>>>  And,
>>>>>>>>>>> we are not aware of any memory leaks.  About the only thing that
>>>>>>>>>>> jumps
>>>>>>>>>>> out
>>>>>>>>>>> at me is the following two lines:
>>>>>>>>>>>
>>>>>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>             MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>
>>>>>>>>>>> What do these do?  Can you comment these out and see if the memory
>>>>>>>>>>> leak
>>>>>>>>>>> still exists?  Since you are passing the modelAuthToken into this
>>>>>>>>>>> method,
>>>>>>>>>>> I
>>>>>>>>>>> don't know what it's doing with the reference and could it be
>>>>>>>>>>> holding
>>>>>>>>>>> onto
>>>>>>>>>>> something to prevent the GC from cleaning up?
>>>>>>>>>>>
>>>>>>>>>>> The rest of your example seems very straight forward with creating
>>>>>>>>>>> and
>>>>>>>>>>> persisting objects.
>>>>>>>>>>>
>>>>>>>>>>> Kevin
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>>>> If you change the 1000 to something like 1000000... does your
>>>>>>>>>>>> application
>>>>>>>>>>>> go
>>>>>>>>>>>> OOM? Are you running in a JSE environment? What is
>>>>>>>>>>>> PersistenceManager?
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <
>>>>>>>>>>>> kurt.stam@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>                         
>>>>>>>>>>>>> BTW I'm running with the cache off
>>>>>>>>>>>>>
>>>>>>>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>>>>>>>
>>>>>>>>>>>>> (that turns it off right?)
>>>>>>>>>>>>>
>>>>>>>>>>>>> --Kurt
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>                           
>>>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the
>>>>>>>>>>>>>> database.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>>>>>>>       EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>>>>>>>>       EntityTransaction tx = em.getTransaction();
>>>>>>>>>>>>>>       try {
>>>>>>>>>>>>>>           tx.begin();
>>>>>>>>>>>>>>               // Generate auth token and store it!
>>>>>>>>>>>>>>           String authInfo = AUTH_TOKEN_PREFIX +
>>>>>>>>>>>>>> UUID.randomUUID();
>>>>>>>>>>>>>>           org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>>>>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>>>>>>>           if (authInfo != null) {
>>>>>>>>>>>>>>               modelAuthToken.setAuthToken(authInfo);
>>>>>>>>>>>>>>               modelAuthToken.setCreated(new Date());
>>>>>>>>>>>>>>               modelAuthToken.setLastUsed(new Date());
>>>>>>>>>>>>>>               modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>>>>>>>               modelAuthToken.setNumberOfUses(0);
>>>>>>>>>>>>>>               modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>>>>>>>                 em.persist(modelAuthToken);
>>>>>>>>>>>>>>           }
>>>>>>>>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>>>>             MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>>>>             tx.commit();
>>>>>>>>>>>>>>                     } finally {
>>>>>>>>>>>>>>           if (tx.isActive()) {
>>>>>>>>>>>>>>               tx.rollback();
>>>>>>>>>>>>>>           }
>>>>>>>>>>>>>>           em.clear();
>>>>>>>>>>>>>>           em.close();
>>>>>>>>>>>>>>       }
>>>>>>>>>>>>>>   }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [ISSUE]
>>>>>>>>>>>>>> After it leaving this code I end up with a 1000
>>>>>>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been
>>>>>>>>>>>>>> using
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This seems to be pretty the most common use case of using an
>>>>>>>>>>>>>> OR-mapping
>>>>>>>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak
>>>>>>>>>>>>>> here.
>>>>>>>>>>>>>> Does
>>>>>>>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an
>>>>>>>>>>>>>> example
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>                             
>>>>>>>>>>>>> that
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>                           
>>>>>>>>>>>>                         
>>>>>>>>>>>>> does not exhibit this behavior? BTW same code using hibernate
>>>>>>>>>>>>> does
>>>>>>>>>>>>> not
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>                           
>>>>>>>>>>>>>> accumulate these objects.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thx,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Kurt
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Apache jUDDI.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>                             
>>>>>>>>>>>>>                           
>>>>>>>>>>>> --
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Rick
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>                         
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>>                     
>>>>>>
>>>>>>             
>>>>>           
>>>       
>>     
>
>   


Re: memory leak? - simple question

Posted by Kevin Sutter <kw...@gmail.com>.
Hi Kurt,
I have not run a profiler to verify the memory leak, but I think I found the
problem...

Although you thought you were running the PCEnhancer during build time, it
was not being performed.  When I looked at the Tomcat log with the original
jar file, the following message was logged which indicates you were falling
back to the "not ready for production" subclassing support:

1844  juddiDatabase  INFO   [http-8080-1] openjpa.Enhance - Creating
subclass for "[class org.apache.juddi.AuthToken]". This means that your
application will be  less efficient and will consume more memory than it
would if you ran the OpenJPA enhancer. Additionally, lazy loading will not
be available for one-to-one and many-to-one persistent attributes in types
using field access; they will be loaded eagerly instead.

I made a few changes to your pom.xml and the "core" module to get the
enhancement to work and I believe the memory leak has went away.  Like I
said, I did not run a profiler, but I added a "finalize" method to your
AuthToken class and I now see that these entities are getting GC'd.

One thing I would do is add the following property to your persistence.xml.

<property name="openjpa.RuntimeUnenhancedClasses" value="warn"/>

This property will produce the following warning if you accidentally fall
into this subclassing support.  We turn this off by default when running
within WebSphere since we really don't want customers to accidentally use
this subclassing.  And, we have turned this off for 1.3.x and trunk due to
the various problems associated with it.  Unfortunately, the 1.2.x branch is
kind of stuck with it...

1688  juddiDatabase  WARN   [http-8080-1] openjpa.Enhance - This
configuration disallows runtime optimization, but the following listed types
were not enhanced at build time or at class load time with a javaagent:
"[class org.apache.juddi.AuthToken]".

And, of course, your webservice no longer works either.  You will get an
HTTP Status 500 error with an error message and call stack that further
explains the error.

To get the maven plugin to work properly...  I made several changes, so
hopefully I remember everything...

o  The number one thing is that the openjpa:enhance goal doesn't seem to get
automatically invoked when you compile.  I have no idea why, but that
doesn't seem to work.

o  You have to make your persistence.xml file available to the "core"
module.  For my testing, I just created a new directory in the "core" module
called "\openjpa-leak\core\src\main\resources\META-INF\persistence.xml".
This will automatically get copied over to "target" and then the
openjpa:enhance goal will find it.

o  How you want to make your persistence.xml file available to both your
"core" module and the "leak-war" module is up to you...  :-)

o  I added a dependency in your core\pom.xml file for the 1.2.1 openjpa.
The documentation [1] says it will default to 1.2.0, if none is specified.

            <dependencies>
                <dependency>
                    <groupId>org.apache.openjpa</groupId>
                    <artifactId>openjpa</artifactId>
                    <version>1.2.1</version>
                </dependency>
            </dependencies>

o  Your plugin properties don't seem to be specified correctly, at least
according to the documentation, but I didn't touch them and things still
seemed to work (once I got around the other problems).

o  I also changed where to find the entities to enhance since this pom.xml
still specified the "model" directory.  There also seems to be confusion as
to whether to specify <classes> or <includes>.  I went with <includes> since
that's what the example showed.

o  It's very easy to see if the enhancement process worked or not since the
size of your entity classes increases by a few thousand bytes.  And, with
the extra property above, this will prevent you from accidentally using the
unenhanced version.

Hope this helps!  Let me know if this gets you around the memory leak.

Kevin

[1]  http://mojo.codehaus.org/openjpa-maven-plugin/usage.html



On Thu, Jan 14, 2010 at 2:12 PM, Kurt T Stam <ku...@gmail.com> wrote:

> 1. Cool that you got it running :)
>
> 2. I just ran with openjpa 1.2.2.SNAPSHOT and still see an accumulation of
> AuthToken objects.
>
> --K
>
>
> Kevin Sutter wrote:
>
>> Hey, this wasn't so difficult...  :-)  Maybe I can still learn new
>> things...  I have Tomcat up and running and I can get your app running.
>>
>> On Thu, Jan 14, 2010 at 1:46 PM, Kevin Sutter <kw...@gmail.com> wrote:
>>
>>
>>
>>> Hi Kurt,
>>> These instructions, of course, assume that I know something about Tomcat.
>>> This would not be a good assumption...  :-)  If you really want me to see
>>> this in action, I will need more information on how to start Tomcat and
>>> how
>>> to load applications.  In the mean time, a trace log of OpenJPA would be
>>> an
>>> excellent start.
>>>
>>> Thanks!
>>> Kevin
>>>
>>>
>>> On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <ku...@gmail.com>
>>> wrote:
>>>
>>>
>>>
>>>> Hi Kevin,
>>>>
>>>> I tried adding the javaagent stuff, and it still ends up leaking. I
>>>> extracted the code into a tiny
>>>> war with just one servlet which persist the authToken. The build creates
>>>> a
>>>> Tomcat distro backed by a
>>>> derby database. You can download it from:
>>>>
>>>> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz>
>>>> <http://people.apache.org/%7Ekstam/leak.tgz>
>>>>
>>>>
>>>> You can build it with: mvn install, you can find the final tomcat in
>>>> tomcat/target/tomcat-1.0.SNAPSHOT.zip
>>>>
>>>> or if you don't want to build it:
>>>> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>>> <http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>>>
>>>>
>>>> After you bring up tomcat it persistent an authToken when you hit
>>>> http://localhost:8080/leak/test
>>>>
>>>> In the profiler I see an org.apache.juddi.AuthToken left behind for each
>>>> request.
>>>>
>>>> Hopefully this little example will pinpoint what's going on.
>>>>
>>>> Thx Kev for sticking with me :)
>>>>
>>>> --Kurt
>>>>
>>>>
>>>>
>>>> Kevin Sutter wrote:
>>>>
>>>>
>>>>
>>>>> Hi Kurt,
>>>>> Since you have the environment to reproduce this, could you run with
>>>>> the
>>>>> OpenJPA trace enabled?  I'm not even sure what I'm looking for at this
>>>>> point.  :-)  But, maybe there's some tracing that would point us in the
>>>>> right direction.
>>>>>
>>>>> <property name="openjpa.Log" value="DefaultLevel=TRACE"/>
>>>>>
>>>>> We have been always been on the watch for memory leaks, so this one is
>>>>> puzzling.  You did mention blocked threads as well.  At this point, you
>>>>> are
>>>>> probably hitting blocked threads due to the GC and Memory problems.
>>>>>  But,
>>>>> we
>>>>> did recently resolve some locking situations [1] in our
>>>>> MetaDataRepository
>>>>> that could affect scaling (under extreme load).  This was resolved in
>>>>> the
>>>>> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release,
>>>>> so
>>>>> that should be coming soon).
>>>>>
>>>>> Does your WebService version of the problem use EntityManager
>>>>> transactions,
>>>>> or is the transaction service coming from Tomcat?  And, does your
>>>>> WebService
>>>>> version also do the em.clear() and em.close()?  Basically, are you
>>>>> using
>>>>> application-managed or container-managed persistence while running as a
>>>>> WebService?
>>>>>
>>>>> I'm very interested in helping to resolve this issue.  Like I
>>>>> mentioned,
>>>>> we
>>>>> do a lot of memory leakage and scalability testing of the OpenJPA
>>>>> solution
>>>>> and we're not aware of anything.  Although there is always the
>>>>> possibility
>>>>> that something was resolved in the 1.2.x branch after the 1.2.1 release
>>>>> that
>>>>> I'm not remembering...
>>>>>
>>>>> Thanks for your help,
>>>>> Kevin
>>>>>
>>>>> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
>>>>>
>>>>> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <ku...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Thanks Kevin,
>>>>>>
>>>>>> We're enhancing at build time:
>>>>>>
>>>>>>
>>>>>>
>>>>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>>>>>
>>>>>> Yeah we've been running load tests and things are nice and stable with
>>>>>> Hibernate but with Openjpa we see increasing memory use, blocking
>>>>>> threads
>>>>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
>>>>>> preference would be to ship with openjpa by default; but our build
>>>>>> supports
>>>>>> both hibernate and openjpa.
>>>>>>
>>>>>> And yes we use openjpa 1.2.1 (latest stable version).
>>>>>>
>>>>>> --Kurt
>>>>>>
>>>>>>
>>>>>> Kevin Sutter wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Interesting detective work, Kurt.  Thanks.
>>>>>>>
>>>>>>> Why the WebService version of the app would behave differently as far
>>>>>>> as
>>>>>>> GC
>>>>>>> is concerned is a mystery.  And, you said that plugging in Hibernate
>>>>>>> into
>>>>>>> this scenario, everything works okay?  Very confusing.
>>>>>>>
>>>>>>> How are you performing the Entity enhancement processing?  Are you
>>>>>>> pre-enhancing via your build process?  Or, are you using the
>>>>>>> -javaagent
>>>>>>> mechanism?  Or, are you falling back to the subclassing support
>>>>>>> within
>>>>>>> OpenJPA?  (See [1] for more information on these questions in case
>>>>>>> they
>>>>>>> don't make sense.)
>>>>>>>
>>>>>>> This would be one area that is different between Hibernate and
>>>>>>> OpenJPA
>>>>>>> --
>>>>>>> enhancement processing.
>>>>>>>
>>>>>>> In the Tomcat environment, you may be falling back to the subclassing
>>>>>>> support (which we do not recommend) and hitting a memory leak with
>>>>>>> that.
>>>>>>>
>>>>>>> You said OpenJPA 1.2.x, right?
>>>>>>>
>>>>>>> Just a couple of thoughts on the subject...
>>>>>>> Kevin
>>>>>>>
>>>>>>> [1]
>>>>>>>
>>>>>>>
>>>>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> The same code executed straight from a java client (inVM) shows no
>>>>>>>> memory
>>>>>>>> leak.
>>>>>>>>
>>>>>>>> So is the fact that it is WebService significant then? What else can
>>>>>>>> be
>>>>>>>> different? I think one thread remains up, and somehow this causes
>>>>>>>> openjpa
>>>>>>>> not being able to clean up after itself. What can I do to debug this
>>>>>>>> more? I
>>>>>>>> can actually see in the profiler that the objects are allocated by
>>>>>>>> the
>>>>>>>> WebService, but why aren't they cleaned up?
>>>>>>>>
>>>>>>>> Thx,
>>>>>>>>
>>>>>>>>
>>>>>>>> --Kurt
>>>>>>>>
>>>>>>>>
>>>>>>>> Kurt T Stam wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Thanks Kevin, thanks for your response.
>>>>>>>>>
>>>>>>>>> I just replaced the static call by:
>>>>>>>>>
>>>>>>>>>            apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>            apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>>>>>>            //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>> apiAuthToken);
>>>>>>>>>
>>>>>>>>> which did not make a difference.
>>>>>>>>>
>>>>>>>>> I'm wondering if the fact that my class is a webservice makes a
>>>>>>>>> difference. I'll try extracting it into
>>>>>>>>> a regular class with a main method and profile that. At least I
>>>>>>>>> know
>>>>>>>>> that
>>>>>>>>> I didn't forget something
>>>>>>>>> completely obvious..
>>>>>>>>>
>>>>>>>>> --Kurt
>>>>>>>>>
>>>>>>>>> Kevin Sutter wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Kurt,
>>>>>>>>>> I agree that this is very common usage of the JPA programming
>>>>>>>>>> model.
>>>>>>>>>>  And,
>>>>>>>>>> we are not aware of any memory leaks.  About the only thing that
>>>>>>>>>> jumps
>>>>>>>>>> out
>>>>>>>>>> at me is the following two lines:
>>>>>>>>>>
>>>>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>             MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>> apiAuthToken);
>>>>>>>>>>
>>>>>>>>>> What do these do?  Can you comment these out and see if the memory
>>>>>>>>>> leak
>>>>>>>>>> still exists?  Since you are passing the modelAuthToken into this
>>>>>>>>>> method,
>>>>>>>>>> I
>>>>>>>>>> don't know what it's doing with the reference and could it be
>>>>>>>>>> holding
>>>>>>>>>> onto
>>>>>>>>>> something to prevent the GC from cleaning up?
>>>>>>>>>>
>>>>>>>>>> The rest of your example seems very straight forward with creating
>>>>>>>>>> and
>>>>>>>>>> persisting objects.
>>>>>>>>>>
>>>>>>>>>> Kevin
>>>>>>>>>>
>>>>>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> If you change the 1000 to something like 1000000... does your
>>>>>>>>>>> application
>>>>>>>>>>> go
>>>>>>>>>>> OOM? Are you running in a JSE environment? What is
>>>>>>>>>>> PersistenceManager?
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <
>>>>>>>>>>> kurt.stam@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> BTW I'm running with the cache off
>>>>>>>>>>>>
>>>>>>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>>>>>>
>>>>>>>>>>>> (that turns it off right?)
>>>>>>>>>>>>
>>>>>>>>>>>> --Kurt
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>>
>>>>>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the
>>>>>>>>>>>>> database.
>>>>>>>>>>>>>
>>>>>>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>>>>>>       EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>>>>>>>       EntityTransaction tx = em.getTransaction();
>>>>>>>>>>>>>       try {
>>>>>>>>>>>>>           tx.begin();
>>>>>>>>>>>>>               // Generate auth token and store it!
>>>>>>>>>>>>>           String authInfo = AUTH_TOKEN_PREFIX +
>>>>>>>>>>>>> UUID.randomUUID();
>>>>>>>>>>>>>           org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>>>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>>>>>>           if (authInfo != null) {
>>>>>>>>>>>>>               modelAuthToken.setAuthToken(authInfo);
>>>>>>>>>>>>>               modelAuthToken.setCreated(new Date());
>>>>>>>>>>>>>               modelAuthToken.setLastUsed(new Date());
>>>>>>>>>>>>>               modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>>>>>>               modelAuthToken.setNumberOfUses(0);
>>>>>>>>>>>>>               modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>>>>>>                 em.persist(modelAuthToken);
>>>>>>>>>>>>>           }
>>>>>>>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>>>             MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>>>             tx.commit();
>>>>>>>>>>>>>                     } finally {
>>>>>>>>>>>>>           if (tx.isActive()) {
>>>>>>>>>>>>>               tx.rollback();
>>>>>>>>>>>>>           }
>>>>>>>>>>>>>           em.clear();
>>>>>>>>>>>>>           em.close();
>>>>>>>>>>>>>       }
>>>>>>>>>>>>>   }
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> [ISSUE]
>>>>>>>>>>>>> After it leaving this code I end up with a 1000
>>>>>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been
>>>>>>>>>>>>> using
>>>>>>>>>>>>> the
>>>>>>>>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This seems to be pretty the most common use case of using an
>>>>>>>>>>>>> OR-mapping
>>>>>>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak
>>>>>>>>>>>>> here.
>>>>>>>>>>>>> Does
>>>>>>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an
>>>>>>>>>>>>> example
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> that
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> does not exhibit this behavior? BTW same code using hibernate
>>>>>>>>>>>> does
>>>>>>>>>>>> not
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> accumulate these objects.
>>>>>>>>>>>>>
>>>>>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thx,
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Kurt
>>>>>>>>>>>>>
>>>>>>>>>>>>> Apache jUDDI.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Rick
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>
>

Re: memory leak? - simple question

Posted by Kurt T Stam <ku...@gmail.com>.
1. Cool that you got it running :)

2. I just ran with openjpa 1.2.2.SNAPSHOT and still see an accumulation 
of AuthToken objects.

--K


Kevin Sutter wrote:
> Hey, this wasn't so difficult...  :-)  Maybe I can still learn new
> things...  I have Tomcat up and running and I can get your app running.
>
> On Thu, Jan 14, 2010 at 1:46 PM, Kevin Sutter <kw...@gmail.com> wrote:
>
>   
>> Hi Kurt,
>> These instructions, of course, assume that I know something about Tomcat.
>> This would not be a good assumption...  :-)  If you really want me to see
>> this in action, I will need more information on how to start Tomcat and how
>> to load applications.  In the mean time, a trace log of OpenJPA would be an
>> excellent start.
>>
>> Thanks!
>> Kevin
>>
>>
>> On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <ku...@gmail.com> wrote:
>>
>>     
>>> Hi Kevin,
>>>
>>> I tried adding the javaagent stuff, and it still ends up leaking. I
>>> extracted the code into a tiny
>>> war with just one servlet which persist the authToken. The build creates a
>>> Tomcat distro backed by a
>>> derby database. You can download it from:
>>>
>>> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz>
>>>
>>> You can build it with: mvn install, you can find the final tomcat in
>>> tomcat/target/tomcat-1.0.SNAPSHOT.zip
>>>
>>> or if you don't want to build it:
>>> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>>
>>> After you bring up tomcat it persistent an authToken when you hit
>>> http://localhost:8080/leak/test
>>>
>>> In the profiler I see an org.apache.juddi.AuthToken left behind for each
>>> request.
>>>
>>> Hopefully this little example will pinpoint what's going on.
>>>
>>> Thx Kev for sticking with me :)
>>>
>>> --Kurt
>>>
>>>
>>>
>>> Kevin Sutter wrote:
>>>
>>>       
>>>> Hi Kurt,
>>>> Since you have the environment to reproduce this, could you run with the
>>>> OpenJPA trace enabled?  I'm not even sure what I'm looking for at this
>>>> point.  :-)  But, maybe there's some tracing that would point us in the
>>>> right direction.
>>>>
>>>> <property name="openjpa.Log" value="DefaultLevel=TRACE"/>
>>>>
>>>> We have been always been on the watch for memory leaks, so this one is
>>>> puzzling.  You did mention blocked threads as well.  At this point, you
>>>> are
>>>> probably hitting blocked threads due to the GC and Memory problems.  But,
>>>> we
>>>> did recently resolve some locking situations [1] in our
>>>> MetaDataRepository
>>>> that could affect scaling (under extreme load).  This was resolved in the
>>>> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so
>>>> that should be coming soon).
>>>>
>>>> Does your WebService version of the problem use EntityManager
>>>> transactions,
>>>> or is the transaction service coming from Tomcat?  And, does your
>>>> WebService
>>>> version also do the em.clear() and em.close()?  Basically, are you using
>>>> application-managed or container-managed persistence while running as a
>>>> WebService?
>>>>
>>>> I'm very interested in helping to resolve this issue.  Like I mentioned,
>>>> we
>>>> do a lot of memory leakage and scalability testing of the OpenJPA
>>>> solution
>>>> and we're not aware of anything.  Although there is always the
>>>> possibility
>>>> that something was resolved in the 1.2.x branch after the 1.2.1 release
>>>> that
>>>> I'm not remembering...
>>>>
>>>> Thanks for your help,
>>>> Kevin
>>>>
>>>> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
>>>>
>>>> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <ku...@gmail.com>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> Thanks Kevin,
>>>>>
>>>>> We're enhancing at build time:
>>>>>
>>>>>
>>>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>>>>
>>>>> Yeah we've been running load tests and things are nice and stable with
>>>>> Hibernate but with Openjpa we see increasing memory use, blocking
>>>>> threads
>>>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
>>>>> preference would be to ship with openjpa by default; but our build
>>>>> supports
>>>>> both hibernate and openjpa.
>>>>>
>>>>> And yes we use openjpa 1.2.1 (latest stable version).
>>>>>
>>>>> --Kurt
>>>>>
>>>>>
>>>>> Kevin Sutter wrote:
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> Interesting detective work, Kurt.  Thanks.
>>>>>>
>>>>>> Why the WebService version of the app would behave differently as far
>>>>>> as
>>>>>> GC
>>>>>> is concerned is a mystery.  And, you said that plugging in Hibernate
>>>>>> into
>>>>>> this scenario, everything works okay?  Very confusing.
>>>>>>
>>>>>> How are you performing the Entity enhancement processing?  Are you
>>>>>> pre-enhancing via your build process?  Or, are you using the -javaagent
>>>>>> mechanism?  Or, are you falling back to the subclassing support within
>>>>>> OpenJPA?  (See [1] for more information on these questions in case they
>>>>>> don't make sense.)
>>>>>>
>>>>>> This would be one area that is different between Hibernate and OpenJPA
>>>>>> --
>>>>>> enhancement processing.
>>>>>>
>>>>>> In the Tomcat environment, you may be falling back to the subclassing
>>>>>> support (which we do not recommend) and hitting a memory leak with
>>>>>> that.
>>>>>>
>>>>>> You said OpenJPA 1.2.x, right?
>>>>>>
>>>>>> Just a couple of thoughts on the subject...
>>>>>> Kevin
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> The same code executed straight from a java client (inVM) shows no
>>>>>>> memory
>>>>>>> leak.
>>>>>>>
>>>>>>> So is the fact that it is WebService significant then? What else can
>>>>>>> be
>>>>>>> different? I think one thread remains up, and somehow this causes
>>>>>>> openjpa
>>>>>>> not being able to clean up after itself. What can I do to debug this
>>>>>>> more? I
>>>>>>> can actually see in the profiler that the objects are allocated by the
>>>>>>> WebService, but why aren't they cleaned up?
>>>>>>>
>>>>>>> Thx,
>>>>>>>
>>>>>>>
>>>>>>> --Kurt
>>>>>>>
>>>>>>>
>>>>>>> Kurt T Stam wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> Thanks Kevin, thanks for your response.
>>>>>>>>
>>>>>>>> I just replaced the static call by:
>>>>>>>>
>>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>             apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>>>>>             //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>> apiAuthToken);
>>>>>>>>
>>>>>>>> which did not make a difference.
>>>>>>>>
>>>>>>>> I'm wondering if the fact that my class is a webservice makes a
>>>>>>>> difference. I'll try extracting it into
>>>>>>>> a regular class with a main method and profile that. At least I know
>>>>>>>> that
>>>>>>>> I didn't forget something
>>>>>>>> completely obvious..
>>>>>>>>
>>>>>>>> --Kurt
>>>>>>>>
>>>>>>>> Kevin Sutter wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> Kurt,
>>>>>>>>> I agree that this is very common usage of the JPA programming model.
>>>>>>>>>  And,
>>>>>>>>> we are not aware of any memory leaks.  About the only thing that
>>>>>>>>> jumps
>>>>>>>>> out
>>>>>>>>> at me is the following two lines:
>>>>>>>>>
>>>>>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>              MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>> apiAuthToken);
>>>>>>>>>
>>>>>>>>> What do these do?  Can you comment these out and see if the memory
>>>>>>>>> leak
>>>>>>>>> still exists?  Since you are passing the modelAuthToken into this
>>>>>>>>> method,
>>>>>>>>> I
>>>>>>>>> don't know what it's doing with the reference and could it be
>>>>>>>>> holding
>>>>>>>>> onto
>>>>>>>>> something to prevent the GC from cleaning up?
>>>>>>>>>
>>>>>>>>> The rest of your example seems very straight forward with creating
>>>>>>>>> and
>>>>>>>>> persisting objects.
>>>>>>>>>
>>>>>>>>> Kevin
>>>>>>>>>
>>>>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>> If you change the 1000 to something like 1000000... does your
>>>>>>>>>> application
>>>>>>>>>> go
>>>>>>>>>> OOM? Are you running in a JSE environment? What is
>>>>>>>>>> PersistenceManager?
>>>>>>>>>>
>>>>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>>> BTW I'm running with the cache off
>>>>>>>>>>>
>>>>>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>>>>>
>>>>>>>>>>> (that turns it off right?)
>>>>>>>>>>>
>>>>>>>>>>> --Kurt
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>
>>>>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the
>>>>>>>>>>>> database.
>>>>>>>>>>>>
>>>>>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>>>>>        EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>>>>>>        EntityTransaction tx = em.getTransaction();
>>>>>>>>>>>>        try {
>>>>>>>>>>>>            tx.begin();
>>>>>>>>>>>>                // Generate auth token and store it!
>>>>>>>>>>>>            String authInfo = AUTH_TOKEN_PREFIX +
>>>>>>>>>>>> UUID.randomUUID();
>>>>>>>>>>>>            org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>>>>>            if (authInfo != null) {
>>>>>>>>>>>>                modelAuthToken.setAuthToken(authInfo);
>>>>>>>>>>>>                modelAuthToken.setCreated(new Date());
>>>>>>>>>>>>                modelAuthToken.setLastUsed(new Date());
>>>>>>>>>>>>                modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>>>>>                modelAuthToken.setNumberOfUses(0);
>>>>>>>>>>>>                modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>>>>>                  em.persist(modelAuthToken);
>>>>>>>>>>>>            }
>>>>>>>>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>>              MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>>              tx.commit();
>>>>>>>>>>>>                      } finally {
>>>>>>>>>>>>            if (tx.isActive()) {
>>>>>>>>>>>>                tx.rollback();
>>>>>>>>>>>>            }
>>>>>>>>>>>>            em.clear();
>>>>>>>>>>>>            em.close();
>>>>>>>>>>>>        }
>>>>>>>>>>>>    }
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [ISSUE]
>>>>>>>>>>>> After it leaving this code I end up with a 1000
>>>>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been
>>>>>>>>>>>> using
>>>>>>>>>>>> the
>>>>>>>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>>>>>>>
>>>>>>>>>>>> This seems to be pretty the most common use case of using an
>>>>>>>>>>>> OR-mapping
>>>>>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak
>>>>>>>>>>>> here.
>>>>>>>>>>>> Does
>>>>>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an
>>>>>>>>>>>> example
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>                         
>>>>>>>>>>> that
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>>                     
>>>>>>>>>>> does not exhibit this behavior? BTW same code using hibernate does
>>>>>>>>>>> not
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>>>> accumulate these objects.
>>>>>>>>>>>>
>>>>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Kurt
>>>>>>>>>>>>
>>>>>>>>>>>> Apache jUDDI.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>                         
>>>>>>>>>>>                       
>>>>>>>>>> --
>>>>>>>>>> Thanks,
>>>>>>>>>> Rick
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>                 
>>>>>           
>>>>
>>>>         
>>>       
>
>   


Re: memory leak? - simple question

Posted by Kevin Sutter <kw...@gmail.com>.
Hey, this wasn't so difficult...  :-)  Maybe I can still learn new
things...  I have Tomcat up and running and I can get your app running.

On Thu, Jan 14, 2010 at 1:46 PM, Kevin Sutter <kw...@gmail.com> wrote:

> Hi Kurt,
> These instructions, of course, assume that I know something about Tomcat.
> This would not be a good assumption...  :-)  If you really want me to see
> this in action, I will need more information on how to start Tomcat and how
> to load applications.  In the mean time, a trace log of OpenJPA would be an
> excellent start.
>
> Thanks!
> Kevin
>
>
> On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <ku...@gmail.com> wrote:
>
>> Hi Kevin,
>>
>> I tried adding the javaagent stuff, and it still ends up leaking. I
>> extracted the code into a tiny
>> war with just one servlet which persist the authToken. The build creates a
>> Tomcat distro backed by a
>> derby database. You can download it from:
>>
>> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz>
>>
>> You can build it with: mvn install, you can find the final tomcat in
>> tomcat/target/tomcat-1.0.SNAPSHOT.zip
>>
>> or if you don't want to build it:
>> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>
>> After you bring up tomcat it persistent an authToken when you hit
>> http://localhost:8080/leak/test
>>
>> In the profiler I see an org.apache.juddi.AuthToken left behind for each
>> request.
>>
>> Hopefully this little example will pinpoint what's going on.
>>
>> Thx Kev for sticking with me :)
>>
>> --Kurt
>>
>>
>>
>> Kevin Sutter wrote:
>>
>>> Hi Kurt,
>>> Since you have the environment to reproduce this, could you run with the
>>> OpenJPA trace enabled?  I'm not even sure what I'm looking for at this
>>> point.  :-)  But, maybe there's some tracing that would point us in the
>>> right direction.
>>>
>>> <property name="openjpa.Log" value="DefaultLevel=TRACE"/>
>>>
>>> We have been always been on the watch for memory leaks, so this one is
>>> puzzling.  You did mention blocked threads as well.  At this point, you
>>> are
>>> probably hitting blocked threads due to the GC and Memory problems.  But,
>>> we
>>> did recently resolve some locking situations [1] in our
>>> MetaDataRepository
>>> that could affect scaling (under extreme load).  This was resolved in the
>>> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so
>>> that should be coming soon).
>>>
>>> Does your WebService version of the problem use EntityManager
>>> transactions,
>>> or is the transaction service coming from Tomcat?  And, does your
>>> WebService
>>> version also do the em.clear() and em.close()?  Basically, are you using
>>> application-managed or container-managed persistence while running as a
>>> WebService?
>>>
>>> I'm very interested in helping to resolve this issue.  Like I mentioned,
>>> we
>>> do a lot of memory leakage and scalability testing of the OpenJPA
>>> solution
>>> and we're not aware of anything.  Although there is always the
>>> possibility
>>> that something was resolved in the 1.2.x branch after the 1.2.1 release
>>> that
>>> I'm not remembering...
>>>
>>> Thanks for your help,
>>> Kevin
>>>
>>> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
>>>
>>> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <ku...@gmail.com>
>>> wrote:
>>>
>>>
>>>
>>>> Thanks Kevin,
>>>>
>>>> We're enhancing at build time:
>>>>
>>>>
>>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>>>
>>>> Yeah we've been running load tests and things are nice and stable with
>>>> Hibernate but with Openjpa we see increasing memory use, blocking
>>>> threads
>>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
>>>> preference would be to ship with openjpa by default; but our build
>>>> supports
>>>> both hibernate and openjpa.
>>>>
>>>> And yes we use openjpa 1.2.1 (latest stable version).
>>>>
>>>> --Kurt
>>>>
>>>>
>>>> Kevin Sutter wrote:
>>>>
>>>>
>>>>
>>>>> Interesting detective work, Kurt.  Thanks.
>>>>>
>>>>> Why the WebService version of the app would behave differently as far
>>>>> as
>>>>> GC
>>>>> is concerned is a mystery.  And, you said that plugging in Hibernate
>>>>> into
>>>>> this scenario, everything works okay?  Very confusing.
>>>>>
>>>>> How are you performing the Entity enhancement processing?  Are you
>>>>> pre-enhancing via your build process?  Or, are you using the -javaagent
>>>>> mechanism?  Or, are you falling back to the subclassing support within
>>>>> OpenJPA?  (See [1] for more information on these questions in case they
>>>>> don't make sense.)
>>>>>
>>>>> This would be one area that is different between Hibernate and OpenJPA
>>>>> --
>>>>> enhancement processing.
>>>>>
>>>>> In the Tomcat environment, you may be falling back to the subclassing
>>>>> support (which we do not recommend) and hitting a memory leak with
>>>>> that.
>>>>>
>>>>> You said OpenJPA 1.2.x, right?
>>>>>
>>>>> Just a couple of thoughts on the subject...
>>>>> Kevin
>>>>>
>>>>> [1]
>>>>>
>>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> The same code executed straight from a java client (inVM) shows no
>>>>>> memory
>>>>>> leak.
>>>>>>
>>>>>> So is the fact that it is WebService significant then? What else can
>>>>>> be
>>>>>> different? I think one thread remains up, and somehow this causes
>>>>>> openjpa
>>>>>> not being able to clean up after itself. What can I do to debug this
>>>>>> more? I
>>>>>> can actually see in the profiler that the objects are allocated by the
>>>>>> WebService, but why aren't they cleaned up?
>>>>>>
>>>>>> Thx,
>>>>>>
>>>>>>
>>>>>> --Kurt
>>>>>>
>>>>>>
>>>>>> Kurt T Stam wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Thanks Kevin, thanks for your response.
>>>>>>>
>>>>>>> I just replaced the static call by:
>>>>>>>
>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>             apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>>>>             //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>> apiAuthToken);
>>>>>>>
>>>>>>> which did not make a difference.
>>>>>>>
>>>>>>> I'm wondering if the fact that my class is a webservice makes a
>>>>>>> difference. I'll try extracting it into
>>>>>>> a regular class with a main method and profile that. At least I know
>>>>>>> that
>>>>>>> I didn't forget something
>>>>>>> completely obvious..
>>>>>>>
>>>>>>> --Kurt
>>>>>>>
>>>>>>> Kevin Sutter wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Kurt,
>>>>>>>> I agree that this is very common usage of the JPA programming model.
>>>>>>>>  And,
>>>>>>>> we are not aware of any memory leaks.  About the only thing that
>>>>>>>> jumps
>>>>>>>> out
>>>>>>>> at me is the following two lines:
>>>>>>>>
>>>>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>              MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>> apiAuthToken);
>>>>>>>>
>>>>>>>> What do these do?  Can you comment these out and see if the memory
>>>>>>>> leak
>>>>>>>> still exists?  Since you are passing the modelAuthToken into this
>>>>>>>> method,
>>>>>>>> I
>>>>>>>> don't know what it's doing with the reference and could it be
>>>>>>>> holding
>>>>>>>> onto
>>>>>>>> something to prevent the GC from cleaning up?
>>>>>>>>
>>>>>>>> The rest of your example seems very straight forward with creating
>>>>>>>> and
>>>>>>>> persisting objects.
>>>>>>>>
>>>>>>>> Kevin
>>>>>>>>
>>>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> If you change the 1000 to something like 1000000... does your
>>>>>>>>> application
>>>>>>>>> go
>>>>>>>>> OOM? Are you running in a JSE environment? What is
>>>>>>>>> PersistenceManager?
>>>>>>>>>
>>>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> BTW I'm running with the cache off
>>>>>>>>>>
>>>>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>>>>
>>>>>>>>>> (that turns it off right?)
>>>>>>>>>>
>>>>>>>>>> --Kurt
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hi guys,
>>>>>>>>>>>
>>>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the
>>>>>>>>>>> database.
>>>>>>>>>>>
>>>>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>>>>        EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>>>>>        EntityTransaction tx = em.getTransaction();
>>>>>>>>>>>        try {
>>>>>>>>>>>            tx.begin();
>>>>>>>>>>>                // Generate auth token and store it!
>>>>>>>>>>>            String authInfo = AUTH_TOKEN_PREFIX +
>>>>>>>>>>> UUID.randomUUID();
>>>>>>>>>>>            org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>>>>            if (authInfo != null) {
>>>>>>>>>>>                modelAuthToken.setAuthToken(authInfo);
>>>>>>>>>>>                modelAuthToken.setCreated(new Date());
>>>>>>>>>>>                modelAuthToken.setLastUsed(new Date());
>>>>>>>>>>>                modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>>>>                modelAuthToken.setNumberOfUses(0);
>>>>>>>>>>>                modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>>>>                  em.persist(modelAuthToken);
>>>>>>>>>>>            }
>>>>>>>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>              MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>              tx.commit();
>>>>>>>>>>>                      } finally {
>>>>>>>>>>>            if (tx.isActive()) {
>>>>>>>>>>>                tx.rollback();
>>>>>>>>>>>            }
>>>>>>>>>>>            em.clear();
>>>>>>>>>>>            em.close();
>>>>>>>>>>>        }
>>>>>>>>>>>    }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [ISSUE]
>>>>>>>>>>> After it leaving this code I end up with a 1000
>>>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been
>>>>>>>>>>> using
>>>>>>>>>>> the
>>>>>>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>>>>>>
>>>>>>>>>>> This seems to be pretty the most common use case of using an
>>>>>>>>>>> OR-mapping
>>>>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak
>>>>>>>>>>> here.
>>>>>>>>>>> Does
>>>>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an
>>>>>>>>>>> example
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> that
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> does not exhibit this behavior? BTW same code using hibernate does
>>>>>>>>>> not
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> accumulate these objects.
>>>>>>>>>>>
>>>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thx,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Kurt
>>>>>>>>>>>
>>>>>>>>>>> Apache jUDDI.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks,
>>>>>>>>> Rick
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>

Re: memory leak? - simple question

Posted by Kevin Sutter <kw...@gmail.com>.
Another quick idea... How about running with the latest 1.2.x snapshot?  Or,
even grabbing the proposed 1.2.2 release (Mike's recent post to dev)?  At
least that would give us a quick read on whether the problem still exists or
not.

On Thu, Jan 14, 2010 at 1:46 PM, Kevin Sutter <kw...@gmail.com> wrote:

> Hi Kurt,
> These instructions, of course, assume that I know something about Tomcat.
> This would not be a good assumption...  :-)  If you really want me to see
> this in action, I will need more information on how to start Tomcat and how
> to load applications.  In the mean time, a trace log of OpenJPA would be an
> excellent start.
>
> Thanks!
> Kevin
>
>
> On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <ku...@gmail.com> wrote:
>
>> Hi Kevin,
>>
>> I tried adding the javaagent stuff, and it still ends up leaking. I
>> extracted the code into a tiny
>> war with just one servlet which persist the authToken. The build creates a
>> Tomcat distro backed by a
>> derby database. You can download it from:
>>
>> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz>
>>
>> You can build it with: mvn install, you can find the final tomcat in
>> tomcat/target/tomcat-1.0.SNAPSHOT.zip
>>
>> or if you don't want to build it:
>> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>
>> After you bring up tomcat it persistent an authToken when you hit
>> http://localhost:8080/leak/test
>>
>> In the profiler I see an org.apache.juddi.AuthToken left behind for each
>> request.
>>
>> Hopefully this little example will pinpoint what's going on.
>>
>> Thx Kev for sticking with me :)
>>
>> --Kurt
>>
>>
>>
>> Kevin Sutter wrote:
>>
>>> Hi Kurt,
>>> Since you have the environment to reproduce this, could you run with the
>>> OpenJPA trace enabled?  I'm not even sure what I'm looking for at this
>>> point.  :-)  But, maybe there's some tracing that would point us in the
>>> right direction.
>>>
>>> <property name="openjpa.Log" value="DefaultLevel=TRACE"/>
>>>
>>> We have been always been on the watch for memory leaks, so this one is
>>> puzzling.  You did mention blocked threads as well.  At this point, you
>>> are
>>> probably hitting blocked threads due to the GC and Memory problems.  But,
>>> we
>>> did recently resolve some locking situations [1] in our
>>> MetaDataRepository
>>> that could affect scaling (under extreme load).  This was resolved in the
>>> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so
>>> that should be coming soon).
>>>
>>> Does your WebService version of the problem use EntityManager
>>> transactions,
>>> or is the transaction service coming from Tomcat?  And, does your
>>> WebService
>>> version also do the em.clear() and em.close()?  Basically, are you using
>>> application-managed or container-managed persistence while running as a
>>> WebService?
>>>
>>> I'm very interested in helping to resolve this issue.  Like I mentioned,
>>> we
>>> do a lot of memory leakage and scalability testing of the OpenJPA
>>> solution
>>> and we're not aware of anything.  Although there is always the
>>> possibility
>>> that something was resolved in the 1.2.x branch after the 1.2.1 release
>>> that
>>> I'm not remembering...
>>>
>>> Thanks for your help,
>>> Kevin
>>>
>>> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
>>>
>>> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <ku...@gmail.com>
>>> wrote:
>>>
>>>
>>>
>>>> Thanks Kevin,
>>>>
>>>> We're enhancing at build time:
>>>>
>>>>
>>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>>>
>>>> Yeah we've been running load tests and things are nice and stable with
>>>> Hibernate but with Openjpa we see increasing memory use, blocking
>>>> threads
>>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
>>>> preference would be to ship with openjpa by default; but our build
>>>> supports
>>>> both hibernate and openjpa.
>>>>
>>>> And yes we use openjpa 1.2.1 (latest stable version).
>>>>
>>>> --Kurt
>>>>
>>>>
>>>> Kevin Sutter wrote:
>>>>
>>>>
>>>>
>>>>> Interesting detective work, Kurt.  Thanks.
>>>>>
>>>>> Why the WebService version of the app would behave differently as far
>>>>> as
>>>>> GC
>>>>> is concerned is a mystery.  And, you said that plugging in Hibernate
>>>>> into
>>>>> this scenario, everything works okay?  Very confusing.
>>>>>
>>>>> How are you performing the Entity enhancement processing?  Are you
>>>>> pre-enhancing via your build process?  Or, are you using the -javaagent
>>>>> mechanism?  Or, are you falling back to the subclassing support within
>>>>> OpenJPA?  (See [1] for more information on these questions in case they
>>>>> don't make sense.)
>>>>>
>>>>> This would be one area that is different between Hibernate and OpenJPA
>>>>> --
>>>>> enhancement processing.
>>>>>
>>>>> In the Tomcat environment, you may be falling back to the subclassing
>>>>> support (which we do not recommend) and hitting a memory leak with
>>>>> that.
>>>>>
>>>>> You said OpenJPA 1.2.x, right?
>>>>>
>>>>> Just a couple of thoughts on the subject...
>>>>> Kevin
>>>>>
>>>>> [1]
>>>>>
>>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> The same code executed straight from a java client (inVM) shows no
>>>>>> memory
>>>>>> leak.
>>>>>>
>>>>>> So is the fact that it is WebService significant then? What else can
>>>>>> be
>>>>>> different? I think one thread remains up, and somehow this causes
>>>>>> openjpa
>>>>>> not being able to clean up after itself. What can I do to debug this
>>>>>> more? I
>>>>>> can actually see in the profiler that the objects are allocated by the
>>>>>> WebService, but why aren't they cleaned up?
>>>>>>
>>>>>> Thx,
>>>>>>
>>>>>>
>>>>>> --Kurt
>>>>>>
>>>>>>
>>>>>> Kurt T Stam wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Thanks Kevin, thanks for your response.
>>>>>>>
>>>>>>> I just replaced the static call by:
>>>>>>>
>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>             apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>>>>             //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>> apiAuthToken);
>>>>>>>
>>>>>>> which did not make a difference.
>>>>>>>
>>>>>>> I'm wondering if the fact that my class is a webservice makes a
>>>>>>> difference. I'll try extracting it into
>>>>>>> a regular class with a main method and profile that. At least I know
>>>>>>> that
>>>>>>> I didn't forget something
>>>>>>> completely obvious..
>>>>>>>
>>>>>>> --Kurt
>>>>>>>
>>>>>>> Kevin Sutter wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Kurt,
>>>>>>>> I agree that this is very common usage of the JPA programming model.
>>>>>>>>  And,
>>>>>>>> we are not aware of any memory leaks.  About the only thing that
>>>>>>>> jumps
>>>>>>>> out
>>>>>>>> at me is the following two lines:
>>>>>>>>
>>>>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>              MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>> apiAuthToken);
>>>>>>>>
>>>>>>>> What do these do?  Can you comment these out and see if the memory
>>>>>>>> leak
>>>>>>>> still exists?  Since you are passing the modelAuthToken into this
>>>>>>>> method,
>>>>>>>> I
>>>>>>>> don't know what it's doing with the reference and could it be
>>>>>>>> holding
>>>>>>>> onto
>>>>>>>> something to prevent the GC from cleaning up?
>>>>>>>>
>>>>>>>> The rest of your example seems very straight forward with creating
>>>>>>>> and
>>>>>>>> persisting objects.
>>>>>>>>
>>>>>>>> Kevin
>>>>>>>>
>>>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> If you change the 1000 to something like 1000000... does your
>>>>>>>>> application
>>>>>>>>> go
>>>>>>>>> OOM? Are you running in a JSE environment? What is
>>>>>>>>> PersistenceManager?
>>>>>>>>>
>>>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> BTW I'm running with the cache off
>>>>>>>>>>
>>>>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>>>>
>>>>>>>>>> (that turns it off right?)
>>>>>>>>>>
>>>>>>>>>> --Kurt
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hi guys,
>>>>>>>>>>>
>>>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the
>>>>>>>>>>> database.
>>>>>>>>>>>
>>>>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>>>>        EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>>>>>        EntityTransaction tx = em.getTransaction();
>>>>>>>>>>>        try {
>>>>>>>>>>>            tx.begin();
>>>>>>>>>>>                // Generate auth token and store it!
>>>>>>>>>>>            String authInfo = AUTH_TOKEN_PREFIX +
>>>>>>>>>>> UUID.randomUUID();
>>>>>>>>>>>            org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>>>>            if (authInfo != null) {
>>>>>>>>>>>                modelAuthToken.setAuthToken(authInfo);
>>>>>>>>>>>                modelAuthToken.setCreated(new Date());
>>>>>>>>>>>                modelAuthToken.setLastUsed(new Date());
>>>>>>>>>>>                modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>>>>                modelAuthToken.setNumberOfUses(0);
>>>>>>>>>>>                modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>>>>                  em.persist(modelAuthToken);
>>>>>>>>>>>            }
>>>>>>>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>              MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>              tx.commit();
>>>>>>>>>>>                      } finally {
>>>>>>>>>>>            if (tx.isActive()) {
>>>>>>>>>>>                tx.rollback();
>>>>>>>>>>>            }
>>>>>>>>>>>            em.clear();
>>>>>>>>>>>            em.close();
>>>>>>>>>>>        }
>>>>>>>>>>>    }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [ISSUE]
>>>>>>>>>>> After it leaving this code I end up with a 1000
>>>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been
>>>>>>>>>>> using
>>>>>>>>>>> the
>>>>>>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>>>>>>
>>>>>>>>>>> This seems to be pretty the most common use case of using an
>>>>>>>>>>> OR-mapping
>>>>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak
>>>>>>>>>>> here.
>>>>>>>>>>> Does
>>>>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an
>>>>>>>>>>> example
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> that
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> does not exhibit this behavior? BTW same code using hibernate does
>>>>>>>>>> not
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> accumulate these objects.
>>>>>>>>>>>
>>>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thx,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Kurt
>>>>>>>>>>>
>>>>>>>>>>> Apache jUDDI.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Thanks,
>>>>>>>>> Rick
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>

Re: memory leak? - simple question

Posted by Kevin Sutter <kw...@gmail.com>.
We must have been crossing in the ether...  :-)  No need for the trace, I
can do that now.  Thanks anyway!

Kevin

On Thu, Jan 14, 2010 at 1:55 PM, Kurt T Stam <ku...@gmail.com> wrote:

> OK :).
>
> 1. Unzip tomcat-1.0.SNAPSHOT.zip
>
> 2. cd tomcat-1.0.SNAPSHOT/bin
>
> 3. ./start.sh
>
> Now, to send in a request open your browser to url:
>
>
> http://localhost:8080/leak/test
>
> Hit refresh to send in another request.
>
> I'll try getting you a trace log. Which packages? Just
> "org.apache.openjpa"?
>
> Cheers,
>
> --Kurt
>
>
> Kevin Sutter wrote:
>
>> Hi Kurt,
>>
>> These instructions, of course, assume that I know something about Tomcat.
>> This would not be a good assumption...  :-)  If you really want me to see
>> this in action, I will need more information on how to start Tomcat and
>> how
>> to load applications.  In the mean time, a trace log of OpenJPA would be
>> an
>> excellent start.
>>
>> Thanks!
>> Kevin
>>
>> On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <ku...@gmail.com>
>> wrote:
>>
>>
>>
>>> Hi Kevin,
>>>
>>> I tried adding the javaagent stuff, and it still ends up leaking. I
>>> extracted the code into a tiny
>>> war with just one servlet which persist the authToken. The build creates
>>> a
>>> Tomcat distro backed by a
>>> derby database. You can download it from:
>>>
>>> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz>
>>> <http://people.apache.org/%7Ekstam/leak.tgz>
>>>
>>>
>>> You can build it with: mvn install, you can find the final tomcat in
>>> tomcat/target/tomcat-1.0.SNAPSHOT.zip
>>>
>>> or if you don't want to build it:
>>> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>> <http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>>
>>>
>>> After you bring up tomcat it persistent an authToken when you hit
>>> http://localhost:8080/leak/test
>>>
>>> In the profiler I see an org.apache.juddi.AuthToken left behind for each
>>> request.
>>>
>>> Hopefully this little example will pinpoint what's going on.
>>>
>>> Thx Kev for sticking with me :)
>>>
>>> --Kurt
>>>
>>>
>>>
>>> Kevin Sutter wrote:
>>>
>>>
>>>
>>>> Hi Kurt,
>>>> Since you have the environment to reproduce this, could you run with the
>>>> OpenJPA trace enabled?  I'm not even sure what I'm looking for at this
>>>> point.  :-)  But, maybe there's some tracing that would point us in the
>>>> right direction.
>>>>
>>>> <property name="openjpa.Log" value="DefaultLevel=TRACE"/>
>>>>
>>>> We have been always been on the watch for memory leaks, so this one is
>>>> puzzling.  You did mention blocked threads as well.  At this point, you
>>>> are
>>>> probably hitting blocked threads due to the GC and Memory problems.
>>>>  But,
>>>> we
>>>> did recently resolve some locking situations [1] in our
>>>> MetaDataRepository
>>>> that could affect scaling (under extreme load).  This was resolved in
>>>> the
>>>> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release,
>>>> so
>>>> that should be coming soon).
>>>>
>>>> Does your WebService version of the problem use EntityManager
>>>> transactions,
>>>> or is the transaction service coming from Tomcat?  And, does your
>>>> WebService
>>>> version also do the em.clear() and em.close()?  Basically, are you using
>>>> application-managed or container-managed persistence while running as a
>>>> WebService?
>>>>
>>>> I'm very interested in helping to resolve this issue.  Like I mentioned,
>>>> we
>>>> do a lot of memory leakage and scalability testing of the OpenJPA
>>>> solution
>>>> and we're not aware of anything.  Although there is always the
>>>> possibility
>>>> that something was resolved in the 1.2.x branch after the 1.2.1 release
>>>> that
>>>> I'm not remembering...
>>>>
>>>> Thanks for your help,
>>>> Kevin
>>>>
>>>> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
>>>>
>>>> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <ku...@gmail.com>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> Thanks Kevin,
>>>>>
>>>>> We're enhancing at build time:
>>>>>
>>>>>
>>>>>
>>>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>>>>
>>>>> Yeah we've been running load tests and things are nice and stable with
>>>>> Hibernate but with Openjpa we see increasing memory use, blocking
>>>>> threads
>>>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
>>>>> preference would be to ship with openjpa by default; but our build
>>>>> supports
>>>>> both hibernate and openjpa.
>>>>>
>>>>> And yes we use openjpa 1.2.1 (latest stable version).
>>>>>
>>>>> --Kurt
>>>>>
>>>>>
>>>>> Kevin Sutter wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Interesting detective work, Kurt.  Thanks.
>>>>>>
>>>>>> Why the WebService version of the app would behave differently as far
>>>>>> as
>>>>>> GC
>>>>>> is concerned is a mystery.  And, you said that plugging in Hibernate
>>>>>> into
>>>>>> this scenario, everything works okay?  Very confusing.
>>>>>>
>>>>>> How are you performing the Entity enhancement processing?  Are you
>>>>>> pre-enhancing via your build process?  Or, are you using the
>>>>>> -javaagent
>>>>>> mechanism?  Or, are you falling back to the subclassing support within
>>>>>> OpenJPA?  (See [1] for more information on these questions in case
>>>>>> they
>>>>>> don't make sense.)
>>>>>>
>>>>>> This would be one area that is different between Hibernate and OpenJPA
>>>>>> --
>>>>>> enhancement processing.
>>>>>>
>>>>>> In the Tomcat environment, you may be falling back to the subclassing
>>>>>> support (which we do not recommend) and hitting a memory leak with
>>>>>> that.
>>>>>>
>>>>>> You said OpenJPA 1.2.x, right?
>>>>>>
>>>>>> Just a couple of thoughts on the subject...
>>>>>> Kevin
>>>>>>
>>>>>> [1]
>>>>>>
>>>>>>
>>>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> The same code executed straight from a java client (inVM) shows no
>>>>>>> memory
>>>>>>> leak.
>>>>>>>
>>>>>>> So is the fact that it is WebService significant then? What else can
>>>>>>> be
>>>>>>> different? I think one thread remains up, and somehow this causes
>>>>>>> openjpa
>>>>>>> not being able to clean up after itself. What can I do to debug this
>>>>>>> more? I
>>>>>>> can actually see in the profiler that the objects are allocated by
>>>>>>> the
>>>>>>> WebService, but why aren't they cleaned up?
>>>>>>>
>>>>>>> Thx,
>>>>>>>
>>>>>>>
>>>>>>> --Kurt
>>>>>>>
>>>>>>>
>>>>>>> Kurt T Stam wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Thanks Kevin, thanks for your response.
>>>>>>>>
>>>>>>>> I just replaced the static call by:
>>>>>>>>
>>>>>>>>            apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>            apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>>>>>            //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>> apiAuthToken);
>>>>>>>>
>>>>>>>> which did not make a difference.
>>>>>>>>
>>>>>>>> I'm wondering if the fact that my class is a webservice makes a
>>>>>>>> difference. I'll try extracting it into
>>>>>>>> a regular class with a main method and profile that. At least I know
>>>>>>>> that
>>>>>>>> I didn't forget something
>>>>>>>> completely obvious..
>>>>>>>>
>>>>>>>> --Kurt
>>>>>>>>
>>>>>>>> Kevin Sutter wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> Kurt,
>>>>>>>>> I agree that this is very common usage of the JPA programming
>>>>>>>>> model.
>>>>>>>>>  And,
>>>>>>>>> we are not aware of any memory leaks.  About the only thing that
>>>>>>>>> jumps
>>>>>>>>> out
>>>>>>>>> at me is the following two lines:
>>>>>>>>>
>>>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>             MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>> apiAuthToken);
>>>>>>>>>
>>>>>>>>> What do these do?  Can you comment these out and see if the memory
>>>>>>>>> leak
>>>>>>>>> still exists?  Since you are passing the modelAuthToken into this
>>>>>>>>> method,
>>>>>>>>> I
>>>>>>>>> don't know what it's doing with the reference and could it be
>>>>>>>>> holding
>>>>>>>>> onto
>>>>>>>>> something to prevent the GC from cleaning up?
>>>>>>>>>
>>>>>>>>> The rest of your example seems very straight forward with creating
>>>>>>>>> and
>>>>>>>>> persisting objects.
>>>>>>>>>
>>>>>>>>> Kevin
>>>>>>>>>
>>>>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> If you change the 1000 to something like 1000000... does your
>>>>>>>>>> application
>>>>>>>>>> go
>>>>>>>>>> OOM? Are you running in a JSE environment? What is
>>>>>>>>>> PersistenceManager?
>>>>>>>>>>
>>>>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <kurt.stam@gmail.com
>>>>>>>>>> >
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> BTW I'm running with the cache off
>>>>>>>>>>>
>>>>>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>>>>>
>>>>>>>>>>> (that turns it off right?)
>>>>>>>>>>>
>>>>>>>>>>> --Kurt
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>
>>>>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the
>>>>>>>>>>>> database.
>>>>>>>>>>>>
>>>>>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>>>>>       EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>>>>>>       EntityTransaction tx = em.getTransaction();
>>>>>>>>>>>>       try {
>>>>>>>>>>>>           tx.begin();
>>>>>>>>>>>>               // Generate auth token and store it!
>>>>>>>>>>>>           String authInfo = AUTH_TOKEN_PREFIX +
>>>>>>>>>>>> UUID.randomUUID();
>>>>>>>>>>>>           org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>>>>>           if (authInfo != null) {
>>>>>>>>>>>>               modelAuthToken.setAuthToken(authInfo);
>>>>>>>>>>>>               modelAuthToken.setCreated(new Date());
>>>>>>>>>>>>               modelAuthToken.setLastUsed(new Date());
>>>>>>>>>>>>               modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>>>>>               modelAuthToken.setNumberOfUses(0);
>>>>>>>>>>>>               modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>>>>>                 em.persist(modelAuthToken);
>>>>>>>>>>>>           }
>>>>>>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>>             MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>>             tx.commit();
>>>>>>>>>>>>                     } finally {
>>>>>>>>>>>>           if (tx.isActive()) {
>>>>>>>>>>>>               tx.rollback();
>>>>>>>>>>>>           }
>>>>>>>>>>>>           em.clear();
>>>>>>>>>>>>           em.close();
>>>>>>>>>>>>       }
>>>>>>>>>>>>   }
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [ISSUE]
>>>>>>>>>>>> After it leaving this code I end up with a 1000
>>>>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been
>>>>>>>>>>>> using
>>>>>>>>>>>> the
>>>>>>>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>>>>>>>
>>>>>>>>>>>> This seems to be pretty the most common use case of using an
>>>>>>>>>>>> OR-mapping
>>>>>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak
>>>>>>>>>>>> here.
>>>>>>>>>>>> Does
>>>>>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an
>>>>>>>>>>>> example
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> that
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> does not exhibit this behavior? BTW same code using hibernate
>>>>>>>>>>> does
>>>>>>>>>>> not
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> accumulate these objects.
>>>>>>>>>>>>
>>>>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Thx,
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Kurt
>>>>>>>>>>>>
>>>>>>>>>>>> Apache jUDDI.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Thanks,
>>>>>>>>>> Rick
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>

Re: memory leak? - simple question

Posted by Kurt T Stam <ku...@gmail.com>.
OK :).

1. Unzip tomcat-1.0.SNAPSHOT.zip

2. cd tomcat-1.0.SNAPSHOT/bin

3. ./start.sh

Now, to send in a request open your browser to url:

http://localhost:8080/leak/test

Hit refresh to send in another request.

I'll try getting you a trace log. Which packages? Just "org.apache.openjpa"?

Cheers,

--Kurt


Kevin Sutter wrote:
> Hi Kurt,
> These instructions, of course, assume that I know something about Tomcat.
> This would not be a good assumption...  :-)  If you really want me to see
> this in action, I will need more information on how to start Tomcat and how
> to load applications.  In the mean time, a trace log of OpenJPA would be an
> excellent start.
>
> Thanks!
> Kevin
>
> On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <ku...@gmail.com> wrote:
>
>   
>> Hi Kevin,
>>
>> I tried adding the javaagent stuff, and it still ends up leaking. I
>> extracted the code into a tiny
>> war with just one servlet which persist the authToken. The build creates a
>> Tomcat distro backed by a
>> derby database. You can download it from:
>>
>> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz>
>>
>> You can build it with: mvn install, you can find the final tomcat in
>> tomcat/target/tomcat-1.0.SNAPSHOT.zip
>>
>> or if you don't want to build it:
>> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>>
>> After you bring up tomcat it persistent an authToken when you hit
>> http://localhost:8080/leak/test
>>
>> In the profiler I see an org.apache.juddi.AuthToken left behind for each
>> request.
>>
>> Hopefully this little example will pinpoint what's going on.
>>
>> Thx Kev for sticking with me :)
>>
>> --Kurt
>>
>>
>>
>> Kevin Sutter wrote:
>>
>>     
>>> Hi Kurt,
>>> Since you have the environment to reproduce this, could you run with the
>>> OpenJPA trace enabled?  I'm not even sure what I'm looking for at this
>>> point.  :-)  But, maybe there's some tracing that would point us in the
>>> right direction.
>>>
>>> <property name="openjpa.Log" value="DefaultLevel=TRACE"/>
>>>
>>> We have been always been on the watch for memory leaks, so this one is
>>> puzzling.  You did mention blocked threads as well.  At this point, you
>>> are
>>> probably hitting blocked threads due to the GC and Memory problems.  But,
>>> we
>>> did recently resolve some locking situations [1] in our MetaDataRepository
>>> that could affect scaling (under extreme load).  This was resolved in the
>>> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so
>>> that should be coming soon).
>>>
>>> Does your WebService version of the problem use EntityManager
>>> transactions,
>>> or is the transaction service coming from Tomcat?  And, does your
>>> WebService
>>> version also do the em.clear() and em.close()?  Basically, are you using
>>> application-managed or container-managed persistence while running as a
>>> WebService?
>>>
>>> I'm very interested in helping to resolve this issue.  Like I mentioned,
>>> we
>>> do a lot of memory leakage and scalability testing of the OpenJPA solution
>>> and we're not aware of anything.  Although there is always the possibility
>>> that something was resolved in the 1.2.x branch after the 1.2.1 release
>>> that
>>> I'm not remembering...
>>>
>>> Thanks for your help,
>>> Kevin
>>>
>>> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
>>>
>>> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <ku...@gmail.com> wrote:
>>>
>>>
>>>
>>>       
>>>> Thanks Kevin,
>>>>
>>>> We're enhancing at build time:
>>>>
>>>>
>>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>>>
>>>> Yeah we've been running load tests and things are nice and stable with
>>>> Hibernate but with Openjpa we see increasing memory use, blocking threads
>>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
>>>> preference would be to ship with openjpa by default; but our build
>>>> supports
>>>> both hibernate and openjpa.
>>>>
>>>> And yes we use openjpa 1.2.1 (latest stable version).
>>>>
>>>> --Kurt
>>>>
>>>>
>>>> Kevin Sutter wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> Interesting detective work, Kurt.  Thanks.
>>>>>
>>>>> Why the WebService version of the app would behave differently as far as
>>>>> GC
>>>>> is concerned is a mystery.  And, you said that plugging in Hibernate
>>>>> into
>>>>> this scenario, everything works okay?  Very confusing.
>>>>>
>>>>> How are you performing the Entity enhancement processing?  Are you
>>>>> pre-enhancing via your build process?  Or, are you using the -javaagent
>>>>> mechanism?  Or, are you falling back to the subclassing support within
>>>>> OpenJPA?  (See [1] for more information on these questions in case they
>>>>> don't make sense.)
>>>>>
>>>>> This would be one area that is different between Hibernate and OpenJPA
>>>>> --
>>>>> enhancement processing.
>>>>>
>>>>> In the Tomcat environment, you may be falling back to the subclassing
>>>>> support (which we do not recommend) and hitting a memory leak with that.
>>>>>
>>>>> You said OpenJPA 1.2.x, right?
>>>>>
>>>>> Just a couple of thoughts on the subject...
>>>>> Kevin
>>>>>
>>>>> [1]
>>>>>
>>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> The same code executed straight from a java client (inVM) shows no
>>>>>> memory
>>>>>> leak.
>>>>>>
>>>>>> So is the fact that it is WebService significant then? What else can be
>>>>>> different? I think one thread remains up, and somehow this causes
>>>>>> openjpa
>>>>>> not being able to clean up after itself. What can I do to debug this
>>>>>> more? I
>>>>>> can actually see in the profiler that the objects are allocated by the
>>>>>> WebService, but why aren't they cleaned up?
>>>>>>
>>>>>> Thx,
>>>>>>
>>>>>>
>>>>>> --Kurt
>>>>>>
>>>>>>
>>>>>> Kurt T Stam wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> Thanks Kevin, thanks for your response.
>>>>>>>
>>>>>>> I just replaced the static call by:
>>>>>>>
>>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>             apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>>>>             //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>> apiAuthToken);
>>>>>>>
>>>>>>> which did not make a difference.
>>>>>>>
>>>>>>> I'm wondering if the fact that my class is a webservice makes a
>>>>>>> difference. I'll try extracting it into
>>>>>>> a regular class with a main method and profile that. At least I know
>>>>>>> that
>>>>>>> I didn't forget something
>>>>>>> completely obvious..
>>>>>>>
>>>>>>> --Kurt
>>>>>>>
>>>>>>> Kevin Sutter wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> Kurt,
>>>>>>>> I agree that this is very common usage of the JPA programming model.
>>>>>>>>  And,
>>>>>>>> we are not aware of any memory leaks.  About the only thing that
>>>>>>>> jumps
>>>>>>>> out
>>>>>>>> at me is the following two lines:
>>>>>>>>
>>>>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>              MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>> apiAuthToken);
>>>>>>>>
>>>>>>>> What do these do?  Can you comment these out and see if the memory
>>>>>>>> leak
>>>>>>>> still exists?  Since you are passing the modelAuthToken into this
>>>>>>>> method,
>>>>>>>> I
>>>>>>>> don't know what it's doing with the reference and could it be holding
>>>>>>>> onto
>>>>>>>> something to prevent the GC from cleaning up?
>>>>>>>>
>>>>>>>> The rest of your example seems very straight forward with creating
>>>>>>>> and
>>>>>>>> persisting objects.
>>>>>>>>
>>>>>>>> Kevin
>>>>>>>>
>>>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> If you change the 1000 to something like 1000000... does your
>>>>>>>>> application
>>>>>>>>> go
>>>>>>>>> OOM? Are you running in a JSE environment? What is
>>>>>>>>> PersistenceManager?
>>>>>>>>>
>>>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>> BTW I'm running with the cache off
>>>>>>>>>>
>>>>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>>>>
>>>>>>>>>> (that turns it off right?)
>>>>>>>>>>
>>>>>>>>>> --Kurt
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>>> Hi guys,
>>>>>>>>>>>
>>>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the
>>>>>>>>>>> database.
>>>>>>>>>>>
>>>>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>>>>        EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>>>>>        EntityTransaction tx = em.getTransaction();
>>>>>>>>>>>        try {
>>>>>>>>>>>            tx.begin();
>>>>>>>>>>>                // Generate auth token and store it!
>>>>>>>>>>>            String authInfo = AUTH_TOKEN_PREFIX +
>>>>>>>>>>> UUID.randomUUID();
>>>>>>>>>>>            org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>>>>            if (authInfo != null) {
>>>>>>>>>>>                modelAuthToken.setAuthToken(authInfo);
>>>>>>>>>>>                modelAuthToken.setCreated(new Date());
>>>>>>>>>>>                modelAuthToken.setLastUsed(new Date());
>>>>>>>>>>>                modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>>>>                modelAuthToken.setNumberOfUses(0);
>>>>>>>>>>>                modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>>>>                  em.persist(modelAuthToken);
>>>>>>>>>>>            }
>>>>>>>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>>              MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>>> apiAuthToken);
>>>>>>>>>>>              tx.commit();
>>>>>>>>>>>                      } finally {
>>>>>>>>>>>            if (tx.isActive()) {
>>>>>>>>>>>                tx.rollback();
>>>>>>>>>>>            }
>>>>>>>>>>>            em.clear();
>>>>>>>>>>>            em.close();
>>>>>>>>>>>        }
>>>>>>>>>>>    }
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [ISSUE]
>>>>>>>>>>> After it leaving this code I end up with a 1000
>>>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been
>>>>>>>>>>> using
>>>>>>>>>>> the
>>>>>>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>>>>>>
>>>>>>>>>>> This seems to be pretty the most common use case of using an
>>>>>>>>>>> OR-mapping
>>>>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak here.
>>>>>>>>>>> Does
>>>>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an
>>>>>>>>>>> example
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>> that
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>                   
>>>>>>>>>> does not exhibit this behavior? BTW same code using hibernate does
>>>>>>>>>> not
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>>> accumulate these objects.
>>>>>>>>>>>
>>>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thx,
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Kurt
>>>>>>>>>>>
>>>>>>>>>>> Apache jUDDI.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>>                     
>>>>>>>>> --
>>>>>>>>> Thanks,
>>>>>>>>> Rick
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>
>>>>>>>>                 
>>>>>>>               
>>>>         
>>>
>>>       
>>     
>
>   


Re: memory leak? - simple question

Posted by Kevin Sutter <kw...@gmail.com>.
Hi Kurt,
These instructions, of course, assume that I know something about Tomcat.
This would not be a good assumption...  :-)  If you really want me to see
this in action, I will need more information on how to start Tomcat and how
to load applications.  In the mean time, a trace log of OpenJPA would be an
excellent start.

Thanks!
Kevin

On Thu, Jan 14, 2010 at 11:30 AM, Kurt T Stam <ku...@gmail.com> wrote:

> Hi Kevin,
>
> I tried adding the javaagent stuff, and it still ends up leaking. I
> extracted the code into a tiny
> war with just one servlet which persist the authToken. The build creates a
> Tomcat distro backed by a
> derby database. You can download it from:
>
> http://people.apache.org/~kstam/leak.tgz<http://people.apache.org/%7Ekstam/leak.tgz>
>
> You can build it with: mvn install, you can find the final tomcat in
> tomcat/target/tomcat-1.0.SNAPSHOT.zip
>
> or if you don't want to build it:
> http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip<http://people.apache.org/%7Ekstam/tomcat-1.0.SNAPSHOT.zip>
>
> After you bring up tomcat it persistent an authToken when you hit
> http://localhost:8080/leak/test
>
> In the profiler I see an org.apache.juddi.AuthToken left behind for each
> request.
>
> Hopefully this little example will pinpoint what's going on.
>
> Thx Kev for sticking with me :)
>
> --Kurt
>
>
>
> Kevin Sutter wrote:
>
>> Hi Kurt,
>> Since you have the environment to reproduce this, could you run with the
>> OpenJPA trace enabled?  I'm not even sure what I'm looking for at this
>> point.  :-)  But, maybe there's some tracing that would point us in the
>> right direction.
>>
>> <property name="openjpa.Log" value="DefaultLevel=TRACE"/>
>>
>> We have been always been on the watch for memory leaks, so this one is
>> puzzling.  You did mention blocked threads as well.  At this point, you
>> are
>> probably hitting blocked threads due to the GC and Memory problems.  But,
>> we
>> did recently resolve some locking situations [1] in our MetaDataRepository
>> that could affect scaling (under extreme load).  This was resolved in the
>> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so
>> that should be coming soon).
>>
>> Does your WebService version of the problem use EntityManager
>> transactions,
>> or is the transaction service coming from Tomcat?  And, does your
>> WebService
>> version also do the em.clear() and em.close()?  Basically, are you using
>> application-managed or container-managed persistence while running as a
>> WebService?
>>
>> I'm very interested in helping to resolve this issue.  Like I mentioned,
>> we
>> do a lot of memory leakage and scalability testing of the OpenJPA solution
>> and we're not aware of anything.  Although there is always the possibility
>> that something was resolved in the 1.2.x branch after the 1.2.1 release
>> that
>> I'm not remembering...
>>
>> Thanks for your help,
>> Kevin
>>
>> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
>>
>> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <ku...@gmail.com> wrote:
>>
>>
>>
>>> Thanks Kevin,
>>>
>>> We're enhancing at build time:
>>>
>>>
>>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>>
>>> Yeah we've been running load tests and things are nice and stable with
>>> Hibernate but with Openjpa we see increasing memory use, blocking threads
>>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
>>> preference would be to ship with openjpa by default; but our build
>>> supports
>>> both hibernate and openjpa.
>>>
>>> And yes we use openjpa 1.2.1 (latest stable version).
>>>
>>> --Kurt
>>>
>>>
>>> Kevin Sutter wrote:
>>>
>>>
>>>
>>>> Interesting detective work, Kurt.  Thanks.
>>>>
>>>> Why the WebService version of the app would behave differently as far as
>>>> GC
>>>> is concerned is a mystery.  And, you said that plugging in Hibernate
>>>> into
>>>> this scenario, everything works okay?  Very confusing.
>>>>
>>>> How are you performing the Entity enhancement processing?  Are you
>>>> pre-enhancing via your build process?  Or, are you using the -javaagent
>>>> mechanism?  Or, are you falling back to the subclassing support within
>>>> OpenJPA?  (See [1] for more information on these questions in case they
>>>> don't make sense.)
>>>>
>>>> This would be one area that is different between Hibernate and OpenJPA
>>>> --
>>>> enhancement processing.
>>>>
>>>> In the Tomcat environment, you may be falling back to the subclassing
>>>> support (which we do not recommend) and hitting a memory leak with that.
>>>>
>>>> You said OpenJPA 1.2.x, right?
>>>>
>>>> Just a couple of thoughts on the subject...
>>>> Kevin
>>>>
>>>> [1]
>>>>
>>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>>
>>>>
>>>>
>>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> The same code executed straight from a java client (inVM) shows no
>>>>> memory
>>>>> leak.
>>>>>
>>>>> So is the fact that it is WebService significant then? What else can be
>>>>> different? I think one thread remains up, and somehow this causes
>>>>> openjpa
>>>>> not being able to clean up after itself. What can I do to debug this
>>>>> more? I
>>>>> can actually see in the profiler that the objects are allocated by the
>>>>> WebService, but why aren't they cleaned up?
>>>>>
>>>>> Thx,
>>>>>
>>>>>
>>>>> --Kurt
>>>>>
>>>>>
>>>>> Kurt T Stam wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Thanks Kevin, thanks for your response.
>>>>>>
>>>>>> I just replaced the static call by:
>>>>>>
>>>>>>             apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>             apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>>>             //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>> apiAuthToken);
>>>>>>
>>>>>> which did not make a difference.
>>>>>>
>>>>>> I'm wondering if the fact that my class is a webservice makes a
>>>>>> difference. I'll try extracting it into
>>>>>> a regular class with a main method and profile that. At least I know
>>>>>> that
>>>>>> I didn't forget something
>>>>>> completely obvious..
>>>>>>
>>>>>> --Kurt
>>>>>>
>>>>>> Kevin Sutter wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Kurt,
>>>>>>> I agree that this is very common usage of the JPA programming model.
>>>>>>>  And,
>>>>>>> we are not aware of any memory leaks.  About the only thing that
>>>>>>> jumps
>>>>>>> out
>>>>>>> at me is the following two lines:
>>>>>>>
>>>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>              MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>> apiAuthToken);
>>>>>>>
>>>>>>> What do these do?  Can you comment these out and see if the memory
>>>>>>> leak
>>>>>>> still exists?  Since you are passing the modelAuthToken into this
>>>>>>> method,
>>>>>>> I
>>>>>>> don't know what it's doing with the reference and could it be holding
>>>>>>> onto
>>>>>>> something to prevent the GC from cleaning up?
>>>>>>>
>>>>>>> The rest of your example seems very straight forward with creating
>>>>>>> and
>>>>>>> persisting objects.
>>>>>>>
>>>>>>> Kevin
>>>>>>>
>>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> If you change the 1000 to something like 1000000... does your
>>>>>>>> application
>>>>>>>> go
>>>>>>>> OOM? Are you running in a JSE environment? What is
>>>>>>>> PersistenceManager?
>>>>>>>>
>>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> BTW I'm running with the cache off
>>>>>>>>>
>>>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>>>
>>>>>>>>> (that turns it off right?)
>>>>>>>>>
>>>>>>>>> --Kurt
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Kurt T Stam wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Hi guys,
>>>>>>>>>>
>>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the
>>>>>>>>>> database.
>>>>>>>>>>
>>>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>>>        EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>>>>        EntityTransaction tx = em.getTransaction();
>>>>>>>>>>        try {
>>>>>>>>>>            tx.begin();
>>>>>>>>>>                // Generate auth token and store it!
>>>>>>>>>>            String authInfo = AUTH_TOKEN_PREFIX +
>>>>>>>>>> UUID.randomUUID();
>>>>>>>>>>            org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>>>            if (authInfo != null) {
>>>>>>>>>>                modelAuthToken.setAuthToken(authInfo);
>>>>>>>>>>                modelAuthToken.setCreated(new Date());
>>>>>>>>>>                modelAuthToken.setLastUsed(new Date());
>>>>>>>>>>                modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>>>                modelAuthToken.setNumberOfUses(0);
>>>>>>>>>>                modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>>>                  em.persist(modelAuthToken);
>>>>>>>>>>            }
>>>>>>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>>              MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>>> apiAuthToken);
>>>>>>>>>>              tx.commit();
>>>>>>>>>>                      } finally {
>>>>>>>>>>            if (tx.isActive()) {
>>>>>>>>>>                tx.rollback();
>>>>>>>>>>            }
>>>>>>>>>>            em.clear();
>>>>>>>>>>            em.close();
>>>>>>>>>>        }
>>>>>>>>>>    }
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [ISSUE]
>>>>>>>>>> After it leaving this code I end up with a 1000
>>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been
>>>>>>>>>> using
>>>>>>>>>> the
>>>>>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>>>>>
>>>>>>>>>> This seems to be pretty the most common use case of using an
>>>>>>>>>> OR-mapping
>>>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak here.
>>>>>>>>>> Does
>>>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an
>>>>>>>>>> example
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> that
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> does not exhibit this behavior? BTW same code using hibernate does
>>>>>>>>> not
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> accumulate these objects.
>>>>>>>>>>
>>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Thx,
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Kurt
>>>>>>>>>>
>>>>>>>>>> Apache jUDDI.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> --
>>>>>>>> Thanks,
>>>>>>>> Rick
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>>
>
>

Re: memory leak? - simple question

Posted by Kurt T Stam <ku...@gmail.com>.
Hi Kevin,

I tried adding the javaagent stuff, and it still ends up leaking. I 
extracted the code into a tiny
war with just one servlet which persist the authToken. The build creates 
a Tomcat distro backed by a
derby database. You can download it from:

http://people.apache.org/~kstam/leak.tgz

You can build it with: mvn install, you can find the final tomcat in 
tomcat/target/tomcat-1.0.SNAPSHOT.zip

or if you don't want to build it:
http://people.apache.org/~kstam/tomcat-1.0.SNAPSHOT.zip

After you bring up tomcat it persistent an authToken when you hit
http://localhost:8080/leak/test

In the profiler I see an org.apache.juddi.AuthToken left behind for each 
request.

Hopefully this little example will pinpoint what's going on.

Thx Kev for sticking with me :)

--Kurt


Kevin Sutter wrote:
> Hi Kurt,
> Since you have the environment to reproduce this, could you run with the
> OpenJPA trace enabled?  I'm not even sure what I'm looking for at this
> point.  :-)  But, maybe there's some tracing that would point us in the
> right direction.
>
> <property name="openjpa.Log" value="DefaultLevel=TRACE"/>
>
> We have been always been on the watch for memory leaks, so this one is
> puzzling.  You did mention blocked threads as well.  At this point, you are
> probably hitting blocked threads due to the GC and Memory problems.  But, we
> did recently resolve some locking situations [1] in our MetaDataRepository
> that could affect scaling (under extreme load).  This was resolved in the
> 1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so
> that should be coming soon).
>
> Does your WebService version of the problem use EntityManager transactions,
> or is the transaction service coming from Tomcat?  And, does your WebService
> version also do the em.clear() and em.close()?  Basically, are you using
> application-managed or container-managed persistence while running as a
> WebService?
>
> I'm very interested in helping to resolve this issue.  Like I mentioned, we
> do a lot of memory leakage and scalability testing of the OpenJPA solution
> and we're not aware of anything.  Although there is always the possibility
> that something was resolved in the 1.2.x branch after the 1.2.1 release that
> I'm not remembering...
>
> Thanks for your help,
> Kevin
>
> [1]  https://issues.apache.org/jira/browse/OPENJPA-250
>
> On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <ku...@gmail.com> wrote:
>
>   
>> Thanks Kevin,
>>
>> We're enhancing at build time:
>>
>> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>>
>> Yeah we've been running load tests and things are nice and stable with
>> Hibernate but with Openjpa we see increasing memory use, blocking threads
>> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
>> preference would be to ship with openjpa by default; but our build supports
>> both hibernate and openjpa.
>>
>> And yes we use openjpa 1.2.1 (latest stable version).
>>
>> --Kurt
>>
>>
>> Kevin Sutter wrote:
>>
>>     
>>> Interesting detective work, Kurt.  Thanks.
>>>
>>> Why the WebService version of the app would behave differently as far as
>>> GC
>>> is concerned is a mystery.  And, you said that plugging in Hibernate into
>>> this scenario, everything works okay?  Very confusing.
>>>
>>> How are you performing the Entity enhancement processing?  Are you
>>> pre-enhancing via your build process?  Or, are you using the -javaagent
>>> mechanism?  Or, are you falling back to the subclassing support within
>>> OpenJPA?  (See [1] for more information on these questions in case they
>>> don't make sense.)
>>>
>>> This would be one area that is different between Hibernate and OpenJPA --
>>> enhancement processing.
>>>
>>> In the Tomcat environment, you may be falling back to the subclassing
>>> support (which we do not recommend) and hitting a memory leak with that.
>>>
>>> You said OpenJPA 1.2.x, right?
>>>
>>> Just a couple of thoughts on the subject...
>>> Kevin
>>>
>>> [1]
>>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>>
>>>
>>>
>>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com> wrote:
>>>
>>>
>>>
>>>       
>>>> The same code executed straight from a java client (inVM) shows no memory
>>>> leak.
>>>>
>>>> So is the fact that it is WebService significant then? What else can be
>>>> different? I think one thread remains up, and somehow this causes openjpa
>>>> not being able to clean up after itself. What can I do to debug this
>>>> more? I
>>>> can actually see in the profiler that the objects are allocated by the
>>>> WebService, but why aren't they cleaned up?
>>>>
>>>> Thx,
>>>>
>>>>
>>>> --Kurt
>>>>
>>>>
>>>> Kurt T Stam wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> Thanks Kevin, thanks for your response.
>>>>>
>>>>> I just replaced the static call by:
>>>>>
>>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>              apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>>              //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>> apiAuthToken);
>>>>>
>>>>> which did not make a difference.
>>>>>
>>>>> I'm wondering if the fact that my class is a webservice makes a
>>>>> difference. I'll try extracting it into
>>>>> a regular class with a main method and profile that. At least I know
>>>>> that
>>>>> I didn't forget something
>>>>> completely obvious..
>>>>>
>>>>> --Kurt
>>>>>
>>>>> Kevin Sutter wrote:
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> Kurt,
>>>>>> I agree that this is very common usage of the JPA programming model.
>>>>>>  And,
>>>>>> we are not aware of any memory leaks.  About the only thing that jumps
>>>>>> out
>>>>>> at me is the following two lines:
>>>>>>
>>>>>>               apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>               MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>> apiAuthToken);
>>>>>>
>>>>>> What do these do?  Can you comment these out and see if the memory leak
>>>>>> still exists?  Since you are passing the modelAuthToken into this
>>>>>> method,
>>>>>> I
>>>>>> don't know what it's doing with the reference and could it be holding
>>>>>> onto
>>>>>> something to prevent the GC from cleaning up?
>>>>>>
>>>>>> The rest of your example seems very straight forward with creating and
>>>>>> persisting objects.
>>>>>>
>>>>>> Kevin
>>>>>>
>>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> If you change the 1000 to something like 1000000... does your
>>>>>>> application
>>>>>>> go
>>>>>>> OOM? Are you running in a JSE environment? What is PersistenceManager?
>>>>>>>
>>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> BTW I'm running with the cache off
>>>>>>>>
>>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>>
>>>>>>>> (that turns it off right?)
>>>>>>>>
>>>>>>>> --Kurt
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Kurt T Stam wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> Hi guys,
>>>>>>>>>
>>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the database.
>>>>>>>>>
>>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>>         EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>>>         EntityTransaction tx = em.getTransaction();
>>>>>>>>>         try {
>>>>>>>>>             tx.begin();
>>>>>>>>>                 // Generate auth token and store it!
>>>>>>>>>             String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
>>>>>>>>>             org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>>             if (authInfo != null) {
>>>>>>>>>                 modelAuthToken.setAuthToken(authInfo);
>>>>>>>>>                 modelAuthToken.setCreated(new Date());
>>>>>>>>>                 modelAuthToken.setLastUsed(new Date());
>>>>>>>>>                 modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>>                 modelAuthToken.setNumberOfUses(0);
>>>>>>>>>                 modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>>                   em.persist(modelAuthToken);
>>>>>>>>>             }
>>>>>>>>>               apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>>               MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>>> apiAuthToken);
>>>>>>>>>               tx.commit();
>>>>>>>>>                       } finally {
>>>>>>>>>             if (tx.isActive()) {
>>>>>>>>>                 tx.rollback();
>>>>>>>>>             }
>>>>>>>>>             em.clear();
>>>>>>>>>             em.close();
>>>>>>>>>         }
>>>>>>>>>     }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [ISSUE]
>>>>>>>>> After it leaving this code I end up with a 1000
>>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been using
>>>>>>>>> the
>>>>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>>>>
>>>>>>>>> This seems to be pretty the most common use case of using an
>>>>>>>>> OR-mapping
>>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak here.
>>>>>>>>> Does
>>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an
>>>>>>>>> example
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> that
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>               
>>>>>>>> does not exhibit this behavior? BTW same code using hibernate does
>>>>>>>> not
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> accumulate these objects.
>>>>>>>>>
>>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thx,
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Kurt
>>>>>>>>>
>>>>>>>>> Apache jUDDI.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>                 
>>>>>>> --
>>>>>>> Thanks,
>>>>>>> Rick
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>
>>>>>>             
>>>>>           
>>>       
>>     
>
>   


Re: memory leak? - simple question

Posted by Kevin Sutter <kw...@gmail.com>.
Hi Kurt,
Since you have the environment to reproduce this, could you run with the
OpenJPA trace enabled?  I'm not even sure what I'm looking for at this
point.  :-)  But, maybe there's some tracing that would point us in the
right direction.

<property name="openjpa.Log" value="DefaultLevel=TRACE"/>

We have been always been on the watch for memory leaks, so this one is
puzzling.  You did mention blocked threads as well.  At this point, you are
probably hitting blocked threads due to the GC and Memory problems.  But, we
did recently resolve some locking situations [1] in our MetaDataRepository
that could affect scaling (under extreme load).  This was resolved in the
1.2.x SNAPSHOT drivers (Mike is currently pushing for a 1.2.2 release, so
that should be coming soon).

Does your WebService version of the problem use EntityManager transactions,
or is the transaction service coming from Tomcat?  And, does your WebService
version also do the em.clear() and em.close()?  Basically, are you using
application-managed or container-managed persistence while running as a
WebService?

I'm very interested in helping to resolve this issue.  Like I mentioned, we
do a lot of memory leakage and scalability testing of the OpenJPA solution
and we're not aware of anything.  Although there is always the possibility
that something was resolved in the 1.2.x branch after the 1.2.1 release that
I'm not remembering...

Thanks for your help,
Kevin

[1]  https://issues.apache.org/jira/browse/OPENJPA-250

On Wed, Jan 13, 2010 at 9:27 PM, Kurt T Stam <ku...@gmail.com> wrote:

> Thanks Kevin,
>
> We're enhancing at build time:
>
> http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml
>
> Yeah we've been running load tests and things are nice and stable with
> Hibernate but with Openjpa we see increasing memory use, blocking threads
> and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. Our
> preference would be to ship with openjpa by default; but our build supports
> both hibernate and openjpa.
>
> And yes we use openjpa 1.2.1 (latest stable version).
>
> --Kurt
>
>
> Kevin Sutter wrote:
>
>> Interesting detective work, Kurt.  Thanks.
>>
>> Why the WebService version of the app would behave differently as far as
>> GC
>> is concerned is a mystery.  And, you said that plugging in Hibernate into
>> this scenario, everything works okay?  Very confusing.
>>
>> How are you performing the Entity enhancement processing?  Are you
>> pre-enhancing via your build process?  Or, are you using the -javaagent
>> mechanism?  Or, are you falling back to the subclassing support within
>> OpenJPA?  (See [1] for more information on these questions in case they
>> don't make sense.)
>>
>> This would be one area that is different between Hibernate and OpenJPA --
>> enhancement processing.
>>
>> In the Tomcat environment, you may be falling back to the subclassing
>> support (which we do not recommend) and hitting a memory leak with that.
>>
>> You said OpenJPA 1.2.x, right?
>>
>> Just a couple of thoughts on the subject...
>> Kevin
>>
>> [1]
>> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>>
>>
>>
>> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com> wrote:
>>
>>
>>
>>> The same code executed straight from a java client (inVM) shows no memory
>>> leak.
>>>
>>> So is the fact that it is WebService significant then? What else can be
>>> different? I think one thread remains up, and somehow this causes openjpa
>>> not being able to clean up after itself. What can I do to debug this
>>> more? I
>>> can actually see in the profiler that the objects are allocated by the
>>> WebService, but why aren't they cleaned up?
>>>
>>> Thx,
>>>
>>>
>>> --Kurt
>>>
>>>
>>> Kurt T Stam wrote:
>>>
>>>
>>>
>>>> Thanks Kevin, thanks for your response.
>>>>
>>>> I just replaced the static call by:
>>>>
>>>>              apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>              apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>>              //MappingModelToApi.mapAuthToken(modelAuthToken,
>>>> apiAuthToken);
>>>>
>>>> which did not make a difference.
>>>>
>>>> I'm wondering if the fact that my class is a webservice makes a
>>>> difference. I'll try extracting it into
>>>> a regular class with a main method and profile that. At least I know
>>>> that
>>>> I didn't forget something
>>>> completely obvious..
>>>>
>>>> --Kurt
>>>>
>>>> Kevin Sutter wrote:
>>>>
>>>>
>>>>
>>>>> Kurt,
>>>>> I agree that this is very common usage of the JPA programming model.
>>>>>  And,
>>>>> we are not aware of any memory leaks.  About the only thing that jumps
>>>>> out
>>>>> at me is the following two lines:
>>>>>
>>>>>               apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>               MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>> apiAuthToken);
>>>>>
>>>>> What do these do?  Can you comment these out and see if the memory leak
>>>>> still exists?  Since you are passing the modelAuthToken into this
>>>>> method,
>>>>> I
>>>>> don't know what it's doing with the reference and could it be holding
>>>>> onto
>>>>> something to prevent the GC from cleaning up?
>>>>>
>>>>> The rest of your example seems very straight forward with creating and
>>>>> persisting objects.
>>>>>
>>>>> Kevin
>>>>>
>>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> If you change the 1000 to something like 1000000... does your
>>>>>> application
>>>>>> go
>>>>>> OOM? Are you running in a JSE environment? What is PersistenceManager?
>>>>>>
>>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> BTW I'm running with the cache off
>>>>>>>
>>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>>
>>>>>>> (that turns it off right?)
>>>>>>>
>>>>>>> --Kurt
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Kurt T Stam wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Hi guys,
>>>>>>>>
>>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the database.
>>>>>>>>
>>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>>         EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>>         EntityTransaction tx = em.getTransaction();
>>>>>>>>         try {
>>>>>>>>             tx.begin();
>>>>>>>>                 // Generate auth token and store it!
>>>>>>>>             String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
>>>>>>>>             org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>>             if (authInfo != null) {
>>>>>>>>                 modelAuthToken.setAuthToken(authInfo);
>>>>>>>>                 modelAuthToken.setCreated(new Date());
>>>>>>>>                 modelAuthToken.setLastUsed(new Date());
>>>>>>>>                 modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>>                 modelAuthToken.setNumberOfUses(0);
>>>>>>>>                 modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>>                   em.persist(modelAuthToken);
>>>>>>>>             }
>>>>>>>>               apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>>               MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>>> apiAuthToken);
>>>>>>>>               tx.commit();
>>>>>>>>                       } finally {
>>>>>>>>             if (tx.isActive()) {
>>>>>>>>                 tx.rollback();
>>>>>>>>             }
>>>>>>>>             em.clear();
>>>>>>>>             em.close();
>>>>>>>>         }
>>>>>>>>     }
>>>>>>>>
>>>>>>>>
>>>>>>>> [ISSUE]
>>>>>>>> After it leaving this code I end up with a 1000
>>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been using
>>>>>>>> the
>>>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>>>
>>>>>>>> This seems to be pretty the most common use case of using an
>>>>>>>> OR-mapping
>>>>>>>> tool, so I find it hard to believe openjpa has a memory leak here.
>>>>>>>> Does
>>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an
>>>>>>>> example
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> that
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>> does not exhibit this behavior? BTW same code using hibernate does
>>>>>>> not
>>>>>>>
>>>>>>>
>>>>>>>> accumulate these objects.
>>>>>>>>
>>>>>>>> We're using openjpa 1.2.1.
>>>>>>>>
>>>>>>>>
>>>>>>>> Thx,
>>>>>>>>
>>>>>>>>
>>>>>>>> Kurt
>>>>>>>>
>>>>>>>> Apache jUDDI.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>> --
>>>>>> Thanks,
>>>>>> Rick
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
>
>

Re: memory leak? - simple question

Posted by Kurt T Stam <ku...@gmail.com>.
Thanks Kevin,

We're enhancing at build time:

http://svn.apache.org/repos/asf/webservices/juddi/trunk/juddi-core/pom.xml

Yeah we've been running load tests and things are nice and stable with 
Hibernate but with Openjpa we see increasing memory use, blocking 
threads and then an OOM. http://issues.apache.org/jira/browse/JUDDI-267. 
Our preference would be to ship with openjpa by default; but our build 
supports both hibernate and openjpa.

And yes we use openjpa 1.2.1 (latest stable version).

--Kurt

Kevin Sutter wrote:
> Interesting detective work, Kurt.  Thanks.
>
> Why the WebService version of the app would behave differently as far as GC
> is concerned is a mystery.  And, you said that plugging in Hibernate into
> this scenario, everything works okay?  Very confusing.
>
> How are you performing the Entity enhancement processing?  Are you
> pre-enhancing via your build process?  Or, are you using the -javaagent
> mechanism?  Or, are you falling back to the subclassing support within
> OpenJPA?  (See [1] for more information on these questions in case they
> don't make sense.)
>
> This would be one area that is different between Hibernate and OpenJPA --
> enhancement processing.
>
> In the Tomcat environment, you may be falling back to the subclassing
> support (which we do not recommend) and hitting a memory leak with that.
>
> You said OpenJPA 1.2.x, right?
>
> Just a couple of thoughts on the subject...
> Kevin
>
> [1]
> http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html
>
>
>
> On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com> wrote:
>
>   
>> The same code executed straight from a java client (inVM) shows no memory
>> leak.
>>
>> So is the fact that it is WebService significant then? What else can be
>> different? I think one thread remains up, and somehow this causes openjpa
>> not being able to clean up after itself. What can I do to debug this more? I
>> can actually see in the profiler that the objects are allocated by the
>> WebService, but why aren't they cleaned up?
>>
>> Thx,
>>
>>
>> --Kurt
>>
>>
>> Kurt T Stam wrote:
>>
>>     
>>> Thanks Kevin, thanks for your response.
>>>
>>> I just replaced the static call by:
>>>
>>>               apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>               apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>>               //MappingModelToApi.mapAuthToken(modelAuthToken,
>>> apiAuthToken);
>>>
>>> which did not make a difference.
>>>
>>> I'm wondering if the fact that my class is a webservice makes a
>>> difference. I'll try extracting it into
>>> a regular class with a main method and profile that. At least I know that
>>> I didn't forget something
>>> completely obvious..
>>>
>>> --Kurt
>>>
>>> Kevin Sutter wrote:
>>>
>>>       
>>>> Kurt,
>>>> I agree that this is very common usage of the JPA programming model.
>>>>  And,
>>>> we are not aware of any memory leaks.  About the only thing that jumps
>>>> out
>>>> at me is the following two lines:
>>>>
>>>>                apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>                MappingModelToApi.mapAuthToken(modelAuthToken,
>>>> apiAuthToken);
>>>>
>>>> What do these do?  Can you comment these out and see if the memory leak
>>>> still exists?  Since you are passing the modelAuthToken into this method,
>>>> I
>>>> don't know what it's doing with the reference and could it be holding
>>>> onto
>>>> something to prevent the GC from cleaning up?
>>>>
>>>> The rest of your example seems very straight forward with creating and
>>>> persisting objects.
>>>>
>>>> Kevin
>>>>
>>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com> wrote:
>>>>
>>>>
>>>>
>>>>         
>>>>> If you change the 1000 to something like 1000000... does your
>>>>> application
>>>>> go
>>>>> OOM? Are you running in a JSE environment? What is PersistenceManager?
>>>>>
>>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>>> BTW I'm running with the cache off
>>>>>>
>>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>>
>>>>>> (that turns it off right?)
>>>>>>
>>>>>> --Kurt
>>>>>>
>>>>>>
>>>>>>
>>>>>> Kurt T Stam wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> Hi guys,
>>>>>>>
>>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the database.
>>>>>>>
>>>>>>> for (int i=1; i<1000; i++) {
>>>>>>>          EntityManager em = PersistenceManager.getEntityManager();
>>>>>>>          EntityTransaction tx = em.getTransaction();
>>>>>>>          try {
>>>>>>>              tx.begin();
>>>>>>>                  // Generate auth token and store it!
>>>>>>>              String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
>>>>>>>              org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>>              if (authInfo != null) {
>>>>>>>                  modelAuthToken.setAuthToken(authInfo);
>>>>>>>                  modelAuthToken.setCreated(new Date());
>>>>>>>                  modelAuthToken.setLastUsed(new Date());
>>>>>>>                  modelAuthToken.setAuthorizedName(publisherId);
>>>>>>>                  modelAuthToken.setNumberOfUses(0);
>>>>>>>                  modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>>                    em.persist(modelAuthToken);
>>>>>>>              }
>>>>>>>                apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>>                MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>>> apiAuthToken);
>>>>>>>                tx.commit();
>>>>>>>                        } finally {
>>>>>>>              if (tx.isActive()) {
>>>>>>>                  tx.rollback();
>>>>>>>              }
>>>>>>>              em.clear();
>>>>>>>              em.close();
>>>>>>>          }
>>>>>>>      }
>>>>>>>
>>>>>>>
>>>>>>> [ISSUE]
>>>>>>> After it leaving this code I end up with a 1000
>>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been using
>>>>>>> the
>>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>>
>>>>>>> This seems to be pretty the most common use case of using an
>>>>>>> OR-mapping
>>>>>>> tool, so I find it hard to believe openjpa has a memory leak here.
>>>>>>> Does
>>>>>>> anyone see what I'm doing wrong? Or can someone point me to an example
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> that
>>>>>>             
>>>>>           
>>>>>> does not exhibit this behavior? BTW same code using hibernate does not
>>>>>>             
>>>>>>> accumulate these objects.
>>>>>>>
>>>>>>> We're using openjpa 1.2.1.
>>>>>>>
>>>>>>>
>>>>>>> Thx,
>>>>>>>
>>>>>>>
>>>>>>> Kurt
>>>>>>>
>>>>>>> Apache jUDDI.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>             
>>>>> --
>>>>> Thanks,
>>>>> Rick
>>>>>
>>>>>
>>>>>
>>>>>           
>>>>
>>>>         
>>>       
>
>   


Re: memory leak? - simple question

Posted by Kevin Sutter <kw...@gmail.com>.
Interesting detective work, Kurt.  Thanks.

Why the WebService version of the app would behave differently as far as GC
is concerned is a mystery.  And, you said that plugging in Hibernate into
this scenario, everything works okay?  Very confusing.

How are you performing the Entity enhancement processing?  Are you
pre-enhancing via your build process?  Or, are you using the -javaagent
mechanism?  Or, are you falling back to the subclassing support within
OpenJPA?  (See [1] for more information on these questions in case they
don't make sense.)

This would be one area that is different between Hibernate and OpenJPA --
enhancement processing.

In the Tomcat environment, you may be falling back to the subclassing
support (which we do not recommend) and hitting a memory leak with that.

You said OpenJPA 1.2.x, right?

Just a couple of thoughts on the subject...
Kevin

[1]
http://webspherepersistence.blogspot.com/2009/02/openjpa-enhancement.html



On Wed, Jan 13, 2010 at 4:25 PM, Kurt T Stam <ku...@gmail.com> wrote:

> The same code executed straight from a java client (inVM) shows no memory
> leak.
>
> So is the fact that it is WebService significant then? What else can be
> different? I think one thread remains up, and somehow this causes openjpa
> not being able to clean up after itself. What can I do to debug this more? I
> can actually see in the profiler that the objects are allocated by the
> WebService, but why aren't they cleaned up?
>
> Thx,
>
>
> --Kurt
>
>
> Kurt T Stam wrote:
>
>> Thanks Kevin, thanks for your response.
>>
>> I just replaced the static call by:
>>
>>               apiAuthToken = new org.uddi.api_v3.AuthToken();
>>               apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>>               //MappingModelToApi.mapAuthToken(modelAuthToken,
>> apiAuthToken);
>>
>> which did not make a difference.
>>
>> I'm wondering if the fact that my class is a webservice makes a
>> difference. I'll try extracting it into
>> a regular class with a main method and profile that. At least I know that
>> I didn't forget something
>> completely obvious..
>>
>> --Kurt
>>
>> Kevin Sutter wrote:
>>
>>> Kurt,
>>> I agree that this is very common usage of the JPA programming model.
>>>  And,
>>> we are not aware of any memory leaks.  About the only thing that jumps
>>> out
>>> at me is the following two lines:
>>>
>>>                apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>                MappingModelToApi.mapAuthToken(modelAuthToken,
>>> apiAuthToken);
>>>
>>> What do these do?  Can you comment these out and see if the memory leak
>>> still exists?  Since you are passing the modelAuthToken into this method,
>>> I
>>> don't know what it's doing with the reference and could it be holding
>>> onto
>>> something to prevent the GC from cleaning up?
>>>
>>> The rest of your example seems very straight forward with creating and
>>> persisting objects.
>>>
>>> Kevin
>>>
>>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com> wrote:
>>>
>>>
>>>
>>>> If you change the 1000 to something like 1000000... does your
>>>> application
>>>> go
>>>> OOM? Are you running in a JSE environment? What is PersistenceManager?
>>>>
>>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>> BTW I'm running with the cache off
>>>>>
>>>>> <property name="openjpa.DataCache" value="false"/>
>>>>>
>>>>> (that turns it off right?)
>>>>>
>>>>> --Kurt
>>>>>
>>>>>
>>>>>
>>>>> Kurt T Stam wrote:
>>>>>
>>>>>
>>>>>
>>>>>> Hi guys,
>>>>>>
>>>>>> [DESCRIPTION] The code below inserts a 1000 records in the database.
>>>>>>
>>>>>> for (int i=1; i<1000; i++) {
>>>>>>          EntityManager em = PersistenceManager.getEntityManager();
>>>>>>          EntityTransaction tx = em.getTransaction();
>>>>>>          try {
>>>>>>              tx.begin();
>>>>>>                  // Generate auth token and store it!
>>>>>>              String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
>>>>>>              org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>>> org.apache.juddi.model.AuthToken();
>>>>>>              if (authInfo != null) {
>>>>>>                  modelAuthToken.setAuthToken(authInfo);
>>>>>>                  modelAuthToken.setCreated(new Date());
>>>>>>                  modelAuthToken.setLastUsed(new Date());
>>>>>>                  modelAuthToken.setAuthorizedName(publisherId);
>>>>>>                  modelAuthToken.setNumberOfUses(0);
>>>>>>                  modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>>                    em.persist(modelAuthToken);
>>>>>>              }
>>>>>>                apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>>                MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>>> apiAuthToken);
>>>>>>                tx.commit();
>>>>>>                        } finally {
>>>>>>              if (tx.isActive()) {
>>>>>>                  tx.rollback();
>>>>>>              }
>>>>>>              em.clear();
>>>>>>              em.close();
>>>>>>          }
>>>>>>      }
>>>>>>
>>>>>>
>>>>>> [ISSUE]
>>>>>> After it leaving this code I end up with a 1000
>>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been using
>>>>>> the
>>>>>> profiler, and these objects cannot be garbage collected.
>>>>>>
>>>>>> This seems to be pretty the most common use case of using an
>>>>>> OR-mapping
>>>>>> tool, so I find it hard to believe openjpa has a memory leak here.
>>>>>> Does
>>>>>> anyone see what I'm doing wrong? Or can someone point me to an example
>>>>>>
>>>>>>
>>>>> that
>>>>
>>>>
>>>>> does not exhibit this behavior? BTW same code using hibernate does not
>>>>>> accumulate these objects.
>>>>>>
>>>>>> We're using openjpa 1.2.1.
>>>>>>
>>>>>>
>>>>>> Thx,
>>>>>>
>>>>>>
>>>>>> Kurt
>>>>>>
>>>>>> Apache jUDDI.
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>> --
>>>> Thanks,
>>>> Rick
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>

Re: memory leak? - simple question

Posted by Kurt T Stam <ku...@gmail.com>.
The same code executed straight from a java client (inVM) shows no 
memory leak.

So is the fact that it is WebService significant then? What else can be 
different? I think one thread remains up, and somehow this causes 
openjpa not being able to clean up after itself. What can I do to debug 
this more? I can actually see in the profiler that the objects are 
allocated by the WebService, but why aren't they cleaned up?

Thx,

--Kurt


Kurt T Stam wrote:
> Thanks Kevin, thanks for your response.
>
> I just replaced the static call by:
>
>                apiAuthToken = new org.uddi.api_v3.AuthToken();
>                apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
>                //MappingModelToApi.mapAuthToken(modelAuthToken, 
> apiAuthToken);
>
> which did not make a difference.
>
> I'm wondering if the fact that my class is a webservice makes a 
> difference. I'll try extracting it into
> a regular class with a main method and profile that. At least I know 
> that I didn't forget something
> completely obvious..
>
> --Kurt
>
> Kevin Sutter wrote:
>> Kurt,
>> I agree that this is very common usage of the JPA programming model.  
>> And,
>> we are not aware of any memory leaks.  About the only thing that 
>> jumps out
>> at me is the following two lines:
>>
>>                 apiAuthToken = new org.uddi.api_v3.AuthToken();
>>                 MappingModelToApi.mapAuthToken(modelAuthToken,
>> apiAuthToken);
>>
>> What do these do?  Can you comment these out and see if the memory leak
>> still exists?  Since you are passing the modelAuthToken into this 
>> method, I
>> don't know what it's doing with the reference and could it be holding 
>> onto
>> something to prevent the GC from cleaning up?
>>
>> The rest of your example seems very straight forward with creating and
>> persisting objects.
>>
>> Kevin
>>
>> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com> wrote:
>>
>>  
>>> If you change the 1000 to something like 1000000... does your 
>>> application
>>> go
>>> OOM? Are you running in a JSE environment? What is PersistenceManager?
>>>
>>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com> 
>>> wrote:
>>>
>>>    
>>>> BTW I'm running with the cache off
>>>>
>>>> <property name="openjpa.DataCache" value="false"/>
>>>>
>>>> (that turns it off right?)
>>>>
>>>> --Kurt
>>>>
>>>>
>>>>
>>>> Kurt T Stam wrote:
>>>>
>>>>      
>>>>> Hi guys,
>>>>>
>>>>> [DESCRIPTION] The code below inserts a 1000 records in the database.
>>>>>
>>>>> for (int i=1; i<1000; i++) {
>>>>>           EntityManager em = PersistenceManager.getEntityManager();
>>>>>           EntityTransaction tx = em.getTransaction();
>>>>>           try {
>>>>>               tx.begin();
>>>>>                   // Generate auth token and store it!
>>>>>               String authInfo = AUTH_TOKEN_PREFIX + 
>>>>> UUID.randomUUID();
>>>>>               org.apache.juddi.model.AuthToken modelAuthToken = new
>>>>> org.apache.juddi.model.AuthToken();
>>>>>               if (authInfo != null) {
>>>>>                   modelAuthToken.setAuthToken(authInfo);
>>>>>                   modelAuthToken.setCreated(new Date());
>>>>>                   modelAuthToken.setLastUsed(new Date());
>>>>>                   modelAuthToken.setAuthorizedName(publisherId);
>>>>>                   modelAuthToken.setNumberOfUses(0);
>>>>>                   modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>>                     em.persist(modelAuthToken);
>>>>>               }
>>>>>                 apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>>                 MappingModelToApi.mapAuthToken(modelAuthToken,
>>>>> apiAuthToken);
>>>>>                 tx.commit();
>>>>>                         } finally {
>>>>>               if (tx.isActive()) {
>>>>>                   tx.rollback();
>>>>>               }
>>>>>               em.clear();
>>>>>               em.close();
>>>>>           }
>>>>>       }
>>>>>
>>>>>
>>>>> [ISSUE]
>>>>> After it leaving this code I end up with a 1000
>>>>> org.apache.juddi.model.AuthToken objects in memory. I've been 
>>>>> using the
>>>>> profiler, and these objects cannot be garbage collected.
>>>>>
>>>>> This seems to be pretty the most common use case of using an 
>>>>> OR-mapping
>>>>> tool, so I find it hard to believe openjpa has a memory leak here. 
>>>>> Does
>>>>> anyone see what I'm doing wrong? Or can someone point me to an 
>>>>> example
>>>>>         
>>> that
>>>    
>>>>> does not exhibit this behavior? BTW same code using hibernate does 
>>>>> not
>>>>> accumulate these objects.
>>>>>
>>>>> We're using openjpa 1.2.1.
>>>>>
>>>>>
>>>>> Thx,
>>>>>
>>>>>
>>>>> Kurt
>>>>>
>>>>> Apache jUDDI.
>>>>>
>>>>>         
>>>>       
>>> -- 
>>> Thanks,
>>> Rick
>>>
>>>     
>>
>>   
>


Re: memory leak? - simple question

Posted by Kurt T Stam <ku...@gmail.com>.
Thanks Kevin, thanks for your response.

I just replaced the static call by:

                apiAuthToken = new org.uddi.api_v3.AuthToken();
                apiAuthToken.setAuthInfo(modelAuthToken.getAuthToken());
                //MappingModelToApi.mapAuthToken(modelAuthToken, 
apiAuthToken);

which did not make a difference.

I'm wondering if the fact that my class is a webservice makes a 
difference. I'll try extracting it into
a regular class with a main method and profile that. At least I know 
that I didn't forget something
completely obvious..

--Kurt

Kevin Sutter wrote:
> Kurt,
> I agree that this is very common usage of the JPA programming model.  And,
> we are not aware of any memory leaks.  About the only thing that jumps out
> at me is the following two lines:
>
>                 apiAuthToken = new org.uddi.api_v3.AuthToken();
>                 MappingModelToApi.mapAuthToken(modelAuthToken,
> apiAuthToken);
>
> What do these do?  Can you comment these out and see if the memory leak
> still exists?  Since you are passing the modelAuthToken into this method, I
> don't know what it's doing with the reference and could it be holding onto
> something to prevent the GC from cleaning up?
>
> The rest of your example seems very straight forward with creating and
> persisting objects.
>
> Kevin
>
> On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com> wrote:
>
>   
>> If you change the 1000 to something like 1000000... does your application
>> go
>> OOM? Are you running in a JSE environment? What is PersistenceManager?
>>
>> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com> wrote:
>>
>>     
>>> BTW I'm running with the cache off
>>>
>>> <property name="openjpa.DataCache" value="false"/>
>>>
>>> (that turns it off right?)
>>>
>>> --Kurt
>>>
>>>
>>>
>>> Kurt T Stam wrote:
>>>
>>>       
>>>> Hi guys,
>>>>
>>>> [DESCRIPTION] The code below inserts a 1000 records in the database.
>>>>
>>>> for (int i=1; i<1000; i++) {
>>>>           EntityManager em = PersistenceManager.getEntityManager();
>>>>           EntityTransaction tx = em.getTransaction();
>>>>           try {
>>>>               tx.begin();
>>>>                   // Generate auth token and store it!
>>>>               String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
>>>>               org.apache.juddi.model.AuthToken modelAuthToken = new
>>>> org.apache.juddi.model.AuthToken();
>>>>               if (authInfo != null) {
>>>>                   modelAuthToken.setAuthToken(authInfo);
>>>>                   modelAuthToken.setCreated(new Date());
>>>>                   modelAuthToken.setLastUsed(new Date());
>>>>                   modelAuthToken.setAuthorizedName(publisherId);
>>>>                   modelAuthToken.setNumberOfUses(0);
>>>>                   modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>>>                     em.persist(modelAuthToken);
>>>>               }
>>>>                 apiAuthToken = new org.uddi.api_v3.AuthToken();
>>>>                 MappingModelToApi.mapAuthToken(modelAuthToken,
>>>> apiAuthToken);
>>>>                 tx.commit();
>>>>                         } finally {
>>>>               if (tx.isActive()) {
>>>>                   tx.rollback();
>>>>               }
>>>>               em.clear();
>>>>               em.close();
>>>>           }
>>>>       }
>>>>
>>>>
>>>> [ISSUE]
>>>> After it leaving this code I end up with a 1000
>>>> org.apache.juddi.model.AuthToken objects in memory. I've been using the
>>>> profiler, and these objects cannot be garbage collected.
>>>>
>>>> This seems to be pretty the most common use case of using an OR-mapping
>>>> tool, so I find it hard to believe openjpa has a memory leak here. Does
>>>> anyone see what I'm doing wrong? Or can someone point me to an example
>>>>         
>> that
>>     
>>>> does not exhibit this behavior? BTW same code using hibernate does not
>>>> accumulate these objects.
>>>>
>>>> We're using openjpa 1.2.1.
>>>>
>>>>
>>>> Thx,
>>>>
>>>>
>>>> Kurt
>>>>
>>>> Apache jUDDI.
>>>>
>>>>         
>>>       
>> --
>> Thanks,
>> Rick
>>
>>     
>
>   


Re: memory leak? - simple question

Posted by Kevin Sutter <kw...@gmail.com>.
Kurt,
I agree that this is very common usage of the JPA programming model.  And,
we are not aware of any memory leaks.  About the only thing that jumps out
at me is the following two lines:

                apiAuthToken = new org.uddi.api_v3.AuthToken();
                MappingModelToApi.mapAuthToken(modelAuthToken,
apiAuthToken);

What do these do?  Can you comment these out and see if the memory leak
still exists?  Since you are passing the modelAuthToken into this method, I
don't know what it's doing with the reference and could it be holding onto
something to prevent the GC from cleaning up?

The rest of your example seems very straight forward with creating and
persisting objects.

Kevin

On Wed, Jan 13, 2010 at 2:09 PM, Rick Curtis <cu...@gmail.com> wrote:

> If you change the 1000 to something like 1000000... does your application
> go
> OOM? Are you running in a JSE environment? What is PersistenceManager?
>
> On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com> wrote:
>
> > BTW I'm running with the cache off
> >
> > <property name="openjpa.DataCache" value="false"/>
> >
> > (that turns it off right?)
> >
> > --Kurt
> >
> >
> >
> > Kurt T Stam wrote:
> >
> >> Hi guys,
> >>
> >> [DESCRIPTION] The code below inserts a 1000 records in the database.
> >>
> >> for (int i=1; i<1000; i++) {
> >>           EntityManager em = PersistenceManager.getEntityManager();
> >>           EntityTransaction tx = em.getTransaction();
> >>           try {
> >>               tx.begin();
> >>                   // Generate auth token and store it!
> >>               String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
> >>               org.apache.juddi.model.AuthToken modelAuthToken = new
> >> org.apache.juddi.model.AuthToken();
> >>               if (authInfo != null) {
> >>                   modelAuthToken.setAuthToken(authInfo);
> >>                   modelAuthToken.setCreated(new Date());
> >>                   modelAuthToken.setLastUsed(new Date());
> >>                   modelAuthToken.setAuthorizedName(publisherId);
> >>                   modelAuthToken.setNumberOfUses(0);
> >>                   modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
> >>                     em.persist(modelAuthToken);
> >>               }
> >>                 apiAuthToken = new org.uddi.api_v3.AuthToken();
> >>                 MappingModelToApi.mapAuthToken(modelAuthToken,
> >> apiAuthToken);
> >>                 tx.commit();
> >>                         } finally {
> >>               if (tx.isActive()) {
> >>                   tx.rollback();
> >>               }
> >>               em.clear();
> >>               em.close();
> >>           }
> >>       }
> >>
> >>
> >> [ISSUE]
> >> After it leaving this code I end up with a 1000
> >> org.apache.juddi.model.AuthToken objects in memory. I've been using the
> >> profiler, and these objects cannot be garbage collected.
> >>
> >> This seems to be pretty the most common use case of using an OR-mapping
> >> tool, so I find it hard to believe openjpa has a memory leak here. Does
> >> anyone see what I'm doing wrong? Or can someone point me to an example
> that
> >> does not exhibit this behavior? BTW same code using hibernate does not
> >> accumulate these objects.
> >>
> >> We're using openjpa 1.2.1.
> >>
> >>
> >> Thx,
> >>
> >>
> >> Kurt
> >>
> >> Apache jUDDI.
> >>
> >
> >
>
>
> --
> Thanks,
> Rick
>

Re: memory leak? - simple question

Posted by Rick Curtis <cu...@gmail.com>.
If you change the 1000 to something like 1000000... does your application go
OOM? Are you running in a JSE environment? What is PersistenceManager?

On Wed, Jan 13, 2010 at 2:05 PM, Kurt T Stam <ku...@gmail.com> wrote:

> BTW I'm running with the cache off
>
> <property name="openjpa.DataCache" value="false"/>
>
> (that turns it off right?)
>
> --Kurt
>
>
>
> Kurt T Stam wrote:
>
>> Hi guys,
>>
>> [DESCRIPTION] The code below inserts a 1000 records in the database.
>>
>> for (int i=1; i<1000; i++) {
>>           EntityManager em = PersistenceManager.getEntityManager();
>>           EntityTransaction tx = em.getTransaction();
>>           try {
>>               tx.begin();
>>                   // Generate auth token and store it!
>>               String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
>>               org.apache.juddi.model.AuthToken modelAuthToken = new
>> org.apache.juddi.model.AuthToken();
>>               if (authInfo != null) {
>>                   modelAuthToken.setAuthToken(authInfo);
>>                   modelAuthToken.setCreated(new Date());
>>                   modelAuthToken.setLastUsed(new Date());
>>                   modelAuthToken.setAuthorizedName(publisherId);
>>                   modelAuthToken.setNumberOfUses(0);
>>                   modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>>                     em.persist(modelAuthToken);
>>               }
>>                 apiAuthToken = new org.uddi.api_v3.AuthToken();
>>                 MappingModelToApi.mapAuthToken(modelAuthToken,
>> apiAuthToken);
>>                 tx.commit();
>>                         } finally {
>>               if (tx.isActive()) {
>>                   tx.rollback();
>>               }
>>               em.clear();
>>               em.close();
>>           }
>>       }
>>
>>
>> [ISSUE]
>> After it leaving this code I end up with a 1000
>> org.apache.juddi.model.AuthToken objects in memory. I've been using the
>> profiler, and these objects cannot be garbage collected.
>>
>> This seems to be pretty the most common use case of using an OR-mapping
>> tool, so I find it hard to believe openjpa has a memory leak here. Does
>> anyone see what I'm doing wrong? Or can someone point me to an example that
>> does not exhibit this behavior? BTW same code using hibernate does not
>> accumulate these objects.
>>
>> We're using openjpa 1.2.1.
>>
>>
>> Thx,
>>
>>
>> Kurt
>>
>> Apache jUDDI.
>>
>
>


-- 
Thanks,
Rick

Re: memory leak? - simple question

Posted by Kurt T Stam <ku...@gmail.com>.
BTW I'm running with the cache off

<property name="openjpa.DataCache" value="false"/>

(that turns it off right?)

--Kurt


Kurt T Stam wrote:
> Hi guys,
>
> [DESCRIPTION] The code below inserts a 1000 records in the database.
>
> for (int i=1; i<1000; i++) {
>            EntityManager em = PersistenceManager.getEntityManager();
>            EntityTransaction tx = em.getTransaction();
>            try {
>                tx.begin();
>                    // Generate auth token and store it!
>                String authInfo = AUTH_TOKEN_PREFIX + UUID.randomUUID();
>                org.apache.juddi.model.AuthToken modelAuthToken = new 
> org.apache.juddi.model.AuthToken();
>                if (authInfo != null) {
>                    modelAuthToken.setAuthToken(authInfo);
>                    modelAuthToken.setCreated(new Date());
>                    modelAuthToken.setLastUsed(new Date());
>                    modelAuthToken.setAuthorizedName(publisherId);
>                    modelAuthToken.setNumberOfUses(0);
>                    modelAuthToken.setTokenState(AUTHTOKEN_ACTIVE);
>                      em.persist(modelAuthToken);
>                }
>                  apiAuthToken = new org.uddi.api_v3.AuthToken();
>                  MappingModelToApi.mapAuthToken(modelAuthToken, 
> apiAuthToken);
>                  tx.commit();
>                          } finally {
>                if (tx.isActive()) {
>                    tx.rollback();
>                }
>                em.clear();
>                em.close();
>            }
>        }
>
>
> [ISSUE]
> After it leaving this code I end up with a 1000 
> org.apache.juddi.model.AuthToken objects in memory. I've been using 
> the profiler, and these objects cannot be garbage collected.
>
> This seems to be pretty the most common use case of using an 
> OR-mapping tool, so I find it hard to believe openjpa has a memory 
> leak here. Does anyone see what I'm doing wrong? Or can someone point 
> me to an example that does not exhibit this behavior? BTW same code 
> using hibernate does not accumulate these objects.
>
> We're using openjpa 1.2.1.
>
>
> Thx,
>
>
> Kurt
>
> Apache jUDDI.