You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by José Renato <jo...@suntech.com.br> on 2006/08/28 19:31:44 UTC
Handling huge amount data
Hi All,
I have an web application deployed with Tomcat 5.0.28. In some cases my
queries can bring back huge amounts of data. Looking Windows Task
Manager I perceived the natural memory increase. I noticed that when I
call my logout method (It calls session.invalidate()) or my user session
expieres the memory is still in use, in other words, the memory is not
cleaned. This process will cause OutOfMemoryException soon. I would like
to undertand how can I discover why this happens and how can I fix it.
Best regards,
José Renato.
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Handling huge amount data
Posted by Robert Harper <ro...@iat-cti.com>.
I've seen the memory rise and fall so if the GC releases memory, it should
be returned to the OS. It would be my guess that you may not be handling
removing all references to an object the way you think you are. You can test
this out by adding a log in the finalize() method of your object and see if
it ever gets called. You could have a reference somewhere in a static class
or some other class that you have not released as well. As long as there is
more than one reference to an object, the GC will not free it. I had found
this to be the case in something I was working on. It is hard to find at
times but you should be able to find it.
The other thing to remember is that the GC is a very low priority thread and
if the system is very busy, it may not run as often as you would like.
Robert S. Harper
Information Access Technology, Inc.
-----Original Message-----
From: Christopher Schultz [mailto:chris@christopherschultz.net]
Sent: Tuesday, August 29, 2006 9:44 AM
To: Tomcat Users List
Subject: Re: Handling huge amount data
José,
> Looking Windows Task
> Manager I perceived the natural memory increase.
>
> I noticed that when I
> call my logout method (It calls session.invalidate()) or my user session
> expieres the memory is still in use, in other words, the memory is not
> cleaned.
Windows only reports the size of the application in memory. Once Java
requests memory from the OS, I don't believe that it ever returns it.
So, the memory size reported by Windows Task Manager will only increase,
never decrease.
You need to look at the memory behavior /inside/ the JVM, which is much
more complicated than what Windows Task Manager displays.
There are some tools that will help you see into a JVM, including simply
using the "verbose GC" settings when starting your JVM, as well as some
other 3rd-party tools that can give you a lot of information about the
JVM upon which Tomcat is running (LabmdaProbe comes to mind:
http://www.lambdaprobe.org/d/index.htm).
Hope that helps,
-chris
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Handling huge amount data
Posted by Christopher Schultz <ch...@christopherschultz.net>.
José,
> Looking Windows Task
> Manager I perceived the natural memory increase.
>
> I noticed that when I
> call my logout method (It calls session.invalidate()) or my user session
> expieres the memory is still in use, in other words, the memory is not
> cleaned.
Windows only reports the size of the application in memory. Once Java
requests memory from the OS, I don't believe that it ever returns it.
So, the memory size reported by Windows Task Manager will only increase,
never decrease.
You need to look at the memory behavior /inside/ the JVM, which is much
more complicated than what Windows Task Manager displays.
There are some tools that will help you see into a JVM, including simply
using the "verbose GC" settings when starting your JVM, as well as some
other 3rd-party tools that can give you a lot of information about the
JVM upon which Tomcat is running (LabmdaProbe comes to mind:
http://www.lambdaprobe.org/d/index.htm).
Hope that helps,
-chris
Re: Handling huge amount data
Posted by anjan bacchu <an...@gmail.com>.
Hi Jose,
Are you using Java 5.x(1.5.x) ? If so, you can watch your JVM's runtime
behavior using the JConsole( in the JAVA_HOME/bin directory) ? It has helped
me earlier on to debug OOM situations.
You need to restart your tomcat so that you can add the required parameters
in your startup.bat/startup.sh script.
Here's what I did to change my tomcat(5.5.17 but also works in any 5.x) to
allow JMX monitoring (from JConsole). I just added the line after the label
doneSetArgs
====================
:doneSetArgs
set JAVA_OPTS=%JAVA_OPTS% -Dcom.sun.management.jmxremote
rem Execute Java with the applicable properties
====================
JConsole will let you monitor your memory usage and also track heap and
non-heap memory. You can also force a gc on your tomcat JVM from JConsole.
It lets you monitor various pools like Eden Space, Survivor, Tenured, Perm,
Code Cache, Perm Gen rw, Perm Gen RO, etc. That should give you some insight
into what is happening within Tomcat.
for more info on JConsole, see
http://www.onjava.com/pub/a/onjava/2004/09/29/tigerjmx.html
BR,
~A
On 8/28/06, José Renato <jo...@suntech.com.br> wrote:
>
> Hi,
>
> I am loading the data to be displayed with display-tag component. The
> data lists has been saved as request attributes. I will look for
> profilers to check this situation better.
>
> Thank a lot, Bye.
>
> David Smith escreveu:
> > If you are sure the sessions are where your memory is going, is there
> > any way to reduce the session size? I would think a query for a large
> > amount of data should be a request scoped item or maybe even put on
> > the presentation layer with a taglib.
> >
> > You could also be hanging onto object references in other parts of
> > your code. Invalidating the session won't clean that up.
> >
> > Also there are profilers around that will give you an idea of memory
> > useage. Google is your friend.
> >
> > --David
> >
> > José Renato wrote:
> >
> >> Hi All,
> >>
> >> I have an web application deployed with Tomcat 5.0.28. In some cases
> >> my queries can bring back huge amounts of data. Looking Windows Task
> >> Manager I perceived the natural memory increase. I noticed that when
> >> I call my logout method (It calls session.invalidate()) or my user
> >> session expieres the memory is still in use, in other words, the
> >> memory is not cleaned. This process will cause OutOfMemoryException
> >> soon. I would like to undertand how can I discover why this happens
> >> and how can I fix it.
> >>
> >> Best regards,
> >> José Renato.
> >>
> >> ---------------------------------------------------------------------
> >> To start a new topic, e-mail: users@tomcat.apache.org
> >> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> >> For additional commands, e-mail: users-help@tomcat.apache.org
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To start a new topic, e-mail: users@tomcat.apache.org
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
--
BR,
Anjan Bacchu
Summit Information Systems
Re: Handling huge amount data
Posted by José Renato <jo...@suntech.com.br>.
Hi,
I am loading the data to be displayed with display-tag component. The
data lists has been saved as request attributes. I will look for
profilers to check this situation better.
Thank a lot, Bye.
David Smith escreveu:
> If you are sure the sessions are where your memory is going, is there
> any way to reduce the session size? I would think a query for a large
> amount of data should be a request scoped item or maybe even put on
> the presentation layer with a taglib.
>
> You could also be hanging onto object references in other parts of
> your code. Invalidating the session won't clean that up.
>
> Also there are profilers around that will give you an idea of memory
> useage. Google is your friend.
>
> --David
>
> José Renato wrote:
>
>> Hi All,
>>
>> I have an web application deployed with Tomcat 5.0.28. In some cases
>> my queries can bring back huge amounts of data. Looking Windows Task
>> Manager I perceived the natural memory increase. I noticed that when
>> I call my logout method (It calls session.invalidate()) or my user
>> session expieres the memory is still in use, in other words, the
>> memory is not cleaned. This process will cause OutOfMemoryException
>> soon. I would like to undertand how can I discover why this happens
>> and how can I fix it.
>>
>> Best regards,
>> José Renato.
>>
>> ---------------------------------------------------------------------
>> To start a new topic, e-mail: users@tomcat.apache.org
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Handling huge amount data
Posted by David Smith <dn...@cornell.edu>.
If you are sure the sessions are where your memory is going, is there
any way to reduce the session size? I would think a query for a large
amount of data should be a request scoped item or maybe even put on the
presentation layer with a taglib.
You could also be hanging onto object references in other parts of your
code. Invalidating the session won't clean that up.
Also there are profilers around that will give you an idea of memory
useage. Google is your friend.
--David
José Renato wrote:
> Hi All,
>
> I have an web application deployed with Tomcat 5.0.28. In some cases
> my queries can bring back huge amounts of data. Looking Windows Task
> Manager I perceived the natural memory increase. I noticed that when I
> call my logout method (It calls session.invalidate()) or my user
> session expieres the memory is still in use, in other words, the
> memory is not cleaned. This process will cause OutOfMemoryException
> soon. I would like to undertand how can I discover why this happens
> and how can I fix it.
>
> Best regards,
> José Renato.
>
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org