You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Leo Donahue <do...@gmail.com> on 2015/09/11 16:36:35 UTC
heap thrashing
Good day,
I see this topic come up from time to time on the list. Can someone point
me to what heap thrashing looks like?
Googled java heap thrashing and looked at the images, but there isn't much
to look at.
I also tried googling for ventricular tachycardia to see if I could find a
similar graph - it's close to what I'm seeing in VisualVM, but not quite.
Is heap thrashing a very "closely spaced" saw tooth pattern?
Leo
Re: heap thrashing
Posted by "André Warnier (tomcat)" <aw...@ice-sa.com>.
On 11.09.2015 16:43, Leo Donahue wrote:
> On Fri, Sep 11, 2015 at 9:36 AM, Leo Donahue <do...@gmail.com> wrote:
>
>> Good day,
>>
>> I see this topic come up from time to time on the list. Can someone point
>> me to what heap thrashing looks like?
>>
>> Googled java heap thrashing and looked at the images, but there isn't much
>> to look at.
>>
>> I also tried googling for ventricular tachycardia to see if I could find a
>> similar graph - it's close to what I'm seeing in VisualVM, but not quite.
>>
>> Is heap thrashing a very "closely spaced" saw tooth pattern?
>>
>> Leo
>>
>
> This is about as close as I can find that is similar to what I'm seeing.
> On the left side of the graph, imagine the spacing so close together that
> it looks like a solid blue read out in the monitor. When I stop the
> webapp, the jvm adjusts itself back to normal. It's only during servicing
> requests that I see the very closely spaced pattern.
>
> http://i.stack.imgur.com/B9oPL.png
>
What about a GC log with timestamps ?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: heap thrashing
Posted by Leo Donahue <do...@gmail.com>.
On Sat, Sep 12, 2015 at 7:49 AM, André Warnier (tomcat) <aw...@ice-sa.com>
wrote:
> On 11.09.2015 18:24, Caldarale, Charles R wrote:
>
>> From: Leo Donahue [mailto:donahulf2@gmail.com]
>>> Subject: Re: heap thrashing
>>>
>>
>> I see this topic come up from time to time on the list. Can someone point
>>> me to what heap thrashing looks like?
>>>
>>
>> Is heap thrashing a very "closely spaced" saw tooth pattern?
>>>
>>
>> Should have mentioned that "heap thrashing" does not have a strict
>> definition. Often, it's used to describe the heap itself expanding and
>> contracting in a cyclic manner. This is most easily avoided by setting the
>> min and max heap size limits to the same value.
>>
>> What you have appears to be just very rapid object creation and garbage
>> collection. Using a larger heap (if you have the RAM for it) could help to
>> reduce the frequency of collections. Fixing the webapp to not consume so
>> much space would be better, of course.
>>
>> - Chuck
>>
>
> In terms of looking at the webapp code which may generate that kind of
> behaviour, while searching for an example on the www, I came across this
> blog post :
>
> http://steve-yegge.blogspot.de/2006/03/execution-in-kingdom-of-nouns.html
>
> Independently of what one may think about the author's opinions and
> treatment of the matter, I find the prose beautiful and witty.
> The pseudo-java code example is of course tongue-in-cheek and contrived,
> but I have seen similar code in the real world, and it would probably
> produce the kind of phenomenon which Leo is seeing.
>
André,
Nice article, hard to believe that was nearly 10 years ago. I also see the
overuse of nouns as verbs and factory patterns that are only producing a
single object and will never add additional ones. I see people using
"private static" in place of a singleton pattern. I see people still
catching Throwable and deferring exceptions as RuntimeExceptions instead of
declaring them. I digress.
Leo
Re: heap thrashing
Posted by "André Warnier (tomcat)" <aw...@ice-sa.com>.
On 11.09.2015 18:24, Caldarale, Charles R wrote:
>> From: Leo Donahue [mailto:donahulf2@gmail.com]
>> Subject: Re: heap thrashing
>
>> I see this topic come up from time to time on the list. Can someone point
>> me to what heap thrashing looks like?
>
>> Is heap thrashing a very "closely spaced" saw tooth pattern?
>
> Should have mentioned that "heap thrashing" does not have a strict definition. Often, it's used to describe the heap itself expanding and contracting in a cyclic manner. This is most easily avoided by setting the min and max heap size limits to the same value.
>
> What you have appears to be just very rapid object creation and garbage collection. Using a larger heap (if you have the RAM for it) could help to reduce the frequency of collections. Fixing the webapp to not consume so much space would be better, of course.
>
> - Chuck
In terms of looking at the webapp code which may generate that kind of behaviour, while
searching for an example on the www, I came across this blog post :
http://steve-yegge.blogspot.de/2006/03/execution-in-kingdom-of-nouns.html
Independently of what one may think about the author's opinions and treatment of the
matter, I find the prose beautiful and witty.
The pseudo-java code example is of course tongue-in-cheek and contrived, but I have seen
similar code in the real world, and it would probably produce the kind of phenomenon which
Leo is seeing.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: heap thrashing
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Leo Donahue [mailto:donahulf2@gmail.com]
> Subject: Re: heap thrashing
> Why does the jvm decide to collect garbage that fast as opposed to letting
> it build and taking care of it later?
Because your webapp has filled up one of the divisions of the heap. Read up on how the Java heap is organized.
> What do you consider to be very rapid object creation?
Enough to cause garbage collection frequent enough to bother you.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: heap thrashing
Posted by Leo Donahue <do...@gmail.com>.
On Fri, Sep 11, 2015 at 11:24 AM, Caldarale, Charles R <
Chuck.Caldarale@unisys.com> wrote:
> > From: Leo Donahue [mailto:donahulf2@gmail.com]
> > Subject: Re: heap thrashing
>
> What you have appears to be just very rapid object creation and garbage
> collection.
>
> - Chuck
>
>
Why does the jvm decide to collect garbage that fast as opposed to letting
it build and taking care of it later?
What do you consider to be very rapid object creation?
Leo
RE: heap thrashing
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Leo Donahue [mailto:donahulf2@gmail.com]
> Subject: Re: heap thrashing
> I see this topic come up from time to time on the list. Can someone point
> me to what heap thrashing looks like?
> Is heap thrashing a very "closely spaced" saw tooth pattern?
Should have mentioned that "heap thrashing" does not have a strict definition. Often, it's used to describe the heap itself expanding and contracting in a cyclic manner. This is most easily avoided by setting the min and max heap size limits to the same value.
What you have appears to be just very rapid object creation and garbage collection. Using a larger heap (if you have the RAM for it) could help to reduce the frequency of collections. Fixing the webapp to not consume so much space would be better, of course.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: heap thrashing
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Leo Donahue [mailto:donahulf2@gmail.com]
> Subject: Re: heap thrashing
> > I see this topic come up from time to time on the list. Can someone point
> > me to what heap thrashing looks like?
> > Is heap thrashing a very "closely spaced" saw tooth pattern?
Very often.
> This is about as close as I can find that is similar to what I'm seeing.
> http://i.stack.imgur.com/B9oPL.png
That is characteristic of something in the webapp consuming lots of short-lived objects (no memory leak). Without an actual heap analysis, you can't tell if it's a small number of large objects or a large number of small ones. Using a heap profiler should help find what's eating the space.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: heap thrashing
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Leo Donahue [mailto:donahulf2@gmail.com]
> Subject: Re: heap thrashing
> "re-using objects" - do you mean declaring static variables or using thread
> locals? Or something else?
In an environment that uses thread pooling (e.g., Tomcat), any use of thread locals has to be done very, very carefully, or they will cause memory leaks and possible crosstalk between requests. Statics have the same issues, even without thread pooling - it's very easy to accidentally place a request-specific item into a static and erroneously share it among requests.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: heap thrashing
Posted by Mark Thomas <ma...@apache.org>.
On 13/09/2015 05:02, Leo Donahue wrote:
> On Fri, Sep 11, 2015 at 11:27 AM, Mark Thomas <ma...@apache.org> wrote:
>
>> On 11/09/2015 15:43, Leo Donahue wrote:
>>> On Fri, Sep 11, 2015 at 9:36 AM, Leo Donahue <do...@gmail.com>
>> wrote:
>>
>> (typically by pooling and re-using objects rather than
>> creating new ones for every request)
>
>
> "re-using objects" - do you mean declaring static variables or using thread
> locals? Or something else?
I mean object pooling. Search the archives for SynchronizedQueue and
SynchronizedStack which were specifically written to reduce GC.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: heap thrashing
Posted by Leo Donahue <do...@gmail.com>.
On Fri, Sep 11, 2015 at 11:27 AM, Mark Thomas <ma...@apache.org> wrote:
> On 11/09/2015 15:43, Leo Donahue wrote:
> > On Fri, Sep 11, 2015 at 9:36 AM, Leo Donahue <do...@gmail.com>
> wrote:
>
> (typically by pooling and re-using objects rather than
> creating new ones for every request)
"re-using objects" - do you mean declaring static variables or using thread
locals? Or something else?
Leo
Re: heap thrashing
Posted by Mark Thomas <ma...@apache.org>.
On 11/09/2015 15:43, Leo Donahue wrote:
> On Fri, Sep 11, 2015 at 9:36 AM, Leo Donahue <do...@gmail.com> wrote:
>
>> Good day,
>>
>> I see this topic come up from time to time on the list. Can someone point
>> me to what heap thrashing looks like?
>>
>> Googled java heap thrashing and looked at the images, but there isn't much
>> to look at.
>>
>> I also tried googling for ventricular tachycardia to see if I could find a
>> similar graph - it's close to what I'm seeing in VisualVM, but not quite.
>>
>> Is heap thrashing a very "closely spaced" saw tooth pattern?
If a lot of garbage is being created very quickly that is the sort of
pattern I'd expect to see.
> This is about as close as I can find that is similar to what I'm seeing.
> On the left side of the graph, imagine the spacing so close together that
> it looks like a solid blue read out in the monitor. When I stop the
> webapp, the jvm adjusts itself back to normal. It's only during servicing
> requests that I see the very closely spaced pattern.
>
> http://i.stack.imgur.com/B9oPL.png
I've done some work on this with Tomcat (search the dev archives for
allocation) pushing requests through it as fast as I could. I found the
JVM could cope fairly well with it. Even after reducing the amount of GC
generated (typically by pooling and re-using objects rather than
creating new ones for every request) then the performance impact was
minimal.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: heap thrashing
Posted by Leo Donahue <do...@gmail.com>.
On Fri, Sep 11, 2015 at 9:36 AM, Leo Donahue <do...@gmail.com> wrote:
> Good day,
>
> I see this topic come up from time to time on the list. Can someone point
> me to what heap thrashing looks like?
>
> Googled java heap thrashing and looked at the images, but there isn't much
> to look at.
>
> I also tried googling for ventricular tachycardia to see if I could find a
> similar graph - it's close to what I'm seeing in VisualVM, but not quite.
>
> Is heap thrashing a very "closely spaced" saw tooth pattern?
>
> Leo
>
This is about as close as I can find that is similar to what I'm seeing.
On the left side of the graph, imagine the spacing so close together that
it looks like a solid blue read out in the monitor. When I stop the
webapp, the jvm adjusts itself back to normal. It's only during servicing
requests that I see the very closely spaced pattern.
http://i.stack.imgur.com/B9oPL.png