You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by "Robinson, Eric" <er...@psmnv.com> on 2009/03/04 11:25:17 UTC
Effect of Heap Size on Performance?
One of our application servers has a small heap size, 64MB. The status
page seems to indicate that the heap usage is somewhere around 48MB,
with 16MB free. Most people these days suggest setting the heap size
much higher, at least 512MB. Is there a way to tell what the optimal
heap size is for our application, and whether the current small size is
effecting performance (aside from trial-and-error or repeated
benchmarking with tools like jmeter)?
--Eric
Disclaimer - March 4, 2009
This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of . Warning: Although has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Effect of Heap Size on Performance?
Posted by Leon Rosenberg <ro...@googlemail.com>.
On Wed, Mar 11, 2009 at 1:56 AM, Caldarale, Charles R
<Ch...@unisys.com> wrote:
>
> It's impossible to prevent at least some objects from eventually migrating to the OldGen, since anything that persists long enough will end up there after some number of minor GCs.
But by choosing a large enough heap size (and evtl. according
strategy) it is possible to optimize the usage of the eden aka young
space and to reduce the amount of minor collections, hence reducing
the chances for unnecessary migration into old space. However, in a
webapp, anything you create during the processing of a request should
remain in the eden and 'die young' (unless you put it in a session or
a cache explicitely).
JMX gives you the access to MemoryPool usage statistics and you can
actually monitor all of heap-spaces separately. Moskito btw, gathers
this info and presents it via its web ui:
http://moskito.anotheria.net/moskitodemo/mui/mskShowProducersByCategory?pCategory=memory
We use this info to create graphs of free memory in relevant spaces
(the tenured/old space, depending on the collector is the most easiest
to monitor, and most relevant to performance) and optimize the
settings unless the memory usage graphs are looking well (not hectic,
well formed sawtooth). With the graph you would also detect a memory
leak much faster, even if the heap size would be too large.
regards
Leon
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Effect of Heap Size on Performance?
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Martin Gainty [mailto:mgainty@hotmail.com]
> Subject: RE: Effect of Heap Size on Performance?
>
> any rules of thumb to follow for restricting heap to eden
> (and not jumping the fence to permgen)?
I think you're confusing PermGen with the old (aka tenured) generation. Only instances of java.lang.Class go into PermGen (plus some internal JVM structures); nothing ever "jumps the fence" into PermGen.
It's impossible to prevent at least some objects from eventually migrating to the OldGen, since anything that persists long enough will end up there after some number of minor GCs.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Effect of Heap Size on Performance?
Posted by Martin Gainty <mg...@hotmail.com>.
any rules of thumb to follow for restricting heap to eden (and not jumping the fence to permgen)?
thx
Martin
______________________________________________
Disclaimer and confidentiality note
Everything in this e-mail and any attachments relates to the official business of Sender. This transmission is of a confidential nature and Sender does not endorse distribution to any party other than intended recipient. Sender does not necessarily endorse content contained within this transmission.
> Date: Tue, 10 Mar 2009 15:54:11 +0100
> Subject: Re: Effect of Heap Size on Performance?
> From: rosenberg.leon@googlemail.com
> To: users@tomcat.apache.org
>
> On Tue, Mar 10, 2009 at 3:41 PM, Christopher Schultz
> <ch...@christopherschultz.net> wrote:
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA1
> >
> > Taylan,
> >
> > On 3/5/2009 5:11 AM, Taylan Develioglu wrote:
> >> I always hold this as a ground rule:
> >>
> >> Increase heapsize as much as possible as long as:
> >
> > My rule has always been to run with the smallest heap you can get away
> > with. We ran our main production app in 64MB of heap (the default for
> > our platform) for 4 years before we got our first OOME. Now we run it
> > with a 192MB heap.
> >
> > A smaller heap means that you'll catch even small memory leaks faster.
> > At least, that's my position.
> >
> > Surprisingly, Chuck hasn't responded (he usually has something to say
> > about GC/heap myths), but I suspect he'd say something like "heap size
> > itself has little effect on the GC's performance... it's really the
> > number of objects that affect the performance.
>
>
> allow me to jump in in place :-)
>
> actually, with the default garbage collector, the amount of the
> objects in the young generation has no effect on GCs performance, the
> size of the young generation does.
> Generally, more free memory, less GC, better performance.
>
> The number of short living objects (and usually 99.9% of your objects
> are short-lived, or should be short lived), is irrelevant, since only
> explicitelly accessible objects survive the young space.
>
> regards
> Leon
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
_________________________________________________________________
Windows Live™ Groups: Create an online spot for your favorite groups to meet.
http://windowslive.com/online/groups?ocid=TXT_TAGLM_WL_groups_032009
Re: Effect of Heap Size on Performance?
Posted by Leon Rosenberg <ro...@googlemail.com>.
On Tue, Mar 10, 2009 at 3:41 PM, Christopher Schultz
<ch...@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Taylan,
>
> On 3/5/2009 5:11 AM, Taylan Develioglu wrote:
>> I always hold this as a ground rule:
>>
>> Increase heapsize as much as possible as long as:
>
> My rule has always been to run with the smallest heap you can get away
> with. We ran our main production app in 64MB of heap (the default for
> our platform) for 4 years before we got our first OOME. Now we run it
> with a 192MB heap.
>
> A smaller heap means that you'll catch even small memory leaks faster.
> At least, that's my position.
>
> Surprisingly, Chuck hasn't responded (he usually has something to say
> about GC/heap myths), but I suspect he'd say something like "heap size
> itself has little effect on the GC's performance... it's really the
> number of objects that affect the performance.
allow me to jump in in place :-)
actually, with the default garbage collector, the amount of the
objects in the young generation has no effect on GCs performance, the
size of the young generation does.
Generally, more free memory, less GC, better performance.
The number of short living objects (and usually 99.9% of your objects
are short-lived, or should be short lived), is irrelevant, since only
explicitelly accessible objects survive the young space.
regards
Leon
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Effect of Heap Size on Performance?
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Taylan Develioglu [mailto:tdevelioglu@ebuddy.com]
> Subject: Re: Effect of Heap Size on Performance?
>
> I think it also gives short-lived objects (for example short
> sessions) a longer time to 'die out', so they won't be moved
> to tenure because survivor space is increased and gc frequency
> is decreased (can anyone confirm this?).
That is the expected effect of having a larger young generation and consequent larger eden and survivor spaces. However, there is a cost of deferring the migration of long-lived objects to tenured: they must be copied from one survivor space to the other on each minor GC, whereas once in tenured, they do not move until a full GC occurs (and perhaps not even then).
It's often cheaper to throw hardware at a performance problem than to do the painstaking analysis of GC behavior and gain the few percentage points of improvement that will bring.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Effect of Heap Size on Performance?
Posted by Taylan Develioglu <td...@ebuddy.com>.
Chris,
We have 100+ application servers in a loadbalancing (application based,
not tomcat) setup. If servers are removed from the load balancing pool
the others need to be able to pick up the load. So the number of
concurrent users is highly dynamic. You can imagine the problem if we
keep the heapsize to a minimum on every server. I'm talking about a
fixed size here ofcourse.
I also don't think the relationship between number of objects and young
gc duration is a linear one.
Increasing the young generation leads to longer gc's. Increasing young
to 682M on a 4G heap from its default size increased gc time approx. 3-4
x (47ms average to 154ms average on one server), but it also decreased
the number of gc's performed by 15-20x.
So eventually, a larger heapsize saved cycles, and subsequently
increased throughput. At least for us.
I think it also gives short-lived objects (for example short sessions) a
longer time to 'die out', so they won't be moved to tenure because
survivor space is increased and gc frequency is decreased (can anyone
confirm this?).
Christopher Schultz wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Taylan,
>
> On 3/5/2009 5:11 AM, Taylan Develioglu wrote:
>
>> I always hold this as a ground rule:
>>
>> Increase heapsize as much as possible as long as:
>>
>
> My rule has always been to run with the smallest heap you can get away
> with. We ran our main production app in 64MB of heap (the default for
> our platform) for 4 years before we got our first OOME. Now we run it
> with a 192MB heap.
>
> A smaller heap means that you'll catch even small memory leaks faster.
> At least, that's my position.
>
> Surprisingly, Chuck hasn't responded (he usually has something to say
> about GC/heap myths), but I suspect he'd say something like "heap size
> itself has little effect on the GC's performance... it's really the
> number of objects that affect the performance. Granted, a larger heap
> invites more objects into it, but generational garbage collection is
> decent enough that the generations rarely grow to such a size that the
> app stalls while the GC runs."
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.9 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iEYEARECAAYFAkm2fBAACgkQ9CaO5/Lv0PCPngCfRfClYEVoDAI57VBbqoBUaAC8
> RDAAn0fztUgMY0d0K0FAdV0uxYzSjDxN
> =EbMZ
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Effect of Heap Size on Performance?
Posted by Oliver Schoett <ol...@capgemini-sdm.com>.
Washburn, Brian J [IT] wrote:
> Also if the application is a web app with any kind of response time requirements I would strongly suggest that you use
>
[set of four options]
My suggestion is to first get a feel for the current gc behaviour with
jstat on the server:
jstat -gc -t -h25 $pid 5000
Then look at the document describing garbage collection on Sun 1.6 JVMs:
http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html
The solution to better GC behaviour may be as simple as choosing the
right collector, as explained in the section "Selecting a Collector".
Regards,
Oliver Schoett
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Effect of Heap Size on Performance?
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Washburn, Brian J [IT] [mailto:Brian.J.Washburn@sprint.com]
> Subject: RE: Effect of Heap Size on Performance?
>
> Also remember to statically size heap regions manually if you
> can as that will always improve GC performance
Don't bet on that; doing so may force excessive minor GCs if you guessed wrong. It is a good idea to set -Xms the same as -Xmx, but unless you have thoroughly analyzed your specific heap usage, don't set the individual generation sizes. Even a minor application change could upset any balance you've achieved.
> -XX:+UseConcMarkSweepGC
Be aware that although the above will virtually eliminate application pause time for a major GC, it will also *reduce* throughput slightly, since it's not as efficient a collection mechanism. Note that if you have any code that calls System.gc() directly (generally a bad idea), you will also want this:
-XX:+ExplicitGCInvokesConcurrent
Also note that using the concurrent collector disables adaptive size policy, making it impossible for the JVM to appropriately respond to changing conditions.
> -XX:+UseParNewGC
Not necessary with the concurrent collector in 1.6 JVMs, since the CMS collector now defaults to parallel on for the young generation.
> -XX:+UseCMSInitiatingOccupancyOnly
> -XX:CMSInitiatingOccupancyFraction=85
These have the risk of not collecting frequently enough, with the possible end result of having to do a full-pause collection. Use these with caution, only after thorough testing.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Effect of Heap Size on Performance?
Posted by "Washburn, Brian J [IT]" <Br...@sprint.com>.
Also remember to statically size heap regions manually if you can as that will always improve GC performance and as for heap sizing keep the heap sized to reduce GC time to less than 1% of the time especially on servers with more than 8 cores. Also if the application is a web app with any kind of response time requirements I would strongly suggest that you use
-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=85 this will keep the app server from having an long pauses that can be detected by users how tend to refresh if current experience is more than 4X normal which can cause extra artificial load.
-----Original Message-----
From: Caldarale, Charles R [mailto:Chuck.Caldarale@unisys.com]
Sent: Tuesday, March 10, 2009 9:57 AM
To: Tomcat Users List
Subject: RE: Effect of Heap Size on Performance?
> From: Christopher Schultz [mailto:chris@christopherschultz.net]
> Subject: Re: Effect of Heap Size on Performance?
>
> A smaller heap means that you'll catch even small memory leaks faster.
Good for debugging, but provokes more frequent GCs, so may not be optimal for production. I tend to use the largest heap that will avoid paging; this is often difficult to determine, unless the system you're running on is used only for the one Java application.
> Granted, a larger heap invites more objects into it
The above requires clarification: the max size of the heap has nothing to do with the number of persistent objects the application creates - that's purely a function of the app; the number of dead (unreachable) objects has negligible effect on GC time.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
This e-mail may contain Sprint Nextel Company proprietary information intended for the sole use of the recipient(s). Any use by others is prohibited. If you are not the intended recipient, please contact the sender and delete all copies of the message.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Effect of Heap Size on Performance?
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Christopher Schultz [mailto:chris@christopherschultz.net]
> Subject: Re: Effect of Heap Size on Performance?
>
> A smaller heap means that you'll catch even small memory leaks faster.
Good for debugging, but provokes more frequent GCs, so may not be optimal for production. I tend to use the largest heap that will avoid paging; this is often difficult to determine, unless the system you're running on is used only for the one Java application.
> Granted, a larger heap invites more objects into it
The above requires clarification: the max size of the heap has nothing to do with the number of persistent objects the application creates - that's purely a function of the app; the number of dead (unreachable) objects has negligible effect on GC time.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Effect of Heap Size on Performance?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Taylan,
On 3/5/2009 5:11 AM, Taylan Develioglu wrote:
> I always hold this as a ground rule:
>
> Increase heapsize as much as possible as long as:
My rule has always been to run with the smallest heap you can get away
with. We ran our main production app in 64MB of heap (the default for
our platform) for 4 years before we got our first OOME. Now we run it
with a 192MB heap.
A smaller heap means that you'll catch even small memory leaks faster.
At least, that's my position.
Surprisingly, Chuck hasn't responded (he usually has something to say
about GC/heap myths), but I suspect he'd say something like "heap size
itself has little effect on the GC's performance... it's really the
number of objects that affect the performance. Granted, a larger heap
invites more objects into it, but generational garbage collection is
decent enough that the generations rarely grow to such a size that the
app stalls while the GC runs."
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAkm2fBAACgkQ9CaO5/Lv0PCPngCfRfClYEVoDAI57VBbqoBUaAC8
RDAAn0fztUgMY0d0K0FAdV0uxYzSjDxN
=EbMZ
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Effect of Heap Size on Performance?
Posted by Taylan Develioglu <td...@ebuddy.com>.
I always hold this as a ground rule:
Increase heapsize as much as possible as long as:
latency goals are met (gc does note take too long)
enough memory is spared for system and vm. (enough to compensate for
spikes, watch your swap usage to determine this)
hope this helps a bit
Peter Crowther wrote:
>> From: Robinson, Eric [eric.robinson@psmnv.com]
>> Is there a way to tell what the optimal
>> heap size is for our application, and whether the current small size is
>> effecting performance (aside from trial-and-error or repeated
>> benchmarking with tools like jmeter)?
>>
>
> Smaller heap sizes may lead to more frequent garbage collection, which costs CPU cycles* and hence affects performance. If your app is spending much of its time in the garbage collector, increasing heap would help - but it's then a process of trial-and-error to work out what the "optimal" size is, as it's a trade-off between %CPU spent in garbage collection and memory.
>
> As a sketch, I'd turn on some of the GC monitoring in the virtual machine and see if you can get a feel for how much time it's spending in the GC. Most of my apps are well below 1%, though I have one (.Net app) that spends c. 98% of its time in the GC as it loads state from disk.
>
> You also need to decide what's "good enough". The difference in "real-world" performance between spending 0.1% and 1% in the GC is negligible; the difference between 10% and 90% is rather obvious!
>
> - Peter
>
> * Some of the cycles would be spent no matter when the garbage happened to be collected; others are overhead each time, though GC algorithms have improved a lot and the overhead's far lower than it used to be.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Effect of Heap Size on Performance?
Posted by Peter Crowther <Pe...@melandra.com>.
> From: Robinson, Eric [eric.robinson@psmnv.com]
> Is there a way to tell what the optimal
> heap size is for our application, and whether the current small size is
> effecting performance (aside from trial-and-error or repeated
> benchmarking with tools like jmeter)?
Smaller heap sizes may lead to more frequent garbage collection, which costs CPU cycles* and hence affects performance. If your app is spending much of its time in the garbage collector, increasing heap would help - but it's then a process of trial-and-error to work out what the "optimal" size is, as it's a trade-off between %CPU spent in garbage collection and memory.
As a sketch, I'd turn on some of the GC monitoring in the virtual machine and see if you can get a feel for how much time it's spending in the GC. Most of my apps are well below 1%, though I have one (.Net app) that spends c. 98% of its time in the GC as it loads state from disk.
You also need to decide what's "good enough". The difference in "real-world" performance between spending 0.1% and 1% in the GC is negligible; the difference between 10% and 90% is rather obvious!
- Peter
* Some of the cycles would be spent no matter when the garbage happened to be collected; others are overhead each time, though GC algorithms have improved a lot and the overhead's far lower than it used to be.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org