You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by "Robinson, Eric" <er...@psmnv.com> on 2011/02/08 17:27:25 UTC
Is IBM Right About Java?
Saw this statement on an IBM web site:
"Most JVMs grow towards the upper heap limit (-Xmx/-mx options) when
more memory is required, and do not return memory to the operating
system, even if the memory is no longer needed, until the JVM process
terminates."
Is this true? If thought returning unneeded memory to the OS was what GC
is all about.
--
Eric Robinson
Disclaimer - February 8, 2011
This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Is IBM Right About Java?
Posted by Pid <pi...@pidster.com>.
On 2/8/11 4:27 PM, Robinson, Eric wrote:
> Saw this statement on an IBM web site:
Please don't hijack threads.
Start a new email to the list, just like you would to a person.
p
Re: Is IBM Right About Java?
Posted by Mark Thomas <ma...@apache.org>.
On 08/02/2011 20:39, Jordan Michaels wrote:
> It's an honest "mistake" - if you even want to call it that. Whether
> it's right for someone like pid to even request that you adhere to this
> concept has been debated on many forums in many places.
On this list, thread-hi-jacking is frowned upon so pid is right to point
this out. Other lists may have other social norms.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Robinson, Eric" <er...@psmnv.com>.
> pid's using a mail client that uses header information to
> track threads.
> So, if you started this thread by hitting "reply" to an
> existing topic then changed the subject and content, the
> header thread tracking info will still be present. Thus, to
> him, it will appear as though you hijacked a thread.
>
> It's an honest "mistake" - if you even want to call it that.
> Whether it's right for someone like pid to even request that
> you adhere to this concept has been debated on many forums in
> many places.
>
I totally understand and completely forgot about that. The last time I
recall anyone mentioning it was years ago. Thanks for the heads-up. I've
been participating in online forums since the Internet was about 1000
nodes strong so I've had my share of debates and have grown somewhat
calloused to the finer points of netiquette. Thank goodness I didn't top
post!
--Eric
Disclaimer - February 8, 2011
This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Is IBM Right About Java?
Posted by Jordan Michaels <jo...@viviotech.net>.
pid's using a mail client that uses header information to track threads.
So, if you started this thread by hitting "reply" to an existing topic
then changed the subject and content, the header thread tracking info
will still be present. Thus, to him, it will appear as though you
hijacked a thread.
It's an honest "mistake" - if you even want to call it that. Whether
it's right for someone like pid to even request that you adhere to this
concept has been debated on many forums in many places.
I've been appreciating Charles' comments as well. They are generally
helpful and informative. =)
-Jordan
On 02/08/2011 12:12 PM, Robinson, Eric wrote:
>>> Saw this statement on an IBM web site:
>>
>> Please don't hijack threads.
>>
>> Start a new email to the list, just like you would to a person.
>>
>>
>
>
> I'm not sure what you are referring to. I started this as a new thread
> and so far it has not wandered off topic at all. I'm especially
> appreciating the comments from Charles Caldarale.
>
>
> --Eric
>
>
>
>
>
>
> Disclaimer - February 8, 2011
> This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
> This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Robinson, Eric" <er...@psmnv.com>.
> > Saw this statement on an IBM web site:
>
> Please don't hijack threads.
>
> Start a new email to the list, just like you would to a person.
>
>
I'm not sure what you are referring to. I started this as a new thread
and so far it has not wandered off topic at all. I'm especially
appreciating the comments from Charles Caldarale.
--Eric
Disclaimer - February 8, 2011
This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Is IBM Right About Java?
Posted by Pid <pi...@pidster.com>.
On 2/8/11 4:27 PM, Robinson, Eric wrote:
> Saw this statement on an IBM web site:
Please don't hijack threads.
Start a new email to the list, just like you would to a person.
p
RE: Is IBM Right About Java?
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Joseph Morgan [mailto:joseph.morgan@ignitesales.com]
> Subject: RE: Is IBM Right About Java?
> if the server experiences a hit on load and increases the
> heap, it remains at that level.
This is controlled by the -Xms setting; if your current heap is less than -Xms, GC will not return memory back to the OS. We normally recommend setting -Xms = -Xmx, to insure that the JVM does *not* return memory it has allocated. The allocation/deallocation is a relatively expensive process, and ought to be minimized where possible. If you're running a JVM as a server of any kind, you should be giving it all the RAM it needs to cover the heap size plus all the other stuff needed to run the process - at least in a production environment - in order to avoid any chance of paging. In a test or development scenario, you may want to compromise, since performance is likely not a top priority.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by Joseph Morgan <jo...@ignitesales.com>.
I have a JVM heap monitor that sends emails when the upper limit
changes, and this appears to be true. The GC still works, that is, used
heap continues to fluctuate up and down within the current upper limit,
but if the server experiences a hit on load and increases the heap, it
remains at that level.
I have not yet seen it release heap, however, our servers have never
grown beyond about 15% of max memory available. I wonder if when it
starts bouncing around 90%-95%, will it start to release heap.
-----Original Message-----
From: Robinson, Eric [mailto:eric.robinson@psmnv.com]
Sent: Tuesday, February 08, 2011 10:27 AM
To: Tomcat Users List
Subject: Is IBM Right About Java?
Saw this statement on an IBM web site:
"Most JVMs grow towards the upper heap limit (-Xmx/-mx options) when
more memory is required, and do not return memory to the operating
system, even if the memory is no longer needed, until the JVM process
terminates."
Is this true? If thought returning unneeded memory to the OS was what GC
is all about.
--
Eric Robinson
Disclaimer - February 8, 2011
This email and any files transmitted with it are confidential and
intended solely for Tomcat Users List. If you are not the named
addressee you should not disseminate, distribute, copy or alter this
email. Any views or opinions presented in this email are solely those of
the author and might not represent those of Physicians' Managed Care or
Physician Select Management. Warning: Although Physicians' Managed Care
or Physician Select Management has taken reasonable precautions to
ensure no viruses are present in this email, the company cannot accept
responsibility for any loss or damage arising from the use of this email
or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Robinson, Eric" <er...@psmnv.com>.
> > The JVM always reserves the maximum heap size as virtual space, but
> > does not allocate more of the heap than the current limit;
> the limit
> > will be adjusted up or down within the -Xms : -Xmx range as load
> > dictates. The unused virtual space is not available for
> any other use.
> >
> > Chuck, d'you happen to know how the virtual space
> reservation impacts
> > swap
> space requirements (if at all)? Although somewhat less
> unpleasant than buying RAM, 200 * 0.5Gbytes is still a lot of
> disk to allocate for swap if you expect it's never going to
> be used. I therefore assume there's not a direct mapping of
> reserved virtual space to reserved space in the swap file.
>
My server should be a good test case. We have 203 instances of java. 95%
of them are configured with -Xms64M -Xmx64M. A handful are configured
with higher values, but none with more than 256M.
A quick...
# ps axo rsz,vsz,comm,user|grep java|sort -n
..shows the java processes are consuming a total of 20GB of resident
memory and 56GB of virtual memory.
'top' shows zero swapping, by the way.
top - 08:24:57 up 128 days, 5:52, 5 users, load average: 1.31, 1.49,
1.40
Tasks: 424 total, 1 running, 422 sleeping, 0 stopped, 1 zombie
Cpu(s): 4.3%us, 1.1%sy, 0.0%ni, 93.6%id, 0.8%wa, 0.0%hi, 0.3%si,
0.0%st
Mem: 33265832k total, 22393776k used, 10872056k free, 297752k buffers
Swap: 2031608k total, 0k used, 2031608k free, 3961280k cached
--Eric
Disclaimer - February 9, 2011
This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Is IBM Right About Java?
Posted by Peter Crowther <pe...@melandra.com>.
On 9 February 2011 01:36, Caldarale, Charles R
<Ch...@unisys.com>wrote:
> The JVM always reserves the maximum heap size as virtual space, but does
> not allocate more of the heap than the current limit; the limit will be
> adjusted up or down within the -Xms : -Xmx range as load dictates. The
> unused virtual space is not available for any other use.
>
> Chuck, d'you happen to know how the virtual space reservation impacts swap
space requirements (if at all)? Although somewhat less unpleasant than
buying RAM, 200 * 0.5Gbytes is still a lot of disk to allocate for swap if
you expect it's never going to be used. I therefore assume there's not a
direct mapping of reserved virtual space to reserved space in the swap file.
- Peter
RE: Is IBM Right About Java?
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: David kerber [mailto:dckerber@verizon.net]
> Subject: Re: Is IBM Right About Java?
> I don't know, but always assumed that GC made the cleared memory
> available for the JVM to use for other things, not the OS.
Not really. The JVM always reserves the maximum heap size as virtual space, but does not allocate more of the heap than the current limit; the limit will be adjusted up or down within the -Xms : -Xmx range as load dictates. The unused virtual space is not available for any other use.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Joseph Morgan [mailto:joseph.morgan@ignitesales.com]
> Subject: RE: Is IBM Right About Java?
> So are you saying it will reduce the upper consumed heap?
It should, if you reduce the -Xms value. Might take a while to observe the effect.
> Our core load is from 9am - 7pm CST, and we have virtually
> nothing from 11pm to 5am.
If all of your JVM instances have the same peak times, you're still going to end up paging.
> it gets to about 15% of max memory available and seems to run
> just fine there.
So set -Xms to something around that value and see what happens.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by Joseph Morgan <jo...@ignitesales.com>.
Chuck,
So are you saying it will reduce the upper consumed heap? I have -Xms =
-Xmx, so maybe that's why I never see a reduction. Our core load is
from 9am - 7pm CST, and we have virtually nothing from 11pm to 5am. So
if low heap usage should ever cause the JVM to drop the consumed heap,
that is when it would, but it has never happened.
Like I said before, though, it gets to about 15% of max memory available
and seems to run just fine there.
-----Original Message-----
From: Caldarale, Charles R [mailto:Chuck.Caldarale@unisys.com]
Sent: Tuesday, February 08, 2011 12:53 PM
To: Tomcat Users List
Subject: RE: Is IBM Right About Java?
> From: Robinson, Eric [mailto:eric.robinson@psmnv.com]
> Subject: RE: Is IBM Right About Java?
> As I understand it, it would actually not work because instances that
> peak at 512M but then go back down to, say, 128M, would still never
> return the unused memory to the OS so it can be used by other
instances.
Not true - that's the whole point of the -Xms setting. If GC can
squeeze the heap down, it will do so and avoid allocations in the now
unused virtual space. There are some heuristics in place within GC to
avoid bouncing up and down, so heap usage has to stay low for a while
before you'll see a lower upper limit and a corresponding reduction in
the real memory usage. (Note that the unused virtual space may not be
returned to the OS, but the real space behind it won't be referenced so
the pages will age out - but only if the OS needs the space for some
other process.)
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY
MATERIAL and is thus for use only by the intended recipient. If you
received this in error, please contact the sender and delete the e-mail
and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Robinson, Eric" <er...@psmnv.com>.
Andre,
> Like, what if they come back next week and say that now they
> need 1 GB heap per instance ?
>
Sometimes I lose sleep thinking about that scenario. :-)
--Eric
Disclaimer - February 8, 2011
This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Is IBM Right About Java?
Posted by André Warnier <aw...@ice-sa.com>.
Now of course, you could just add some RAM, and send the bill for the upgrade to the
unreasonable software vendor.
Or to the customer, or to whoever decided to upgrade to this new version which requires
more resources.
Like, what if they come back next week and say that now they need 1 GB heap per instance ?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Is IBM Right About Java?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Daniel,
On 2/8/2011 4:56 PM, Daniel Savard wrote:
> However, increasing this limit for all instances may lead to a
> situation where enough instances are claiming more memory at the same
> time and beyond the physical memory available.
Exactly. With the low-memory limit on each JVM, you are basically
limited to OOME on a single JVM, while a high-memory limit on each JVM
means that they can more easily interfere with each other due to paging.
Of course, low-memory limits can still have this effect but if you do
your math right -- system_memory / ((64MiB + slop_factor) * JVM_count)
then you should be just fine. Your vendor's demand that you increase the
heap space ruins that calculation.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk1Rxw0ACgkQ9CaO5/Lv0PBubwCfaKKd1IvVoGgiFsilvq9VvEs1
MN4Anj+2AU/g4PKJRh1Ia1rJQDJ1lAgs
=wqAJ
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Robinson, Eric" <er...@psmnv.com>.
> Daniel,
>
> On 2/8/2011 4:56 PM, Daniel Savard wrote:
> > However, increasing this limit for all instances may lead to a
> > situation where enough instances are claiming more memory
> at the same
> > time and beyond the physical memory available.
>
> Exactly. With the low-memory limit on each JVM, you are
> basically limited to OOME on a single JVM, while a
> high-memory limit on each JVM means that they can more easily
> interfere with each other due to paging.
>
Okay, that's what I figured, so I think we're all on the same page.
Thanks much. I guess the only way to be 100% certain is to try it, watch
it carefully, and be ready to reconfigure and restart all the tomcats if
it looks like things are going to get dicey (like if I see iowait and/or
swap usage start to climb). Thankfully, because of the way we do load
balancing, this can be accomplished gracefully without severely
impacting customers.
--Eric
Disclaimer - February 8, 2011
This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Is IBM Right About Java?
Posted by Daniel Savard <da...@gmail.com>.
2011/2/8 Robinson, Eric <er...@psmnv.com>
>
> Obviously I need to understand this better. If all instances are set to
> -Xmx512M and then one instance peaks to 512M and even tries to go above
> that (and therefore generates an OOME) how does that impact the other
> instances? Does an OOME mean there is no more memory available from the
> OS? I thought it just meant it had used the max memory allowed by the
> -Xmx setting. That should not cause a problem for other instances should
> it?
>
> --Eric
Eric,
what Chris is trying to explain is that a single instance using 512M
may not right then have adverse effects on others as long as the real
memory available is not exhausted and paging has started at the OS
level. However, increasing this limit for all instances may lead to a
situation where enough instances are claiming more memory at the same
time and beyond the physical memory available. At that point, the
system (OS) will start paging, provided you have paging space. It may
be still runnable and usable at this point until too many memory is
claimed and consume at the same time the OS enter the thrashing state.
Paging is not necessarily bad, thrashing is the point where paging
activity is so high no useful work but paging take place. The system
is no longer usable at this point.
Hope I was able to clarify Chris' point a little bit.
Regards,
--
-----------------
Daniel Savard
CiDS Inc.
Montreal, QC
Canada
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Robinson, Eric" <er...@psmnv.com>.
Chris,
> On 2/8/2011 3:16 PM, Robinson, Eric wrote:
> > This is great feedback, Chuck. Right now we have 200+ instances
> > running on the same server, most of which are set to 64MB
> min and max
> > heap. Like I daid, we do not see OOM messages or paging,
> and top shows
> > about 27GB of 32GB in use. Based on your comments, I would
> expect that
> > changing the instances to 32M min and 512M max would not cause a
> > negative impact. I will give this a try and report the effect.
>
> Here's the problem, though: if you leave your system as it
> is, and one (or more) of your VM/webapps suffers an OOME,
> only that one VM/webapp suffers. If you increase the -Xmx for
> /all/ instances, then in cases where memory usage spikes, an
> individual instance could potentially negatively impact all
> other instances because of memory thrashing.
>
> - -chris
Obviously I need to understand this better. If all instances are set to
-Xmx512M and then one instance peaks to 512M and even tries to go above
that (and therefore generates an OOME) how does that impact the other
instances? Does an OOME mean there is no more memory available from the
OS? I thought it just meant it had used the max memory allowed by the
-Xmx setting. That should not cause a problem for other instances should
it?
--Eric
Disclaimer - February 8, 2011
This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Is IBM Right About Java?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Eric,
On 2/8/2011 3:16 PM, Robinson, Eric wrote:
> This is great feedback, Chuck. Right now we have 200+ instances running
> on the same server, most of which are set to 64MB min and max heap. Like
> I daid, we do not see OOM messages or paging, and top shows about 27GB
> of 32GB in use. Based on your comments, I would expect that changing the
> instances to 32M min and 512M max would not cause a negative impact. I
> will give this a try and report the effect.
Here's the problem, though: if you leave your system as it is, and one
(or more) of your VM/webapps suffers an OOME, only that one VM/webapp
suffers. If you increase the -Xmx for /all/ instances, then in cases
where memory usage spikes, an individual instance could potentially
negatively impact all other instances because of memory thrashing.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk1RtMkACgkQ9CaO5/Lv0PDDEACfR/gp8G0Qcc1qu715V+KLBW+6
178An1OPGcQZaa8bdvZG8CaMYaKOHbwP
=PFcJ
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Robinson, Eric" <er...@psmnv.com>.
> > As I understand it, it would actually not work because
> instances that
> > peak at 512M but then go back down to, say, 128M, would still never
> > return the unused memory to the OS so it can be used by
> other instances.
>
> Not true - that's the whole point of the -Xms setting. If GC
> can squeeze the heap down, it will do so and avoid
> allocations in the now unused virtual space. There are some
> heuristics in place within GC to avoid bouncing up and down,
> so heap usage has to stay low for a while before you'll see a
> lower upper limit and a corresponding reduction in the real
> memory usage. (Note that the unused virtual space may not be
> returned to the OS, but the real space behind it won't be
> referenced so the pages will age out - but only if the OS
> needs the space for some other process.)
>
> - Chuck
>
>
This is great feedback, Chuck. Right now we have 200+ instances running
on the same server, most of which are set to 64MB min and max heap. Like
I daid, we do not see OOM messages or paging, and top shows about 27GB
of 32GB in use. Based on your comments, I would expect that changing the
instances to 32M min and 512M max would not cause a negative impact. I
will give this a try and report the effect.
--Eric
Disclaimer - February 8, 2011
This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Robinson, Eric [mailto:eric.robinson@psmnv.com]
> Subject: RE: Is IBM Right About Java?
> As I understand it, it would actually not work because instances that
> peak at 512M but then go back down to, say, 128M, would still never
> return the unused memory to the OS so it can be used by other instances.
Not true - that's the whole point of the -Xms setting. If GC can squeeze the heap down, it will do so and avoid allocations in the now unused virtual space. There are some heuristics in place within GC to avoid bouncing up and down, so heap usage has to stay low for a while before you'll see a lower upper limit and a corresponding reduction in the real memory usage. (Note that the unused virtual space may not be returned to the OS, but the real space behind it won't be referenced so the pages will age out - but only if the OS needs the space for some other process.)
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Robinson, Eric" <er...@psmnv.com>.
> On 2/8/2011 12:27 PM, Robinson, Eric wrote:
> >>> I would like to set all JVMs to -Xms=32M -Xmx=512M and let
> >> the system
> >>> figure out how much memory each instance really needs.
> >>
> >> That should work, at the expense of some thrashing of the
> heap size
> >> as the load waxes and wanes.
> >>
> >
> > As I understand it, it would actually not work because
> instances that
> > peak at 512M but then go back down to, say, 128M, would still never
> > return the unused memory to the OS so it can be used by
> other instances.
> > Consequently, we would get a lot of disk paging.
>
> Have you benchmarked the new version to see how high the
> memory use actually gets? It might not have much real effect
> if it doesn't go very high.
>
>
Yes, we've benchmarked it and the memory usage is not significantly
higher than with the old version. The problem is the software company's
arbitrary requirements. They think the product needs 512MB RAM. You know
how software companies are. It's slow? Well then throw more RAM at it!
It doesn't matter if actuall USES the memory. It's just a typical
software company knee jerk reaction, but one that we have to abide by.
--Eric
Disclaimer - February 8, 2011
This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Is IBM Right About Java?
Posted by David kerber <dc...@verizon.net>.
On 2/8/2011 12:27 PM, Robinson, Eric wrote:
>>> I would like to set all JVMs to -Xms=32M -Xmx=512M and let
>> the system
>>> figure out how much memory each instance really needs.
>>
>> That should work, at the expense of some thrashing of the
>> heap size as the load waxes and wanes.
>>
>
> As I understand it, it would actually not work because instances that
> peak at 512M but then go back down to, say, 128M, would still never
> return the unused memory to the OS so it can be used by other instances.
> Consequently, we would get a lot of disk paging.
Have you benchmarked the new version to see how high the memory use
actually gets? It might not have much real effect if it doesn't go very
high.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Robinson, Eric" <er...@psmnv.com>.
> > I would like to set all JVMs to -Xms=32M -Xmx=512M and let
> the system
> > figure out how much memory each instance really needs.
>
> That should work, at the expense of some thrashing of the
> heap size as the load waxes and wanes.
>
As I understand it, it would actually not work because instances that
peak at 512M but then go back down to, say, 128M, would still never
return the unused memory to the OS so it can be used by other instances.
Consequently, we would get a lot of disk paging.
--Eric
Disclaimer - February 8, 2011
This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Robinson, Eric [mailto:eric.robinson@psmnv.com]
> Subject: RE: Is IBM Right About Java?
> I would like to set all JVMs to -Xms=32M -Xmx=512M and let the
> system figure out how much memory each instance really needs.
That should work, at the expense of some thrashing of the heap size as the load waxes and wanes.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Is IBM Right About Java?
Posted by "Robinson, Eric" <er...@psmnv.com>.
Thanks for the input, everyone. I see there is mostly agreement on this
question. Once the OS allocates memory to the JVM, the OS never gets it
back. GC does not return memory to the OS.
This is unfortunate behavior for those of us who stack multiple
instances on tomcat/java on the same server as it limits the number of
possible running instances.
We have servers with more than 200 instances on tomcat running on them
(8-core, 32GB). For most instances, Xms and Xmx are 64M. Performance is
good and we do not get OOM errors. However, the software vendor is now
arbitrarily requiring us to change the -Xmx value to 512M for all
running instances. Ideally, I would like to set all JVMs to -Xms=32M
-Xmx=512M and let the system figure out how much memory each instance
really needs. If IBM's statement is unalterably true, then this will
have a negative impact on the number of instances we can stack. :-(
--
Eric Robinson
Disclaimer - February 8, 2011
This email and any files transmitted with it are confidential and intended solely for Tomcat Users List. If you are not the named addressee you should not disseminate, distribute, copy or alter this email. Any views or opinions presented in this email are solely those of the author and might not represent those of Physicians' Managed Care or Physician Select Management. Warning: Although Physicians' Managed Care or Physician Select Management has taken reasonable precautions to ensure no viruses are present in this email, the company cannot accept responsibility for any loss or damage arising from the use of this email or attachments.
This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Is IBM Right About Java?
Posted by David kerber <dc...@verizon.net>.
On 2/8/2011 11:27 AM, Robinson, Eric wrote:
> Saw this statement on an IBM web site:
>
> "Most JVMs grow towards the upper heap limit (-Xmx/-mx options) when
> more memory is required, and do not return memory to the operating
> system, even if the memory is no longer needed, until the JVM process
> terminates."
>
> Is this true? If thought returning unneeded memory to the OS was what GC
> is all about.
>
> --
> Eric Robinson
I don't know, but always assumed that GC made the cleared memory
available for the JVM to use for other things, not the OS.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Is IBM Right About Java?
Posted by Marco Castillo <ma...@vdkit.net>.
Dear Eric:
Nop, the GC process returns the memory to the memory pool allocated by the
JVM, so the JVM never exhaust its memory resources. The Xmx parameter indeed
indicates the maximum amount of memory from the system that the JVM can use
and the IBM assertion is true, the JVM never returns the memory allocated to
the system but don't fear, this is actually a standard design behaviour for
this kind of processes!!
Regards
Ing. Marco Antonio Castillo
Chief Design Engineer
Van Der Kaaden IT Consulting
Guatemala, Guatemala C.A.
tel: +502 22382710
mobile: +502 59186971
e-mail: mabcastillo@vdkit.net
On Tue, Feb 8, 2011 at 10:27 AM, Robinson, Eric <er...@psmnv.com>wrote:
> Saw this statement on an IBM web site:
>
> "Most JVMs grow towards the upper heap limit (-Xmx/-mx options) when
> more memory is required, and do not return memory to the operating
> system, even if the memory is no longer needed, until the JVM process
> terminates."
>
> Is this true? If thought returning unneeded memory to the OS was what GC
> is all about.
>
> --
> Eric Robinson
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Disclaimer - February 8, 2011
> This email and any files transmitted with it are confidential and intended
> solely for Tomcat Users List. If you are not the named addressee you should
> not disseminate, distribute, copy or alter this email. Any views or opinions
> presented in this email are solely those of the author and might not
> represent those of Physicians' Managed Care or Physician Select Management.
> Warning: Although Physicians' Managed Care or Physician Select Management
> has taken reasonable precautions to ensure no viruses are present in this
> email, the company cannot accept responsibility for any loss or damage
> arising from the use of this email or attachments.
> This disclaimer was added by Policy Patrol: http://www.policypatrol.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>