You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Fady Haikal <fa...@gmail.com> on 2017/08/16 13:34:26 UTC

Tomcat memory

Dear Team,
I'm facing an issue that tomcat from task manager is consuming around 60 GB
of memory while from Oracle Java Mission Control is showing maximum 10 GB
(Attached screenshots), and from time to time the server hang due to
insufficent memory.
can you please advise why the above is showing like that? and why the
memory of the server is not reducing?

Tomcat Version: 8.0.30
JDK: 1.7.0_67

Regards,
Fady

Re: Tomcat memory

Posted by "André Warnier (tomcat)" <aw...@ice-sa.com>.
On 16.08.2017 15:34, Fady Haikal wrote:
> Dear Team,
> I'm facing an issue that tomcat from task manager is consuming around 60 GB of memory
> while from Oracle Java Mission Control is showing maximum 10 GB (Attached screenshots),
> and from time to time the server hang due to insufficent memory.
> can you please advise why the above is showing like that? and why the memory of the server
> is not reducing?
>
> Tomcat Version: 8.0.30
> JDK: 1.7.0_67
>

Maybe one preliminary response here, just to orient your further thinking and analysis, is 
that Tomcat itself will never use 60 GB of memory, nor even 10 GB of memory, nor even 1 GB 
of memory. Properly configured, and with one basic and correct webapp, Tomcat would 
certainly use much less than 1 GB.
What /may/ be using more memory is :
- parameters used when starting the Java JVM which runs Tomcat, related to the size of the 
Java Heap etc.
- and then, more likely than not, /the web applications/ which you are running within Tomcat

As to your second question (why Java's OS memory usage is not decreasing), maybe have a 
look here :
http://www.stefankrause.net/wp/?p=14
and/or here :
https://coderanch.com/t/609676/java/Unused-java-memory-released-OS
and a number of other similar responses on the WWW.
Summary :
It looks like being an area of some uncertainty and discussion on the WWW, but the general 
consensus seems to be that once Java has obtained some additional memory from the OS, it 
never (or rarely) releases it, until the Java process itself exits.
Note that this is /Java/, not Tomcat-specific.



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat memory

Posted by Suvendu Sekhar Mondal <su...@gmail.com>.
Hi Fady,

On Thu, Aug 17, 2017 at 12:16 PM, Fady Haikal <fa...@gmail.com> wrote:
> @Suvendu,
> I took a heap dump from Java VisualVM but honestly i didnt know how i
> should analyse it, please some help here

Acquire the software:
André has already given you some pointer. My favorite is Eclipse
MAT[http://www.eclipse.org/mat]. :)
- Download the standalone version.
- To open a large dump, you need to have a server with sufficient
amount of memory. I assume you have such machine. :)
- Backup MemoryAnalyzer.ini. Then Remove everything inside it and add
following in two lines...make sure there is no space before and after
those args:
                   -vmargs
                   -Xmx14g
It will create a VM for MAT with 14GB heap. For me it opens up 6GB
heap in 5-8min. Adjust heap size as required.

Analysis:
You need to change Memory Look at the object size in 'Dominator Tree'
view(by looking at the Retained and Shallow Heap Size) and use 'path
to GC root' option to see what is keeping them alive. If MAT shows
that small number of objects are consuming most of the heap, then you
are lucky. If very small objects are filling up the heap then your job
will become tedious. There is one 'Leak Suspect' view but it may or
may not be accurate - depending on your application architecture.

I will strongly suggest that you go through some documentation and how
to use MAT stuffs so that you become more familiar with it :
http://www.eclipse.org/mat/documentation

There are lots of docs available over Internet to help you in analysis part.

> also please find below the java configuration i used:
> -XX:PermSize=10240m
> -XX:MaxPermSize=10240m
> -XX:ReservedCodeCacheSize=512m
> -XX:+UseParNewGC
> -XX:+UseConcMarkSweepGC
> -XX:+CMSParallelRemarkEnabled
> -XX:TargetSurvivorRatio=90
>
> Initial and Max Memory Pool: 100000MB
>
> Regards,
> Fady
>

So you have created a huge JVM. This is the high level formula I use
to predict approximate JVM process size:
JDK 1.7 JVM Process Size = [-Xmx] + [-XX:MaxPermSize] + [-Xss] *
number_of_threads + JIT’ed code + GC + Other Stuffs

In your case, from above parameter, (max)approximate size could be
100000MB+10240MB+some more memory. Because I do not know how many
threads you have opened. By default I guess JDK 1.7 allocates 1MB
stack space per thread. Again, I will suggest to enable NMT if
possible. It tells us where most of the memory is being consumed which
you can not trace using heap dumps.

You are using CMS. I have seen that it can create problem if it is not
properly tuned. I hope you have came up with those GC params after
good amount of performance testing.

One last thing, I agree with André, most probably you are dealing with
JVM problem not Tomcat. Please keep us posted about your findings.

Thanks!

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat memory

Posted by "André Warnier (tomcat)" <aw...@ice-sa.com>.
On 17.08.2017 12:57, Leon Rosenberg wrote:
> Fady, one thing,
>
> analyzing heap dumps is hard especially a 10GB dump, you will need at least
> 40 Gb of memory an about 10 hours to start jhat.

Hmm. Good to know.  Strange that the documentation didn't say anything about that.. :-)

> What is fast is analyzing a histogram. A histogram is a list of all classes
> in your JVM and amount of memory they use. It is very easy to use:
>

There is maybe another suggestion : start the Java JVM with MUCH smaller heap settings 
(like -Xmx1G), and set the parameter which tells it to generate a heap dump on OOM.
Chances are that whatever application generates the runaway memory usage and OOM at 60GB, 
will also generate it then (only much faster).


> jmap -histo:live <pid> >outputfile
> the :live parameter is important, otherwise you will see dead objects as
> well. Keep in mind a major gc will be triggered.
>
> The output looks like this:
>
>   num     #instances         #bytes  class name
> ----------------------------------------------
>     1:         98573       13165976  [C
>     2:          4376       11325928  [B
>     3:         96526        2316624  java.lang.String
>     4:         40522        1296704  java.util.HashMap$Node
>     5:         10954         807552  [Ljava.lang.Object;
>     6:          7494         781480  java.lang.Class
>     7:          6778         655464  [Ljava.util.HashMap$Node;
> ...
>
> 3103:             1             16
> sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter
> 3104:             1             16  sun.util.resources.LocaleData
> 3105:             1             16
> sun.util.resources.LocaleData$LocaleDataResourceBundleControl
> Total        456560       37330672
>
> This won't point you to exact direction of memory leak or resource
> consumption, but you will see what classes are consuming memory. So if you
> find out that class foo.bar.XYZBean has 10.000.000 instances and occupies 2
> gb of memory, you know where to look.
>
> Keep in mind that only memory directly used by a class is counted, so every
> string will appear twice, as string and as byte array.
>
> If you see that the used memory amount is growing, take multiple
> histograms, every 10 minutes or so and analyse the differences. I wrote a
> little script years ago for myself (
> https://github.com/dvayanu/ldt/blob/c8b3c2b6f61de5c583db503f6fd5e2d8aa8b9aa0/java/ldt/histo/HistoDiffReader.java)
> that takes two histograms and prints out the differences. This way you can
> see if new instances of a specific class(es) are accumulated over time. In
> this case you have a memory leak.
>
> regards
> Leon
>
>
>
>
>
>
>
>
> On Thu, Aug 17, 2017 at 12:09 PM, André Warnier (tomcat) <aw...@ice-sa.com>
> wrote:
>
>> On 17.08.2017 11:21, Fady Haikal wrote:
>>
>>> Team,
>>> Please i need some help her
>>>
>>
>> Maybe start here ? http://lmgtfy.com/?q=analyse+tomcat+heap+dump
>>
>> (and this looks like it might help you :
>> http://docs.oracle.com/javase/7/docs/technotes/tools/share/jhat.html)
>>
>> To restate the obvious :
>> - this list here is manned by volunteers, who donate their time trying to
>> help people who have problems with Tomcat, and who are doing their best, but
>> - the list works best with people who also help themselves
>> - we do not have access to your system, and we do not know your
>> applications
>> - we can only try to help, on the base of the information which you provide
>> - there are 3 (possible) components to your problem :
>>    a) the Java JVM which runs Tomcat
>>    b) Tomcat itself, which runs your applications
>>    c) the applications themselves
>> Of these :
>>    a) is controlled entirely by you, and is in fact the one which
>> ultimately uses all that memory (but we do not know why either). Below you
>> are showing the startup parameters of the JVM. They are not the default
>> settings. So where do these parameters come from, and do you understand
>> them ? (if you don't, you first need help with Java, not with Tomcat)
>>    b) That is really the focus of this list. And we know that Tomcat, by
>> itself, does not use 60GB or 10GB of memory, nor anywhere near it
>>    c) we know nothing about, but it is likely that it is something there,
>> which causes Tomcat (and Java) to use all that memory
>> And about (c), maybe it is normal that it/they use this gigantic amount of
>> memory. Maybe one of the applications allocates some gigantic array of data
>> over time. Maybe one of these applications is really badly programmed and
>> causes enormous "memory leaks". How would we know ? Are these your
>> applications ? If not, have you contacted the people responsible for these
>> applications, and described your problem to them ?
>> (because again, with 99& probability, that is where the problem is)
>>
>> Try to collect some more specific data, and when you have it, post it
>> here, and maybe someone will have an idea.  But as it is, the only thing
>> known so far is that on your system, the Java JVM which runs tomcat is
>> using up a lot of memory, increasingly over time, and that it ends up
>> crashing with an OOM.
>> That is not enough information for us to be able to help you.
>>
>> If you are desperate and in a hurry, maybe you need first to find a Java
>> JVM specialist, who can help you poinpoint the problem more specifically.
>>
>>
>>
>>
>>> Regards,
>>>
>>>
>>> On Thu, Aug 17, 2017 at 9:46 AM, Fady Haikal <fa...@gmail.com>
>>> wrote:
>>>
>>> @Suvendu,
>>>> I took a heap dump from Java VisualVM but honestly i didnt know how i
>>>> should analyse it, please some help here
>>>>
>>>> also please find below the java configuration i used:
>>>> -XX:PermSize=10240m
>>>> -XX:MaxPermSize=10240m
>>>> -XX:ReservedCodeCacheSize=512m
>>>> -XX:+UseParNewGC
>>>> -XX:+UseConcMarkSweepGC
>>>> -XX:+CMSParallelRemarkEnabled
>>>> -XX:TargetSurvivorRatio=90
>>>>
>>>> Initial and Max Memory Pool: 100000MB
>>>>
>>>> Regards,
>>>> Fady
>>>>
>>>>
>>>> On Wed, Aug 16, 2017 at 5:46 PM, Suvendu Sekhar Mondal <
>>>> suv3ndu@gmail.com>
>>>> wrote:
>>>>
>>>> Hi Fady,
>>>>>
>>>>> On Aug 16, 2017 7:04 PM, "Fady Haikal" <fa...@gmail.com> wrote:
>>>>>
>>>>> Dear Team,
>>>>> I'm facing an issue that tomcat from task manager is consuming around 60
>>>>> GB
>>>>> of memory while from Oracle Java Mission Control is showing maximum 10
>>>>> GB
>>>>> (Attached screenshots), and from time to time the server hang due to
>>>>> insufficent memory.
>>>>> can you please advise why the above is showing like that? and why the
>>>>> memory of the server is not reducing?
>>>>>
>>>>> Tomcat Version: 8.0.30
>>>>> JDK: 1.7.0_67
>>>>>
>>>>>
>>>>> I couldn't find the screenshot. Anyway​, the issue you mentioned is very
>>>>> interesting. Usually I saw some difference(in MB) between task manager
>>>>> data
>>>>> and JMC data...but not like this.  Can you please let us know about the
>>>>> Java Heap, Permgen and thread stack size configuration?
>>>>>
>>>>> For starter, I will suggest to take a heap dump and look into it. It
>>>>> will
>>>>> show you which objects are live and holding the memory. Also it will be
>>>>> awesome if you can enable Native Memory Tracking for your TC instance.
>>>>>
>>>>> Thanks!
>>>>> Suvendu
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat memory

Posted by "André Warnier (tomcat)" <aw...@ice-sa.com>.
On 17.08.2017 14:21, Suvendu Sekhar Mondal wrote:
>> According to this however :
>> https://eclipsesource.com/blogs/2013/01/31/creating-tomcat-heap-dumps-on-windows/
>> jmap does not work under Windows, if Tomcat is running as a Service
>> (seems likely in the OP's case)
>> I don't know if that's true, have not checked it.
>
> Andre, That is not correct. I use those jmap commands against our
> Tomcat Windows services pretty frequently. Although I can not tell
> about what it used to do in 2013. :)

Ok, well, that's good to know also, and forget my above comment.
That shows how Google is for many things a splendid search tool, but unfortunately in many 
cases it delivers a lot of noise. (Note that in this case, the page as a whole seems 
rather nice and helpful, except for this bit of outdated information thus).

>
>
> On Thu, Aug 17, 2017 at 5:31 PM, André Warnier (tomcat) <aw...@ice-sa.com> wrote:
>> On 17.08.2017 12:57, Leon Rosenberg wrote:
>>>
>>> Fady, one thing,
>>>
>>> analyzing heap dumps is hard especially a 10GB dump, you will need at
>>> least
>>> 40 Gb of memory an about 10 hours to start jhat.
>>> What is fast is analyzing a histogram. A histogram is a list of all
>>> classes
>>> in your JVM and amount of memory they use. It is very easy to use:
>>>
>>> jmap -histo:live <pid> >outputfile
>>
>>
>> According to this however :
>> https://eclipsesource.com/blogs/2013/01/31/creating-tomcat-heap-dumps-on-windows/
>> jmap does not work under Windows, if Tomcat is running as a Service
>> (seems likely in the OP's case)
>> I don't know if that's true, have not checked it.
>>
>>
>>> the :live parameter is important, otherwise you will see dead objects as
>>> well. Keep in mind a major gc will be triggered.
>>>
>>> The output looks like this:
>>>
>>>    num     #instances         #bytes  class name
>>> ----------------------------------------------
>>>      1:         98573       13165976  [C
>>>      2:          4376       11325928  [B
>>>      3:         96526        2316624  java.lang.String
>>>      4:         40522        1296704  java.util.HashMap$Node
>>>      5:         10954         807552  [Ljava.lang.Object;
>>>      6:          7494         781480  java.lang.Class
>>>      7:          6778         655464  [Ljava.util.HashMap$Node;
>>> ...
>>>
>>> 3103:             1             16
>>> sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter
>>> 3104:             1             16  sun.util.resources.LocaleData
>>> 3105:             1             16
>>> sun.util.resources.LocaleData$LocaleDataResourceBundleControl
>>> Total        456560       37330672
>>>
>>> This won't point you to exact direction of memory leak or resource
>>> consumption, but you will see what classes are consuming memory. So if you
>>> find out that class foo.bar.XYZBean has 10.000.000 instances and occupies
>>> 2
>>> gb of memory, you know where to look.
>>>
>>> Keep in mind that only memory directly used by a class is counted, so
>>> every
>>> string will appear twice, as string and as byte array.
>>>
>>> If you see that the used memory amount is growing, take multiple
>>> histograms, every 10 minutes or so and analyse the differences. I wrote a
>>> little script years ago for myself (
>>>
>>> https://github.com/dvayanu/ldt/blob/c8b3c2b6f61de5c583db503f6fd5e2d8aa8b9aa0/java/ldt/histo/HistoDiffReader.java)
>>> that takes two histograms and prints out the differences. This way you can
>>> see if new instances of a specific class(es) are accumulated over time. In
>>> this case you have a memory leak.
>>>
>>> regards
>>> Leon
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Thu, Aug 17, 2017 at 12:09 PM, André Warnier (tomcat) <aw...@ice-sa.com>
>>> wrote:
>>>
>>>> On 17.08.2017 11:21, Fady Haikal wrote:
>>>>
>>>>> Team,
>>>>> Please i need some help her
>>>>>
>>>>
>>>> Maybe start here ? http://lmgtfy.com/?q=analyse+tomcat+heap+dump
>>>>
>>>> (and this looks like it might help you :
>>>> http://docs.oracle.com/javase/7/docs/technotes/tools/share/jhat.html)
>>>>
>>>> To restate the obvious :
>>>> - this list here is manned by volunteers, who donate their time trying to
>>>> help people who have problems with Tomcat, and who are doing their best,
>>>> but
>>>> - the list works best with people who also help themselves
>>>> - we do not have access to your system, and we do not know your
>>>> applications
>>>> - we can only try to help, on the base of the information which you
>>>> provide
>>>> - there are 3 (possible) components to your problem :
>>>>     a) the Java JVM which runs Tomcat
>>>>     b) Tomcat itself, which runs your applications
>>>>     c) the applications themselves
>>>> Of these :
>>>>     a) is controlled entirely by you, and is in fact the one which
>>>> ultimately uses all that memory (but we do not know why either). Below
>>>> you
>>>> are showing the startup parameters of the JVM. They are not the default
>>>> settings. So where do these parameters come from, and do you understand
>>>> them ? (if you don't, you first need help with Java, not with Tomcat)
>>>>     b) That is really the focus of this list. And we know that Tomcat, by
>>>> itself, does not use 60GB or 10GB of memory, nor anywhere near it
>>>>     c) we know nothing about, but it is likely that it is something there,
>>>> which causes Tomcat (and Java) to use all that memory
>>>> And about (c), maybe it is normal that it/they use this gigantic amount
>>>> of
>>>> memory. Maybe one of the applications allocates some gigantic array of
>>>> data
>>>> over time. Maybe one of these applications is really badly programmed and
>>>> causes enormous "memory leaks". How would we know ? Are these your
>>>> applications ? If not, have you contacted the people responsible for
>>>> these
>>>> applications, and described your problem to them ?
>>>> (because again, with 99& probability, that is where the problem is)
>>>>
>>>> Try to collect some more specific data, and when you have it, post it
>>>> here, and maybe someone will have an idea.  But as it is, the only thing
>>>> known so far is that on your system, the Java JVM which runs tomcat is
>>>> using up a lot of memory, increasingly over time, and that it ends up
>>>> crashing with an OOM.
>>>> That is not enough information for us to be able to help you.
>>>>
>>>> If you are desperate and in a hurry, maybe you need first to find a Java
>>>> JVM specialist, who can help you poinpoint the problem more specifically.
>>>>
>>>>
>>>>
>>>>
>>>>> Regards,
>>>>>
>>>>>
>>>>> On Thu, Aug 17, 2017 at 9:46 AM, Fady Haikal <fa...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> @Suvendu,
>>>>>>
>>>>>> I took a heap dump from Java VisualVM but honestly i didnt know how i
>>>>>> should analyse it, please some help here
>>>>>>
>>>>>> also please find below the java configuration i used:
>>>>>> -XX:PermSize=10240m
>>>>>> -XX:MaxPermSize=10240m
>>>>>> -XX:ReservedCodeCacheSize=512m
>>>>>> -XX:+UseParNewGC
>>>>>> -XX:+UseConcMarkSweepGC
>>>>>> -XX:+CMSParallelRemarkEnabled
>>>>>> -XX:TargetSurvivorRatio=90
>>>>>>
>>>>>> Initial and Max Memory Pool: 100000MB
>>>>>>
>>>>>> Regards,
>>>>>> Fady
>>>>>>
>>>>>>
>>>>>> On Wed, Aug 16, 2017 at 5:46 PM, Suvendu Sekhar Mondal <
>>>>>> suv3ndu@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Hi Fady,
>>>>>>>
>>>>>>>
>>>>>>> On Aug 16, 2017 7:04 PM, "Fady Haikal" <fa...@gmail.com> wrote:
>>>>>>>
>>>>>>> Dear Team,
>>>>>>> I'm facing an issue that tomcat from task manager is consuming around
>>>>>>> 60
>>>>>>> GB
>>>>>>> of memory while from Oracle Java Mission Control is showing maximum 10
>>>>>>> GB
>>>>>>> (Attached screenshots), and from time to time the server hang due to
>>>>>>> insufficent memory.
>>>>>>> can you please advise why the above is showing like that? and why the
>>>>>>> memory of the server is not reducing?
>>>>>>>
>>>>>>> Tomcat Version: 8.0.30
>>>>>>> JDK: 1.7.0_67
>>>>>>>
>>>>>>>
>>>>>>> I couldn't find the screenshot. Anyway, the issue you mentioned is
>>>>>>> very
>>>>>>> interesting. Usually I saw some difference(in MB) between task manager
>>>>>>> data
>>>>>>> and JMC data...but not like this.  Can you please let us know about
>>>>>>> the
>>>>>>> Java Heap, Permgen and thread stack size configuration?
>>>>>>>
>>>>>>> For starter, I will suggest to take a heap dump and look into it. It
>>>>>>> will
>>>>>>> show you which objects are live and holding the memory. Also it will
>>>>>>> be
>>>>>>> awesome if you can enable Native Memory Tracking for your TC instance.
>>>>>>>
>>>>>>> Thanks!
>>>>>>> Suvendu
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>>
>>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat memory

Posted by Suvendu Sekhar Mondal <su...@gmail.com>.
> According to this however :
> https://eclipsesource.com/blogs/2013/01/31/creating-tomcat-heap-dumps-on-windows/
> jmap does not work under Windows, if Tomcat is running as a Service
> (seems likely in the OP's case)
> I don't know if that's true, have not checked it.

Andre, That is not correct. I use those jmap commands against our
Tomcat Windows services pretty frequently. Although I can not tell
about what it used to do in 2013. :)


On Thu, Aug 17, 2017 at 5:31 PM, André Warnier (tomcat) <aw...@ice-sa.com> wrote:
> On 17.08.2017 12:57, Leon Rosenberg wrote:
>>
>> Fady, one thing,
>>
>> analyzing heap dumps is hard especially a 10GB dump, you will need at
>> least
>> 40 Gb of memory an about 10 hours to start jhat.
>> What is fast is analyzing a histogram. A histogram is a list of all
>> classes
>> in your JVM and amount of memory they use. It is very easy to use:
>>
>> jmap -histo:live <pid> >outputfile
>
>
> According to this however :
> https://eclipsesource.com/blogs/2013/01/31/creating-tomcat-heap-dumps-on-windows/
> jmap does not work under Windows, if Tomcat is running as a Service
> (seems likely in the OP's case)
> I don't know if that's true, have not checked it.
>
>
>> the :live parameter is important, otherwise you will see dead objects as
>> well. Keep in mind a major gc will be triggered.
>>
>> The output looks like this:
>>
>>   num     #instances         #bytes  class name
>> ----------------------------------------------
>>     1:         98573       13165976  [C
>>     2:          4376       11325928  [B
>>     3:         96526        2316624  java.lang.String
>>     4:         40522        1296704  java.util.HashMap$Node
>>     5:         10954         807552  [Ljava.lang.Object;
>>     6:          7494         781480  java.lang.Class
>>     7:          6778         655464  [Ljava.util.HashMap$Node;
>> ...
>>
>> 3103:             1             16
>> sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter
>> 3104:             1             16  sun.util.resources.LocaleData
>> 3105:             1             16
>> sun.util.resources.LocaleData$LocaleDataResourceBundleControl
>> Total        456560       37330672
>>
>> This won't point you to exact direction of memory leak or resource
>> consumption, but you will see what classes are consuming memory. So if you
>> find out that class foo.bar.XYZBean has 10.000.000 instances and occupies
>> 2
>> gb of memory, you know where to look.
>>
>> Keep in mind that only memory directly used by a class is counted, so
>> every
>> string will appear twice, as string and as byte array.
>>
>> If you see that the used memory amount is growing, take multiple
>> histograms, every 10 minutes or so and analyse the differences. I wrote a
>> little script years ago for myself (
>>
>> https://github.com/dvayanu/ldt/blob/c8b3c2b6f61de5c583db503f6fd5e2d8aa8b9aa0/java/ldt/histo/HistoDiffReader.java)
>> that takes two histograms and prints out the differences. This way you can
>> see if new instances of a specific class(es) are accumulated over time. In
>> this case you have a memory leak.
>>
>> regards
>> Leon
>>
>>
>>
>>
>>
>>
>>
>>
>> On Thu, Aug 17, 2017 at 12:09 PM, André Warnier (tomcat) <aw...@ice-sa.com>
>> wrote:
>>
>>> On 17.08.2017 11:21, Fady Haikal wrote:
>>>
>>>> Team,
>>>> Please i need some help her
>>>>
>>>
>>> Maybe start here ? http://lmgtfy.com/?q=analyse+tomcat+heap+dump
>>>
>>> (and this looks like it might help you :
>>> http://docs.oracle.com/javase/7/docs/technotes/tools/share/jhat.html)
>>>
>>> To restate the obvious :
>>> - this list here is manned by volunteers, who donate their time trying to
>>> help people who have problems with Tomcat, and who are doing their best,
>>> but
>>> - the list works best with people who also help themselves
>>> - we do not have access to your system, and we do not know your
>>> applications
>>> - we can only try to help, on the base of the information which you
>>> provide
>>> - there are 3 (possible) components to your problem :
>>>    a) the Java JVM which runs Tomcat
>>>    b) Tomcat itself, which runs your applications
>>>    c) the applications themselves
>>> Of these :
>>>    a) is controlled entirely by you, and is in fact the one which
>>> ultimately uses all that memory (but we do not know why either). Below
>>> you
>>> are showing the startup parameters of the JVM. They are not the default
>>> settings. So where do these parameters come from, and do you understand
>>> them ? (if you don't, you first need help with Java, not with Tomcat)
>>>    b) That is really the focus of this list. And we know that Tomcat, by
>>> itself, does not use 60GB or 10GB of memory, nor anywhere near it
>>>    c) we know nothing about, but it is likely that it is something there,
>>> which causes Tomcat (and Java) to use all that memory
>>> And about (c), maybe it is normal that it/they use this gigantic amount
>>> of
>>> memory. Maybe one of the applications allocates some gigantic array of
>>> data
>>> over time. Maybe one of these applications is really badly programmed and
>>> causes enormous "memory leaks". How would we know ? Are these your
>>> applications ? If not, have you contacted the people responsible for
>>> these
>>> applications, and described your problem to them ?
>>> (because again, with 99& probability, that is where the problem is)
>>>
>>> Try to collect some more specific data, and when you have it, post it
>>> here, and maybe someone will have an idea.  But as it is, the only thing
>>> known so far is that on your system, the Java JVM which runs tomcat is
>>> using up a lot of memory, increasingly over time, and that it ends up
>>> crashing with an OOM.
>>> That is not enough information for us to be able to help you.
>>>
>>> If you are desperate and in a hurry, maybe you need first to find a Java
>>> JVM specialist, who can help you poinpoint the problem more specifically.
>>>
>>>
>>>
>>>
>>>> Regards,
>>>>
>>>>
>>>> On Thu, Aug 17, 2017 at 9:46 AM, Fady Haikal <fa...@gmail.com>
>>>> wrote:
>>>>
>>>> @Suvendu,
>>>>>
>>>>> I took a heap dump from Java VisualVM but honestly i didnt know how i
>>>>> should analyse it, please some help here
>>>>>
>>>>> also please find below the java configuration i used:
>>>>> -XX:PermSize=10240m
>>>>> -XX:MaxPermSize=10240m
>>>>> -XX:ReservedCodeCacheSize=512m
>>>>> -XX:+UseParNewGC
>>>>> -XX:+UseConcMarkSweepGC
>>>>> -XX:+CMSParallelRemarkEnabled
>>>>> -XX:TargetSurvivorRatio=90
>>>>>
>>>>> Initial and Max Memory Pool: 100000MB
>>>>>
>>>>> Regards,
>>>>> Fady
>>>>>
>>>>>
>>>>> On Wed, Aug 16, 2017 at 5:46 PM, Suvendu Sekhar Mondal <
>>>>> suv3ndu@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Hi Fady,
>>>>>>
>>>>>>
>>>>>> On Aug 16, 2017 7:04 PM, "Fady Haikal" <fa...@gmail.com> wrote:
>>>>>>
>>>>>> Dear Team,
>>>>>> I'm facing an issue that tomcat from task manager is consuming around
>>>>>> 60
>>>>>> GB
>>>>>> of memory while from Oracle Java Mission Control is showing maximum 10
>>>>>> GB
>>>>>> (Attached screenshots), and from time to time the server hang due to
>>>>>> insufficent memory.
>>>>>> can you please advise why the above is showing like that? and why the
>>>>>> memory of the server is not reducing?
>>>>>>
>>>>>> Tomcat Version: 8.0.30
>>>>>> JDK: 1.7.0_67
>>>>>>
>>>>>>
>>>>>> I couldn't find the screenshot. Anyway, the issue you mentioned is
>>>>>> very
>>>>>> interesting. Usually I saw some difference(in MB) between task manager
>>>>>> data
>>>>>> and JMC data...but not like this.  Can you please let us know about
>>>>>> the
>>>>>> Java Heap, Permgen and thread stack size configuration?
>>>>>>
>>>>>> For starter, I will suggest to take a heap dump and look into it. It
>>>>>> will
>>>>>> show you which objects are live and holding the memory. Also it will
>>>>>> be
>>>>>> awesome if you can enable Native Memory Tracking for your TC instance.
>>>>>>
>>>>>> Thanks!
>>>>>> Suvendu
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>>> For additional commands, e-mail: users-help@tomcat.apache.org
>>>
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat memory

Posted by "André Warnier (tomcat)" <aw...@ice-sa.com>.
On 17.08.2017 12:57, Leon Rosenberg wrote:
> Fady, one thing,
>
> analyzing heap dumps is hard especially a 10GB dump, you will need at least
> 40 Gb of memory an about 10 hours to start jhat.
> What is fast is analyzing a histogram. A histogram is a list of all classes
> in your JVM and amount of memory they use. It is very easy to use:
>
> jmap -histo:live <pid> >outputfile

According to this however :
https://eclipsesource.com/blogs/2013/01/31/creating-tomcat-heap-dumps-on-windows/
jmap does not work under Windows, if Tomcat is running as a Service
(seems likely in the OP's case)
I don't know if that's true, have not checked it.

> the :live parameter is important, otherwise you will see dead objects as
> well. Keep in mind a major gc will be triggered.
>
> The output looks like this:
>
>   num     #instances         #bytes  class name
> ----------------------------------------------
>     1:         98573       13165976  [C
>     2:          4376       11325928  [B
>     3:         96526        2316624  java.lang.String
>     4:         40522        1296704  java.util.HashMap$Node
>     5:         10954         807552  [Ljava.lang.Object;
>     6:          7494         781480  java.lang.Class
>     7:          6778         655464  [Ljava.util.HashMap$Node;
> ...
>
> 3103:             1             16
> sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter
> 3104:             1             16  sun.util.resources.LocaleData
> 3105:             1             16
> sun.util.resources.LocaleData$LocaleDataResourceBundleControl
> Total        456560       37330672
>
> This won't point you to exact direction of memory leak or resource
> consumption, but you will see what classes are consuming memory. So if you
> find out that class foo.bar.XYZBean has 10.000.000 instances and occupies 2
> gb of memory, you know where to look.
>
> Keep in mind that only memory directly used by a class is counted, so every
> string will appear twice, as string and as byte array.
>
> If you see that the used memory amount is growing, take multiple
> histograms, every 10 minutes or so and analyse the differences. I wrote a
> little script years ago for myself (
> https://github.com/dvayanu/ldt/blob/c8b3c2b6f61de5c583db503f6fd5e2d8aa8b9aa0/java/ldt/histo/HistoDiffReader.java)
> that takes two histograms and prints out the differences. This way you can
> see if new instances of a specific class(es) are accumulated over time. In
> this case you have a memory leak.
>
> regards
> Leon
>
>
>
>
>
>
>
>
> On Thu, Aug 17, 2017 at 12:09 PM, André Warnier (tomcat) <aw...@ice-sa.com>
> wrote:
>
>> On 17.08.2017 11:21, Fady Haikal wrote:
>>
>>> Team,
>>> Please i need some help her
>>>
>>
>> Maybe start here ? http://lmgtfy.com/?q=analyse+tomcat+heap+dump
>>
>> (and this looks like it might help you :
>> http://docs.oracle.com/javase/7/docs/technotes/tools/share/jhat.html)
>>
>> To restate the obvious :
>> - this list here is manned by volunteers, who donate their time trying to
>> help people who have problems with Tomcat, and who are doing their best, but
>> - the list works best with people who also help themselves
>> - we do not have access to your system, and we do not know your
>> applications
>> - we can only try to help, on the base of the information which you provide
>> - there are 3 (possible) components to your problem :
>>    a) the Java JVM which runs Tomcat
>>    b) Tomcat itself, which runs your applications
>>    c) the applications themselves
>> Of these :
>>    a) is controlled entirely by you, and is in fact the one which
>> ultimately uses all that memory (but we do not know why either). Below you
>> are showing the startup parameters of the JVM. They are not the default
>> settings. So where do these parameters come from, and do you understand
>> them ? (if you don't, you first need help with Java, not with Tomcat)
>>    b) That is really the focus of this list. And we know that Tomcat, by
>> itself, does not use 60GB or 10GB of memory, nor anywhere near it
>>    c) we know nothing about, but it is likely that it is something there,
>> which causes Tomcat (and Java) to use all that memory
>> And about (c), maybe it is normal that it/they use this gigantic amount of
>> memory. Maybe one of the applications allocates some gigantic array of data
>> over time. Maybe one of these applications is really badly programmed and
>> causes enormous "memory leaks". How would we know ? Are these your
>> applications ? If not, have you contacted the people responsible for these
>> applications, and described your problem to them ?
>> (because again, with 99& probability, that is where the problem is)
>>
>> Try to collect some more specific data, and when you have it, post it
>> here, and maybe someone will have an idea.  But as it is, the only thing
>> known so far is that on your system, the Java JVM which runs tomcat is
>> using up a lot of memory, increasingly over time, and that it ends up
>> crashing with an OOM.
>> That is not enough information for us to be able to help you.
>>
>> If you are desperate and in a hurry, maybe you need first to find a Java
>> JVM specialist, who can help you poinpoint the problem more specifically.
>>
>>
>>
>>
>>> Regards,
>>>
>>>
>>> On Thu, Aug 17, 2017 at 9:46 AM, Fady Haikal <fa...@gmail.com>
>>> wrote:
>>>
>>> @Suvendu,
>>>> I took a heap dump from Java VisualVM but honestly i didnt know how i
>>>> should analyse it, please some help here
>>>>
>>>> also please find below the java configuration i used:
>>>> -XX:PermSize=10240m
>>>> -XX:MaxPermSize=10240m
>>>> -XX:ReservedCodeCacheSize=512m
>>>> -XX:+UseParNewGC
>>>> -XX:+UseConcMarkSweepGC
>>>> -XX:+CMSParallelRemarkEnabled
>>>> -XX:TargetSurvivorRatio=90
>>>>
>>>> Initial and Max Memory Pool: 100000MB
>>>>
>>>> Regards,
>>>> Fady
>>>>
>>>>
>>>> On Wed, Aug 16, 2017 at 5:46 PM, Suvendu Sekhar Mondal <
>>>> suv3ndu@gmail.com>
>>>> wrote:
>>>>
>>>> Hi Fady,
>>>>>
>>>>> On Aug 16, 2017 7:04 PM, "Fady Haikal" <fa...@gmail.com> wrote:
>>>>>
>>>>> Dear Team,
>>>>> I'm facing an issue that tomcat from task manager is consuming around 60
>>>>> GB
>>>>> of memory while from Oracle Java Mission Control is showing maximum 10
>>>>> GB
>>>>> (Attached screenshots), and from time to time the server hang due to
>>>>> insufficent memory.
>>>>> can you please advise why the above is showing like that? and why the
>>>>> memory of the server is not reducing?
>>>>>
>>>>> Tomcat Version: 8.0.30
>>>>> JDK: 1.7.0_67
>>>>>
>>>>>
>>>>> I couldn't find the screenshot. Anyway​, the issue you mentioned is very
>>>>> interesting. Usually I saw some difference(in MB) between task manager
>>>>> data
>>>>> and JMC data...but not like this.  Can you please let us know about the
>>>>> Java Heap, Permgen and thread stack size configuration?
>>>>>
>>>>> For starter, I will suggest to take a heap dump and look into it. It
>>>>> will
>>>>> show you which objects are live and holding the memory. Also it will be
>>>>> awesome if you can enable Native Memory Tracking for your TC instance.
>>>>>
>>>>> Thanks!
>>>>> Suvendu
>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat memory

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Leon,

On 8/17/17 6:57 AM, Leon Rosenberg wrote:
> Fady, one thing,
> 
> analyzing heap dumps is hard especially a 10GB dump, you will need
> at least 40 Gb of memory an about 10 hours to start jhat. What is
> fast is analyzing a histogram. A histogram is a list of all
> classes in your JVM and amount of memory they use. It is very easy
> to use:
> 
> jmap -histo:live <pid> >outputfile the :live parameter is
> important, otherwise you will see dead objects as well. Keep in
> mind a major gc will be triggered.
> 
> The output looks like this:
> 
> num     #instances         #bytes  class name 
> ---------------------------------------------- 1:         98573
> 13165976  [C 2:          4376       11325928  [B 3:         96526
> 2316624  java.lang.String 4:         40522        1296704
> java.util.HashMap$Node 5:         10954         807552
> [Ljava.lang.Object; 6:          7494         781480
> java.lang.Class 7:          6778         655464
> [Ljava.util.HashMap$Node; ...
> 
> 3103:             1             16 
> sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter 
> 3104:             1             16  sun.util.resources.LocaleData 
> 3105:             1             16 
> sun.util.resources.LocaleData$LocaleDataResourceBundleControl Total
> 456560       37330672
> 
> This won't point you to exact direction of memory leak or resource 
> consumption, but you will see what classes are consuming memory. So
> if you find out that class foo.bar.XYZBean has 10.000.000 instances
> and occupies 2 gb of memory, you know where to look.
> 
> Keep in mind that only memory directly used by a class is counted,
> so every string will appear twice, as string and as byte array.
> 
> If you see that the used memory amount is growing, take multiple 
> histograms, every 10 minutes or so and analyse the differences. I
> wrote a little script years ago for myself ( 
> https://github.com/dvayanu/ldt/blob/c8b3c2b6f61de5c583db503f6fd5e2d8aa
8b9aa0/java/ldt/histo/HistoDiffReader.java)
>
> 
that takes two histograms and prints out the differences. This way you c
an
> see if new instances of a specific class(es) are accumulated over
> time. In this case you have a memory leak.

The OP's report sounds more like a great disparity between native
memory versus heap memory usage. The Java heap, of course, is only a
part of the actual (virtual) RAM used by the process; there are plenty
of non-heap memory resources required by the JVM for everything. I've
not found a great tool for inspecting native memory spaces in a JVM.

My recommendation would be that the OP make sure he/she has the latest
possible version of the JVM available, just in case there is a memory
leak in the JVM itself. The same should be done with all other
libraries in case any memory leaks exist and have been fixed.

- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlmW7WAACgkQHPApP6U8
pFhd9RAAsjPO/BTFX2RRZH0SRry2b0Pb1tMlGwPBW0tEesQRnLn+zS5l3FaSrwJJ
7Ar9fJhcwIiGZdDq15hURg9vVJFoEEfh3jhhOTH8DafDFfjneEnwzrvUcdyyrlpL
8tyXByTpk0H51HPaEkhm0YJlSY6rSQuVOfVNOY+HrFSQiDxidqU5l5igUdnmczk1
0Z+AXuOB+oF1E1XkihPfDpX1TwsImuGKylBe7Rmng4KWA8J8d9Xo17bXH2DfgDOm
zt0iSTYZ3Hus1T84zpO4qCm9NWvYvROMACcTweYJ8VsgXkjnacVZhSP4NA3vlGF4
TuzxnyUWngbB82ERoiD0BY99406fcxvAyyJ1H1EQ7Z76UlRWKaQgGWK3U/Fb4kI0
6ugXV9daNmGYGar9rHmSGccUzkyqzOnjinFOF3OYaAs8eyFPvXlB0TvXWbQUQESe
Jx4YK+vmtGVIUlM/c4selCPPKpryOkRfjjqBdDOnrJYWbp6bYlMtRpzO19TpCOIA
qve6SWCVhEhzviXbW+xs0GH2QQHuveDzc5XIZC7ysX3Y5kTdJlOvH0j9pWCeMaQ5
1tAKj2jiI/m9VirjsFpxC0hy8lRB1wqrJQdaIRwWtaMEO7zVy20igyEMrw/tXOcv
6K14O6hMQPqaE1ge8xFk+IzEzyceaCzmv3xw8C5FRlIntzlFFBM=
=KFMx
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat memory

Posted by Leon Rosenberg <ro...@gmail.com>.
Fady, one thing,

analyzing heap dumps is hard especially a 10GB dump, you will need at least
40 Gb of memory an about 10 hours to start jhat.
What is fast is analyzing a histogram. A histogram is a list of all classes
in your JVM and amount of memory they use. It is very easy to use:

jmap -histo:live <pid> >outputfile
the :live parameter is important, otherwise you will see dead objects as
well. Keep in mind a major gc will be triggered.

The output looks like this:

 num     #instances         #bytes  class name
----------------------------------------------
   1:         98573       13165976  [C
   2:          4376       11325928  [B
   3:         96526        2316624  java.lang.String
   4:         40522        1296704  java.util.HashMap$Node
   5:         10954         807552  [Ljava.lang.Object;
   6:          7494         781480  java.lang.Class
   7:          6778         655464  [Ljava.util.HashMap$Node;
...

3103:             1             16
sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter
3104:             1             16  sun.util.resources.LocaleData
3105:             1             16
sun.util.resources.LocaleData$LocaleDataResourceBundleControl
Total        456560       37330672

This won't point you to exact direction of memory leak or resource
consumption, but you will see what classes are consuming memory. So if you
find out that class foo.bar.XYZBean has 10.000.000 instances and occupies 2
gb of memory, you know where to look.

Keep in mind that only memory directly used by a class is counted, so every
string will appear twice, as string and as byte array.

If you see that the used memory amount is growing, take multiple
histograms, every 10 minutes or so and analyse the differences. I wrote a
little script years ago for myself (
https://github.com/dvayanu/ldt/blob/c8b3c2b6f61de5c583db503f6fd5e2d8aa8b9aa0/java/ldt/histo/HistoDiffReader.java)
that takes two histograms and prints out the differences. This way you can
see if new instances of a specific class(es) are accumulated over time. In
this case you have a memory leak.

regards
Leon








On Thu, Aug 17, 2017 at 12:09 PM, André Warnier (tomcat) <aw...@ice-sa.com>
wrote:

> On 17.08.2017 11:21, Fady Haikal wrote:
>
>> Team,
>> Please i need some help her
>>
>
> Maybe start here ? http://lmgtfy.com/?q=analyse+tomcat+heap+dump
>
> (and this looks like it might help you :
> http://docs.oracle.com/javase/7/docs/technotes/tools/share/jhat.html)
>
> To restate the obvious :
> - this list here is manned by volunteers, who donate their time trying to
> help people who have problems with Tomcat, and who are doing their best, but
> - the list works best with people who also help themselves
> - we do not have access to your system, and we do not know your
> applications
> - we can only try to help, on the base of the information which you provide
> - there are 3 (possible) components to your problem :
>   a) the Java JVM which runs Tomcat
>   b) Tomcat itself, which runs your applications
>   c) the applications themselves
> Of these :
>   a) is controlled entirely by you, and is in fact the one which
> ultimately uses all that memory (but we do not know why either). Below you
> are showing the startup parameters of the JVM. They are not the default
> settings. So where do these parameters come from, and do you understand
> them ? (if you don't, you first need help with Java, not with Tomcat)
>   b) That is really the focus of this list. And we know that Tomcat, by
> itself, does not use 60GB or 10GB of memory, nor anywhere near it
>   c) we know nothing about, but it is likely that it is something there,
> which causes Tomcat (and Java) to use all that memory
> And about (c), maybe it is normal that it/they use this gigantic amount of
> memory. Maybe one of the applications allocates some gigantic array of data
> over time. Maybe one of these applications is really badly programmed and
> causes enormous "memory leaks". How would we know ? Are these your
> applications ? If not, have you contacted the people responsible for these
> applications, and described your problem to them ?
> (because again, with 99& probability, that is where the problem is)
>
> Try to collect some more specific data, and when you have it, post it
> here, and maybe someone will have an idea.  But as it is, the only thing
> known so far is that on your system, the Java JVM which runs tomcat is
> using up a lot of memory, increasingly over time, and that it ends up
> crashing with an OOM.
> That is not enough information for us to be able to help you.
>
> If you are desperate and in a hurry, maybe you need first to find a Java
> JVM specialist, who can help you poinpoint the problem more specifically.
>
>
>
>
>> Regards,
>>
>>
>> On Thu, Aug 17, 2017 at 9:46 AM, Fady Haikal <fa...@gmail.com>
>> wrote:
>>
>> @Suvendu,
>>> I took a heap dump from Java VisualVM but honestly i didnt know how i
>>> should analyse it, please some help here
>>>
>>> also please find below the java configuration i used:
>>> -XX:PermSize=10240m
>>> -XX:MaxPermSize=10240m
>>> -XX:ReservedCodeCacheSize=512m
>>> -XX:+UseParNewGC
>>> -XX:+UseConcMarkSweepGC
>>> -XX:+CMSParallelRemarkEnabled
>>> -XX:TargetSurvivorRatio=90
>>>
>>> Initial and Max Memory Pool: 100000MB
>>>
>>> Regards,
>>> Fady
>>>
>>>
>>> On Wed, Aug 16, 2017 at 5:46 PM, Suvendu Sekhar Mondal <
>>> suv3ndu@gmail.com>
>>> wrote:
>>>
>>> Hi Fady,
>>>>
>>>> On Aug 16, 2017 7:04 PM, "Fady Haikal" <fa...@gmail.com> wrote:
>>>>
>>>> Dear Team,
>>>> I'm facing an issue that tomcat from task manager is consuming around 60
>>>> GB
>>>> of memory while from Oracle Java Mission Control is showing maximum 10
>>>> GB
>>>> (Attached screenshots), and from time to time the server hang due to
>>>> insufficent memory.
>>>> can you please advise why the above is showing like that? and why the
>>>> memory of the server is not reducing?
>>>>
>>>> Tomcat Version: 8.0.30
>>>> JDK: 1.7.0_67
>>>>
>>>>
>>>> I couldn't find the screenshot. Anyway​, the issue you mentioned is very
>>>> interesting. Usually I saw some difference(in MB) between task manager
>>>> data
>>>> and JMC data...but not like this.  Can you please let us know about the
>>>> Java Heap, Permgen and thread stack size configuration?
>>>>
>>>> For starter, I will suggest to take a heap dump and look into it. It
>>>> will
>>>> show you which objects are live and holding the memory. Also it will be
>>>> awesome if you can enable Native Memory Tracking for your TC instance.
>>>>
>>>> Thanks!
>>>> Suvendu
>>>>
>>>>
>>>
>>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>

Re: Tomcat memory

Posted by "André Warnier (tomcat)" <aw...@ice-sa.com>.
On 17.08.2017 11:21, Fady Haikal wrote:
> Team,
> Please i need some help her

Maybe start here ? http://lmgtfy.com/?q=analyse+tomcat+heap+dump

(and this looks like it might help you :
http://docs.oracle.com/javase/7/docs/technotes/tools/share/jhat.html)

To restate the obvious :
- this list here is manned by volunteers, who donate their time trying to help people who 
have problems with Tomcat, and who are doing their best, but
- the list works best with people who also help themselves
- we do not have access to your system, and we do not know your applications
- we can only try to help, on the base of the information which you provide
- there are 3 (possible) components to your problem :
   a) the Java JVM which runs Tomcat
   b) Tomcat itself, which runs your applications
   c) the applications themselves
Of these :
   a) is controlled entirely by you, and is in fact the one which ultimately uses all that 
memory (but we do not know why either). Below you are showing the startup parameters of 
the JVM. They are not the default settings. So where do these parameters come from, and do 
you understand them ? (if you don't, you first need help with Java, not with Tomcat)
   b) That is really the focus of this list. And we know that Tomcat, by itself, does not 
use 60GB or 10GB of memory, nor anywhere near it
   c) we know nothing about, but it is likely that it is something there, which causes 
Tomcat (and Java) to use all that memory
And about (c), maybe it is normal that it/they use this gigantic amount of memory. Maybe 
one of the applications allocates some gigantic array of data over time. Maybe one of 
these applications is really badly programmed and causes enormous "memory leaks". How 
would we know ? Are these your applications ? If not, have you contacted the people 
responsible for these applications, and described your problem to them ?
(because again, with 99& probability, that is where the problem is)

Try to collect some more specific data, and when you have it, post it here, and maybe 
someone will have an idea.  But as it is, the only thing known so far is that on your 
system, the Java JVM which runs tomcat is using up a lot of memory, increasingly over 
time, and that it ends up crashing with an OOM.
That is not enough information for us to be able to help you.

If you are desperate and in a hurry, maybe you need first to find a Java JVM specialist, 
who can help you poinpoint the problem more specifically.


>
> Regards,
>
>
> On Thu, Aug 17, 2017 at 9:46 AM, Fady Haikal <fa...@gmail.com> wrote:
>
>> @Suvendu,
>> I took a heap dump from Java VisualVM but honestly i didnt know how i
>> should analyse it, please some help here
>>
>> also please find below the java configuration i used:
>> -XX:PermSize=10240m
>> -XX:MaxPermSize=10240m
>> -XX:ReservedCodeCacheSize=512m
>> -XX:+UseParNewGC
>> -XX:+UseConcMarkSweepGC
>> -XX:+CMSParallelRemarkEnabled
>> -XX:TargetSurvivorRatio=90
>>
>> Initial and Max Memory Pool: 100000MB
>>
>> Regards,
>> Fady
>>
>>
>> On Wed, Aug 16, 2017 at 5:46 PM, Suvendu Sekhar Mondal <su...@gmail.com>
>> wrote:
>>
>>> Hi Fady,
>>>
>>> On Aug 16, 2017 7:04 PM, "Fady Haikal" <fa...@gmail.com> wrote:
>>>
>>> Dear Team,
>>> I'm facing an issue that tomcat from task manager is consuming around 60
>>> GB
>>> of memory while from Oracle Java Mission Control is showing maximum 10 GB
>>> (Attached screenshots), and from time to time the server hang due to
>>> insufficent memory.
>>> can you please advise why the above is showing like that? and why the
>>> memory of the server is not reducing?
>>>
>>> Tomcat Version: 8.0.30
>>> JDK: 1.7.0_67
>>>
>>>
>>> I couldn't find the screenshot. Anyway​, the issue you mentioned is very
>>> interesting. Usually I saw some difference(in MB) between task manager
>>> data
>>> and JMC data...but not like this.  Can you please let us know about the
>>> Java Heap, Permgen and thread stack size configuration?
>>>
>>> For starter, I will suggest to take a heap dump and look into it. It will
>>> show you which objects are live and holding the memory. Also it will be
>>> awesome if you can enable Native Memory Tracking for your TC instance.
>>>
>>> Thanks!
>>> Suvendu
>>>
>>
>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Tomcat memory

Posted by Fady Haikal <fa...@gmail.com>.
Team,
Please i need some help her

Regards,


On Thu, Aug 17, 2017 at 9:46 AM, Fady Haikal <fa...@gmail.com> wrote:

> @Suvendu,
> I took a heap dump from Java VisualVM but honestly i didnt know how i
> should analyse it, please some help here
>
> also please find below the java configuration i used:
> -XX:PermSize=10240m
> -XX:MaxPermSize=10240m
> -XX:ReservedCodeCacheSize=512m
> -XX:+UseParNewGC
> -XX:+UseConcMarkSweepGC
> -XX:+CMSParallelRemarkEnabled
> -XX:TargetSurvivorRatio=90
>
> Initial and Max Memory Pool: 100000MB
>
> Regards,
> Fady
>
>
> On Wed, Aug 16, 2017 at 5:46 PM, Suvendu Sekhar Mondal <su...@gmail.com>
> wrote:
>
>> Hi Fady,
>>
>> On Aug 16, 2017 7:04 PM, "Fady Haikal" <fa...@gmail.com> wrote:
>>
>> Dear Team,
>> I'm facing an issue that tomcat from task manager is consuming around 60
>> GB
>> of memory while from Oracle Java Mission Control is showing maximum 10 GB
>> (Attached screenshots), and from time to time the server hang due to
>> insufficent memory.
>> can you please advise why the above is showing like that? and why the
>> memory of the server is not reducing?
>>
>> Tomcat Version: 8.0.30
>> JDK: 1.7.0_67
>>
>>
>> I couldn't find the screenshot. Anyway​, the issue you mentioned is very
>> interesting. Usually I saw some difference(in MB) between task manager
>> data
>> and JMC data...but not like this.  Can you please let us know about the
>> Java Heap, Permgen and thread stack size configuration?
>>
>> For starter, I will suggest to take a heap dump and look into it. It will
>> show you which objects are live and holding the memory. Also it will be
>> awesome if you can enable Native Memory Tracking for your TC instance.
>>
>> Thanks!
>> Suvendu
>>
>
>

Re: Tomcat memory

Posted by Fady Haikal <fa...@gmail.com>.
@Suvendu,
I took a heap dump from Java VisualVM but honestly i didnt know how i
should analyse it, please some help here

also please find below the java configuration i used:
-XX:PermSize=10240m
-XX:MaxPermSize=10240m
-XX:ReservedCodeCacheSize=512m
-XX:+UseParNewGC
-XX:+UseConcMarkSweepGC
-XX:+CMSParallelRemarkEnabled
-XX:TargetSurvivorRatio=90

Initial and Max Memory Pool: 100000MB

Regards,
Fady


On Wed, Aug 16, 2017 at 5:46 PM, Suvendu Sekhar Mondal <su...@gmail.com>
wrote:

> Hi Fady,
>
> On Aug 16, 2017 7:04 PM, "Fady Haikal" <fa...@gmail.com> wrote:
>
> Dear Team,
> I'm facing an issue that tomcat from task manager is consuming around 60 GB
> of memory while from Oracle Java Mission Control is showing maximum 10 GB
> (Attached screenshots), and from time to time the server hang due to
> insufficent memory.
> can you please advise why the above is showing like that? and why the
> memory of the server is not reducing?
>
> Tomcat Version: 8.0.30
> JDK: 1.7.0_67
>
>
> I couldn't find the screenshot. Anyway​, the issue you mentioned is very
> interesting. Usually I saw some difference(in MB) between task manager data
> and JMC data...but not like this.  Can you please let us know about the
> Java Heap, Permgen and thread stack size configuration?
>
> For starter, I will suggest to take a heap dump and look into it. It will
> show you which objects are live and holding the memory. Also it will be
> awesome if you can enable Native Memory Tracking for your TC instance.
>
> Thanks!
> Suvendu
>

Re: Tomcat memory

Posted by Suvendu Sekhar Mondal <su...@gmail.com>.
Hi Fady,

On Aug 16, 2017 7:04 PM, "Fady Haikal" <fa...@gmail.com> wrote:

Dear Team,
I'm facing an issue that tomcat from task manager is consuming around 60 GB
of memory while from Oracle Java Mission Control is showing maximum 10 GB
(Attached screenshots), and from time to time the server hang due to
insufficent memory.
can you please advise why the above is showing like that? and why the
memory of the server is not reducing?

Tomcat Version: 8.0.30
JDK: 1.7.0_67


I couldn't find the screenshot. Anyway​, the issue you mentioned is very
interesting. Usually I saw some difference(in MB) between task manager data
and JMC data...but not like this.  Can you please let us know about the
Java Heap, Permgen and thread stack size configuration?

For starter, I will suggest to take a heap dump and look into it. It will
show you which objects are live and holding the memory. Also it will be
awesome if you can enable Native Memory Tracking for your TC instance.

Thanks!
Suvendu