You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Doron Tsur <qb...@gmail.com> on 2011/11/28 19:09:04 UTC
Tomcat None Page Pool creeps up until OS crashes
Hi All
I've been having an issue in one of our production servers. The Non Paged
Pool creeps up slowly until the system hangs and than crash.
Tomcat: 6.0.16
Java: 1.5.16
OS: Windows Server 2003 R2 SP2
This is a production server that uses high bandwidth. Poking around
Poolmon.exe I was able to find that
1. the AfdP tag is consuming a lot of NP (Non Paged) pool.
2. This is a tag concerning the windows socket driver (afd.sys).
3. I have seen an Microsoft hotfix which seems to be related to this issue
http://support.microsoft.com/kb/931311 . The only issue is that the afd.sys
file mention in the hotfix is from a previous version from the one
installed. 5.2.3790.4008 vs 5.2.3790.4898
We have many production server running our tomcat. We have only encountered
this issue in two of them, those servers have the same OS/Java/Tomcat
installed. One server uses high bandwidth and the other uses much less.
Here I am mumbling about Microsoft and win socket what do I want from you
guys?
As far as I understand the issue is can happen in 3 areas. The java part
interfacing with the driver, the driver it's self and the network card
interfacing with the driver. I say tomcat because this is the process that
consumes all NP pool (about 170MB). I would like to explore with you guys
the Tomcat to winsock option
1. Have you ever encountered this issue?
2. Do you know of any application level/Tomcat configuration work around to
solve this issue?
3. Do you think that there is something wrong in the way Tomcat interfaces
with winsock?
4. Do you believe that Tomcat APR might resolve this issue?
Any tip or information about this issue will be highly appreciated, I've
scattered the web and banging my head over this issue for a long time.
Good Day,
qballer
Re: Tomcat None Page Pool creeps up until OS crashes
Posted by Pid <pi...@pidster.com>.
On 29/11/2011 09:32, Doron Tsur wrote:
> On Tue, Nov 29, 2011 at 9:52 AM, Pid * <pi...@pidster.com> wrote:
>
>> On 28 Nov 2011, at 18:09, Doron Tsur <qb...@gmail.com> wrote:
>>
>>> Hi All
>>>
>>> I've been having an issue in one of our production servers. The Non Paged
>>> Pool creeps up slowly until the system hangs and than crash.
>>>
>>> Tomcat: 6.0.16
>>
>> So the latest release is 6.0.33. Yours is over 3 years old.
>>
>>> Java: 1.5.16
>>
>> Java 5 is no longer supported, and your version is old, even so.
>>
>>> OS: Windows Server 2003 R2 SP2
>>>
>>> This is a production server that uses high bandwidth. Poking around
>>> Poolmon.exe I was able to find that
>>>
>>> 1. the AfdP tag is consuming a lot of NP (Non Paged) pool.
>>> 2. This is a tag concerning the windows socket driver (afd.sys).
>>> 3. I have seen an Microsoft hotfix which seems to be related to this
>> issue
>>> http://support.microsoft.com/kb/931311 . The only issue is that the
>> afd.sys
>>> file mention in the hotfix is from a previous version from the one
>>> installed. 5.2.3790.4008 vs 5.2.3790.4898
>>
>> ... right.
I re-read the above KB article. That patch is dated 2007. Are you sure
it isn't already installed on your system?
If you haven't patched that server since 2007 you have bigger problems
than one Java process consuming more memory than expected.
>>> We have many production server running our tomcat.
Excellent. Hopefully you'll be updating them to a more recent version
than the pre-stable one you're using now?
>> We have only encountered
>>> this issue in two of them, those servers have the same OS/Java/Tomcat
>>> installed. One server uses high bandwidth and the other uses much less.
So does the other one crash too?
>>> Here I am mumbling about Microsoft and win socket what do I want from you
>>> guys?
>>
>> Good question.
>>
>>> As far as I understand the issue is can happen in 3 areas. The java part
>>> interfacing with the driver, the driver it's self and the network card
>>> interfacing with the driver. I say tomcat because this is the process
>> that
>>> consumes all NP pool (about 170MB). I would like to explore with you
>> guys
>>> the Tomcat to winsock option
>>>
>>> 1. Have you ever encountered this issue?
>>
>> Which issue, exactly?
> Issue is Non Paged pool of tomcat increases until the OS crashes
Please enable JMX and use a tool like JConsole (found in the JDK/bin
directory) to connect to Tomcat and observe the memory used in different
areas of the process.
http://tomcat.apache.org/tomcat-6.0-doc/monitoring.html
As per my suggestion, below.
p
>>> 2. Do you know of any application level/Tomcat configuration work around
>> to
>>> solve this issue?
>>> 3. Do you think that there is something wrong in the way Tomcat
>> interfaces
>>> with winsock?
>>
>> Tomcat doesn't. Java does.
>>
>>> 4. Do you believe that Tomcat APR might resolve this issue?
>>
>> Not sure I understand what the issue is.
>>
>>> Any tip or information about this issue will be highly appreciated, I've
>>> scattered the web and banging my head over this issue for a long time.
>>
>> How are you starting Tomcat, e.g. using the service wrapper?
>>
>> Enable JMX (there's notes on tomcat.apache.org) and use JConsole to
>> observe memory usage inside the JVM.
>>
>> If any of the JVM memory areas shows that it's consuming more memory,
>> the app has a memory leak.
>>
>> If not, you have confirmed that the issue is outside Java.
>>
>>
>> p
>>
>>
>>> Good Day,
>>> qballer
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
--
[key:62590808]
Re: Tomcat None Page Pool creeps up until OS crashes
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Doron,
On 11/29/11 4:32 AM, Doron Tsur wrote:
> Issue is Non Paged pool of tomcat increases until the OS crashes
Wow, the OS crashes? That really sounds like an OS bug to me.
Why do you suspect afd.dll is the cause (or symptom) of this problem?
Do you have a core dump file or something like that?
Please provide:
* Tomcat configuration (specifically, the <Connector> definitions)
* JVM settings (specifically, those connected to memory usage, GC, etc.)
* Any limits that are by default or explicitly set for things like
buffers by the OS for your JVM process
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk7U9nYACgkQ9CaO5/Lv0PCKUACggB+ofV3fMsL5s9yJPCh4zwoR
dbwAoIGDMYc8kERwW+IjMjcK3CF1oclT
=SuU9
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat None Page Pool creeps up until OS crashes
Posted by Pid <pi...@pidster.com>.
On 29/11/2011 09:32, Doron Tsur wrote:
> Issue is Non Paged pool of tomcat increases until the OS crashes
Please post replies below the question so people can follow the
conversation.
p
--
[key:62590808]
Re: Tomcat None Page Pool creeps up until OS crashes
Posted by Doron Tsur <qb...@gmail.com>.
Issue is Non Paged pool of tomcat increases until the OS crashes
On Tue, Nov 29, 2011 at 9:52 AM, Pid * <pi...@pidster.com> wrote:
> On 28 Nov 2011, at 18:09, Doron Tsur <qb...@gmail.com> wrote:
>
> > Hi All
> >
> > I've been having an issue in one of our production servers. The Non Paged
> > Pool creeps up slowly until the system hangs and than crash.
> >
> > Tomcat: 6.0.16
>
> So the latest release is 6.0.33. Yours is over 3 years old.
>
> > Java: 1.5.16
>
> Java 5 is no longer supported, and your version is old, even so.
>
> > OS: Windows Server 2003 R2 SP2
> >
> > This is a production server that uses high bandwidth. Poking around
> > Poolmon.exe I was able to find that
> >
> > 1. the AfdP tag is consuming a lot of NP (Non Paged) pool.
> > 2. This is a tag concerning the windows socket driver (afd.sys).
> > 3. I have seen an Microsoft hotfix which seems to be related to this
> issue
> > http://support.microsoft.com/kb/931311 . The only issue is that the
> afd.sys
> > file mention in the hotfix is from a previous version from the one
> > installed. 5.2.3790.4008 vs 5.2.3790.4898
>
> ... right.
>
>
> > We have many production server running our tomcat. We have only
> encountered
> > this issue in two of them, those servers have the same OS/Java/Tomcat
> > installed. One server uses high bandwidth and the other uses much less.
> >
> > Here I am mumbling about Microsoft and win socket what do I want from you
> > guys?
>
> Good question.
>
> > As far as I understand the issue is can happen in 3 areas. The java part
> > interfacing with the driver, the driver it's self and the network card
> > interfacing with the driver. I say tomcat because this is the process
> that
> > consumes all NP pool (about 170MB). I would like to explore with you
> guys
> > the Tomcat to winsock option
> >
> > 1. Have you ever encountered this issue?
>
> Which issue, exactly?
>
> > 2. Do you know of any application level/Tomcat configuration work around
> to
> > solve this issue?
> > 3. Do you think that there is something wrong in the way Tomcat
> interfaces
> > with winsock?
>
> Tomcat doesn't. Java does.
>
> > 4. Do you believe that Tomcat APR might resolve this issue?
>
> Not sure I understand what the issue is.
>
> > Any tip or information about this issue will be highly appreciated, I've
> > scattered the web and banging my head over this issue for a long time.
>
> How are you starting Tomcat, e.g. using the service wrapper?
>
> Enable JMX (there's notes on tomcat.apache.org) and use JConsole to
> observe memory usage inside the JVM.
>
> If any of the JVM memory areas shows that it's consuming more memory,
> the app has a memory leak.
>
> If not, you have confirmed that the issue is outside Java.
>
>
> p
>
>
> > Good Day,
> > qballer
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Tomcat None Page Pool creeps up until OS crashes
Posted by Pid * <pi...@pidster.com>.
On 28 Nov 2011, at 18:09, Doron Tsur <qb...@gmail.com> wrote:
> Hi All
>
> I've been having an issue in one of our production servers. The Non Paged
> Pool creeps up slowly until the system hangs and than crash.
>
> Tomcat: 6.0.16
So the latest release is 6.0.33. Yours is over 3 years old.
> Java: 1.5.16
Java 5 is no longer supported, and your version is old, even so.
> OS: Windows Server 2003 R2 SP2
>
> This is a production server that uses high bandwidth. Poking around
> Poolmon.exe I was able to find that
>
> 1. the AfdP tag is consuming a lot of NP (Non Paged) pool.
> 2. This is a tag concerning the windows socket driver (afd.sys).
> 3. I have seen an Microsoft hotfix which seems to be related to this issue
> http://support.microsoft.com/kb/931311 . The only issue is that the afd.sys
> file mention in the hotfix is from a previous version from the one
> installed. 5.2.3790.4008 vs 5.2.3790.4898
... right.
> We have many production server running our tomcat. We have only encountered
> this issue in two of them, those servers have the same OS/Java/Tomcat
> installed. One server uses high bandwidth and the other uses much less.
>
> Here I am mumbling about Microsoft and win socket what do I want from you
> guys?
Good question.
> As far as I understand the issue is can happen in 3 areas. The java part
> interfacing with the driver, the driver it's self and the network card
> interfacing with the driver. I say tomcat because this is the process that
> consumes all NP pool (about 170MB). I would like to explore with you guys
> the Tomcat to winsock option
>
> 1. Have you ever encountered this issue?
Which issue, exactly?
> 2. Do you know of any application level/Tomcat configuration work around to
> solve this issue?
> 3. Do you think that there is something wrong in the way Tomcat interfaces
> with winsock?
Tomcat doesn't. Java does.
> 4. Do you believe that Tomcat APR might resolve this issue?
Not sure I understand what the issue is.
> Any tip or information about this issue will be highly appreciated, I've
> scattered the web and banging my head over this issue for a long time.
How are you starting Tomcat, e.g. using the service wrapper?
Enable JMX (there's notes on tomcat.apache.org) and use JConsole to
observe memory usage inside the JVM.
If any of the JVM memory areas shows that it's consuming more memory,
the app has a memory leak.
If not, you have confirmed that the issue is outside Java.
p
> Good Day,
> qballer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org