You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Anoop kumar V <an...@gmail.com> on 2005/03/07 17:20:07 UTC

Tomcat 4.1.30 crashing on production site - need help

Hi All,

For the past couple of months we have been facing a peculiar problem
on our production environment. And we have been unable to resolve this
issue so far.

We have an application hosted on Tomcat 4.1.30 and using JDK1.4.2_06.
We also have an Apache WS - (think it is v2.0.2). Everyday approx
after every 24 hours of uptime the website slows down and hangs... On
checking the tomcat logs, we see an OutOfMemoryException.

Restarting Tomcat was one way to momentarily fix the issue and have
the site up - but due to Apache forked processes we are not able to
restart Tomcat too. Even if could our Tomcat takes about 30-40 mins to
load/startup due to some user data loading on startup.

So that leaves the only option of killing certain threads/processes
and then starting Tomcat. which works fine - only we have been doing
this every night for the past more than a month.

We also got a thread dump just to check what is going on.. and I have
pasted that info herewith. Sun has suggested that we install all
patches to the JDK.

I just wanted to find out if anyone had faced a similar issue and if
there was something they did that fixed the issue. Was Tomcat the
culprit?? We will go ahead and install the patches very soon - but
wanted to be ready in case that didnt fix the issue - did anyone
experience an improvement once the patches were installed??
*****************************
Here is the Exception Trace: 

An unexpected exception has been detected in native code outside the VM. 
Unexpected Signal : 11 occurred at PC=0xFF380CBC 
Function=memcpy+0x7E0 
Library=/usr/platform/sun4u/lib/libc_psr.so.1 

Current Java thread: 
        at COM.ibm.db2.jdbc.app.DB2PreparedStatement.SQLBindChar(Native Method) 
        at COM.ibm.db2.jdbc.app.DB2PreparedStatement.execute2(DB2PreparedStatement.java:1020)
        - locked <0x9ec56ef8> (a COM.ibm.db2.jdbc.app.DB2Connection) 
        at COM.ibm.db2.jdbc.app.DB2PreparedStatement.executeUpdate(DB2PreparedStatement.java:807)
        at com.waveset.repository.RelationalDataStore$Item.setPersistentObject(RelationalDataStore.java:999)
        at com.waveset.repository.AbstractDataStore.setItems(AbstractDataStore.java:4926)
        at com.waveset.repository.AbstractDataStore.set(AbstractDataStore.java:1804)
        at com.waveset.repository.AbstractDataStore.checkin(AbstractDataStore.java:1758)
        at com.waveset.repository.AbstractDataStore.set(AbstractDataStore.java:1804)
        at com.waveset.repository.AbstractDataStore.checkin(AbstractDataStore.java:1758)
        at com.waveset.repository.AbstractDataStore.checkin(AbstractDataStore.java:1401)
        at com.waveset.repository.ServerRepository.checkin(ServerRepository.java:2276)
        at com.waveset.session.InternalSession.checkinObject(InternalSession.java:581)
        at com.waveset.task.Scheduler.storeExecutingTask(Scheduler.java:1515) 
        at com.waveset.task.Scheduler.executeReadyTask(Scheduler.java:2918) 
        at com.waveset.task.Scheduler.processReadyTasks(Scheduler.java:2837) 
        at com.waveset.task.Scheduler.processTasks(Scheduler.java:1031) 
        at com.waveset.task.Scheduler.run(Scheduler.java:866) 

Dynamic libraries: 
0x10000         /www4/j2sdk1_4_2_06/java/usr/j2se/bin/java 
0xff360000      /usr/lib/libthread.so.1 
0xff39a000      /usr/lib/libdl.so.1 
0xff280000      /usr/lib/libc.so.1 
0xff380000      /usr/platform/SUNW,Ultra-80/lib/libc_psr.so.1 
0xfec00000     
/www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/client/libjvm.so
0xff240000      /usr/lib/libCrun.so.1 
0xff210000      /usr/lib/libsocket.so.1 
0xff100000      /usr/lib/libnsl.so.1 
0xff1e0000      /usr/lib/libm.so.1 
0xff1c0000      /usr/lib/libsched.so.1 
0xff270000      /usr/lib/libw.so.1 
0xff0d0000      /usr/lib/libmp.so.2 
0xff0b0000      /usr/lib/librt.so.1 
0xff090000      /usr/lib/libaio.so.1 
0xff060000      /usr/lib/libmd5.so.1 
0xff040000      /usr/platform/SUNW,Ultra-80/lib/libmd5_psr.so.1 
0xfebd0000     
/www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/native_threads/libhpi.so
0xfeb80000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libverify.so 
0xfeb40000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libjava.so 
0xfeb20000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libzip.so 
0x737d0000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libnet.so 
0x74910000      /opt/IBMdb2/V7.1/java12/libdb2jdbc.so 
0x71c00000      /opt/IBMdb2/V7.1/lib/libdb2.so.1 
0x73ea0000      /usr/lib/libresolv.so.2 

Heap at VM Abort: 
Heap 
 def new generation   total 226112K, used 67456K [0x75800000,
0x83b80000, 0x83b80000)
  eden space 219264K,  27% used [0x75800000, 0x79331618, 0x82e20000) 
  from space 6848K, 100% used [0x82e20000, 0x834d0000, 0x834d0000) 
  to   space 6848K,   0% used [0x834d0000, 0x834d0000, 0x83b80000) 
 tenured generation   total 1864192K, used 1152200K [0x83b80000,
0xf5800000, 0xf5800000)
   the space 1864192K,  61% used [0x83b80000, 0xca0b2380, 0xca0b2400,
0xf5800000)
 compacting perm gen  total 21248K, used 20838K [0xf5800000,
0xf6cc0000, 0xf9800000)
   the space 21248K,  98% used [0xf5800000, 0xf6c59a40, 0xf6c59c00, 0xf6cc0000) 

Local Time = Mon Feb  7 23:44:49 2005 
Elapsed Time = 42200 
# 
# The exception above was detected in native code outside the VM 
# 
# Java VM: Java HotSpot(TM) Client VM (1.4.2_06-b03 mixed mode) 
# 
# An error report file has been saved as hs_err_pid13341.log. 

-- 
Thanks and best regards,
Anoop

---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-user-help@jakarta.apache.org


Tomcat 4.1 and auto-redirect/URL rewrite

Posted by David Wall <d....@computer.org>.
I have a page in a tomcat 4.1 webapp that has a rather lengthy path name, 
but I'd like to be able to email a much shorter link to avoid long links 
being broken across lines by some older email clients.

With Apache HTTPD's rewrite module, this could be accomplished.  Is there 
something similar in TC?  I'd like to map something like:

https://www.myhost.com/app/pickup?c=12345678901234567890

to

https://www.myhost.com/app/custom/customerName/applicationName/welcome.jsp?c=12345678901234567890

I know I could create a servlet that does an auto-redirect, but I'm looking 
to see if TC has anything that more directly will rewrite the URLs.

Thanks,
David 


---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-user-help@jakarta.apache.org


Re: Tomcat 4.1.30 crashing on production site - need help

Posted by Anoop kumar V <an...@gmail.com>.
So - JDK with or without the patches may not be the issue here?? Did
anyone install patches for JDK 1.4.2_06 and see any improvement or
anything??

-Anoop


On Mon, 07 Mar 2005 18:50:49 +0100, David Tonhofer, m-plify S.A.
<d....@m-plify.com> wrote:
> It might also be that Tomcat needs more memory than the JVM will allocate.
> We had that problem.
> 
> With the Sun JVM, the options "-Xss128k -Xmx128m" increased the stack and max
> heap size and there no longer was a problem.
> 
> Try to set CATALINA_OPTS in bin/startup.sh:
> 
> CATALINA_OPTS="$CATALINA_OPTS -server -Xss128k -Xmx128m"
> 
> Otherwise, try to get MRTG to graph your Tomcat's memory usage, like it's
> done in the attached PNG.
> 
> Best regards,
> 
>  -- DAvid
> 
> 
> --On Monday, March 07, 2005 4:49 PM +0000 James Sys <ja...@MySpareKey.co.uk> wrote:
> 
> >
> > I've seen similar problems in a non-Tomcat Java application.
> >
> > The root cause was a memory leak. Although Java has good garbage collection,
> >  it relies on object references being released.
> >
> > This is generally fine, but some buggy applications can add references to
> >  large object to the application and forget to release them. This causes
> >  memory to be consumed, the app so slow and eventually the app or server to
> >  lock.
> >
> > With Tomcat, I can imagine this happening with objects continually being
> >  added to, say, an ArrayList in the Application context and not being
> >  released.
> >
> > I believe there are various tools around to check object allocation and
> >  release. So if you see objects of a certain type have a gradually increasing
> >  number, you may well have a memory leak.
> >
> > Hope this helps.
> >
> > Regards,
> >
> > James.
> > =====
> >
> >
> > Hi All,
> >
> > For the past couple of months we have been facing a peculiar problem
> > on our production environment. And we have been unable to resolve this
> > issue so far.
> >
> > We have an application hosted on Tomcat 4.1.30 and using JDK1.4.2_06.
> > We also have an Apache WS - (think it is v2.0.2). Everyday approx
> > after every 24 hours of uptime the website slows down and hangs... On
> > checking the tomcat logs, we see an OutOfMemoryException.
> >
> > Restarting Tomcat was one way to momentarily fix the issue and have
> > the site up - but due to Apache forked processes we are not able to
> > restart Tomcat too. Even if could our Tomcat takes about 30-40 mins to
> > load/startup due to some user data loading on startup.
> >
> > So that leaves the only option of killing certain threads/processes
> > and then starting Tomcat. which works fine - only we have been doing
> > this every night for the past more than a month.
> >
> > We also got a thread dump just to check what is going on.. and I have
> > pasted that info herewith. Sun has suggested that we install all
> > patches to the JDK.
> >
> > I just wanted to find out if anyone had faced a similar issue and if
> > there was something they did that fixed the issue. Was Tomcat the
> > culprit?? We will go ahead and install the patches very soon - but
> > wanted to be ready in case that didnt fix the issue - did anyone
> > experience an improvement once the patches were installed??
> > *****************************
> > Here is the Exception Trace:
> >
> > An unexpected exception has been detected in native code outside the VM.
> > Unexpected Signal : 11 occurred at PC=0xFF380CBC
> > Function=memcpy+0x7E0
> > Library=/usr/platform/sun4u/lib/libc_psr.so.1
> >
> > Current Java thread:
> >         at COM.ibm.db2.jdbc.app.DB2PreparedStatement.SQLBindChar(Native
> >  Method) at
> >  COM.ibm.db2.jdbc.app.DB2PreparedStatement.execute2(DB2PreparedStatement.java
> >
> > :1020) - locked <0x9ec56ef8> (a COM.ibm.db2.jdbc.app.DB2Connection) at
> >
> >  COM.ibm.db2.jdbc.app.DB2PreparedStatement.executeUpdate(DB2PreparedStatement
> > .java:807) at
> >  com.waveset.repository.RelationalDataStore$Item.setPersistentObject(Relation
> > alDataStore.java:999) at
> >  com.waveset.repository.AbstractDataStore.setItems(AbstractDataStore.java:492
> > 6) at
> >  com.waveset.repository.AbstractDataStore.set(AbstractDataStore.java:1804) at
> >  com.waveset.repository.AbstractDataStore.checkin(AbstractDataStore.java:1758
> > ) at
> >  com.waveset.repository.AbstractDataStore.set(AbstractDataStore.java:1804) at
> >  com.waveset.repository.AbstractDataStore.checkin(AbstractDataStore.java:1758
> > ) at
> >  com.waveset.repository.AbstractDataStore.checkin(AbstractDataStore.java:1401
> > ) at
> >  com.waveset.repository.ServerRepository.checkin(ServerRepository.java:2276)
> >  at
> >  com.waveset.session.InternalSession.checkinObject(InternalSession.java:581)
> >  at com.waveset.task.Scheduler.storeExecutingTask(Scheduler.java:1515) at
> >  com.waveset.task.Scheduler.executeReadyTask(Scheduler.java:2918) at
> >  com.waveset.task.Scheduler.processReadyTasks(Scheduler.java:2837) at
> >  com.waveset.task.Scheduler.processTasks(Scheduler.java:1031) at
> >  com.waveset.task.Scheduler.run(Scheduler.java:866)
> >
> > Dynamic libraries:
> > 0x10000         /www4/j2sdk1_4_2_06/java/usr/j2se/bin/java
> > 0xff360000      /usr/lib/libthread.so.1
> > 0xff39a000      /usr/lib/libdl.so.1
> > 0xff280000      /usr/lib/libc.so.1
> > 0xff380000      /usr/platform/SUNW,Ultra-80/lib/libc_psr.so.1
> > 0xfec00000
> > /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/client/libjvm.so
> > 0xff240000      /usr/lib/libCrun.so.1
> > 0xff210000      /usr/lib/libsocket.so.1
> > 0xff100000      /usr/lib/libnsl.so.1
> > 0xff1e0000      /usr/lib/libm.so.1
> > 0xff1c0000      /usr/lib/libsched.so.1
> > 0xff270000      /usr/lib/libw.so.1
> > 0xff0d0000      /usr/lib/libmp.so.2
> > 0xff0b0000      /usr/lib/librt.so.1
> > 0xff090000      /usr/lib/libaio.so.1
> > 0xff060000      /usr/lib/libmd5.so.1
> > 0xff040000      /usr/platform/SUNW,Ultra-80/lib/libmd5_psr.so.1
> > 0xfebd0000
> > /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/native_threads/libhpi.so
> > 0xfeb80000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libverify.so
> > 0xfeb40000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libjava.so
> > 0xfeb20000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libzip.so
> > 0x737d0000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libnet.so
> > 0x74910000      /opt/IBMdb2/V7.1/java12/libdb2jdbc.so
> > 0x71c00000      /opt/IBMdb2/V7.1/lib/libdb2.so.1
> > 0x73ea0000      /usr/lib/libresolv.so.2
> >
> > Heap at VM Abort:
> > Heap
> >  def new generation   total 226112K, used 67456K [0x75800000,
> > 0x83b80000, 0x83b80000)
> >   eden space 219264K,  27% used [0x75800000, 0x79331618, 0x82e20000)
> >   from space 6848K, 100% used [0x82e20000, 0x834d0000, 0x834d0000)
> >   to   space 6848K,   0% used [0x834d0000, 0x834d0000, 0x83b80000)
> >  tenured generation   total 1864192K, used 1152200K [0x83b80000,
> > 0xf5800000, 0xf5800000)
> >    the space 1864192K,  61% used [0x83b80000, 0xca0b2380, 0xca0b2400,
> > 0xf5800000)
> >  compacting perm gen  total 21248K, used 20838K [0xf5800000,
> > 0xf6cc0000, 0xf9800000)
> >    the space 21248K,  98% used [0xf5800000, 0xf6c59a40, 0xf6c59c00,
> >  0xf6cc0000)
> >
> > Local Time = Mon Feb  7 23:44:49 2005
> > Elapsed Time = 42200
> >#
> ># The exception above was detected in native code outside the VM
> >#
> ># Java VM: Java HotSpot(TM) Client VM (1.4.2_06-b03 mixed mode)
> >#
> ># An error report file has been saved as hs_err_pid13341.log.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: tomcat-user-help@jakarta.apache.org
> >
> >
> >
> >
> 
> -- David Tonhofer
> 
>    M-PLIFY S.A.
>    Resp. Informatique
>    47, av. de la Liberté
>    L-1931 Luxembourg
>    Tel: +352 261846-52
>    Fax: +352 261846-46
>    Mob: +352 021-139031
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-user-help@jakarta.apache.org
> 
> 


-- 
Thanks and best regards,
Anoop

---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-user-help@jakarta.apache.org


Re: Tomcat 4.1.30 crashing on production site - need help

Posted by "David Tonhofer, m-plify S.A." <d....@m-plify.com>.
It might also be that Tomcat needs more memory than the JVM will allocate.
We had that problem.

With the Sun JVM, the options "-Xss128k -Xmx128m" increased the stack and max
heap size and there no longer was a problem.

Try to set CATALINA_OPTS in bin/startup.sh:

CATALINA_OPTS="$CATALINA_OPTS -server -Xss128k -Xmx128m"

Otherwise, try to get MRTG to graph your Tomcat's memory usage, like it's
done in the attached PNG.

Best regards,

 -- DAvid



--On Monday, March 07, 2005 4:49 PM +0000 James Sys <ja...@MySpareKey.co.uk> wrote:

>
> I've seen similar problems in a non-Tomcat Java application.
>
> The root cause was a memory leak. Although Java has good garbage collection,
>  it relies on object references being released.
>
> This is generally fine, but some buggy applications can add references to
>  large object to the application and forget to release them. This causes
>  memory to be consumed, the app so slow and eventually the app or server to
>  lock.
>
> With Tomcat, I can imagine this happening with objects continually being
>  added to, say, an ArrayList in the Application context and not being
>  released.
>
> I believe there are various tools around to check object allocation and
>  release. So if you see objects of a certain type have a gradually increasing
>  number, you may well have a memory leak.
>
> Hope this helps.
>
> Regards,
>
> James.
> =====
>
>
> Hi All,
>
> For the past couple of months we have been facing a peculiar problem
> on our production environment. And we have been unable to resolve this
> issue so far.
>
> We have an application hosted on Tomcat 4.1.30 and using JDK1.4.2_06.
> We also have an Apache WS - (think it is v2.0.2). Everyday approx
> after every 24 hours of uptime the website slows down and hangs... On
> checking the tomcat logs, we see an OutOfMemoryException.
>
> Restarting Tomcat was one way to momentarily fix the issue and have
> the site up - but due to Apache forked processes we are not able to
> restart Tomcat too. Even if could our Tomcat takes about 30-40 mins to
> load/startup due to some user data loading on startup.
>
> So that leaves the only option of killing certain threads/processes
> and then starting Tomcat. which works fine - only we have been doing
> this every night for the past more than a month.
>
> We also got a thread dump just to check what is going on.. and I have
> pasted that info herewith. Sun has suggested that we install all
> patches to the JDK.
>
> I just wanted to find out if anyone had faced a similar issue and if
> there was something they did that fixed the issue. Was Tomcat the
> culprit?? We will go ahead and install the patches very soon - but
> wanted to be ready in case that didnt fix the issue - did anyone
> experience an improvement once the patches were installed??
> *****************************
> Here is the Exception Trace:
>
> An unexpected exception has been detected in native code outside the VM.
> Unexpected Signal : 11 occurred at PC=0xFF380CBC
> Function=memcpy+0x7E0
> Library=/usr/platform/sun4u/lib/libc_psr.so.1
>
> Current Java thread:
>         at COM.ibm.db2.jdbc.app.DB2PreparedStatement.SQLBindChar(Native
>  Method) at
>  COM.ibm.db2.jdbc.app.DB2PreparedStatement.execute2(DB2PreparedStatement.java
>
> :1020) - locked <0x9ec56ef8> (a COM.ibm.db2.jdbc.app.DB2Connection) at
>
>  COM.ibm.db2.jdbc.app.DB2PreparedStatement.executeUpdate(DB2PreparedStatement
> .java:807) at
>  com.waveset.repository.RelationalDataStore$Item.setPersistentObject(Relation
> alDataStore.java:999) at
>  com.waveset.repository.AbstractDataStore.setItems(AbstractDataStore.java:492
> 6) at
>  com.waveset.repository.AbstractDataStore.set(AbstractDataStore.java:1804) at
>  com.waveset.repository.AbstractDataStore.checkin(AbstractDataStore.java:1758
> ) at
>  com.waveset.repository.AbstractDataStore.set(AbstractDataStore.java:1804) at
>  com.waveset.repository.AbstractDataStore.checkin(AbstractDataStore.java:1758
> ) at
>  com.waveset.repository.AbstractDataStore.checkin(AbstractDataStore.java:1401
> ) at
>  com.waveset.repository.ServerRepository.checkin(ServerRepository.java:2276)
>  at
>  com.waveset.session.InternalSession.checkinObject(InternalSession.java:581)
>  at com.waveset.task.Scheduler.storeExecutingTask(Scheduler.java:1515) at
>  com.waveset.task.Scheduler.executeReadyTask(Scheduler.java:2918) at
>  com.waveset.task.Scheduler.processReadyTasks(Scheduler.java:2837) at
>  com.waveset.task.Scheduler.processTasks(Scheduler.java:1031) at
>  com.waveset.task.Scheduler.run(Scheduler.java:866)
>
> Dynamic libraries:
> 0x10000         /www4/j2sdk1_4_2_06/java/usr/j2se/bin/java
> 0xff360000      /usr/lib/libthread.so.1
> 0xff39a000      /usr/lib/libdl.so.1
> 0xff280000      /usr/lib/libc.so.1
> 0xff380000      /usr/platform/SUNW,Ultra-80/lib/libc_psr.so.1
> 0xfec00000
> /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/client/libjvm.so
> 0xff240000      /usr/lib/libCrun.so.1
> 0xff210000      /usr/lib/libsocket.so.1
> 0xff100000      /usr/lib/libnsl.so.1
> 0xff1e0000      /usr/lib/libm.so.1
> 0xff1c0000      /usr/lib/libsched.so.1
> 0xff270000      /usr/lib/libw.so.1
> 0xff0d0000      /usr/lib/libmp.so.2
> 0xff0b0000      /usr/lib/librt.so.1
> 0xff090000      /usr/lib/libaio.so.1
> 0xff060000      /usr/lib/libmd5.so.1
> 0xff040000      /usr/platform/SUNW,Ultra-80/lib/libmd5_psr.so.1
> 0xfebd0000
> /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/native_threads/libhpi.so
> 0xfeb80000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libverify.so
> 0xfeb40000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libjava.so
> 0xfeb20000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libzip.so
> 0x737d0000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libnet.so
> 0x74910000      /opt/IBMdb2/V7.1/java12/libdb2jdbc.so
> 0x71c00000      /opt/IBMdb2/V7.1/lib/libdb2.so.1
> 0x73ea0000      /usr/lib/libresolv.so.2
>
> Heap at VM Abort:
> Heap
>  def new generation   total 226112K, used 67456K [0x75800000,
> 0x83b80000, 0x83b80000)
>   eden space 219264K,  27% used [0x75800000, 0x79331618, 0x82e20000)
>   from space 6848K, 100% used [0x82e20000, 0x834d0000, 0x834d0000)
>   to   space 6848K,   0% used [0x834d0000, 0x834d0000, 0x83b80000)
>  tenured generation   total 1864192K, used 1152200K [0x83b80000,
> 0xf5800000, 0xf5800000)
>    the space 1864192K,  61% used [0x83b80000, 0xca0b2380, 0xca0b2400,
> 0xf5800000)
>  compacting perm gen  total 21248K, used 20838K [0xf5800000,
> 0xf6cc0000, 0xf9800000)
>    the space 21248K,  98% used [0xf5800000, 0xf6c59a40, 0xf6c59c00,
>  0xf6cc0000)
>
> Local Time = Mon Feb  7 23:44:49 2005
> Elapsed Time = 42200
>#
># The exception above was detected in native code outside the VM
>#
># Java VM: Java HotSpot(TM) Client VM (1.4.2_06-b03 mixed mode)
>#
># An error report file has been saved as hs_err_pid13341.log.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: tomcat-user-help@jakarta.apache.org
>
>
>
>



-- David Tonhofer

   M-PLIFY S.A.
   Resp. Informatique
   47, av. de la Liberté
   L-1931 Luxembourg
   Tel: +352 261846-52
   Fax: +352 261846-46
   Mob: +352 021-139031


Re: Tomcat 4.1.30 crashing on production site - need help

Posted by James Sys <ja...@MySpareKey.co.uk>.
I've seen similar problems in a non-Tomcat Java application.

The root cause was a memory leak. Although Java has good garbage collection,
 it relies on object references being released.

This is generally fine, but some buggy applications can add references to
 large object to the application and forget to release them. This causes
 memory to be consumed, the app so slow and eventually the app or server to
 lock.

With Tomcat, I can imagine this happening with objects continually being
 added to, say, an ArrayList in the Application context and not being
 released.

I believe there are various tools around to check object allocation and
 release. So if you see objects of a certain type have a gradually increasing
 number, you may well have a memory leak.

Hope this helps.

Regards,

James.
=====


Hi All,

For the past couple of months we have been facing a peculiar problem
on our production environment. And we have been unable to resolve this
issue so far.

We have an application hosted on Tomcat 4.1.30 and using JDK1.4.2_06.
We also have an Apache WS - (think it is v2.0.2). Everyday approx
after every 24 hours of uptime the website slows down and hangs... On
checking the tomcat logs, we see an OutOfMemoryException.

Restarting Tomcat was one way to momentarily fix the issue and have
the site up - but due to Apache forked processes we are not able to
restart Tomcat too. Even if could our Tomcat takes about 30-40 mins to
load/startup due to some user data loading on startup.

So that leaves the only option of killing certain threads/processes
and then starting Tomcat. which works fine - only we have been doing
this every night for the past more than a month.

We also got a thread dump just to check what is going on.. and I have
pasted that info herewith. Sun has suggested that we install all
patches to the JDK.

I just wanted to find out if anyone had faced a similar issue and if
there was something they did that fixed the issue. Was Tomcat the
culprit?? We will go ahead and install the patches very soon - but
wanted to be ready in case that didnt fix the issue - did anyone
experience an improvement once the patches were installed??
*****************************
Here is the Exception Trace:

An unexpected exception has been detected in native code outside the VM.
Unexpected Signal : 11 occurred at PC=0xFF380CBC
Function=memcpy+0x7E0
Library=/usr/platform/sun4u/lib/libc_psr.so.1

Current Java thread:
        at COM.ibm.db2.jdbc.app.DB2PreparedStatement.SQLBindChar(Native
 Method) at
 COM.ibm.db2.jdbc.app.DB2PreparedStatement.execute2(DB2PreparedStatement.java

:1020) - locked <0x9ec56ef8> (a COM.ibm.db2.jdbc.app.DB2Connection) at

 COM.ibm.db2.jdbc.app.DB2PreparedStatement.executeUpdate(DB2PreparedStatement
.java:807) at
 com.waveset.repository.RelationalDataStore$Item.setPersistentObject(Relation
alDataStore.java:999) at
 com.waveset.repository.AbstractDataStore.setItems(AbstractDataStore.java:492
6) at
 com.waveset.repository.AbstractDataStore.set(AbstractDataStore.java:1804) at
 com.waveset.repository.AbstractDataStore.checkin(AbstractDataStore.java:1758
) at
 com.waveset.repository.AbstractDataStore.set(AbstractDataStore.java:1804) at
 com.waveset.repository.AbstractDataStore.checkin(AbstractDataStore.java:1758
) at
 com.waveset.repository.AbstractDataStore.checkin(AbstractDataStore.java:1401
) at
 com.waveset.repository.ServerRepository.checkin(ServerRepository.java:2276)
 at
 com.waveset.session.InternalSession.checkinObject(InternalSession.java:581)
 at com.waveset.task.Scheduler.storeExecutingTask(Scheduler.java:1515) at
 com.waveset.task.Scheduler.executeReadyTask(Scheduler.java:2918) at
 com.waveset.task.Scheduler.processReadyTasks(Scheduler.java:2837) at
 com.waveset.task.Scheduler.processTasks(Scheduler.java:1031) at
 com.waveset.task.Scheduler.run(Scheduler.java:866)

Dynamic libraries:
0x10000         /www4/j2sdk1_4_2_06/java/usr/j2se/bin/java
0xff360000      /usr/lib/libthread.so.1
0xff39a000      /usr/lib/libdl.so.1
0xff280000      /usr/lib/libc.so.1
0xff380000      /usr/platform/SUNW,Ultra-80/lib/libc_psr.so.1
0xfec00000
/www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/client/libjvm.so
0xff240000      /usr/lib/libCrun.so.1
0xff210000      /usr/lib/libsocket.so.1
0xff100000      /usr/lib/libnsl.so.1
0xff1e0000      /usr/lib/libm.so.1
0xff1c0000      /usr/lib/libsched.so.1
0xff270000      /usr/lib/libw.so.1
0xff0d0000      /usr/lib/libmp.so.2
0xff0b0000      /usr/lib/librt.so.1
0xff090000      /usr/lib/libaio.so.1
0xff060000      /usr/lib/libmd5.so.1
0xff040000      /usr/platform/SUNW,Ultra-80/lib/libmd5_psr.so.1
0xfebd0000
/www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/native_threads/libhpi.so
0xfeb80000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libverify.so
0xfeb40000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libjava.so
0xfeb20000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libzip.so
0x737d0000      /www4/j2sdk1_4_2_06/java/usr/j2se/jre/lib/sparc/libnet.so
0x74910000      /opt/IBMdb2/V7.1/java12/libdb2jdbc.so
0x71c00000      /opt/IBMdb2/V7.1/lib/libdb2.so.1
0x73ea0000      /usr/lib/libresolv.so.2

Heap at VM Abort:
Heap
 def new generation   total 226112K, used 67456K [0x75800000,
0x83b80000, 0x83b80000)
  eden space 219264K,  27% used [0x75800000, 0x79331618, 0x82e20000)
  from space 6848K, 100% used [0x82e20000, 0x834d0000, 0x834d0000)
  to   space 6848K,   0% used [0x834d0000, 0x834d0000, 0x83b80000)
 tenured generation   total 1864192K, used 1152200K [0x83b80000,
0xf5800000, 0xf5800000)
   the space 1864192K,  61% used [0x83b80000, 0xca0b2380, 0xca0b2400,
0xf5800000)
 compacting perm gen  total 21248K, used 20838K [0xf5800000,
0xf6cc0000, 0xf9800000)
   the space 21248K,  98% used [0xf5800000, 0xf6c59a40, 0xf6c59c00,
 0xf6cc0000)

Local Time = Mon Feb  7 23:44:49 2005
Elapsed Time = 42200
#
# The exception above was detected in native code outside the VM
#
# Java VM: Java HotSpot(TM) Client VM (1.4.2_06-b03 mixed mode)
#
# An error report file has been saved as hs_err_pid13341.log.


---------------------------------------------------------------------
To unsubscribe, e-mail: tomcat-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: tomcat-user-help@jakarta.apache.org