You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Wayne Pope <wa...@gmail.com> on 2009/12/15 10:14:30 UTC
Very slow shutdown - 3 mins
Hi,
we're running 3 tomcat instances on Solaris. We are experiencing extremely
show shutdowns. I've done a thread dump, but as far as I can tell all looks
ok. Does anyone know what is causing this?
We're calling /opt/ec/tomcat/bin/shutdown.sh to shut down the instance.
I see the line: 'INFO: Stopping Coyote HTTP/1.1 on http-8082' appear
quickly. However the java process hangs about for at least another 3
minutes.
Here's the thread dump:
Dec 15, 2009 7:35:23 AM org.apache.coyote.http11.Http11Protocol pause
INFO: Pausing Coyote HTTP/1.1 on http-8082
Dec 15, 2009 7:35:24 AM org.apache.catalina.core.StandardService stop
INFO: Stopping service Catalina
Dec 15, 2009 7:35:26 AM org.apache.coyote.http11.Http11Protocol destroy
INFO: Stopping Coyote HTTP/1.1 on http-8082
2009-12-15 07:36:20
Full thread dump Java HotSpot(TM) Server VM (11.2-b01 mixed mode):
"DestroyJavaVM" prio=3 tid=0x08070000 nid=0x2 waiting on condition
[0x00000000..0xfe46ebb0]
java.lang.Thread.State: RUNNABLE
"http-8082-1" daemon prio=3 tid=0x0848a800 nid=0xba in Object.wait()
[0xe396d000..0xe396dbf0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
- locked <0xf9e92aa8> (a
org.apache.tomcat.util.net.JIoEndpoint$Worker)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449)
at java.lang.Thread.run(Thread.java:619)
"Java2D Disposer" daemon prio=3 tid=0x08dd0000 nid=0x50 in Object.wait()
[0xe407b000..0xe407baf0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0xec1c6cc8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at sun.java2d.Disposer.run(Disposer.java:125)
at java.lang.Thread.run(Thread.java:619)
"pool-1-thread-10" prio=3 tid=0x085b8800 nid=0x38 waiting on condition
[0xe44d9000..0xe44d9af0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xeb57f170> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"pool-1-thread-9" prio=3 tid=0x08e0b400 nid=0x37 waiting on condition
[0xe497d000..0xe497db70]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xeb57f170> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"pool-1-thread-8" prio=3 tid=0x09550c00 nid=0x36 waiting on condition
[0xe49ce000..0xe49ce9f0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xeb57f170> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"pool-1-thread-7" prio=3 tid=0x08e74400 nid=0x35 waiting on condition
[0xe45ae000..0xe45aea70]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xeb57f170> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"pool-1-thread-6" prio=3 tid=0x08739000 nid=0x34 waiting on condition
[0xe4612000..0xe46128f0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xeb57f170> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"pool-1-thread-5" prio=3 tid=0x08b5e400 nid=0x32 waiting on condition
[0xe4663000..0xe4663bf0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xeb57f170> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"pool-1-thread-4" prio=3 tid=0x08681400 nid=0x31 waiting on condition
[0xe46b4000..0xe46b4c70]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xeb57f170> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"MultiThreadedHttpConnectionManager cleanup" daemon prio=3 tid=0x0824a400
nid=0x30 in Object.wait() [0xe4705000..0xe4705af0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0xeb8d2bd8> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at
org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$ReferenceQueueThread.run(MultiThreadedHttpConnectionManager.java:1122)
"pool-1-thread-3" prio=3 tid=0x08b5f800 nid=0x2f waiting on condition
[0xe4756000..0xe4756b70]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xeb57f170> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"pool-1-thread-2" prio=3 tid=0x0839f800 nid=0x2d waiting on condition
[0xe47f8000..0xe47f8a70]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xeb57f170> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"pool-1-thread-1" prio=3 tid=0x087f6c00 nid=0x2a waiting on condition
[0xe4849000..0xe4849bf0]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xeb57f170> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:358)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"pool-3-thread-1" prio=3 tid=0x083fe400 nid=0x29 waiting on condition
[0xe489a000..0xe489ac70]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0xeb80e2a0> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1925)
at java.util.concurrent.DelayQueue.take(DelayQueue.java:160)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907)
at java.lang.Thread.run(Thread.java:619)
"TP-Processor4" daemon prio=3 tid=0x08a28000 nid=0x1f in Object.wait()
[0xe4ed7000..0xe4ed7b70]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at org.apache.jk.common.ChannelSocket.accept(ChannelSocket.java:301)
- locked <0xeb4458c8> (a org.apache.jk.common.ChannelSocket)
at
org.apache.jk.common.ChannelSocket.acceptConnections(ChannelSocket.java:661)
at
org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(ChannelSocket.java:872)
at
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
at java.lang.Thread.run(Thread.java:619)
"http-8082-Acceptor-0" daemon prio=3 tid=0x08535400 nid=0x1b waiting on
condition [0xe5438000..0xe5438970]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at
org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:309)
at java.lang.Thread.run(Thread.java:619)
"MySQL Statement Cancellation Timer" daemon prio=3 tid=0x08498400 nid=0x18
in Object.wait() [0xe4f79000..0xe4f79af0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xeb191718> (a java.util.TaskQueue)
at java.lang.Object.wait(Object.java:485)
at java.util.TimerThread.mainLoop(Timer.java:483)
- locked <0xeb191718> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:462)
"Low Memory Detector" daemon prio=3 tid=0x081ab400 nid=0x11 runnable
[0x00000000..0x00000000]
java.lang.Thread.State: RUNNABLE
"CompilerThread1" daemon prio=3 tid=0x081a8800 nid=0x10 waiting on condition
[0x00000000..0xe55b2048]
java.lang.Thread.State: RUNNABLE
"CompilerThread0" daemon prio=3 tid=0x081a7000 nid=0xf waiting on condition
[0x00000000..0xe55f30c8]
java.lang.Thread.State: RUNNABLE
"Signal Dispatcher" daemon prio=3 tid=0x081a5c00 nid=0xe waiting on
condition [0x00000000..0x00000000]
java.lang.Thread.State: RUNNABLE
"Finalizer" daemon prio=3 tid=0x08190400 nid=0xd in Object.wait()
[0xe5695000..0xe5695a70]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:116)
- locked <0xeb06e320> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:132)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)
"Reference Handler" daemon prio=3 tid=0x0818ec00 nid=0xc in Object.wait()
[0xe56e6000..0xe56e68f0]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0xeb06b6b8> (a java.lang.ref.Reference$Lock)
"VM Thread" prio=3 tid=0x0818bc00 nid=0xb runnable
"GC task thread#0 (ParallelGC)" prio=3 tid=0x08076800 nid=0x3 runnable
"GC task thread#1 (ParallelGC)" prio=3 tid=0x08077c00 nid=0x4 runnable
"GC task thread#2 (ParallelGC)" prio=3 tid=0x08079000 nid=0x5 runnable
"GC task thread#3 (ParallelGC)" prio=3 tid=0x0807a400 nid=0x6 runnable
"GC task thread#4 (ParallelGC)" prio=3 tid=0x0807b800 nid=0x7 runnable
"GC task thread#5 (ParallelGC)" prio=3 tid=0x0807cc00 nid=0x8 runnable
"GC task thread#6 (ParallelGC)" prio=3 tid=0x0807e000 nid=0x9 runnable
"GC task thread#7 (ParallelGC)" prio=3 tid=0x0807f800 nid=0xa runnable
"VM Periodic Task Thread" prio=3 tid=0x081ad400 nid=0x12 waiting on
condition
JNI global references: 852
Heap
PSYoungGen total 12608K, used 2968K [0xf9200000, 0xfa200000,
0xfb000000)
eden space 8832K, 5% used [0xf9200000,0xf9272180,0xf9aa0000)
from space 3776K, 66% used [0xf9e50000,0xfa0c41e8,0xfa200000)
to space 3776K, 0% used [0xf9aa0000,0xf9aa0000,0xf9e50000)
PSOldGen total 116736K, used 91584K [0xeae00000, 0xf2000000,
0xf9200000)
object space 116736K, 78% used [0xeae00000,0xf0770288,0xf2000000)
PSPermGen total 86016K, used 47160K [0xe5a00000, 0xeae00000,
0xeae00000)
object space 86016K, 54% used [0xe5a00000,0xe880e200,0xeae00000)
many thanks
--
View this message in context: http://old.nabble.com/Very-slow-shutdown---3-mins-tp26791794p26791794.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Very slow shutdown - 3 mins
Posted by Wayne Pope <wa...@gmail.com>.
thanks guys,
I'll ask dev whats going on.
el al wrote:
>
> You have many threads that are not part of Tomcat, as Chuck mentions.
> Whatever you start in the webapp you have to close as well. You can
> close via a shutdown listener (Context listener), or if you use
> something like Spring which already has built in shutdown hook,
> implement their shutdown interface or register a callback.
> When you shutdown a java executor (which can be seen in the thread
> dump), notice that to be safe you can do a two steps shutdown. Step 1
> does a graceful shutdown and waits. When the graceful shutdown times
> out and the pool has not finishes, step 2, “hard” shutdown to finish
> quickly. This will give you the benefit of finishing pending tasks and
> limit the amount of time allowed before you kill the pool. A hard shut
> down also means that tasks that are still waiting in the pool will not
> be executed, so use with caution.
>
> This also goes for everything that you create that potentially has
> persistent resource usage, such as connection pools, HttpClient, etc.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
>
--
View this message in context: http://old.nabble.com/Very-slow-shutdown---3-mins-tp26791794p26809018.html
Sent from the Tomcat - User mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Very slow shutdown - 3 mins
Posted by Elli Albek <el...@sustainlane.com>.
You have many threads that are not part of Tomcat, as Chuck mentions.
Whatever you start in the webapp you have to close as well. You can
close via a shutdown listener (Context listener), or if you use
something like Spring which already has built in shutdown hook,
implement their shutdown interface or register a callback.
When you shutdown a java executor (which can be seen in the thread
dump), notice that to be safe you can do a two steps shutdown. Step 1
does a graceful shutdown and waits. When the graceful shutdown times
out and the pool has not finishes, step 2, “hard” shutdown to finish
quickly. This will give you the benefit of finishing pending tasks and
limit the amount of time allowed before you kill the pool. A hard shut
down also means that tasks that are still waiting in the pool will not
be executed, so use with caution.
This also goes for everything that you create that potentially has
persistent resource usage, such as connection pools, HttpClient, etc.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Very slow shutdown - 3 mins
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Wayne Pope [mailto:waynemailinglists@gmail.com]
> Subject: Very slow shutdown - 3 mins
>
> I've done a thread dump, but as far as I can tell all
> looks ok.
Not to me. You have numerous threads apparently spawned by a webapp that are still sitting around. Whatever webapp is responsible for them needs to implement a ServletContextListener to terminate those threads when the webapp is stopped. Tomcat knows nothing about them, so it can't do it.
Some of the threads of interest that do not belong to Tomcat or the JVM:
> "pool-1-thread-10" prio=3 tid=0x085b8800 nid=0x38
> "pool-1-thread-9" prio=3 tid=0x08e0b400 nid=0x37
> "pool-1-thread-8" prio=3 tid=0x09550c00 nid=0x36
> "pool-1-thread-7" prio=3 tid=0x08e74400 nid=0x35
> "pool-1-thread-6" prio=3 tid=0x08739000 nid=0x34
> "pool-1-thread-5" prio=3 tid=0x08b5e400 nid=0x32
> "pool-1-thread-4" prio=3 tid=0x08681400 nid=0x31
> "pool-1-thread-3" prio=3 tid=0x08b5f800 nid=0x2f
> "pool-1-thread-2" prio=3 tid=0x0839f800 nid=0x2d
> "pool-1-thread-1" prio=3 tid=0x087f6c00 nid=0x2a
> "pool-3-thread-1" prio=3 tid=0x083fe400 nid=0x29
The above appear to be associated with some outbound HTTP call mechanism. Since they're not daemon threads, they won't go away nicely until someone tells them to (or Tomcat gets tired of waiting).
> "MultiThreadedHttpConnectionManager cleanup" daemon prio=3 tid=0x0824a400 nid=0x30
The manager for the above pooled threads.
> "MySQL Statement Cancellation Timer" daemon prio=3 tid=0x08498400 nid=0x18
Again, not a thread Tomcat creates, but at least it's a daemon.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org