You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Chaitanya Sabbineni <sa...@gmail.com> on 2017/07/20 06:57:38 UTC
Tomcat is stopping on its own even though stop script is not executed
Hi All,
i am using tomcat 7.0.069 version where tomcat is stopping at a specified
time(02:00AM) even though the stop script of the tomcat is not executed.
Below is the *error *which i can see in the tomcat log at 2:00 AM when the
server is stopped.In order to start the sever we need to execute the start
script.
*INFO: Illegal access: this web application instance has been stopped
already. Could not load java.util.concurrent.locks.Lock. The eventual
following stack trace is caused by an error thrown for debugging purposes
as well as to attempt to terminate the thread which caused the illegal
access, and has no functional impact.*
*java.lang.IllegalStateException*
at
org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1777)
at
org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1735)
at
j.aj.passj.core.events.passEventManager.isListeningTo(passEventManager.java:429)
at
com.j.aj.core.vud.VUDManager.refreshVUDConsumptionStore(VUDManager.java:1066)
at
j.aj.passj.core.vud.VUDManager.refreshVUDConsumptionMap(VUDManager.java:1043)
at
j.aj.passj.core.vud.VUDCycleTracker$VUDTimerTask.run(VUDCycleTracker.java:54)
at java.util.TimerThread.mainLoop(Timer.java:555)
at java.util.TimerThread.run(Timer.java:505)
Below is the *thread dump* which i collected and i am attaching over here
2017-07-17 10:28:03
Full thread dump Java HotSpot(TM) 64-Bit Server VM (25.60-b23 mixed mode):
"RMI TCP Connection(2)-10.600.12.89" #79 daemon prio=9 os_prio=0
tid=0x00007f883073e800 nid=0xd67 runnable [0x00007f8844cd8000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
- locked <0x00000000f76931c8> (a java.io.BufferedInputStream)
at java.io.FilterInputStream.read(FilterInputStream.java:83)
at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:550)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$79(TCPTransport.java:683)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler$$Lambda$19/2023115504.run(Unknown
Source)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- <0x00000000f72af0c8> (a java.util.concurrent.ThreadPoolExecutor$Worker)
"JMX server connection timeout 78" #78 daemon prio=9 os_prio=0
tid=0x00007f883c5ed000 nid=0xcf7 in Object.wait() [0x00007f88135f9000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at
com.sun.jmx.remote.internal.ServerCommunicatorAdmin$Timeout.run(ServerCommunicatorAdmin.java:168)
- locked <0x00000000f7376720> (a [I)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- None
"RMI Scheduler(0)" #77 daemon prio=9 os_prio=0 tid=0x00007f883c123000
nid=0xcf6 waiting on condition [0x00007f880d3b5000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000f7270338> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- None
"RMI TCP Connection(idle)" #76 daemon prio=9 os_prio=0
tid=0x00007f8830745000 nid=0xcf5 waiting on condition [0x00007f8818364000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000f72625c8> (a
java.util.concurrent.SynchronousQueue$TransferStack)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
at
java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- None
"RMI TCP Accept-0" #75 daemon prio=9 os_prio=0 tid=0x00007f883400d000
nid=0xcf3 runnable [0x00007f8813dfd000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:409)
at java.net.ServerSocket.implAccept(ServerSocket.java:545)
at java.net.ServerSocket.accept(ServerSocket.java:513)
at
sun.management.jmxremote.LocalRMIServerSocketFactory$1.accept(LocalRMIServerSocketFactory.java:52)
at
sun.rmi.transport.tcp.TCPTransport$AcceptLoop.executeAcceptLoop(TCPTransport.java:400)
at sun.rmi.transport.tcp.TCPTransport$AcceptLoop.run(TCPTransport.java:372)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- None
"Attach Listener" #74 daemon prio=9 os_prio=0 tid=0x00007f8840179000
nid=0x88c waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"DestroyJavaVM" #73 prio=5 os_prio=0 tid=0x00007f8870116800 nid=0x4714
waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Java2D Disposer" #69 daemon prio=10 os_prio=0 tid=0x00007f8840253000
nid=0x61cc in Object.wait() [0x00007f880dc30000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x00000000e381afd0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
at sun.java2d.Disposer.run(Disposer.java:148)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- None
"Timer-3" #55 daemon prio=5 os_prio=0 tid=0x00007f883074c800 nid=0x194a in
Object.wait() [0x00007f881031d000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:552)
- locked <0x00000000e2ccd318> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:505)
Locked ownable synchronizers:
- None
"TraceConfigurationUpdateMonitor" #54 daemon prio=5 os_prio=0
tid=0x00007f88305a6000 nid=0x1943 runnable [0x00007f881091e000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
- locked <0x00000000e2a4aed8> (a java.io.BufferedInputStream)
at java.io.DataInputStream.readByte(DataInputStream.java:265)
at com.j.o.pl.trace.TraceCfg$ConfigThread.run(Unknown Source)
Locked ownable synchronizers:
- None
"PASS_DEFAULT_GROUP_com.j.passj.core.config.scheduler.LicenseRemovalAfterExpiry"
#53 daemon prio=5 os_prio=0 tid=0x00007f882461a000 nid=0x1942 in
Object.wait() [0x00007f8810b1f000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:552)
- locked <0x00000000e2ccd988> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:505)
Locked ownable synchronizers:
- None
"pool-4-thread-1" #52 prio=5 os_prio=0 tid=0x00007f8830332800 nid=0x1941
waiting on condition [0x00007f8810d20000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e2ccdf28> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- None
"pool-5-thread-1" #51 prio=5 os_prio=0 tid=0x00007f8830337000 nid=0x1940
waiting on condition [0x00007f8811321000]
java.lang.Thread.State: TIMED_WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000e2cce8f8> (a
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at
java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Locked ownable synchronizers:
- None
"Timer-2" #50 daemon prio=5 os_prio=0 tid=0x00007f88302e5000 nid=0x193f in
Object.wait() [0x00007f8811522000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:552)
- locked <0x00000000e2cceed8> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:505)
Locked ownable synchronizers:
- None
"Timer-1" #44 daemon prio=5 os_prio=0 tid=0x00007f8830015800 nid=0x1931 in
Object.wait() [0x00007f881175c000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:552)
- locked <0x00000000e2ccfbd8> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:505)
Locked ownable synchronizers:
- None
"TraceConfigurationUpdateMonitor" #21 daemon prio=5 os_prio=0
tid=0x00007f88246cc800 nid=0x5499 runnable [0x00007f8818965000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
- locked <0x00000000e23ee0b0> (a java.io.BufferedInputStream)
at java.io.DataInputStream.readByte(DataInputStream.java:265)
at com.j.o.xp.trace.TraceCfg$ConfigThread.run(Unknown Source)
Locked ownable synchronizers:
- None
"TraceConfigurationUpdateMonitor" #19 daemon prio=5 os_prio=0
tid=0x00007f88243a8800 nid=0x5421 runnable [0x00007f8818e68000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
- locked <0x00000000e1e80888> (a java.io.BufferedInputStream)
at java.io.DataInputStream.readByte(DataInputStream.java:265)
at com.j.o.xp.trace.TraceCfg$ConfigThread.run(Unknown Source)
Locked ownable synchronizers:
- None
"TraceConfigurationUpdateMonitor" #16 daemon prio=5 os_prio=0
tid=0x00007f88245ee800 nid=0x53e0 runnable [0x00007f881f9fd000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
- locked <0x00000000e184ef28> (a java.io.BufferedInputStream)
at java.io.DataInputStream.readByte(DataInputStream.java:265)
at com.j.o.xp.trace.TraceCfg$ConfigThread.run(Unknown Source)
Locked ownable synchronizers:
- None
"Timer-0" #14 daemon prio=5 os_prio=0 tid=0x00007f882457f800 nid=0x53df in
Object.wait() [0x00007f881fdfe000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.util.TimerThread.mainLoop(Timer.java:552)
- locked <0x00000000e184f4e0> (a java.util.TaskQueue)
at java.util.TimerThread.run(Timer.java:505)
Locked ownable synchronizers:
- None
"GC Daemon" #10 daemon prio=2 os_prio=0 tid=0x00007f887054c800 nid=0x48d6
in Object.wait() [0x00007f8845737000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x00000000e0160c40> (a sun.misc.GC$LatencyLock)
at sun.misc.GC$Daemon.run(GC.java:117)
- locked <0x00000000e0160c40> (a sun.misc.GC$LatencyLock)
Locked ownable synchronizers:
- None
"Service Thread" #7 daemon prio=9 os_prio=0 tid=0x00007f88700bc000
nid=0x471d runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"C1 CompilerThread1" #6 daemon prio=9 os_prio=0 tid=0x00007f88700b7000
nid=0x471c waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"C2 CompilerThread0" #5 daemon prio=9 os_prio=0 tid=0x00007f88700b4000
nid=0x471b waiting on condition [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Signal Dispatcher" #4 daemon prio=9 os_prio=0 tid=0x00007f88700b2800
nid=0x471a runnable [0x0000000000000000]
java.lang.Thread.State: RUNNABLE
Locked ownable synchronizers:
- None
"Finalizer" #3 daemon prio=8 os_prio=0 tid=0x00007f887007a800 nid=0x4719 in
Object.wait() [0x00007f88607b5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x00000000e04a7d30> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:164)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:209)
Locked ownable synchronizers:
- None
"Reference Handler" #2 daemon prio=10 os_prio=0 tid=0x00007f8870078000
nid=0x4718 in Object.wait() [0x00007f88609b6000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:157)
- locked <0x00000000e04a7d60> (a java.lang.ref.Reference$Lock)
Locked ownable synchronizers:
- None
"VM Thread" os_prio=0 tid=0x00007f8870073000 nid=0x4717 runnable
"GC task thread#0 (ParallelGC)" os_prio=0 tid=0x00007f887001e800 nid=0x4715
runnable
"GC task thread#1 (ParallelGC)" os_prio=0 tid=0x00007f8870020000 nid=0x4716
runnable
"VM Periodic Task Thread" os_prio=0 tid=0x00007f88700be800 nid=0x471e
waiting on condition
JNI global references: 519
Can anyone please let me know what is the problem or can any one suggest a
solution.
Thanks & Regards,
Chaitanya
Re: Tomcat is stopping on its own even though stop script is not
executed
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Chaitanya,
On 7/20/17 2:40 PM, Chaitanya Sabbineni wrote:
> Hi All,
>
> This is to add that I found some interesting thing. I order to
> enable debugging in tomcat I changed the logging level from Fine to
> Finest .Upon changing the log level we see that tomcat is not
> getting stopped on its own.This is he strange behaviour we
> observed.
>
> can you please elaborate what actually this mean
>
> Fix your application so that the Tomcat JVM exits cleanly
Yes this would certainly be a good thing to do.
> Use a thread pool. Manage the thread pool in a servlet context
> listener (creation, destruction).
>
> can you let me know is there any way way to kill the timer thread
> during the stop of tomcat.
>
> And can you let me know what actually the timer thread task in the
> error log.I mean if can point me to that snippet.
There are many way to create a timer in Java. Usually it's either your
code, or some code in a library you use. You will have to determine
where the timer is being created, and how to shut it down.
All good libraries that launch timers have some kind of "shut down"
command you can give to them. You just need to make sure that when
your application is being shut-down, you make sure to shut-down all of
the various timers that may be running.
Usually something like this:
public class TimerKiller implements ServletContextListener {
public void init(ServletContext ctx) { }
public void destroy(ServletContext ctx) {
org.library.timer.TimerManager.shutdown();
}
}
That's very hypothetical... you will need to research each of the
timer threads that are running, determine how to shut them down, and
then implement a ServletContextListener (and configure it in web.xml!)
that calls the right method(s) to shut-down those timers.
- -chris
> On Thu, 20 Jul 2017 11:37 pm Mark Eggers,
> <it...@yahoo.com.invalid> wrote:
>
>> Chaitanya,
>>
>> This will be long and somewhat speculative.
>>
>> On 7/20/2017 9:00 AM, Christopher Schultz wrote:
>>> Chaitanya,
>>>
>>> On 7/20/17 11:03 AM, Chaitanya Sabbineni wrote:
>>>> Stop script in the sense it's Catalina script only but we
>>>> usually stop tomcat using the command Catalina.sh stop. But
>>>> in our case we are not manually executing this script to stop
>>>> tomcat and tomcat is stopping on its own.
>>>
>>>> our main problem here us tomcat is stopping on its own and it
>>>> needs a restart.
>>>
>>> Right.
>>>
>>>> If I understand you correct you are telling TimerThread that
>>>> does not stop when the application is shut down. Can you let
>>>> me know what actually the timer thread mean. And moreover if
>>>> the timer thread didn't stop ideally tomcat shouldn't stop
>>>> but in our case it's stopping.
>>>
>>> Tomcat is stopping but the JVM is not. If your application were
>>> to shut-down cleanly, then the JVM would exit as well. This is
>>> unrelated to your real problem (unexpected Tomcat shutdown),
>>> but you might want to look into fixing that, because it makes
>>> your application impossible to reload without risking serious
>>> heap space problems.
>>>
>>>> Yes my question is why Tomcat is being shut down at all.
>>>
>>>> Yes when ever tomcat is stopping on own(not daily) it stops
>>>> at 02:00 . You mentioned that your guess is that we are
>>>> using a service runner that is configured to bounce your
>>>> services at 02:00.Can let me know what this service runner is
>>>> and how to check it.
>>>
>>> I know nothing about your environment. Until you mentioned
>>> "catalina.sh stop" above, I didn't even know you were on a
>>> UNIX-like environment. Honestly, I assumed you were on Windows
>>> because "mysterious service stoppage" has Microsoft Windows
>>> behavior written all over it.
>>>
>>> There are two ways to trigger a Tomcat shut down:
>>>
>>> 1. Send a TERM signal to the process 2. Connect to Tomcat's
>>> shutdown listener (default: port 8005) and give the shutdown
>>> command (default: "SHUTDOWN")
>>>
>>> You can eliminate one of those possibilities by setting the
>>> shutdown port in server.xml to "" (empty) which will disable
>>> this type of shutdow n:
>>>
>>> <Server port="" ...
>>>
>>> You cannot disable the other type of shutdown... any user on
>>> the system who can send a TERM signal to your process could
>>> terminate Tomcat .
>>>
>>> As for catching whoever is shutting down your Tomcat, you may
>>> want to look at who has administrative access to your server,
>>> and who has access to the user running your Tomcat server.
>>>
>>> Check your syslog to find sudo and cron events that might be
>>> automatically shutting-down Tomcat.
>>>
>>> If you want to catch a TCP connection, you will likely have to
>>> enable tcpwrappers, iptables, ipfw, etc. to log connections to
>>> port 8005. Those logs will only tell you that the command is
>>> being sent, not who is sending it.
>>>
>>> -chris
>>
>> I am going to go out on a limb here and try to explain things.
>> Please note that this is all based upon reading between the
>> lines, and may not at all reflect what is actually going on.
>>
>> Overview --------
>>
>> I suspect the following:
>>
>> 1. Logrotate of catalina.out at 2 AM 2. Tomcat JVM fails to exit,
>> then restart
>>
>> Detail ------
>>
>> 1. Logrotate (or other log rotation utility)
>>
>> There are several ways that one can use to rotate catalina.out.
>> See the following:
>>
>> https://wiki.apache.org/tomcat/FAQ/Logging#Q10
>>
>> Some system admins actually stop Tomcat, rotate the logs, and
>> then start Tomcat. This has the advantage over the logrotate's
>> copytruncate option in that there is no possibility of partial
>> log entries.
>>
>> 2. JVM fails to exit
>>
>> From your error log, you have a TimeerTask thread that is not
>> shutting down. This prevents the JVM from exiting (see Chris's
>> comments). I suspect that this then prevents the start script
>> from starting Tomcat again (depending on the script).
>>
>> Solutions ---------
>>
>> 1. Fix your application so that the Tomcat JVM exits cleanly
>>
>> Use a thread pool. Manage the thread pool in a servlet context
>> listener (creation, destruction).
>>
>> This should be done in addition to anything else.
>>
>> 2. Talk to your system admin to see if log rotation is being
>> used
>>
>> Use copytruncate with logrotate rather than stopping and starting
>> the Tomca service.
>>
>> This is assuming that you're using logrotate, and that there is
>> a logrotate process that kicks off every morning at 2 AM.
>>
>> 3. Use another method for rotating catalina.out
>>
>> There are other methods for rotating catalina.out mentioned in
>> the link above.
>>
>> Again, this is assuming that your system admin has implemented
>> some log rotation which is causing the problem.
>>
>> 4. See Chris's comments above concerning potential security
>> issues
>>
>> Finally -------
>>
>> Your catalina.out file should be small, and consist of startup /
>> shutdown messages from Tomcat. Other (application) information
>> should go into application-specific log files. This means that
>> you should implement some sort of logging for your applications.
>>
>> In other words, there should be little need to periodically
>> rotate catalina.out.
>>
>> . . . just my two cents /mde/
>>
>>
>
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJZch4KAAoJEBzwKT+lPKRYl9wP/iJstzt2Yoymcvm0R3xeXL+B
W3S+U9L93PT4DTs9e/1wEvCDCAudwebmhNuTAqHtgwdm65LAJ06eQwRsPpIeYXUB
m+Ja5Pc4wM3bIDMupO3KBACiW2cAJ665oOQCf28R1RFqrgKJy8bn/yYxPXQBMCs1
JXA4AhjQvQTogap/DoPRv9HTEK9pO9SRdkPam+p3ahjJoPijFGDY8kgzP7IMtS2/
lKXnipfdX3NS7ngqcXu1ThsWvZmK2+CRLn4IIBht37S+nU0NaG4T+5m2Q1cGxfXi
zpMg6NnwB3EBpy4KLJYSloOCSsd20JIiKQibgunKLvXKkc073f3avXEljVOICSI8
UcGEIWWQHo8XjDvQ3+eFK6E9y6C2FL9YqzXyPDZ/HvB5nQx4f8BwL87D/nevW9jO
s8Xs/2BC1LOMYmjzBkyro8Q3nsWUneYBROM5MX74k+OyxVf8o93f+TV/8sdQdBy4
DNXh3z/7cIG3PesLYgx4NTAthlGbe5XBJBXAdhJMGOWNq2YC8ISh/tV+MeLcIpEC
qsKkITlA+PlKX6iK6bKB+fpE3/GkUUn+GFFyZyCLghPxkVcWvd5yHz1CfmiR+Tkz
7ebyBBMEIQ2Jipxb4v/4u9Xs0CpCg/g54YeaKCg7W3Sy0p83BZGv/El3Zndto7MP
pc0DBU0spS2GOiUxNkIp
=F41+
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat is stopping on its own even though stop script is not executed
Posted by Chaitanya Sabbineni <sa...@gmail.com>.
Hi All,
This is to add that I found some interesting thing. I order to enable
debugging in tomcat I changed the logging level from Fine to Finest .Upon
changing the log level we see that tomcat is not getting stopped on its
own.This is he strange behaviour we observed.
can you please elaborate what actually this mean
Fix your application so that the Tomcat JVM exits cleanly
Use a thread pool. Manage the thread pool in a servlet context listener
(creation, destruction).
can you let me know is there any way way to kill the timer thread during
the stop of tomcat.
And can you let me know what actually the timer thread task in the error
log.I mean if can point me to that snippet.
On Thu, 20 Jul 2017 11:37 pm Mark Eggers, <it...@yahoo.com.invalid>
wrote:
> Chaitanya,
>
> This will be long and somewhat speculative.
>
> On 7/20/2017 9:00 AM, Christopher Schultz wrote:
> > Chaitanya,
> >
> > On 7/20/17 11:03 AM, Chaitanya Sabbineni wrote:
> >> Stop script in the sense it's Catalina script only but we usually
> >> stop tomcat using the command Catalina.sh stop. But in our case we
> >> are not manually executing this script to stop tomcat and tomcat is
> >> stopping on its own.
> >
> >> our main problem here us tomcat is stopping on its own and it needs
> >> a restart.
> >
> > Right.
> >
> >> If I understand you correct you are telling TimerThread that does
> >> not stop when the application is shut down. Can you let me know
> >> what actually the timer thread mean. And moreover if the timer
> >> thread didn't stop ideally tomcat shouldn't stop but in our case
> >> it's stopping.
> >
> > Tomcat is stopping but the JVM is not. If your application were to
> > shut-down cleanly, then the JVM would exit as well. This is unrelated
> > to your real problem (unexpected Tomcat shutdown), but you might want
> > to look into fixing that, because it makes your application impossible
> > to reload without risking serious heap space problems.
> >
> >> Yes my question is why Tomcat is being shut down at all.
> >
> >> Yes when ever tomcat is stopping on own(not daily) it stops at
> >> 02:00 . You mentioned that your guess is that we are using a
> >> service runner that is configured to bounce your services at
> >> 02:00.Can let me know what this service runner is and how to check
> >> it.
> >
> > I know nothing about your environment. Until you mentioned
> > "catalina.sh stop" above, I didn't even know you were on a UNIX-like
> > environment. Honestly, I assumed you were on Windows because
> > "mysterious service stoppage" has Microsoft Windows behavior written
> > all over it.
> >
> > There are two ways to trigger a Tomcat shut down:
> >
> > 1. Send a TERM signal to the process
> > 2. Connect to Tomcat's shutdown listener (default: port 8005) and give
> > the shutdown command (default: "SHUTDOWN")
> >
> > You can eliminate one of those possibilities by setting the shutdown
> > port in server.xml to "" (empty) which will disable this type of shutdow
> > n:
> >
> > <Server port="" ...
> >
> > You cannot disable the other type of shutdown... any user on the
> > system who can send a TERM signal to your process could terminate Tomcat
> > .
> >
> > As for catching whoever is shutting down your Tomcat, you may want to
> > look at who has administrative access to your server, and who has
> > access to the user running your Tomcat server.
> >
> > Check your syslog to find sudo and cron events that might be
> > automatically shutting-down Tomcat.
> >
> > If you want to catch a TCP connection, you will likely have to enable
> > tcpwrappers, iptables, ipfw, etc. to log connections to port 8005.
> > Those logs will only tell you that the command is being sent, not who
> > is sending it.
> >
> > -chris
>
> I am going to go out on a limb here and try to explain things. Please
> note that this is all based upon reading between the lines, and may not
> at all reflect what is actually going on.
>
> Overview
> --------
>
> I suspect the following:
>
> 1. Logrotate of catalina.out at 2 AM
> 2. Tomcat JVM fails to exit, then restart
>
> Detail
> ------
>
> 1. Logrotate (or other log rotation utility)
>
> There are several ways that one can use to rotate catalina.out. See the
> following:
>
> https://wiki.apache.org/tomcat/FAQ/Logging#Q10
>
> Some system admins actually stop Tomcat, rotate the logs, and then start
> Tomcat. This has the advantage over the logrotate's copytruncate option
> in that there is no possibility of partial log entries.
>
> 2. JVM fails to exit
>
> From your error log, you have a TimeerTask thread that is not shutting
> down. This prevents the JVM from exiting (see Chris's comments). I
> suspect that this then prevents the start script from starting Tomcat
> again (depending on the script).
>
> Solutions
> ---------
>
> 1. Fix your application so that the Tomcat JVM exits cleanly
>
> Use a thread pool. Manage the thread pool in a servlet context listener
> (creation, destruction).
>
> This should be done in addition to anything else.
>
> 2. Talk to your system admin to see if log rotation is being used
>
> Use copytruncate with logrotate rather than stopping and starting the
> Tomca service.
>
> This is assuming that you're using logrotate, and that there is a
> logrotate process that kicks off every morning at 2 AM.
>
> 3. Use another method for rotating catalina.out
>
> There are other methods for rotating catalina.out mentioned in the link
> above.
>
> Again, this is assuming that your system admin has implemented some log
> rotation which is causing the problem.
>
> 4. See Chris's comments above concerning potential security issues
>
> Finally
> -------
>
> Your catalina.out file should be small, and consist of startup /
> shutdown messages from Tomcat. Other (application) information should go
> into application-specific log files. This means that you should
> implement some sort of logging for your applications.
>
> In other words, there should be little need to periodically rotate
> catalina.out.
>
> . . . just my two cents
> /mde/
>
>
Re: Tomcat is stopping on its own even though stop script is not
executed
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Chaitanya,
On 7/21/17 11:44 AM, Chaitanya Sabbineni wrote:
> Thanks a lot for the help.
>
> can you let me know how to check if there is a way to check if any
> log rotator is configured.
Look in /etc/logrotate* for log rotation settings
Look in /etc/cron* or system cron jobs
Look in /var/spool/* for user crontabs
If you get despirate, there is always:
# find / -type f -print0 | xargs -0 grep -i "catalina"
- -chris
> On Fri, 21 Jul 2017 8:54 pm Christopher Schultz, <
> chris@christopherschultz.net> wrote:
>
> Coty,
>
> On 7/21/17 8:51 AM, Coty Sutherland wrote:
>>>> On Thu, Jul 20, 2017 at 3:05 PM, Chaitanya Sabbineni
>>>> <sa...@gmail.com> wrote:
>>>>> Hi,
>>>>>
>>>>> just to explain you what exactly is happening. We are
>>>>> taking back up of the server at 12:00 AM daily.which is a
>>>>> type of offline backup.After backup server is running fine
>>>>> for 2 hours and exactly at 2:00am server is stopping on its
>>>>> own. This issue of stopping is not occurring daily.And when
>>>>> ever this issue occur it is at 2:00 AM.
>>>>
>>>> Always stopping at 2:00 AM is very suspicious. As you're on
>>>> linux and I've seen user experience similar things before
>>>> before, please verify that you don't have a cronjob that's
>>>> killing/restarting the service in addition to the previous
>>>> suggestion to check for logrotate. You said it doesn't stop
>>>> daily, but does it stop weekly or every N days? I'm sure that
>>>> there's some sort of pattern to this in addition to stopping
>>>> at a specific time.
>
> +1
>
> This is almost certainly some kind of log-rotation or other
> scheduled thing. Maybe it only happens on some days because the
> log-rotator only bothers to rotate the log when it exceeds a
> certain size. In that case, it checks every day at 02:00 but only
> tries to bounce Tomcat if the log is a certain size.
>
> Since the application doesn't shut-down cleanly, "bouncing" Tomcat
> fails .
>
> This is all pure speculation, but such a scenario would fit the
> evidence .
>
> -chris
>>
>> ---------------------------------------------------------------------
>>
>>
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
>>
>>
>
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJZciWQAAoJEBzwKT+lPKRY6dIQAL7cCmG9hXNMDyqvngPf0hxs
rlyOfNEgUmEHAiwgdpe7gFkVZhNXnB6w20RtFSHV4nqbLGNCi9Vx0bgypdpPOPps
heF5+ab6mrSG0b1SAA7WtuCYRBWYrmTlkemEG0HJdz6NXbgeIxM1eE31hQs067do
tdfnOSgN64b4IQDqt5Qt8TLJbafHcoAIvFB8jbQZxL+EFullM1EID4buZDVZonDp
0Wdkn28bjLxFxCzS/fFXcNnp65/o58OxSR3c8Etgod7zeNwgwHOKUoaQFKXbW4rH
Xfqav97CDt1F+0pXtuYYPBK+Q8R284wE0A53pFYCq4nwWAdmzzRM5MIfsSYEqvFY
b4L/4qY6PKYc1iOn2mRys1VkgJNz/hNS/f9FFQTUVUkM34hhrW/3BCKVxa2sWqds
71EPold/WGsjD8kb0uD5qxcvC7bcmTFFTXCx5i+Zgloo8SZo1Rdbxd3lO/BKnJ9N
C4h5WYI6Ojt4kGd3PeLTkgQTLrmE2YybVEZnilPiCWH02B+CgkmUQm66UBmexJa6
LfIVJvWkzHNd7ceBC7KsJlEMum0DGc3mTIxsC2nP8m3fLIutBrX/ySbB7cE4iabh
ZzCM1QkCubyoEXEGcqYJ2AHBe1i1cphfMNFsNNzoMmM5ixFa0fln5AJsbWFMnL+g
lU9RVuuhZC0gOtiIsvLz
=+yFy
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat is stopping on its own even though stop script is not executed
Posted by Chaitanya Sabbineni <sa...@gmail.com>.
Hi All,
Thanks a lot for the help.
can you let me know how to check if there is a way to check if any log
rotator is configured.
Thanks & Regards,
Chaitanya
On Fri, 21 Jul 2017 8:54 pm Christopher Schultz, <
chris@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Coty,
>
> On 7/21/17 8:51 AM, Coty Sutherland wrote:
> > On Thu, Jul 20, 2017 at 3:05 PM, Chaitanya Sabbineni
> > <sa...@gmail.com> wrote:
> >> Hi,
> >>
> >> just to explain you what exactly is happening. We are taking back
> >> up of the server at 12:00 AM daily.which is a type of offline
> >> backup.After backup server is running fine for 2 hours and
> >> exactly at 2:00am server is stopping on its own. This issue of
> >> stopping is not occurring daily.And when ever this issue occur it
> >> is at 2:00 AM.
> >
> > Always stopping at 2:00 AM is very suspicious. As you're on linux
> > and I've seen user experience similar things before before, please
> > verify that you don't have a cronjob that's killing/restarting the
> > service in addition to the previous suggestion to check for
> > logrotate. You said it doesn't stop daily, but does it stop weekly
> > or every N days? I'm sure that there's some sort of pattern to this
> > in addition to stopping at a specific time.
>
> +1
>
> This is almost certainly some kind of log-rotation or other scheduled
> thing. Maybe it only happens on some days because the log-rotator only
> bothers to rotate the log when it exceeds a certain size. In that
> case, it checks every day at 02:00 but only tries to bounce Tomcat if
> the log is a certain size.
>
> Since the application doesn't shut-down cleanly, "bouncing" Tomcat fails
> .
>
> This is all pure speculation, but such a scenario would fit the evidence
> .
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJZchydAAoJEBzwKT+lPKRYFI8P/RbVaRqNb9V0lZBU1bvZuTPc
> mtlrU7zvJ8lQd0/XNff0z6YRwLpx0YoB881ZWjROkR6ybZTce+CXDxi9wySnPsvR
> ij1slZhQhm1SBEb1hvWmYWop9Up17rb3O3XBF0EVkFuc/S8U0ZLD4lTSH+kjyCm4
> OpTWV+K+rUd87MGMO9CNQ114aRvHPuNObBDF2rjnC/sxJOlDV40hh+UmL2il4vgP
> NeLUZeQ0qfCPwHpjduKdCdUHIs5BrVgFz1gf3MYbGSK9VTAr0FGfJdrU+IKhsKrG
> pXWVsmLVN/EzkArxhwsLxoILwBIJsBiXse+uUOe1RryL0P4KNxYkI1GZRIjqCcKn
> bZN6Y7Jf2X+qpJ2zN0OmRWnGrjvJpvH+J0SAcvJDsZKRECJWnODUiAl4FDJXTDh+
> sdHhhR9KOzJua07OGJMIhXJVHIP7/8e+T1unHus1BWIms5bS+4hnz/CzZRaFsRHx
> O/Focpfd3vZj0YbkDcM5YZDu0FzwURptU8DM9eDOGDhCF/KDqDaVDDjwluPqOZLA
> sQV+l7nq639qVGcDaMEElT0EuFzbX4ik4zUwIhp9nVdt2uHTUy9Lceu4WGu5+SjV
> 50ykbwfRrN1p1f6qgeTqfUHl6d9O317+nnaGEUiJJIStjsTZGsamgRKYVZ3cbPIm
> bQjuv+aLAA4MvRpYTwBh
> =O9nD
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Tomcat is stopping on its own even though stop script is not
executed
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Coty,
On 7/21/17 8:51 AM, Coty Sutherland wrote:
> On Thu, Jul 20, 2017 at 3:05 PM, Chaitanya Sabbineni
> <sa...@gmail.com> wrote:
>> Hi,
>>
>> just to explain you what exactly is happening. We are taking back
>> up of the server at 12:00 AM daily.which is a type of offline
>> backup.After backup server is running fine for 2 hours and
>> exactly at 2:00am server is stopping on its own. This issue of
>> stopping is not occurring daily.And when ever this issue occur it
>> is at 2:00 AM.
>
> Always stopping at 2:00 AM is very suspicious. As you're on linux
> and I've seen user experience similar things before before, please
> verify that you don't have a cronjob that's killing/restarting the
> service in addition to the previous suggestion to check for
> logrotate. You said it doesn't stop daily, but does it stop weekly
> or every N days? I'm sure that there's some sort of pattern to this
> in addition to stopping at a specific time.
+1
This is almost certainly some kind of log-rotation or other scheduled
thing. Maybe it only happens on some days because the log-rotator only
bothers to rotate the log when it exceeds a certain size. In that
case, it checks every day at 02:00 but only tries to bounce Tomcat if
the log is a certain size.
Since the application doesn't shut-down cleanly, "bouncing" Tomcat fails
.
This is all pure speculation, but such a scenario would fit the evidence
.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJZchydAAoJEBzwKT+lPKRYFI8P/RbVaRqNb9V0lZBU1bvZuTPc
mtlrU7zvJ8lQd0/XNff0z6YRwLpx0YoB881ZWjROkR6ybZTce+CXDxi9wySnPsvR
ij1slZhQhm1SBEb1hvWmYWop9Up17rb3O3XBF0EVkFuc/S8U0ZLD4lTSH+kjyCm4
OpTWV+K+rUd87MGMO9CNQ114aRvHPuNObBDF2rjnC/sxJOlDV40hh+UmL2il4vgP
NeLUZeQ0qfCPwHpjduKdCdUHIs5BrVgFz1gf3MYbGSK9VTAr0FGfJdrU+IKhsKrG
pXWVsmLVN/EzkArxhwsLxoILwBIJsBiXse+uUOe1RryL0P4KNxYkI1GZRIjqCcKn
bZN6Y7Jf2X+qpJ2zN0OmRWnGrjvJpvH+J0SAcvJDsZKRECJWnODUiAl4FDJXTDh+
sdHhhR9KOzJua07OGJMIhXJVHIP7/8e+T1unHus1BWIms5bS+4hnz/CzZRaFsRHx
O/Focpfd3vZj0YbkDcM5YZDu0FzwURptU8DM9eDOGDhCF/KDqDaVDDjwluPqOZLA
sQV+l7nq639qVGcDaMEElT0EuFzbX4ik4zUwIhp9nVdt2uHTUy9Lceu4WGu5+SjV
50ykbwfRrN1p1f6qgeTqfUHl6d9O317+nnaGEUiJJIStjsTZGsamgRKYVZ3cbPIm
bQjuv+aLAA4MvRpYTwBh
=O9nD
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat is stopping on its own even though stop script is not executed
Posted by Coty Sutherland <cs...@redhat.com>.
On Thu, Jul 20, 2017 at 3:05 PM, Chaitanya Sabbineni
<sa...@gmail.com> wrote:
> Hi,
>
> just to explain you what exactly is happening. We are taking back up of the
> server at 12:00 AM daily.which is a type of offline backup.After backup
> server is running fine for 2 hours and exactly at 2:00am server is stopping
> on its own. This issue of stopping is not occurring daily.And when ever
> this issue occur it is at 2:00 AM.
Always stopping at 2:00 AM is very suspicious. As you're on linux and
I've seen user experience similar things before before, please verify
that you don't have a cronjob that's killing/restarting the service in
addition to the previous suggestion to check for logrotate. You said
it doesn't stop daily, but does it stop weekly or every N days? I'm
sure that there's some sort of pattern to this in addition to stopping
at a specific time.
> In order to check more logs we tried to increase the logging level from
> Fine to Finest and upon changing of the logging level ,server is not
> stopping on its own.
>
> when I listed the top command in Linux which will return all the running or
> active process it's not listing tomcat which means its stopped.
>
> I checked the available memory at that time and its 139mb.
>
> From the error log can anyone help in pointing to the timer task that is
> preventing the jvm to stop.
>
> Thanks in advance
>
> On Thu, 20 Jul 2017 11:37 pm Mark Eggers, <it...@yahoo.com.invalid>
> wrote:
>
>> Chaitanya,
>>
>> This will be long and somewhat speculative.
>>
>> On 7/20/2017 9:00 AM, Christopher Schultz wrote:
>> > Chaitanya,
>> >
>> > On 7/20/17 11:03 AM, Chaitanya Sabbineni wrote:
>> >> Stop script in the sense it's Catalina script only but we usually
>> >> stop tomcat using the command Catalina.sh stop. But in our case we
>> >> are not manually executing this script to stop tomcat and tomcat is
>> >> stopping on its own.
>> >
>> >> our main problem here us tomcat is stopping on its own and it needs
>> >> a restart.
>> >
>> > Right.
>> >
>> >> If I understand you correct you are telling TimerThread that does
>> >> not stop when the application is shut down. Can you let me know
>> >> what actually the timer thread mean. And moreover if the timer
>> >> thread didn't stop ideally tomcat shouldn't stop but in our case
>> >> it's stopping.
>> >
>> > Tomcat is stopping but the JVM is not. If your application were to
>> > shut-down cleanly, then the JVM would exit as well. This is unrelated
>> > to your real problem (unexpected Tomcat shutdown), but you might want
>> > to look into fixing that, because it makes your application impossible
>> > to reload without risking serious heap space problems.
>> >
>> >> Yes my question is why Tomcat is being shut down at all.
>> >
>> >> Yes when ever tomcat is stopping on own(not daily) it stops at
>> >> 02:00 . You mentioned that your guess is that we are using a
>> >> service runner that is configured to bounce your services at
>> >> 02:00.Can let me know what this service runner is and how to check
>> >> it.
>> >
>> > I know nothing about your environment. Until you mentioned
>> > "catalina.sh stop" above, I didn't even know you were on a UNIX-like
>> > environment. Honestly, I assumed you were on Windows because
>> > "mysterious service stoppage" has Microsoft Windows behavior written
>> > all over it.
>> >
>> > There are two ways to trigger a Tomcat shut down:
>> >
>> > 1. Send a TERM signal to the process
>> > 2. Connect to Tomcat's shutdown listener (default: port 8005) and give
>> > the shutdown command (default: "SHUTDOWN")
>> >
>> > You can eliminate one of those possibilities by setting the shutdown
>> > port in server.xml to "" (empty) which will disable this type of shutdow
>> > n:
>> >
>> > <Server port="" ...
>> >
>> > You cannot disable the other type of shutdown... any user on the
>> > system who can send a TERM signal to your process could terminate Tomcat
>> > .
>> >
>> > As for catching whoever is shutting down your Tomcat, you may want to
>> > look at who has administrative access to your server, and who has
>> > access to the user running your Tomcat server.
>> >
>> > Check your syslog to find sudo and cron events that might be
>> > automatically shutting-down Tomcat.
>> >
>> > If you want to catch a TCP connection, you will likely have to enable
>> > tcpwrappers, iptables, ipfw, etc. to log connections to port 8005.
>> > Those logs will only tell you that the command is being sent, not who
>> > is sending it.
>> >
>> > -chris
>>
>> I am going to go out on a limb here and try to explain things. Please
>> note that this is all based upon reading between the lines, and may not
>> at all reflect what is actually going on.
>>
>> Overview
>> --------
>>
>> I suspect the following:
>>
>> 1. Logrotate of catalina.out at 2 AM
>> 2. Tomcat JVM fails to exit, then restart
>>
>> Detail
>> ------
>>
>> 1. Logrotate (or other log rotation utility)
>>
>> There are several ways that one can use to rotate catalina.out. See the
>> following:
>>
>> https://wiki.apache.org/tomcat/FAQ/Logging#Q10
>>
>> Some system admins actually stop Tomcat, rotate the logs, and then start
>> Tomcat. This has the advantage over the logrotate's copytruncate option
>> in that there is no possibility of partial log entries.
>>
>> 2. JVM fails to exit
>>
>> From your error log, you have a TimeerTask thread that is not shutting
>> down. This prevents the JVM from exiting (see Chris's comments). I
>> suspect that this then prevents the start script from starting Tomcat
>> again (depending on the script).
>>
>> Solutions
>> ---------
>>
>> 1. Fix your application so that the Tomcat JVM exits cleanly
>>
>> Use a thread pool. Manage the thread pool in a servlet context listener
>> (creation, destruction).
>>
>> This should be done in addition to anything else.
>>
>> 2. Talk to your system admin to see if log rotation is being used
>>
>> Use copytruncate with logrotate rather than stopping and starting the
>> Tomca service.
>>
>> This is assuming that you're using logrotate, and that there is a
>> logrotate process that kicks off every morning at 2 AM.
>>
>> 3. Use another method for rotating catalina.out
>>
>> There are other methods for rotating catalina.out mentioned in the link
>> above.
>>
>> Again, this is assuming that your system admin has implemented some log
>> rotation which is causing the problem.
>>
>> 4. See Chris's comments above concerning potential security issues
>>
>> Finally
>> -------
>>
>> Your catalina.out file should be small, and consist of startup /
>> shutdown messages from Tomcat. Other (application) information should go
>> into application-specific log files. This means that you should
>> implement some sort of logging for your applications.
>>
>> In other words, there should be little need to periodically rotate
>> catalina.out.
>>
>> . . . just my two cents
>> /mde/
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat is stopping on its own even though stop script is not executed
Posted by Chaitanya Sabbineni <sa...@gmail.com>.
Hi,
just to explain you what exactly is happening. We are taking back up of the
server at 12:00 AM daily.which is a type of offline backup.After backup
server is running fine for 2 hours and exactly at 2:00am server is stopping
on its own. This issue of stopping is not occurring daily.And when ever
this issue occur it is at 2:00 AM.
In order to check more logs we tried to increase the logging level from
Fine to Finest and upon changing of the logging level ,server is not
stopping on its own.
when I listed the top command in Linux which will return all the running or
active process it's not listing tomcat which means its stopped.
I checked the available memory at that time and its 139mb.
From the error log can anyone help in pointing to the timer task that is
preventing the jvm to stop.
Thanks in advance
On Thu, 20 Jul 2017 11:37 pm Mark Eggers, <it...@yahoo.com.invalid>
wrote:
> Chaitanya,
>
> This will be long and somewhat speculative.
>
> On 7/20/2017 9:00 AM, Christopher Schultz wrote:
> > Chaitanya,
> >
> > On 7/20/17 11:03 AM, Chaitanya Sabbineni wrote:
> >> Stop script in the sense it's Catalina script only but we usually
> >> stop tomcat using the command Catalina.sh stop. But in our case we
> >> are not manually executing this script to stop tomcat and tomcat is
> >> stopping on its own.
> >
> >> our main problem here us tomcat is stopping on its own and it needs
> >> a restart.
> >
> > Right.
> >
> >> If I understand you correct you are telling TimerThread that does
> >> not stop when the application is shut down. Can you let me know
> >> what actually the timer thread mean. And moreover if the timer
> >> thread didn't stop ideally tomcat shouldn't stop but in our case
> >> it's stopping.
> >
> > Tomcat is stopping but the JVM is not. If your application were to
> > shut-down cleanly, then the JVM would exit as well. This is unrelated
> > to your real problem (unexpected Tomcat shutdown), but you might want
> > to look into fixing that, because it makes your application impossible
> > to reload without risking serious heap space problems.
> >
> >> Yes my question is why Tomcat is being shut down at all.
> >
> >> Yes when ever tomcat is stopping on own(not daily) it stops at
> >> 02:00 . You mentioned that your guess is that we are using a
> >> service runner that is configured to bounce your services at
> >> 02:00.Can let me know what this service runner is and how to check
> >> it.
> >
> > I know nothing about your environment. Until you mentioned
> > "catalina.sh stop" above, I didn't even know you were on a UNIX-like
> > environment. Honestly, I assumed you were on Windows because
> > "mysterious service stoppage" has Microsoft Windows behavior written
> > all over it.
> >
> > There are two ways to trigger a Tomcat shut down:
> >
> > 1. Send a TERM signal to the process
> > 2. Connect to Tomcat's shutdown listener (default: port 8005) and give
> > the shutdown command (default: "SHUTDOWN")
> >
> > You can eliminate one of those possibilities by setting the shutdown
> > port in server.xml to "" (empty) which will disable this type of shutdow
> > n:
> >
> > <Server port="" ...
> >
> > You cannot disable the other type of shutdown... any user on the
> > system who can send a TERM signal to your process could terminate Tomcat
> > .
> >
> > As for catching whoever is shutting down your Tomcat, you may want to
> > look at who has administrative access to your server, and who has
> > access to the user running your Tomcat server.
> >
> > Check your syslog to find sudo and cron events that might be
> > automatically shutting-down Tomcat.
> >
> > If you want to catch a TCP connection, you will likely have to enable
> > tcpwrappers, iptables, ipfw, etc. to log connections to port 8005.
> > Those logs will only tell you that the command is being sent, not who
> > is sending it.
> >
> > -chris
>
> I am going to go out on a limb here and try to explain things. Please
> note that this is all based upon reading between the lines, and may not
> at all reflect what is actually going on.
>
> Overview
> --------
>
> I suspect the following:
>
> 1. Logrotate of catalina.out at 2 AM
> 2. Tomcat JVM fails to exit, then restart
>
> Detail
> ------
>
> 1. Logrotate (or other log rotation utility)
>
> There are several ways that one can use to rotate catalina.out. See the
> following:
>
> https://wiki.apache.org/tomcat/FAQ/Logging#Q10
>
> Some system admins actually stop Tomcat, rotate the logs, and then start
> Tomcat. This has the advantage over the logrotate's copytruncate option
> in that there is no possibility of partial log entries.
>
> 2. JVM fails to exit
>
> From your error log, you have a TimeerTask thread that is not shutting
> down. This prevents the JVM from exiting (see Chris's comments). I
> suspect that this then prevents the start script from starting Tomcat
> again (depending on the script).
>
> Solutions
> ---------
>
> 1. Fix your application so that the Tomcat JVM exits cleanly
>
> Use a thread pool. Manage the thread pool in a servlet context listener
> (creation, destruction).
>
> This should be done in addition to anything else.
>
> 2. Talk to your system admin to see if log rotation is being used
>
> Use copytruncate with logrotate rather than stopping and starting the
> Tomca service.
>
> This is assuming that you're using logrotate, and that there is a
> logrotate process that kicks off every morning at 2 AM.
>
> 3. Use another method for rotating catalina.out
>
> There are other methods for rotating catalina.out mentioned in the link
> above.
>
> Again, this is assuming that your system admin has implemented some log
> rotation which is causing the problem.
>
> 4. See Chris's comments above concerning potential security issues
>
> Finally
> -------
>
> Your catalina.out file should be small, and consist of startup /
> shutdown messages from Tomcat. Other (application) information should go
> into application-specific log files. This means that you should
> implement some sort of logging for your applications.
>
> In other words, there should be little need to periodically rotate
> catalina.out.
>
> . . . just my two cents
> /mde/
>
>
Re: Tomcat is stopping on its own even though stop script is not
executed
Posted by Mark Eggers <it...@yahoo.com.INVALID>.
Chaitanya,
This will be long and somewhat speculative.
On 7/20/2017 9:00 AM, Christopher Schultz wrote:
> Chaitanya,
>
> On 7/20/17 11:03 AM, Chaitanya Sabbineni wrote:
>> Stop script in the sense it's Catalina script only but we usually
>> stop tomcat using the command Catalina.sh stop. But in our case we
>> are not manually executing this script to stop tomcat and tomcat is
>> stopping on its own.
>
>> our main problem here us tomcat is stopping on its own and it needs
>> a restart.
>
> Right.
>
>> If I understand you correct you are telling TimerThread that does
>> not stop when the application is shut down. Can you let me know
>> what actually the timer thread mean. And moreover if the timer
>> thread didn't stop ideally tomcat shouldn't stop but in our case
>> it's stopping.
>
> Tomcat is stopping but the JVM is not. If your application were to
> shut-down cleanly, then the JVM would exit as well. This is unrelated
> to your real problem (unexpected Tomcat shutdown), but you might want
> to look into fixing that, because it makes your application impossible
> to reload without risking serious heap space problems.
>
>> Yes my question is why Tomcat is being shut down at all.
>
>> Yes when ever tomcat is stopping on own(not daily) it stops at
>> 02:00 . You mentioned that your guess is that we are using a
>> service runner that is configured to bounce your services at
>> 02:00.Can let me know what this service runner is and how to check
>> it.
>
> I know nothing about your environment. Until you mentioned
> "catalina.sh stop" above, I didn't even know you were on a UNIX-like
> environment. Honestly, I assumed you were on Windows because
> "mysterious service stoppage" has Microsoft Windows behavior written
> all over it.
>
> There are two ways to trigger a Tomcat shut down:
>
> 1. Send a TERM signal to the process
> 2. Connect to Tomcat's shutdown listener (default: port 8005) and give
> the shutdown command (default: "SHUTDOWN")
>
> You can eliminate one of those possibilities by setting the shutdown
> port in server.xml to "" (empty) which will disable this type of shutdow
> n:
>
> <Server port="" ...
>
> You cannot disable the other type of shutdown... any user on the
> system who can send a TERM signal to your process could terminate Tomcat
> .
>
> As for catching whoever is shutting down your Tomcat, you may want to
> look at who has administrative access to your server, and who has
> access to the user running your Tomcat server.
>
> Check your syslog to find sudo and cron events that might be
> automatically shutting-down Tomcat.
>
> If you want to catch a TCP connection, you will likely have to enable
> tcpwrappers, iptables, ipfw, etc. to log connections to port 8005.
> Those logs will only tell you that the command is being sent, not who
> is sending it.
>
> -chris
I am going to go out on a limb here and try to explain things. Please
note that this is all based upon reading between the lines, and may not
at all reflect what is actually going on.
Overview
--------
I suspect the following:
1. Logrotate of catalina.out at 2 AM
2. Tomcat JVM fails to exit, then restart
Detail
------
1. Logrotate (or other log rotation utility)
There are several ways that one can use to rotate catalina.out. See the
following:
https://wiki.apache.org/tomcat/FAQ/Logging#Q10
Some system admins actually stop Tomcat, rotate the logs, and then start
Tomcat. This has the advantage over the logrotate's copytruncate option
in that there is no possibility of partial log entries.
2. JVM fails to exit
From your error log, you have a TimeerTask thread that is not shutting
down. This prevents the JVM from exiting (see Chris's comments). I
suspect that this then prevents the start script from starting Tomcat
again (depending on the script).
Solutions
---------
1. Fix your application so that the Tomcat JVM exits cleanly
Use a thread pool. Manage the thread pool in a servlet context listener
(creation, destruction).
This should be done in addition to anything else.
2. Talk to your system admin to see if log rotation is being used
Use copytruncate with logrotate rather than stopping and starting the
Tomca service.
This is assuming that you're using logrotate, and that there is a
logrotate process that kicks off every morning at 2 AM.
3. Use another method for rotating catalina.out
There are other methods for rotating catalina.out mentioned in the link
above.
Again, this is assuming that your system admin has implemented some log
rotation which is causing the problem.
4. See Chris's comments above concerning potential security issues
Finally
-------
Your catalina.out file should be small, and consist of startup /
shutdown messages from Tomcat. Other (application) information should go
into application-specific log files. This means that you should
implement some sort of logging for your applications.
In other words, there should be little need to periodically rotate
catalina.out.
. . . just my two cents
/mde/
Re: Tomcat is stopping on its own even though stop script is not
executed
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Chaitanya,
On 7/20/17 11:03 AM, Chaitanya Sabbineni wrote:
> Stop script in the sense it's Catalina script only but we usually
> stop tomcat using the command Catalina.sh stop. But in our case we
> are not manually executing this script to stop tomcat and tomcat is
> stopping on its own.
>
> our main problem here us tomcat is stopping on its own and it needs
> a restart.
Right.
> If I understand you correct you are telling TimerThread that does
> not stop when the application is shut down. Can you let me know
> what actually the timer thread mean. And moreover if the timer
> thread didn't stop ideally tomcat shouldn't stop but in our case
> it's stopping.
Tomcat is stopping but the JVM is not. If your application were to
shut-down cleanly, then the JVM would exit as well. This is unrelated
to your real problem (unexpected Tomcat shutdown), but you might want
to look into fixing that, because it makes your application impossible
to reload without risking serious heap space problems.
> Yes my question is why Tomcat is being shut down at all.
>
> Yes when ever tomcat is stopping on own(not daily) it stops at
> 02:00 . You mentioned that your guess is that we are using a
> service runner that is configured to bounce your services at
> 02:00.Can let me know what this service runner is and how to check
> it.
I know nothing about your environment. Until you mentioned
"catalina.sh stop" above, I didn't even know you were on a UNIX-like
environment. Honestly, I assumed you were on Windows because
"mysterious service stoppage" has Microsoft Windows behavior written
all over it.
There are two ways to trigger a Tomcat shut down:
1. Send a TERM signal to the process
2. Connect to Tomcat's shutdown listener (default: port 8005) and give
the shutdown command (default: "SHUTDOWN")
You can eliminate one of those possibilities by setting the shutdown
port in server.xml to "" (empty) which will disable this type of shutdow
n:
<Server port="" ...
You cannot disable the other type of shutdown... any user on the
system who can send a TERM signal to your process could terminate Tomcat
.
As for catching whoever is shutting down your Tomcat, you may want to
look at who has administrative access to your server, and who has
access to the user running your Tomcat server.
Check your syslog to find sudo and cron events that might be
automatically shutting-down Tomcat.
If you want to catch a TCP connection, you will likely have to enable
tcpwrappers, iptables, ipfw, etc. to log connections to port 8005.
Those logs will only tell you that the command is being sent, not who
is sending it.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJZcNOlAAoJEBzwKT+lPKRYsC8QAL14/tZgd/Mg/DXFmGBLXfUh
CpCYGPfgEpOLPrEqxbjdpP0172v9tp0+i/DXSJE4kHYUgr3mK0U3Y5Nzyonhttdd
yeloPlZfPvVp9VewDd4RlXGda3D/g0r7+6eFXX14Hj1xBVLNitOQ1sRPF/JL9aJS
Itjz0q3B6t2ZLIS6sXiOnmMfFjQs986C1R+YVKTkst1l6k+wwyqCPWXCGm/lWLqq
MwtTOPq159BGmWqh7T2jHTE/eZ7f81Kzw4mXSh0jPsO3da+2mElzES0LPWoSh3fi
5mkuDUjOqp9OEEPLxkM5HiOEIgy9z0K8WNrSoAKtZpRNezHNmA7zx5rT+rug+aI/
2GgTBBcPPjuhorO+/IrdneDFpUO3wpcHpkFkoMlqK4e3UaJ5GHByHAF4mn5ssQ38
iQMAwIP7cWtzdEPy0NsjzU7bkMrawNi95U6KDYuZjspz/HEzrgqJKEc4+o3rArVg
bd1p8L0PzR3io3Qp9LqnADbEPAbFFSUrj7yqvAdmFTm3bk1A5Z4ljCFnFKI8EG+Z
UILGSNovPwKrcyAFcYkdr37xZWObW+BnLtOt8wSuyWr8PDBoTkkMp30KJ+wMXI4P
aqnPFdiX88W33LoAIByJ0uDNMnxxyUHVXaWkQ2oGagoJ7DwnK58KredV2peaoeyY
GISWZgEaqQxipyI0uZg6
=p4Sz
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat is stopping on its own even though stop script is not executed
Posted by Chaitanya Sabbineni <sa...@gmail.com>.
Hi All,
can you please let me know on how to identify which component or the
application is making the main thread in the tomcat to stop. Is there way
to check what would be the reason because of which tomcat is stopping.
Thanks & Regards,
Chaitanya
On Thu, 20 Jul 2017 9:09 pm Christopher Schultz, <
chris@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Suvendu,
>
> On 7/20/17 11:25 AM, Suvendu Sekhar Mondal wrote:
> > Chaitanya,
> >
> > On Thu, Jul 20, 2017 at 8:33 PM, Chaitanya Sabbineni
> > <sa...@gmail.com> wrote:
> >> Hi Chris,
> >>
> >> Stop script in the sense it's Catalina script only but we usually
> >> stop tomcat using the command Catalina.sh stop. But in our case
> >> we are not manually executing this script to stop tomcat and
> >> tomcat is stopping on its own.
> >>
> >> our main problem here us tomcat is stopping on its own and it
> >> needs a restart.
> >>
> >> If I understand you correct you are telling TimerThread that does
> >> not stop when the application is shut down. Can you let me know
> >> what actually the timer thread mean. And moreover if the timer
> >> thread didn't stop ideally tomcat shouldn't stop but in our case
> >> it's stopping.
> >>
> >> Yes my question is why Tomcat is being shut down at all.
> >>
> >> Yes when ever tomcat is stopping on own(not daily) it stops at
> >> 02:00 . You mentioned that your guess is that we are using a
> >> service runner that is configured to bounce your services at
> >> 02:00.Can let me know what this service runner is and how to
> >> check it.
> >>
> >
> > First, is that a full thread dump? One interesting thing in your
> > dumps is that there is no "main" thread. This is the first time I
> > am seeing a thread dump like this.
>
> The main thread has exited, because Tomcat has done all it will do in
> order to shut down. System.exit is not called, only the main thread
> ends. The TimerTask threads are non-daemon threads, so they keep the
> JVM from shutting down.
>
> > Second thing is, it seems that you are running some type of
> > scheduler. Can you please check if that is not doing anything
> > crazy?
> > "PASS_DEFAULT_GROUP_com.j.passj.core.config.scheduler.LicenseRemovalAf
> terExpiry"
> >
> >
> #53 daemon prio=5 os_prio=0 tid=0x00007f882461a000 nid=0x1942 in
> > Object.wait() [0x00007f8810b1f000]
> >
> > One last thing, can you please check Tomcat is not shutting down
> > due to OutOfMemory?
>
> Tomcat would not do that.
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJZcM6lAAoJEBzwKT+lPKRYZiEQAIoC2tAGloNk76PznITq3Nu8
> sIz4mxVS1+NGFdSSuCcIqk1kJM8FfUwHIxlmRg4JyrmkJTbZuaSJtzT6Xypeuj2/
> uXKa+z2Me33hSLuJlO5rm1Q92X7tMsm3xqo6Nsg5bmrk0euLi3+W/57MT3cNYeTi
> FzAxJpYxnr6isGhejnXexYzHdJYayHLHFU2CVNphugcsA2zk1mTbg4bgi+ROhAXB
> 7QJY9H5BzMVB7N2Ix3oNb9feM8vtm13qjMDHEGXMoNtBPLrCV9u4EQ3wo+Vm1Ajp
> 6/uTY/s7jtxPtk8Cq2aX7cIVAsVVJFnpdExobihk2TmGtAbNmiGiyu4puJcVb15L
> MlF+KCLTtVOKudptFk9CJ0vv9fiZVRE7Wm979/aBXpOjx4dhXdSH/xUyouQLmM5v
> fwIBZv4w05Y9lZ1cxmIfnM3yluxMm0y55MopFSr7d45RwVL1O4pmR06cnb/kvu4c
> z/l2W5LDgJOxPeBrjAAyepPTUxpf5mLTjMx3D1Mmaw2FYBr58FT9zz5oPcBwSTJs
> 6EYgw/ckVJ5x18QR1gi9mODS/G5qAt/mAJ0KYEis9zQL281nXkdEaad6QM538MgT
> rjCjrtwvMER2f+1jkqR4merm3WbIExid1xqQpzEGdlu1whqROreiuOxoX6HwR2jg
> hnzMpL9pTpxEYJLTuu0y
> =d5ho
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Tomcat is stopping on its own even though stop script is not
executed
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Suvendu,
On 7/20/17 11:25 AM, Suvendu Sekhar Mondal wrote:
> Chaitanya,
>
> On Thu, Jul 20, 2017 at 8:33 PM, Chaitanya Sabbineni
> <sa...@gmail.com> wrote:
>> Hi Chris,
>>
>> Stop script in the sense it's Catalina script only but we usually
>> stop tomcat using the command Catalina.sh stop. But in our case
>> we are not manually executing this script to stop tomcat and
>> tomcat is stopping on its own.
>>
>> our main problem here us tomcat is stopping on its own and it
>> needs a restart.
>>
>> If I understand you correct you are telling TimerThread that does
>> not stop when the application is shut down. Can you let me know
>> what actually the timer thread mean. And moreover if the timer
>> thread didn't stop ideally tomcat shouldn't stop but in our case
>> it's stopping.
>>
>> Yes my question is why Tomcat is being shut down at all.
>>
>> Yes when ever tomcat is stopping on own(not daily) it stops at
>> 02:00 . You mentioned that your guess is that we are using a
>> service runner that is configured to bounce your services at
>> 02:00.Can let me know what this service runner is and how to
>> check it.
>>
>
> First, is that a full thread dump? One interesting thing in your
> dumps is that there is no "main" thread. This is the first time I
> am seeing a thread dump like this.
The main thread has exited, because Tomcat has done all it will do in
order to shut down. System.exit is not called, only the main thread
ends. The TimerTask threads are non-daemon threads, so they keep the
JVM from shutting down.
> Second thing is, it seems that you are running some type of
> scheduler. Can you please check if that is not doing anything
> crazy?
> "PASS_DEFAULT_GROUP_com.j.passj.core.config.scheduler.LicenseRemovalAf
terExpiry"
>
>
#53 daemon prio=5 os_prio=0 tid=0x00007f882461a000 nid=0x1942 in
> Object.wait() [0x00007f8810b1f000]
>
> One last thing, can you please check Tomcat is not shutting down
> due to OutOfMemory?
Tomcat would not do that.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJZcM6lAAoJEBzwKT+lPKRYZiEQAIoC2tAGloNk76PznITq3Nu8
sIz4mxVS1+NGFdSSuCcIqk1kJM8FfUwHIxlmRg4JyrmkJTbZuaSJtzT6Xypeuj2/
uXKa+z2Me33hSLuJlO5rm1Q92X7tMsm3xqo6Nsg5bmrk0euLi3+W/57MT3cNYeTi
FzAxJpYxnr6isGhejnXexYzHdJYayHLHFU2CVNphugcsA2zk1mTbg4bgi+ROhAXB
7QJY9H5BzMVB7N2Ix3oNb9feM8vtm13qjMDHEGXMoNtBPLrCV9u4EQ3wo+Vm1Ajp
6/uTY/s7jtxPtk8Cq2aX7cIVAsVVJFnpdExobihk2TmGtAbNmiGiyu4puJcVb15L
MlF+KCLTtVOKudptFk9CJ0vv9fiZVRE7Wm979/aBXpOjx4dhXdSH/xUyouQLmM5v
fwIBZv4w05Y9lZ1cxmIfnM3yluxMm0y55MopFSr7d45RwVL1O4pmR06cnb/kvu4c
z/l2W5LDgJOxPeBrjAAyepPTUxpf5mLTjMx3D1Mmaw2FYBr58FT9zz5oPcBwSTJs
6EYgw/ckVJ5x18QR1gi9mODS/G5qAt/mAJ0KYEis9zQL281nXkdEaad6QM538MgT
rjCjrtwvMER2f+1jkqR4merm3WbIExid1xqQpzEGdlu1whqROreiuOxoX6HwR2jg
hnzMpL9pTpxEYJLTuu0y
=d5ho
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat is stopping on its own even though stop script is not executed
Posted by Chaitanya Sabbineni <sa...@gmail.com>.
Hi,
I checked the memory ,we are running on the 4 gb ram machine where 3.8 gb
is almost full we are left with 150 mb free memory. If that is issue with
memory then it should stop during the start time only but when ever it stop
its stopping at 2:00 am.
On Thu, 20 Jul 2017 8:56 pm Suvendu Sekhar Mondal, <su...@gmail.com>
wrote:
> Chaitanya,
>
> On Thu, Jul 20, 2017 at 8:33 PM, Chaitanya Sabbineni
> <sa...@gmail.com> wrote:
> > Hi Chris,
> >
> > Stop script in the sense it's Catalina script only but we usually stop
> > tomcat using the command Catalina.sh stop. But in our case we are not
> > manually executing this script to stop tomcat and tomcat is stopping on
> its
> > own.
> >
> > our main problem here us tomcat is stopping on its own and it needs a
> > restart.
> >
> > If I understand you correct you are telling TimerThread that does not
> stop
> > when the application is shut down. Can you let me know what actually the
> > timer thread mean. And moreover if the timer thread didn't stop ideally
> > tomcat shouldn't stop but in our case it's stopping.
> >
> > Yes my question is why Tomcat is being shut down at all.
> >
> > Yes when ever tomcat is stopping on own(not daily) it stops at 02:00 .
> You
> > mentioned that your guess is that we are using a service runner that is
> > configured to bounce your services at 02:00.Can let me know what this
> > service runner is and how to check it.
> >
>
> First, is that a full thread dump? One interesting thing in your dumps
> is that there is no "main" thread. This is the first time I am seeing
> a thread dump like this.
>
> Second thing is, it seems that you are running some type of scheduler.
> Can you please check if that is not doing anything crazy?
>
> "PASS_DEFAULT_GROUP_com.j.passj.core.config.scheduler.LicenseRemovalAfterExpiry"
> #53 daemon prio=5 os_prio=0 tid=0x00007f882461a000 nid=0x1942 in
> Object.wait() [0x00007f8810b1f000]
>
> One last thing, can you please check Tomcat is not shutting down due
> to OutOfMemory?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Tomcat is stopping on its own even though stop script is not executed
Posted by Suvendu Sekhar Mondal <su...@gmail.com>.
Chaitanya,
On Thu, Jul 20, 2017 at 8:33 PM, Chaitanya Sabbineni
<sa...@gmail.com> wrote:
> Hi Chris,
>
> Stop script in the sense it's Catalina script only but we usually stop
> tomcat using the command Catalina.sh stop. But in our case we are not
> manually executing this script to stop tomcat and tomcat is stopping on its
> own.
>
> our main problem here us tomcat is stopping on its own and it needs a
> restart.
>
> If I understand you correct you are telling TimerThread that does not stop
> when the application is shut down. Can you let me know what actually the
> timer thread mean. And moreover if the timer thread didn't stop ideally
> tomcat shouldn't stop but in our case it's stopping.
>
> Yes my question is why Tomcat is being shut down at all.
>
> Yes when ever tomcat is stopping on own(not daily) it stops at 02:00 . You
> mentioned that your guess is that we are using a service runner that is
> configured to bounce your services at 02:00.Can let me know what this
> service runner is and how to check it.
>
First, is that a full thread dump? One interesting thing in your dumps
is that there is no "main" thread. This is the first time I am seeing
a thread dump like this.
Second thing is, it seems that you are running some type of scheduler.
Can you please check if that is not doing anything crazy?
"PASS_DEFAULT_GROUP_com.j.passj.core.config.scheduler.LicenseRemovalAfterExpiry"
#53 daemon prio=5 os_prio=0 tid=0x00007f882461a000 nid=0x1942 in
Object.wait() [0x00007f8810b1f000]
One last thing, can you please check Tomcat is not shutting down due
to OutOfMemory?
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat is stopping on its own even though stop script is not executed
Posted by Chaitanya Sabbineni <sa...@gmail.com>.
Hi Chris,
Stop script in the sense it's Catalina script only but we usually stop
tomcat using the command Catalina.sh stop. But in our case we are not
manually executing this script to stop tomcat and tomcat is stopping on its
own.
our main problem here us tomcat is stopping on its own and it needs a
restart.
If I understand you correct you are telling TimerThread that does not stop
when the application is shut down. Can you let me know what actually the
timer thread mean. And moreover if the timer thread didn't stop ideally
tomcat shouldn't stop but in our case it's stopping.
Yes my question is why Tomcat is being shut down at all.
Yes when ever tomcat is stopping on own(not daily) it stops at 02:00 . You
mentioned that your guess is that we are using a service runner that is
configured to bounce your services at 02:00.Can let me know what this
service runner is and how to check it.
Thanks & Regards,
Chaitanya
On Thu, 20 Jul 2017 8:10 pm Christopher Schultz, <
chris@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Chaitanya,
>
> On 7/20/17 2:57 AM, Chaitanya Sabbineni wrote:
> > i am using tomcat 7.0.069 version where tomcat is stopping at a
> > specified time(02:00AM) even though the stop script of the tomcat
> > is not executed.
>
> Why do you say that the stop script is not executed?
>
> > Below is the *error *which i can see in the tomcat log at 2:00 AM
> > when the server is stopped.In order to start the sever we need to
> > execute the start script.
>
> So Tomcat is shutting down, but you have a TimerThread that does not
> stop when the application is shut-down. I'd say you have a bug in your
> application because of that, but I think your real question is why
> Tomcat is being shut down at all.
>
> > Can anyone please let me know what is the problem or can any one
> > suggest a solution.
>
> 02:00 sounds suspicious. My guess is that you are using a service
> runner that is configured to bounce your services at 02:00, and you
> just don't know about it.
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQIcBAEBCAAGBQJZcMDJAAoJEBzwKT+lPKRY5EwP/1iTW/nQCuYbX0siCl8xoa5Y
> ZGDZCTWzSFK/4gaZL6R871REbv6KvfpL2+zIzMMnuXvVXSpvbZocfE57LuLRSfEL
> rN2ChW/OWkF3K4A0E7mMKws/fdpuglbSZlfoGAzTlDbBPHxYC/1F9xraNrVXi2gt
> 1VPJgPs9v+UlSZIpFbk8NcrB7Ury8cW7BfMsaCRBmCIXATSzjBh98Igmfg5hSnwH
> ndYIIxg47sd9Nno4nvx5KZfE0KC2ybcGjdjDSfkHlgfjcOhTrQJEJeA5etcQdktr
> vMg03lWYkCzBKAXLP2gsfU7Fk9TtMAnRzWouTC8ikPVff6kPcMR2K4AKwhyBYIRg
> /B0tiEWywnO6dwLjIHT8o3ylc4cMpX2O793hwRx5pM5f5RXrSYCPWQlk38l8D3tP
> 5ru2evgM9smTbWAcu9LNwjYMY6mqdydHDXoKlNkOoXYlSuQbyRA/KKYvq0OgISy1
> cHyrQRmNgauu8YqC0olYi5UwmRgG10tAouhup2O+UI/zSUuuTbhxvK+jQGYT/4NK
> BwiHoloZI+zwKI9ZWhPLm8lp2QxpygzzTPZAImUEbbsJB5u1/mwIxJ874uZ+kDDU
> l1nHheEVoRYU4nwPwjIulbfUdKoC9ZFdGqQPOptK25t0QCH3MkCCALqsSbwI9JUS
> Hd/SJscNg86hdO8O/EjQ
> =7bU2
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
Re: Tomcat is stopping on its own even though stop script is not
executed
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Chaitanya,
On 7/20/17 2:57 AM, Chaitanya Sabbineni wrote:
> i am using tomcat 7.0.069 version where tomcat is stopping at a
> specified time(02:00AM) even though the stop script of the tomcat
> is not executed.
Why do you say that the stop script is not executed?
> Below is the *error *which i can see in the tomcat log at 2:00 AM
> when the server is stopped.In order to start the sever we need to
> execute the start script.
So Tomcat is shutting down, but you have a TimerThread that does not
stop when the application is shut-down. I'd say you have a bug in your
application because of that, but I think your real question is why
Tomcat is being shut down at all.
> Can anyone please let me know what is the problem or can any one
> suggest a solution.
02:00 sounds suspicious. My guess is that you are using a service
runner that is configured to bounce your services at 02:00, and you
just don't know about it.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJZcMDJAAoJEBzwKT+lPKRY5EwP/1iTW/nQCuYbX0siCl8xoa5Y
ZGDZCTWzSFK/4gaZL6R871REbv6KvfpL2+zIzMMnuXvVXSpvbZocfE57LuLRSfEL
rN2ChW/OWkF3K4A0E7mMKws/fdpuglbSZlfoGAzTlDbBPHxYC/1F9xraNrVXi2gt
1VPJgPs9v+UlSZIpFbk8NcrB7Ury8cW7BfMsaCRBmCIXATSzjBh98Igmfg5hSnwH
ndYIIxg47sd9Nno4nvx5KZfE0KC2ybcGjdjDSfkHlgfjcOhTrQJEJeA5etcQdktr
vMg03lWYkCzBKAXLP2gsfU7Fk9TtMAnRzWouTC8ikPVff6kPcMR2K4AKwhyBYIRg
/B0tiEWywnO6dwLjIHT8o3ylc4cMpX2O793hwRx5pM5f5RXrSYCPWQlk38l8D3tP
5ru2evgM9smTbWAcu9LNwjYMY6mqdydHDXoKlNkOoXYlSuQbyRA/KKYvq0OgISy1
cHyrQRmNgauu8YqC0olYi5UwmRgG10tAouhup2O+UI/zSUuuTbhxvK+jQGYT/4NK
BwiHoloZI+zwKI9ZWhPLm8lp2QxpygzzTPZAImUEbbsJB5u1/mwIxJ874uZ+kDDU
l1nHheEVoRYU4nwPwjIulbfUdKoC9ZFdGqQPOptK25t0QCH3MkCCALqsSbwI9JUS
Hd/SJscNg86hdO8O/EjQ
=7bU2
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org