You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by James Lampert <ja...@touchtonecorp.com> on 2012/11/27 18:10:11 UTC

Shutdown.sh doesn't. At least not reliably. (7.0.25)

Running Tomcat on various AS/400s (V6R1 or later OS), we've found that 
shutdown.sh doesn't reliably shut down the server, and we frequently 
have to shut it down forcibly (i.e., finding CATALINA on a WRKACTJOB, 
and giving it a "4" with "OPTION(*IMMED)").

Port 8009 does appear to be open before we call shutdown.sh, and it does 
appear to be closed after we call it.

Any idea what the problem could be? Or where I should look for 
indications of what it could be?

--
JHHL

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


RE: Shutdown.sh doesn't. At least not reliably. (7.0.25)

Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: James Lampert [mailto:jamesl@touchtonecorp.com] 
> Subject: Shutdown.sh doesn't. At least not reliably. (7.0.25)

> Or where I should look for indications of what it could be?

Can you take a thread dump of the Tomcat process after shutdown.sh has been used?
http://wiki.apache.org/tomcat/HowTo#How_do_I_obtain_a_thread_dump_of_my_running_webapp_.3F

Can you try it on more recent Tomcat?

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Shutdown.sh doesn't. At least not reliably. (7.0.25)

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

James,

On 11/27/12 3:56 PM, James Lampert wrote:
> Christopher Schultz wrote:
> 
>> Maybe use DMPJVM? Sorry for the through-Google link [PDF]:
> 
> THANKS! Never heard of such a thing until you brought it to my 
> attention, and it's enough of an eye-opener that I would have
> gladly forgiven even a through-LMGTFY link.
> 
> DMPJVM looks like it may be just the thing. I don't want to risk 
> unnecessarily disrupting anybody right now, but later this
> afternoon, I'm going to run some test shutdowns.

If you get it figured out, please amend the wiki to include
instructions for AS/400. I don't have an AS/400 system handy, so I
can't do it myself. I'd probably go mad in the process, anyway ;)

I found some instructions for iSeries on the SAP website
(http://www.stechno.net/sap-notes.html?view=sapnote&id=710154) but I
wasn't going to copy-and-paste that or even link to it unless it
actually made some kind of sense.

> Among the responses I got on java400-l@midrange.com, one fellow
> user remarked that he'd been seeing the same phenomenon, and
> another had this comment:
>> Tomcat won't shut down while there are non-daemon threads
>> running.
> 
> Still another turned me on to WRKJVMJOB (yet another command I'd
> never heard of). Applying option 11 of that ("Display Threads") to
> the QP0ZSPWT job (the only active QP0ZSPWT in the system, evidently
> slaved to the CATALINA job), I get: Thread      Name
> Status 0000515F    main              TIMW 00005162    JIT
> Compilatio >  THDW 00005164    JVMTI event re >  THDW 00005165
> Signal Dispatc >  THDW 00005167    Gc Slave Thread   THDW 00005168
> i5/OS informat >  JVAW 0000516A    MemoryPoolMXBe >  THDW 0000516B
> Attach API wai >  SEMW 0000516C    GC Daemon         THDW 0000516D
> Finalizer thread  THDW 00005172    ContainerBackg >  THDW 00005173
> http-bio-8080- >  TIMW 00005174    http-bio-8080- >  THDW 00005175
> http-bio-443-A >  TIMW 00005176    http-bio-443-A >  THDW 00005177
> ajp-bio-8009-A >  TIMW 00005178    ajp-bio-8009-A >  THDW 000051BB
> http-bio-443-e >  THDW 000051BC    http-bio-443-e >  THDW 000051BD
> http-bio-443-e >  THDW 000051BE    http-bio-443-e >  THDW 000051BF
> http-bio-8080- >  THDW 000051C0    http-bio-8080- >  THDW 000051C1
> http-bio-8080- >  THDW 000051C4    http-bio-443-e >  THDW 000051DD
> http-bio-443-e >  THDW 000051DE    http-bio-443-e >  THDW 000051DF
> http-bio-443-e >  THDW 000051E0    http-bio-443-e >  THDW 000051E1
> http-bio-443-e >  THDW 000051E8    Java2D Disposer   THDW 000051F6
> http-bio-8080- >  THDW 000051F7    http-bio-8080- >  THDW 000051F8
> http-bio-8080- >  THDW 000051F9    http-bio-8080- >  THDW 000051FA
> http-bio-8080- >  THDW 000051FC    http-bio-8080- >  THDW 0000520B
> http-bio-8080- >  THDW (funny, a few seconds ago, it didn't have
> nearly that many threads; evidently it is indeed currently in use).
> I note that option 12 appears to be plumbed into DMPJVM.

That certainly looks like a thread list at least. Now you just need to
try to get the stack trace of each of those threads.

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEARECAAYFAlC1RPQACgkQ9CaO5/Lv0PDZ8QCgkpIOtziAQN6EmyIP0st/uJmx
5C8AmwZyyt+UlH8/+93MddR5VJqO26Dr
=VXVF
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Shutdown.sh doesn't. At least not reliably. (7.0.25)

Posted by James Lampert <ja...@touchtonecorp.com>.
Christopher Schultz wrote:

> Maybe use DMPJVM? Sorry for the through-Google link [PDF]:

THANKS! Never heard of such a thing until you brought it to my 
attention, and it's enough of an eye-opener that I would have gladly 
forgiven even a through-LMGTFY link.

DMPJVM looks like it may be just the thing. I don't want to risk 
unnecessarily disrupting anybody right now, but later this afternoon, 
I'm going to run some test shutdowns.

Among the responses I got on java400-l@midrange.com, one fellow user 
remarked that he'd been seeing the same phenomenon, and another had this 
comment:
> Tomcat won't shut down while there are non-daemon threads running.

Still another turned me on to WRKJVMJOB (yet another command I'd never 
heard of). Applying option 11 of that ("Display Threads") to the 
QP0ZSPWT job (the only active QP0ZSPWT in the system, evidently slaved 
to the CATALINA job), I get:
  Thread      Name              Status
  0000515F    main              TIMW
  00005162    JIT Compilatio >  THDW
  00005164    JVMTI event re >  THDW
  00005165    Signal Dispatc >  THDW
  00005167    Gc Slave Thread   THDW
  00005168    i5/OS informat >  JVAW
  0000516A    MemoryPoolMXBe >  THDW
  0000516B    Attach API wai >  SEMW
  0000516C    GC Daemon         THDW
  0000516D    Finalizer thread  THDW
  00005172    ContainerBackg >  THDW
  00005173    http-bio-8080- >  TIMW
  00005174    http-bio-8080- >  THDW
  00005175    http-bio-443-A >  TIMW
  00005176    http-bio-443-A >  THDW
  00005177    ajp-bio-8009-A >  TIMW
  00005178    ajp-bio-8009-A >  THDW
  000051BB    http-bio-443-e >  THDW
  000051BC    http-bio-443-e >  THDW
  000051BD    http-bio-443-e >  THDW
  000051BE    http-bio-443-e >  THDW
  000051BF    http-bio-8080- >  THDW
  000051C0    http-bio-8080- >  THDW
  000051C1    http-bio-8080- >  THDW
  000051C4    http-bio-443-e >  THDW
  000051DD    http-bio-443-e >  THDW
  000051DE    http-bio-443-e >  THDW
  000051DF    http-bio-443-e >  THDW
  000051E0    http-bio-443-e >  THDW
  000051E1    http-bio-443-e >  THDW
  000051E8    Java2D Disposer   THDW
  000051F6    http-bio-8080- >  THDW
  000051F7    http-bio-8080- >  THDW
  000051F8    http-bio-8080- >  THDW
  000051F9    http-bio-8080- >  THDW
  000051FA    http-bio-8080- >  THDW
  000051FC    http-bio-8080- >  THDW
  0000520B    http-bio-8080- >  THDW
(funny, a few seconds ago, it didn't have nearly that many threads; 
evidently it is indeed currently in use). I note that option 12 appears 
to be plumbed into DMPJVM.

--
JHHL

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Shutdown.sh doesn't. At least not reliably. (7.0.25)

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

James,

On 11/27/12 1:13 PM, James Lampert wrote:
> Konstantin Kolinko wrote:
> 
>> 2. Shutdown command is sent to port 8005. (8009 is a port used by
>> AJP protocol connector).
> 
> Thanks. That at least clears up a misconception on my part.
> Researching it cleared up another misconception on my part: that
> the Catalina job directly owns the ports. Actually, the ports are
> owned by a QP0ZSPWT job (that is itself probably owned by the
> Catalina job.)
> 
> Once I know how to get a thread dump on an AS/400 (the link didn't
> cover that; I've got a question out to the Java-400 list at
> Midrange.com about that)

Maybe use DMPJVM? Sorry for the through-Google link [PDF]:

http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=9&ved=0CFsQFjAI&url=http%3A%2F%2Fpublib.boulder.ibm.com%2Fhtml%2Fas400%2Fv4r5%2Fic2924%2Finfo%2Fjava%2Frzaha%2Frzaha.pdf&ei=RQ-1UI-4OdO-0QGJtICADA&usg=AFQjCNE2Ai3Hltz7kngTIjlv7t3OjmF3Zg&cad=rja

- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEARECAAYFAlC1EAkACgkQ9CaO5/Lv0PD23gCgp78aBQBPaXX8NuHhAqDHnyBq
1IcAnRxXIFHDm0goWOZCutw4z2BoNEqP
=XYDi
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Shutdown.sh doesn't. At least not reliably. (7.0.25)

Posted by James Lampert <ja...@touchtonecorp.com>.
Konstantin Kolinko wrote:

> 2. Shutdown command is sent to port 8005. (8009 is a port used by AJP
> protocol connector).

Thanks. That at least clears up a misconception on my part. Researching 
it cleared up another misconception on my part: that the Catalina job 
directly owns the ports. Actually, the ports are owned by a QP0ZSPWT job 
(that is itself probably owned by the Catalina job.)

Once I know how to get a thread dump on an AS/400 (the link didn't cover 
that; I've got a question out to the Java-400 list at Midrange.com about 
that), I'll try another shutdown/restart on our box, and see what 
happens to port 8005 and the QP0ZSPWT job.

--
JHHL

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Shutdown.sh doesn't. At least not reliably. (7.0.25)

Posted by Konstantin Kolinko <kn...@gmail.com>.
2012/11/27 James Lampert <ja...@touchtonecorp.com>:
> Running Tomcat on various AS/400s (V6R1 or later OS), we've found that
> shutdown.sh doesn't reliably shut down the server, and we frequently have to
> shut it down forcibly (i.e., finding CATALINA on a WRKACTJOB, and giving it
> a "4" with "OPTION(*IMMED)").
>
> Port 8009 does appear to be open before we call shutdown.sh, and it does
> appear to be closed after we call it.
>
> Any idea what the problem could be? Or where I should look for indications
> of what it could be?
>

1. catalina.sh and shutdown.sh should be able to kill a hung process,
but only if you call them with "-force" and if you have CATALINA_PID
variable defined in your configuration.

Do you have CATALINA_PID configured?

A clean shutdown might be impossible if Tomcat encounters an OutOfMemoryError.

2. Shutdown command is sent to port 8005. (8009 is a port used by AJP
protocol connector).

Best regards,
Konstantin Kolinko

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


RE: Shutdown.sh doesn't. At least not reliably. (7.0.25)

Posted by "Carrillo, Dan" <dc...@idanalytics.com>.
If you take a thread dump you'll be able to see the state of both Tomcat's threads and your own application's threads to see where it's hanging. 

-----Original Message-----
From: James Lampert [mailto:jamesl@touchtonecorp.com] 
Sent: Tuesday, November 27, 2012 9:10 AM
To: Tomcat Users List
Subject: Shutdown.sh doesn't. At least not reliably. (7.0.25)

Running Tomcat on various AS/400s (V6R1 or later OS), we've found that shutdown.sh doesn't reliably shut down the server, and we frequently have to shut it down forcibly (i.e., finding CATALINA on a WRKACTJOB, and giving it a "4" with "OPTION(*IMMED)").

Port 8009 does appear to be open before we call shutdown.sh, and it does appear to be closed after we call it.

Any idea what the problem could be? Or where I should look for indications of what it could be?

--
JHHL

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org