You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Christopher Schultz <ch...@christopherschultz.net> on 2014/12/27 17:01:13 UTC
Re: mod_jk busyness , sending more request's to backend tomcats
causing high cpu
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Rajesh,
On 12/24/14 7:50 AM, Rajesh Cherukuri wrote:
> Qe have noticed that mod_jk method busyness is sending more
> requests to backend tomcat and causing hung threads and high CPU.
>
> [I] need to understand why it's doing that.
Are you saying that you are getting N requests to httpd from clients
and Tomcat is receiving M x N requests where M > 1.0?
That suggests that the Tomcat thread is not returning anything within
a certain amount of time and mod_jk is re-trying the request on
another host.
What is your Tomcat version? mod_jk version? httpd version? Can you
post your mod_jk configuration (everything in workers.properties) and
tell us about where the threads are stuck on the Tomcat side (take a
thread dump and look at where the threads are stuck). Are they all
stuck in the same place?
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
iQIcBAEBCAAGBQJUntfJAAoJEBzwKT+lPKRYtAEQAKZFhibbMepBa5QL3hCeCfwH
mLpK2OKc5Pr+ErlF0yIHScZr56uM7v6bHyCH3Ut2z+M5E8W+IlSJI13tYewp5dMg
ccvYh9LLYpWvIl9ZcdKG7MqSHOsF2zqb3FijjYDMBQwgU6onY0oINLDhOl7Led+t
3tjxZLig5fHzaBWQ7c04Fp96awIghHHp6N5O/kGv4/aeLV+jHi5uxg9cL5G2LMiW
ZTKMvdkn8YyAhmJYEGHy6T6Ypba+qsnGsHjUvWPfhb7TTTDPqFgoPPaux5g6XqKM
5SP3vOzRmhk+w62IcIPJzh5x/jo60ia/fOTVuRS9IL/UG1B+X9rnGwXupOaeE/jB
O5FzfLMh6nmqQ2h0QD1if+bd/JojoHABa68bfnJZ6OejHYJ+eHqPrutpRT4A0cQR
4mskyQbt1wtTwtmioU9sPqVqJWWlviKa573691uw2HSrQMjOZrD9RFZ/iwzeRowC
kO7Q2MdkBK1vKhc0xSh2bZsr3hNC+gbABNHE4SoaZRDh48pghC9T7UOjb95tXf0z
VHYJy05CmWr2dmmF8KOe7Gyri3tGrTKyQ/PtOdz/ppWqz+/co9xbv3kzOxT3W4mz
2r5QfYoOAnNd9ank2q0rbVB9sTDQlD0U5NoXhqAOxwulFglrraelMSLUJXML2TEn
xg50hO7JZj9rYklYV5gF
=hQzk
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk busyness , sending more request's to backend tomcats
causing high cpu
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Rajesh,
On 12/28/14 1:54 AM, Rajesh Cherukuri wrote:
> Thanks for the response here are the details , i can also see that
> the backend tomcat which has high cpu has processed less number of
> request when compared with other tomcat's i checked this from
> tomcat access logs , in the monotoring tool i can see that the
> tomcat which has issues have taken more hits/sec when compared to
> others
Maybe your web application doesn't work well under load...
> as of now i am suspecting the busyness method has sent more request
> to the tomcat that served less number of request's (the tomcat
> which has high cpu) , which is unable to serve the requests due
> to hung threads to DB
If you have threads hanging waiting on db responses, then maybe you
should look there... mod_jk isn't the problem.
(Unless you suspect that there is a problem with the "busyness"
algorithm. If you think there is a problem with the busyness
algorithm, then you'll need to provide a LOT of data. As of yet, you
have provided exactly zero data. The mod_jk status worker can provide
a great deal of information about the state of your workers.)
> *Tomcat version * 6.0
6.0.what?
> *Apache Version * 2.2
2.2.what?
What about mod_jk? Version number?
> *Thread dump anylisis*
>
>> From thread dump i can see that most of the threads are waiting
>> on DB ,and
> other are mod_jk threads Application uses c3po mchange to connect
> to DB
I have no idea why people use c3p0. It's considered pre-alpha at this
point. If your threads are stuck waiting for db connections (and not
waiting for the db to complete some actual work), then you might
consider swapping-out your connection pool for one that a bit more
reliable.
> worker.propertires
>
>
> *here is my work.property file* worker.xxx.type=lb
> worker.xxx.method=Busyness Definite common values for all xxxlb
> workers
>
> *worker.xxxlb.common.type=ajp13 worker.xxxlb.common.lbfactor=1
> worker.xxxlb.common.ping_mode=A
> worker.xxxlb.common.socket_connect_timeout=10000
> worker.xxxlb.common.connection_pool_size=6*
>
> *worker.xxxlb.common.connection_pool_timeout=600
> worker.xxxlb.common.socket_keepalive=True*
Nothing seems amiss here. You need to give more information.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: GPGTools - http://gpgtools.org
iQIcBAEBCAAGBQJUoBoGAAoJEBzwKT+lPKRYWqQQAIt76zjfPhfCRfjkdf4txWwm
l801mc6RLVGCJT37vAH6FeCC9hMUTcAGNRnPjo3X7E5aZd+CDxQ129TGd/qbFSCv
PKp39Fg2x4Idv2YTuKWxBXTxngqoHpYotz6KaKMt1cbSVuaWhJk1t6nrwD597JI4
r1ZsjuqCvtexsw6/uPoKCCXioKnmLr9IGOPIWxmZrGtrt7mxPtdnkWHHvNVe+0Ct
RbB5TDWYCLWllpjE7sXNokYagGk5Ozfh8MFYGrDNkANCWnD9aL4XC2Z2sgglPBND
UEl7YY0UYPvgP72ISJ6vo3TpjaGE9R7aDSE1tEhPQH1+2hf+rRReJlC00Ynom2Jg
5IgC0uk87Vt26iUIdx+KJIuR6fGOKtKPexTolFq0iZEyzGhZ/wuxPczSuVgtNLS2
Qla4byTfE4Cx0YYylLdJTGYhPFAtCDCVZSp5BlQQxqruGN77+gsn0zZoXVT7pPH3
c0uL3poikjM3UQDNnbSzlxv+37qPS+ziAeAFNKcvNpx9z8dHrPP+Xk6NGGTEaQ+x
YLY6xuBF+3AqObGrg56zEcHOWsPrWD2oW8mb43S6y0DiZTq2tf2i5wWoofPk2Q3E
YacbaRqzlkNNBV8RBVq+qCVhPFoPFOthSwDbMcxw2NZRiv4l9rD9qNRvzsZpEb39
rgl+JKOYuwXRsNF7TCtf
=eE8h
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: mod_jk busyness , sending more request's to backend tomcats
causing high cpu
Posted by Rajesh Cherukuri <ra...@gmail.com>.
Chris
Thanks for the response here are the details , i can also see that the
backend tomcat which has high cpu has processed less number of request when
compared with other tomcat's i checked this from tomcat access logs , in
the monotoring tool i can see that the tomcat which has issues have taken
more hits/sec when compared to others
as of now i am suspecting the busyness method has sent more
request to the tomcat that served less number of request's (the tomcat
which has high cpu) , which is unable to serve the requests due to hung
threads to DB
*Tomcat version *
6.0
*Apache Version *
2.2
*Thread dump anylisis*
>From thread dump i can see that most of the threads are waiting on DB ,and
other are mod_jk threads Application uses c3po mchange to connect to DB
worker.propertires
*here is my work.property file*
worker.xxx.type=lb worker.xxx.method=Busyness Definite common values for
all xxxlb workers
*worker.xxxlb.common.type=ajp13 worker.xxxlb.common.lbfactor=1
worker.xxxlb.common.ping_mode=A
worker.xxxlb.common.socket_connect_timeout=10000
worker.xxxlb.common.connection_pool_size=6*
*worker.xxxlb.common.connection_pool_timeout=600
worker.xxxlb.common.socket_keepalive=True*
On Sat, Dec 27, 2014 at 9:31 PM, Christopher Schultz <
chris@christopherschultz.net> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> Rajesh,
>
> On 12/24/14 7:50 AM, Rajesh Cherukuri wrote:
> > Qe have noticed that mod_jk method busyness is sending more
> > requests to backend tomcat and causing hung threads and high CPU.
> >
> > [I] need to understand why it's doing that.
>
> Are you saying that you are getting N requests to httpd from clients
> and Tomcat is receiving M x N requests where M > 1.0?
>
> That suggests that the Tomcat thread is not returning anything within
> a certain amount of time and mod_jk is re-trying the request on
> another host.
>
> What is your Tomcat version? mod_jk version? httpd version? Can you
> post your mod_jk configuration (everything in workers.properties) and
> tell us about where the threads are stuck on the Tomcat side (take a
> thread dump and look at where the threads are stuck). Are they all
> stuck in the same place?
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1
> Comment: GPGTools - http://gpgtools.org
>
> iQIcBAEBCAAGBQJUntfJAAoJEBzwKT+lPKRYtAEQAKZFhibbMepBa5QL3hCeCfwH
> mLpK2OKc5Pr+ErlF0yIHScZr56uM7v6bHyCH3Ut2z+M5E8W+IlSJI13tYewp5dMg
> ccvYh9LLYpWvIl9ZcdKG7MqSHOsF2zqb3FijjYDMBQwgU6onY0oINLDhOl7Led+t
> 3tjxZLig5fHzaBWQ7c04Fp96awIghHHp6N5O/kGv4/aeLV+jHi5uxg9cL5G2LMiW
> ZTKMvdkn8YyAhmJYEGHy6T6Ypba+qsnGsHjUvWPfhb7TTTDPqFgoPPaux5g6XqKM
> 5SP3vOzRmhk+w62IcIPJzh5x/jo60ia/fOTVuRS9IL/UG1B+X9rnGwXupOaeE/jB
> O5FzfLMh6nmqQ2h0QD1if+bd/JojoHABa68bfnJZ6OejHYJ+eHqPrutpRT4A0cQR
> 4mskyQbt1wtTwtmioU9sPqVqJWWlviKa573691uw2HSrQMjOZrD9RFZ/iwzeRowC
> kO7Q2MdkBK1vKhc0xSh2bZsr3hNC+gbABNHE4SoaZRDh48pghC9T7UOjb95tXf0z
> VHYJy05CmWr2dmmF8KOe7Gyri3tGrTKyQ/PtOdz/ppWqz+/co9xbv3kzOxT3W4mz
> 2r5QfYoOAnNd9ank2q0rbVB9sTDQlD0U5NoXhqAOxwulFglrraelMSLUJXML2TEn
> xg50hO7JZj9rYklYV5gF
> =hQzk
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>