You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Leandro Dardini <l....@comune.prato.it> on 2008/08/21 12:27:08 UTC

Newer version of mod_jk much slower than 1.2.13

I am sorry for the long email, but I like to be complete. 

I just stay with mod_jk 1.2.13 for a long time. It is used between
apache 2.0.52 and tomcat 5.5.26 each one on Linux Redhat ES4 64 bit.
There are two apache with dns load balancing and four tomcat with mod_jk
balancing. Last day I upgrade to 1.2.26 and get a very noticeable
slowdown of data trasfer from tomcat to apache. Each version in
recompiled from source from the same compilation farm. Following there
is a benchmark and few comments:

-------------------------

[root@servizio tmp]# time wget -O /dev/null
http://elenco1.comune.prato.it/eltbinj/servlet/ricercaelenco
--12:05:49--
http://elenco1.comune.prato.it/eltbinj/servlet/ricercaelenco
           => `/dev/null'
Risoluzione di elenco1.comune.prato.it in corso... 193.43.107.14
Connessione a elenco1.comune.prato.it|193.43.107.14:80... connesso.
HTTP richiesta inviata, aspetto la risposta... 200 OK
Lunghezza: non specificato [text/html]

    [                                   <=>
] 53,048         7.17K/s

12:05:56 (7.55 KB/s) - "/dev/null" salvato [53048]


real    0m6.931s
user    0m0.011s
sys     0m0.063s
[root@servizio tmp]# time wget -O /dev/null
http://elenco.comune.prato.it/eltbinj/servlet/ricercaelenco
--12:06:01--
http://elenco.comune.prato.it/eltbinj/servlet/ricercaelenco
           => `/dev/null'
Risoluzione di elenco.comune.prato.it in corso... 193.43.107.15
Connessione a elenco.comune.prato.it|193.43.107.15:80... connesso.
HTTP richiesta inviata, aspetto la risposta... 200 OK
Lunghezza: non specificato [text/html]

    [ <=>
] 53,048        --.--K/s

12:06:01 (572.81 KB/s) - "/dev/null" salvato [53048]


real    0m0.195s
user    0m0.001s
sys     0m0.003s

--------------------------

elenco.comune.prato.it is managed by old 1.2.13 version.
elenco1.comune.prato.it is managed by the newer 1.2.26. All machines are
on the same gigabit network and load is zero on all of them. The test
has ben repeated a good number of times without differences. If I put
the mod_jk 1.2.13 back on elenco1, I get the same good performance, so
it is definitively due to the different version of mod_jk.

For who (like me before testing and re testing), don't belive, this is a
test on elenco1 after replacing mod_jk 1.2.26 with the old 1.2.13
version.

------------------------------

[root@servizio tmp]# time wget -O /dev/null
http://elenco1.comune.prato.it/eltbinj/servlet/ricercaelenco
--12:16:47--
http://elenco1.comune.prato.it/eltbinj/servlet/ricercaelenco
           => `/dev/null'
Risoluzione di elenco1.comune.prato.it in corso... 193.43.107.14
Connessione a elenco1.comune.prato.it|193.43.107.14:80... connesso.
HTTP richiesta inviata, aspetto la risposta... 200 OK
Lunghezza: non specificato [text/html]

    [ <=>
] 53,048        --.--K/s

12:16:48 (346.64 KB/s) - "/dev/null" salvato [53048]


real    0m0.257s
user    0m0.003s
sys     0m0.003s

------------------------------

Unfortunately I don't have old version from 1.2.13 to 1.2.26, so I
cannot say you when the slowdown happens. Following there is my
workers.properties. The worker used for elenco/elenco1 is wlbs.

------------------------------

[root@apache1 ~]# cat /usr1/web/admin/conf/workers.properties
# workers.properties.minimal -
#
# This file provides minimal jk configuration properties needed to
# connect to Tomcat.
#
# The workers that jk should create and work with
#

worker.list=ajp13w_01_01,ajp13w_02_01,ajp13w_03_01,ajp13w_04_01,ajp13w_i
g_01_01,ajp13w_ig_02_01,ajp13w_is_01_01,ajp13w_is_02_01,ajp13w_sondaggi_
01_01,ajp13w_sondaggi_02_01,wlbs,wlbig,wlbis,wlbsondaggi,jkstatus,ajp13w
_ws_01_01,ajp13w_ws_02_01,wlbws,ajp13w_geoserver_01_01,ajp13w_geoserver_
02_01,wlbgeoserver,ajp13w_ip_01_01,ajp13w_ip_02_01,wlbip,ajp13w_safe_01_
01,ajp13w_safe_02_01,ajp13w_safe_03_01,ajp13w_safe_04_01,wlbsafe,ajp13w_
je_03_01,ajp13w_je_04_01,wlbjente


# Defining a worker named ajp13w and of type ajp13
# Note that the name and the type do not have to match.
#

worker.ajp13w_01_01.type=ajp13
worker.ajp13w_01_01.host=tomcat01.comune.prato.it
worker.ajp13w_01_01.port=8009
worker.ajp13w_01_01.lbfactor=1
worker.ajp13w_01_01.socket_timeout=1
worker.ajp13w_01_01.socket_keepalive=true
worker.ajp13w_01_01.connection_pool_timeout=60

worker.ajp13w_02_01.type=ajp13
worker.ajp13w_02_01.host=tomcat02.comune.prato.it
worker.ajp13w_02_01.port=8009
worker.ajp13w_02_01.lbfactor=1
worker.ajp13w_02_01.socket_timeout=1
worker.ajp13w_02_01.socket_keepalive=true
worker.ajp13w_02_01.connection_pool_timeout=60

worker.ajp13w_03_01.type=ajp13
worker.ajp13w_03_01.host=tomcat03.comune.prato.it
worker.ajp13w_03_01.port=8009
worker.ajp13w_03_01.lbfactor=1
worker.ajp13w_03_01.socket_timeout=1
worker.ajp13w_03_01.socket_keepalive=true
worker.ajp13w_03_01.connection_pool_timeout=60

worker.ajp13w_04_01.type=ajp13
worker.ajp13w_04_01.host=tomcat04.comune.prato.it
worker.ajp13w_04_01.port=8009
worker.ajp13w_04_01.lbfactor=1
worker.ajp13w_04_01.socket_timeout=1
worker.ajp13w_04_01.socket_keepalive=true
worker.ajp13w_04_01.connection_pool_timeout=60

worker.ajp13w_je_03_01.type=ajp13
worker.ajp13w_je_03_01.host=tomcat03.comune.prato.it
worker.ajp13w_je_03_01.port=8016
worker.ajp13w_je_03_01.lbfactor=1
worker.ajp13w_je_03_01.socket_timeout=1
worker.ajp13w_je_03_01.socket_keepalive=true
worker.ajp13w_je_03_01.connection_pool_timeout=60

worker.ajp13w_je_04_01.type=ajp13
worker.ajp13w_je_04_01.host=tomcat04.comune.prato.it
worker.ajp13w_je_04_01.port=8016
worker.ajp13w_je_04_01.lbfactor=1
worker.ajp13w_je_04_01.socket_timeout=1
worker.ajp13w_je_04_01.socket_keepalive=true
worker.ajp13w_je_04_01.connection_pool_timeout=60

worker.ajp13w_safe_01_01.type=ajp13
worker.ajp13w_safe_01_01.host=tomcat01.comune.prato.it
worker.ajp13w_safe_01_01.port=8017
worker.ajp13w_safe_01_01.lbfactor=1
worker.ajp13w_safe_01_01.socket_timeout=1
worker.ajp13w_safe_01_01.socket_keepalive=true
worker.ajp13w_safe_01_01.connection_pool_timeout=60

worker.ajp13w_safe_02_01.type=ajp13
worker.ajp13w_safe_02_01.host=tomcat02.comune.prato.it
worker.ajp13w_safe_02_01.port=8017
worker.ajp13w_safe_02_01.lbfactor=1
worker.ajp13w_safe_02_01.socket_timeout=1
worker.ajp13w_safe_02_01.socket_keepalive=true
worker.ajp13w_safe_02_01.connection_pool_timeout=60

worker.ajp13w_safe_03_01.type=ajp13
worker.ajp13w_safe_03_01.host=tomcat03.comune.prato.it
worker.ajp13w_safe_03_01.port=8017
worker.ajp13w_safe_03_01.lbfactor=1
worker.ajp13w_safe_03_01.socket_timeout=1
worker.ajp13w_safe_03_01.socket_keepalive=true
worker.ajp13w_safe_03_01.connection_pool_timeout=60

worker.ajp13w_safe_04_01.type=ajp13
worker.ajp13w_safe_04_01.host=tomcat04.comune.prato.it
worker.ajp13w_safe_04_01.port=8017
worker.ajp13w_safe_04_01.lbfactor=1
worker.ajp13w_safe_04_01.socket_timeout=1
worker.ajp13w_safe_04_01.socket_keepalive=true
worker.ajp13w_safe_04_01.connection_pool_timeout=60

worker.ajp13w_ws_01_01.type=ajp13
worker.ajp13w_ws_01_01.host=tomcat01.comune.prato.it
worker.ajp13w_ws_01_01.port=8010
worker.ajp13w_ws_01_01.lbfactor=1
worker.ajp13w_ws_01_01.socket_timeout=1
worker.ajp13w_ws_01_01.socket_keepalive=true
worker.ajp13w_ws_01_01.connection_pool_timeout=60

worker.ajp13w_ws_02_01.type=ajp13
worker.ajp13w_ws_02_01.host=tomcat02.comune.prato.it
worker.ajp13w_ws_02_01.port=8010
worker.ajp13w_ws_02_01.lbfactor=1
worker.ajp13w_ws_02_01.socket_timeout=1
worker.ajp13w_ws_02_01.socket_keepalive=true
worker.ajp13w_ws_02_01.connection_pool_timeout=60

worker.ajp13w_geoserver_01_01.type=ajp13
worker.ajp13w_geoserver_01_01.host=tomcat01.comune.prato.it
worker.ajp13w_geoserver_01_01.port=8014
worker.ajp13w_geoserver_01_01.lbfactor=1
worker.ajp13w_geoserver_01_01.socket_timeout=1
worker.ajp13w_geoserver_01_01.socket_keepalive=true
worker.ajp13w_geoserver_01_01.connection_pool_timeout=60

worker.ajp13w_geoserver_02_01.type=ajp13
worker.ajp13w_geoserver_02_01.host=tomcat02.comune.prato.it
worker.ajp13w_geoserver_02_01.port=8014
worker.ajp13w_geoserver_02_01.lbfactor=1
worker.ajp13w_geoserver_02_01.socket_timeout=1
worker.ajp13w_geoserver_02_01.socket_keepalive=true
worker.ajp13w_geoserver_02_01.connection_pool_timeout=60

worker.ajp13w_ig_01_01.type=ajp13
worker.ajp13w_ig_01_01.host=tomcat01.comune.prato.it
worker.ajp13w_ig_01_01.port=8011
worker.ajp13w_ig_01_01.lbfactor=1

worker.ajp13w_ig_02_01.type=ajp13
worker.ajp13w_ig_02_01.host=tomcat02.comune.prato.it
worker.ajp13w_ig_02_01.port=8011
worker.ajp13w_ig_02_01.lbfactor=1

worker.ajp13w_ip_01_01.type=ajp13
worker.ajp13w_ip_01_01.host=tomcat01.comune.prato.it
worker.ajp13w_ip_01_01.port=8015
worker.ajp13w_ip_01_01.lbfactor=1

worker.ajp13w_ip_02_01.type=ajp13
worker.ajp13w_ip_02_01.host=tomcat02.comune.prato.it
worker.ajp13w_ip_02_01.port=8015
worker.ajp13w_ip_02_01.lbfactor=1

worker.ajp13w_is_01_01.type=ajp13
worker.ajp13w_is_01_01.host=tomcat01.comune.prato.it
worker.ajp13w_is_01_01.port=8012
worker.ajp13w_is_01_01.lbfactor=1

worker.ajp13w_is_02_01.type=ajp13
worker.ajp13w_is_02_01.host=tomcat02.comune.prato.it
worker.ajp13w_is_02_01.port=8012
worker.ajp13w_is_02_01.lbfactor=1

worker.ajp13w_sondaggi_01_01.type=ajp13
worker.ajp13w_sondaggi_01_01.host=tomcat01.comune.prato.it
worker.ajp13w_sondaggi_01_01.port=8013
worker.ajp13w_sondaggi_01_01.lbfactor=1

worker.ajp13w_sondaggi_02_01.type=ajp13
worker.ajp13w_sondaggi_02_01.host=tomcat02.comune.prato.it
worker.ajp13w_sondaggi_02_01.port=8013
worker.ajp13w_sondaggi_02_01.lbfactor=1

#
# Defining a load balancer
#

worker.wlbs.type=lb
worker.wlbs.sticky_session=1
worker.wlbs.balance_workers=ajp13w_01_01,ajp13w_02_01,ajp13w_03_01,ajp13
w_04_01
worker.wlbs.socket_timeout=1

worker.wlbjente.type=lb
worker.wlbjente.sticky_session=1
worker.wlbjente.balance_workers=ajp13w_je_03_01,ajp13w_je_04_01
worker.wlbjente.socket_timeout=1

worker.wlbsafe.type=lb
worker.wlbsafe.sticky_session=1
worker.wlbsafe.balance_workers=ajp13w_safe_01_01,ajp13w_safe_02_01,ajp13
w_safe_03_01,ajp13w_safe_04_01
worker.wlbsafe.socket_timeout=1

worker.wlbig.type=lb
worker.wlbig.sticky_session=1
worker.wlbig.balance_workers=ajp13w_ig_01_01,ajp13w_ig_02_01

worker.wlbip.type=lb
#worker.wlbip.sticky_session=1
worker.wlbip.balance_workers=ajp13w_ip_01_01,ajp13w_ip_02_01

worker.wlbis.type=lb
worker.wlbis.sticky_session=1
worker.wlbis.balance_workers=ajp13w_is_01_01,ajp13w_is_02_01

worker.wlbsondaggi.type=lb
worker.wlbsondaggi.sticky_session=1
worker.wlbsondaggi.balance_workers=ajp13w_sondaggi_01_01,ajp13w_sondaggi
_02_01

worker.wlbws.type=lb
worker.wlbws.balance_workers=ajp13w_ws_01_01,ajp13w_ws_02_01
worker.wlbws.socket_timeout=1

worker.wlbgeoserver.type=lb
worker.wlbgeoserver.sticky_session=1
worker.wlbgeoserver.balance_workers=ajp13w_geoserver_01_01,ajp13w_geoser
ver_02_01

#
# Define status worker
#

worker.jkstatus.type=status

---------------------

Thank you for your patient.

Leandro

---------------------
Dott. Leandro Dardini
Sistema Informativo
Comune di Prato
Via Cairoli, 16
59100 Prato PO
Tel: +39 05741835216
Fax: +39 05721835212 

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


R: R: R: Newer version of mod_jk much slower than 1.2.13

Posted by Leandro Dardini <l....@comune.prato.it>.
> -----Messaggio originale-----
> Da: Rainer Jung [mailto:rainer.jung@kippdata.de] 
> Inviato: venerdì 22 agosto 2008 21.09
> A: Tomcat Users List
> Oggetto: Re: R: R: Newer version of mod_jk much slower than 1.2.13
> 
> Hi Leandro,
> 
> Leandro Dardini schrieb:
> > LoadModule jk_module modules/mod_jk.so
> > 
> > JkWorkersFile /etc/httpd/conf/workers.properties
>  > JkLogFile /etc/httpd/logs/mod_jk.log
>  > JkLogLevel info
> 
> OK
> 
> > About the trace level, full it is over 700Kbyte (gzipped), 
> I send it 
> > directly to you. Thank you very much if you can look at it.
> > 
> > 
> > Looking to tcpdump flow of packets between client and apache web 
> > server, I notice a lots of small packets. With older version of 
> > mod_jk, there are instead fewer packets of bigger size.
> 
> Yes, that's the problem, but I don't see, how mod_jk 
> influences that, it's a decision of the backend how to chunk 
> the data. Maybe we now stream them back sooner than before, 
> but I doubt it.
> 
> I asked about a log file for the old and for the new version. 
> Could you provide the one for the old version too?
> 
> There are 10630 packets received by mod_jk from the backend, 
> and 6608 of those are empty (no data, only AJP protocol header).
> 
> The first packet is received after about 200 milliseconds, 
> each further packet takes 1-2 milliseconds. I want to compare 
> with the log when you use 1.2.13. If possible please send the 
> exact same request with the same httpd and the same backend, 
> only using 1.2.13 instead of 1.2.26.
> 
> Regards,
> 
> Rainer
> 
> 
> 
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org To 
> unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 
> 

The saga continues. Privately I send you the mod_jk.log using the latest versione of mod_jk with trace debug. 

A very big thank you.

Leandro

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: R: R: Newer version of mod_jk much slower than 1.2.13

Posted by Rainer Jung <ra...@kippdata.de>.
Hi Leandro,

Leandro Dardini schrieb:
> LoadModule jk_module modules/mod_jk.so
> 
> JkWorkersFile /etc/httpd/conf/workers.properties
 > JkLogFile /etc/httpd/logs/mod_jk.log
 > JkLogLevel info

OK

> About the trace level, full it is over 700Kbyte (gzipped), I send it
> directly to you. Thank you very much if you can look at it.
> 
> 
> Looking to tcpdump flow of packets between client and apache web
> server, I notice a lots of small packets. With older version of
> mod_jk, there are instead fewer packets of bigger size.

Yes, that's the problem, but I don't see, how mod_jk influences that, 
it's a decision of the backend how to chunk the data. Maybe we now 
stream them back sooner than before, but I doubt it.

I asked about a log file for the old and for the new version. Could you 
provide the one for the old version too?

There are 10630 packets received by mod_jk from the backend, and 6608 of 
those are empty (no data, only AJP protocol header).

The first packet is received after about 200 milliseconds, each further 
packet takes 1-2 milliseconds. I want to compare with the log when you 
use 1.2.13. If possible please send the exact same request with the same 
httpd and the same backend, only using 1.2.13 instead of 1.2.26.

Regards,

Rainer



---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


R: R: Newer version of mod_jk much slower than 1.2.13

Posted by Leandro Dardini <l....@comune.prato.it>.


-----Messaggio originale-----
Da: Rainer Jung [mailto:rainer.jung@kippdata.de]
Inviato: ven 22/08/2008 14.21
A: Tomcat Users List
Oggetto: Re: R: Newer version of mod_jk much slower than 1.2.13
 
> I get rid of the balancer and set in apache configuration file:
> 
>         JkMount /eltbinj/* ajp13w_01_01
> 
> With the 1.2.13 the result is obviously the same high speed (I unset the lang for not italian (lucky) people):
...
> 13:13:00 (630.19 KB/s) - `/dev/null' saved [53048]
> 
> With the 1.2.26 the result is the same:
...
> 13:14:53 (7.35 KB/s) - `/dev/null' saved [53048]
> 
> So in conclusion, it is not a matter of load balancer.

Thanks for helping narrowing it down, also the information about it only 
happening for dynamic content is interesting (maybe it has to do with 
chunked encoding).

I could analyze the packet flow between Apache and Tomcat. If you set 
JkLogLevel to trace (and if trace doesn't exist for 1.2.13 to debug 
there), you get very long log output. If you do the wget test once with 
each version and with the increased log level, I could have a look at 
those log files. They will contain a full packet dump of the 
communication and at least for 1.2.26 millisecond timestamps.

Nevertheless I'm still curious about your JK directives for Apache. I 
guess JkMount is not everything ;)

Regards,

Rainer

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Sorry for the ugly mailer, but I am using a webmail now from home. About the jk directives for apache, here they are:

LoadModule jk_module modules/mod_jk.so

JkWorkersFile /etc/httpd/conf/workers.properties
JkLogFile /etc/httpd/logs/mod_jk.log
JkLogLevel info

About the trace level, full it is over 700Kbyte (gzipped), I send it directly to you. Thank you very much if you can look at it.

172.16.1.49 is the host making the wget

193.43.107.13 is the apache server 

193.43.107.16 is the tomcat server

A piece following:

[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [debug] init_ws_service::mod_jk.c (888): Service protocol=HTTP/1.0 method=GET host=(null) a
ddr=172.16.1.49 name=elenco1.comune.prato.it port=80 auth=(null) user=(null) laddr=193.43.107.14 raddr=172.16.1.49 uri=/eltbinj/servlet/ricer
caelenco
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [trace] get_endpoint::jk_ajp13_worker.c (73): enter
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [trace] ajp_get_endpoint::jk_ajp_common.c (2552): enter
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [debug] ajp_get_endpoint::jk_ajp_common.c (2579): acquired connection pool slot=0
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [trace] ajp_get_endpoint::jk_ajp_common.c (2582): exit
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [trace] get_endpoint::jk_ajp13_worker.c (75): exit
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [trace] ajp_service::jk_ajp_common.c (1966): enter
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [trace] ajp_marshal_into_msgb::jk_ajp_common.c (373): enter
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [debug] ajp_marshal_into_msgb::jk_ajp_common.c (553): ajp marshaling done
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [trace] ajp_marshal_into_msgb::jk_ajp_common.c (554): exit
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [debug] ajp_service::jk_ajp_common.c (2050): processing ajp13w_01_01 with 2 retries
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [trace] ajp_send_request::jk_ajp_common.c (1263): enter
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [trace] ajp_next_connection::jk_ajp_common.c (737): enter
[Fri Aug 22 18:10:57.827 2008] [23127:2537062720] [trace] ajp_next_connection::jk_ajp_common.c (764): exit
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [debug] ajp_send_request::jk_ajp_common.c (1352): (ajp13w_01_01) all endpoints are disconne
cted, detected by connect check (0), cping (0), send (0)
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [trace] ajp_connect_to_endpoint::jk_ajp_common.c (859): enter
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [trace] jk_open_socket::jk_connect.c (426): enter
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [debug] jk_open_socket::jk_connect.c (448): socket TCP_NODELAY set to On
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [debug] jk_open_socket::jk_connect.c (462): socket SO_KEEPALIVE set to On
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [debug] jk_open_socket::jk_connect.c (513): timeout 1 set for socket=17
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [debug] jk_open_socket::jk_connect.c (548): trying to connect socket 17 to 193.43.107.16:8009
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [trace] nb_connect::jk_connect.c (218): enter
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [trace] nb_connect::jk_connect.c (264): exit
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [debug] jk_open_socket::jk_connect.c (574): socket 17 connected to 193.43.107.16:8009
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [trace] jk_open_socket::jk_connect.c (577): exit
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [debug] ajp_connect_to_endpoint::jk_ajp_common.c (878): Connected socket 17 to (193.43.107.16:8009)
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [trace] ajp_connect_to_endpoint::jk_ajp_common.c (908): exit
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [trace] ajp_connection_tcp_send_message::jk_ajp_common.c (928): enter
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): sending to ajp13 pos=4 len=
190 max=8192
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 0000    12 34 00 BA 02 02 0
0 08 48 54 54 50 2F 31 2E 30  - .4......HTTP/1.0
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 0010    00 00 1E 2F 65 6C 7
4 62 69 6E 6A 2F 73 65 72 76  - .../eltbinj/serv
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 0020    6C 65 74 2F 72 69 6
3 65 72 63 61 65 6C 65 6E 63  - let/ricercaelenc
[Fri Aug 22 18:10:57.828 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 0030    6F 00 00 0B 31 37 3
2 2E 31 36 2E 31 2E 34 39 00  - o...172.16.1.49.
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 0040    FF FF 00 17 65 6C 6
5 6E 63 6F 31 2E 63 6F 6D 75  - ....elenco1.comu
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 0050    6E 65 2E 70 72 61 7
4 6F 2E 69 74 00 00 50 00 00  - ne.prato.it..P..
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 0060    05 A0 0E 00 1E 57 6
7 65 74 2F 31 2E 31 30 2E 32  - .....Wget/1.10.2
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 0070    20 28 52 65 64 20 4
8 61 74 20 6D 6F 64 69 66 69  - .(Red.Hat.modifi
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 0080    65 64 29 00 A0 01 0
0 03 2A 2F 2A 00 A0 0B 00 17  - ed).....*/*.....
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 0090    65 6C 65 6E 63 6F 3
1 2E 63 6F 6D 75 6E 65 2E 70  - elenco1.comune.p
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 00a0    72 61 74 6F 2E 69 7
4 00 A0 06 00 0A 4B 65 65 70  - rato.it.....Keep
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [debug] ajp_connection_tcp_send_message::jk_ajp_common.c (934): 00b0    2D 41 6C 69 76 65 0
0 A0 08 00 01 30 00 FF 00 00  - -Alive.....0....
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [trace] jk_tcp_socket_sendfull::jk_connect.c (719): enter
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [trace] jk_tcp_socket_sendfull::jk_connect.c (747): exit
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [trace] ajp_connection_tcp_send_message::jk_ajp_common.c (961): exit
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [debug] ajp_send_request::jk_ajp_common.c (1395): (ajp13w_01_01) request body to send 0 - r
equest body to resend 0
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [trace] ajp_send_request::jk_ajp_common.c (1506): exit
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [trace] ajp_get_reply::jk_ajp_common.c (1718): enter
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (992): enter
[Fri Aug 22 18:10:57.829 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.021 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.021 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.021 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.021 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): received from ajp13 pos=0 l
en=56 max=8192
[Fri Aug 22 18:10:58.021 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0000    04 00 C8 00 02 4F 4
B 00 00 01 00 0C 43 6F 6E 74  - .....OK.....Cont
[Fri Aug 22 18:10:58.021 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0010    65 6E 74 2D 54 79 7
0 65 00 00 1C 74 65 78 74 2F  - ent-Type...text/
[Fri Aug 22 18:10:58.021 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0020    68 74 6D 6C 3B 63 6
8 61 72 73 65 74 3D 49 53 4F  - html;charset=ISO
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0030    2D 38 38 35 39 2D 3
1 00 00 00 00 00 00 00 00 00  - -8859-1.........
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (1123): exit
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1536): enter
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [trace] ajp_unmarshal_response::jk_ajp_common.c (590): enter
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [debug] ajp_unmarshal_response::jk_ajp_common.c (608): status = 200
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [debug] ajp_unmarshal_response::jk_ajp_common.c (615): Number of headers is = 1
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [debug] ajp_unmarshal_response::jk_ajp_common.c (671): Header[0] [Content-Type] = [text/htm
l;charset=ISO-8859-1]
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [trace] ajp_unmarshal_response::jk_ajp_common.c (678): exit
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (992): enter
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): received from ajp13 pos=0 l
en=4 max=8192
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0000    03 00 00 00 00 00 0
0 00 00 00 00 00 00 00 00 00  - ................
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (1123): exit
[Fri Aug 22 18:10:58.022 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1536): enter
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1680): exit
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (992): enter
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): received from ajp13 pos=0 len=73 max=8192
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0000    03 00 45 3C 21 44 4
F 43 54 59 50 45 20 48 54 4D  - ..E<!DOCTYPE.HTM
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0010    4C 20 50 55 42 4C 4
9 43 20 22 2D 2F 2F 57 33 43  - L.PUBLIC."-//W3C
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0020    2F 2F 44 54 44 20 4
8 54 4D 4C 20 34 2E 30 31 20  - //DTD.HTML.4.01.
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0030    54 72 61 6E 73 69 7
4 69 6F 6E 61 6C 2F 2F 45 4E  - Transitional//EN
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0040    22 3E 0A 3C 68 74 6
D 6C 00 00 00 00 00 00 00 00  - ">.<html........
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (1123): exit
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1536): enter
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [debug] ws_write::mod_jk.c (455): written 69 out of 69
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1680): exit
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (992): enter
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.025 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.026 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): received from ajp13 pos=0 l
en=4 max=8192
[Fri Aug 22 18:10:58.026 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0000    03 00 00 00 00 00 0
0 00 00 00 00 00 00 00 00 00  - ................
[Fri Aug 22 18:10:58.026 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (1123): exit
[Fri Aug 22 18:10:58.026 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1536): enter
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1680): exit
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (992): enter
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): received from ajp13 pos=0 l
en=13 max=8192
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0000    03 00 09 3E 0A 20 2
0 3C 68 65 61 64 00 00 00 00  - ...>...<head....
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (1123): exit
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1536): enter
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [debug] ws_write::mod_jk.c (455): written 9 out of 9
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1680): exit
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (992): enter
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): received from ajp13 pos=0 l
en=4 max=8192
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0000    03 00 00 00 00 00 0
0 00 00 00 00 00 00 00 00 00  - ................
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (1123): exit
[Fri Aug 22 18:10:58.027 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1536): enter
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1680): exit
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (992): enter
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): received from ajp13 pos=0 l
en=93 max=8192
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0000    03 00 59 3E 0A 20 2
0 20 20 3C 4D 45 54 41 20 68  - ..Y>.....<META.h
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0010    74 74 70 2D 65 71 7
5 69 76 3D 22 43 6F 6E 74 65  - ttp-equiv="Conte
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0020    6E 74 2D 54 79 70 6
5 22 20 63 6F 6E 74 65 6E 74  - nt-Type".content
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0030    3D 22 74 65 78 74 2
F 68 74 6D 6C 3B 20 63 68 61  - ="text/html;.cha
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0040    72 73 65 74 3D 49 5
3 4F 2D 38 38 35 39 2D 31 22  - rset=ISO-8859-1"
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0050    3E 0A 20 20 20 20 3
C 74 69 74 6C 65 00 00 00 00  - >.....<title....
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (1123): exit
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1536): enter
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [debug] ws_write::mod_jk.c (455): written 89 out of 89
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1680): exit
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (992): enter
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): received from ajp13 pos=0 l
en=4 max=8192
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0000    03 00 00 00 00 00 0
0 00 00 00 00 00 00 00 00 00  - ................
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (1123): exit
[Fri Aug 22 18:10:58.029 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1536): enter
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1680): exit
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (992): enter
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): received from ajp13 pos=0 l
en=13 max=8192
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0000    03 00 09 3E 52 69 6
3 65 72 63 61 20 00 00 00 00  - ...>Ricerca.....
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (1123): exit
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1536): enter
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [debug] ws_write::mod_jk.c (455): written 9 out of 9
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1680): exit
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (992): enter
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.031 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.032 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): received from ajp13 pos=0 l
en=4 max=8192
[Fri Aug 22 18:10:58.032 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): 0000    03 00 00 00 00 00 0
0 00 00 00 00 00 00 00 00 00  - ................
[Fri Aug 22 18:10:58.032 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (1123): exit
[Fri Aug 22 18:10:58.032 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1536): enter
[Fri Aug 22 18:10:58.033 2008] [23127:2537062720] [trace] ajp_process_callback::jk_ajp_common.c (1680): exit
[Fri Aug 22 18:10:58.033 2008] [23127:2537062720] [trace] ajp_connection_tcp_get_message::jk_ajp_common.c (992): enter
[Fri Aug 22 18:10:58.033 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.033 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.033 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (768): enter
[Fri Aug 22 18:10:58.033 2008] [23127:2537062720] [trace] jk_tcp_socket_recvfull::jk_connect.c (796): exit
[Fri Aug 22 18:10:58.033 2008] [23127:2537062720] [debug] ajp_connection_tcp_get_message::jk_ajp_common.c (1117): received from ajp13 pos=0 l
en=12 max=8192

... and so on

Looking to tcpdump flow of packets between client and apache web server, I notice a lots of small packets. With older version of mod_jk, there are instead fewer packets of bigger size.

Leandro

Re: R: Newer version of mod_jk much slower than 1.2.13

Posted by Rainer Jung <ra...@kippdata.de>.
> I get rid of the balancer and set in apache configuration file:
> 
>         JkMount /eltbinj/* ajp13w_01_01
> 
> With the 1.2.13 the result is obviously the same high speed (I unset the lang for not italian (lucky) people):
...
> 13:13:00 (630.19 KB/s) - `/dev/null' saved [53048]
> 
> With the 1.2.26 the result is the same:
...
> 13:14:53 (7.35 KB/s) - `/dev/null' saved [53048]
> 
> So in conclusion, it is not a matter of load balancer.

Thanks for helping narrowing it down, also the information about it only 
happening for dynamic content is interesting (maybe it has to do with 
chunked encoding).

I could analyze the packet flow between Apache and Tomcat. If you set 
JkLogLevel to trace (and if trace doesn't exist for 1.2.13 to debug 
there), you get very long log output. If you do the wget test once with 
each version and with the increased log level, I could have a look at 
those log files. They will contain a full packet dump of the 
communication and at least for 1.2.26 millisecond timestamps.

Nevertheless I'm still curious about your JK directives for Apache. I 
guess JkMount is not everything ;)

Regards,

Rainer

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


R: Newer version of mod_jk much slower than 1.2.13

Posted by Leandro Dardini <l....@comune.prato.it>.
> -----Messaggio originale-----
> Da: Rainer Jung [mailto:rainer.jung@kippdata.de] 
> Inviato: venerdì 22 agosto 2008 12.15
> A: Tomcat Users List
> Oggetto: Re: Newer version of mod_jk much slower than 1.2.13
> 
> Can you throw our the load balancer and retry with only one 
> direct worker? That would be helpful.
> 
> Also: please post your platform (type/version) and your JK 
> directives (for Apache).
> 
> The 7 seconds is by far to long for 50KB.
> 
> Regards,
> 
> Rainer
> 
> ---------------------------------------------------------------------
> To start a new topic, e-mail: users@tomcat.apache.org To 
> unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 
> 

I get rid of the balancer and set in apache configuration file:

        JkMount /eltbinj/* ajp13w_01_01

With the 1.2.13 the result is obviously the same high speed (I unset the lang for not italian (lucky) people):

[root@servizio ~]# time wget -O /dev/null http://elenco.comune.prato.it/eltbinj/servlet/ricercaelenco
--13:13:00--  http://elenco.comune.prato.it/eltbinj/servlet/ricercaelenco
           => `/dev/null'
Resolving elenco.comune.prato.it... 193.43.107.15
Connecting to elenco.comune.prato.it|193.43.107.15|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]

    [ <=>                                                                                                               ] 53,048        --.--K/s

13:13:00 (630.19 KB/s) - `/dev/null' saved [53048]


real    0m0.168s
user    0m0.000s
sys     0m0.002s

With the 1.2.26 the result is the same:

[root@servizio ~]# time wget -O /dev/null http://elenco1.comune.prato.it/eltbinj/servlet/ricercaelenco
--13:14:46--  http://elenco1.comune.prato.it/eltbinj/servlet/ricercaelenco
           => `/dev/null'
Resolving elenco1.comune.prato.it... 193.43.107.14
Connecting to elenco1.comune.prato.it|193.43.107.14|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]

    [                                    <=>                                                                            ] 53,048         7.36K/s

13:14:53 (7.35 KB/s) - `/dev/null' saved [53048]


real    0m7.113s
user    0m0.020s
sys     0m0.092s

So in conclusion, it is not a matter of load balancer.

However I get a bit further with tests: it seems the slowdown happens only when tomcat is spending some time building the answer. The "ricercaelenco" servlet perform a query on a database. I try pulling a generic file from tomcat and the result are pretty the same:

[root@servizio ~]# time wget -O /dev/null http://elenco.comune.prato.it/eltbinj/htdocs/test.dat
--13:18:21--  http://elenco.comune.prato.it/eltbinj/htdocs/test.dat
           => `/dev/null'
Resolving elenco.comune.prato.it... 193.43.107.15
Connecting to elenco.comune.prato.it|193.43.107.15|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 102,400,000 (98M) [text/plain]

100%[==================================================================================================================>] 102,400,000    8.06M/s    ETA 00:00

13:18:34 (7.53 MB/s) - `/dev/null' saved [102400000/102400000]


real    0m13.034s
user    0m0.107s
sys     0m0.448s
[root@servizio ~]# time wget -O /dev/null http://elenco1.comune.prato.it/eltbinj/htdocs/test.dat
--13:18:38--  http://elenco1.comune.prato.it/eltbinj/htdocs/test.dat
           => `/dev/null'
Resolving elenco1.comune.prato.it... 193.43.107.14
Connecting to elenco1.comune.prato.it|193.43.107.14|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 102,400,000 (98M) [text/plain]

100%[==================================================================================================================>] 102,400,000    6.88M/s    ETA 00:00

13:18:52 (6.70 MB/s) - `/dev/null' saved [102400000/102400000]


real    0m14.601s
user    0m0.097s
sys     0m0.435s

About my environment, all is running on Redhat ES4 64bit on X86_64 platform. There are two machine running "stock redhat" apache 2.0.52 and (now) one machine running "stock apache.org" tomcat 5.5.26. Following the java options on starting tomcat:

JAVA_OPTS="-Xms1024M -Xmx1024M -XX:MaxPermSize=512M -Dcom.sun.management.jmxremote -XX:GCTimeRatio=99 -verbose:gc"

Leandro

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org


Re: Newer version of mod_jk much slower than 1.2.13

Posted by Rainer Jung <ra...@kippdata.de>.
Can you throw our the load balancer and retry with only one direct 
worker? That would be helpful.

Also: please post your platform (type/version) and your JK directives 
(for Apache).

The 7 seconds is by far to long for 50KB.

Regards,

Rainer

---------------------------------------------------------------------
To start a new topic, e-mail: users@tomcat.apache.org
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org