You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Alexandre Chapellon <al...@mana.pf> on 2010/08/26 03:28:47 UTC
pick load
Hello,
I'm quite new to tomcat and have an old webapps running on tomcat 4.1
and jvm 1.4.2 with apach2.2 in front ofthem (using modjk).
I'm trying to get ready for a comming pick load I will have to face.
I Try to do some benchmark using ab and the jkstatus worker.
Whatever the configuration of my connecter (both on the apache or tomcat
side) I never go upper than 20 requests / second.
Here are few parameters I changed in order to get better performances:
-Apache2 (worker):
increased ServerLimit (64), ThreadLimit (256), MaxClients (2048),
ThreadsPerChild (128)
set to a non zero value MaxRequestsPerChild (500)
- modjk (1.2.30):
set to non-zero value worker.selfcare.connection_pool_timeout=60
-Tomcat AJP13 Connector:
acceptCount="50" enableLookups="false" maxProcessors="500"
bufferSize="4096" socketBuffer="20000"
Unfortunately this doesn't help and am still stuck with 20req/s when the
machines' load is not that high and 60% of CPU at most is used during
stress test.
I've googled around but can't find anything else about increasing
performances of apache/tomcat... Help much appreciated
Regards
P.S: right now am using ab to send 2000 request with 50 concurrents.
Re: pick load
Posted by André Warnier <aw...@ice-sa.com>.
Alexandre Chapellon wrote:
> Hello,
>
> I'm quite new to tomcat and have an old webapps running on tomcat 4.1
> and jvm 1.4.2 with apach2.2 in front ofthem (using modjk).
> I'm trying to get ready for a comming pick load I will have to face.
> I Try to do some benchmark using ab and the jkstatus worker.
> Whatever the configuration of my connecter (both on the apache or tomcat
> side) I never go upper than 20 requests / second.
> Here are few parameters I changed in order to get better performances:
>
...
Hi.
It is very likely that by changing the default parameters of Apache, mod_jk and Tomcat
without first knowing exactly where the "slowness" is happening, you are making the
situation worse rather than better.
It is also very likely that the webapp itself is the limiting factor.
Apache + mod_jk + Tomcat, in their default configuration, by themselves, can handle
several hundred requests per second. If you do not believe this, then try it yourself :
set up a static html page inside tomcat's ROOT webapp, make sure that the mod_jk
configuration forwards these requests to Tomcat, and then use ab to make requests to that
static page, just to convince yourself.
Then use one of the Tomcat standard example webapps, and do the same.
Then, use Tomcat's AccessLogValve
(http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html#Access_Log_Valve) to compare
the processing times between static pages, an example webapp and your webapp.
When you really understand where the problem seems to happen, then come back and ask more
precise questions.
The point is : the default configuration of Apache, mod_jk and Tomcat is *already* tuned
to perform well in most standard situations. If there existed any way "in general" to tune
these things better, and the Apache, mod_jk and Tomcat developers knew it, then of course
*that* would become the default configuration.
The point is also : nobody here has any idea of what your old webapp is doing; only you
know that. If it is making 10,000 disk accesses each time it processes a request, then
maybe what you need is a faster disk. If it builds a huge table in memory each time it is
called, then maybe you need more memory. Etc..
One last thing : Tomcat 4.1 and the JVM 1.4.2, both, are about 10 years old, and not
supported anymore. The current versions are free, and supported. Do yourself (and
everyone else) a favor and make your testing on the latest versions. Your webapp should
work just as well (or better) with them.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: pick load
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Rainer,
On 8/26/2010 3:25 AM, Rainer Jung wrote:
> On 26.08.2010 03:28, Alexandre Chapellon wrote:
>> Whatever the configuration of my connecter (both on the apache or tomcat
>> side) I never go upper than 20 requests / second.
>> Here are few parameters I changed in order to get better performances:
>
> Take thread dumps of the Tomcat JVM and check what your applicaion is
> actually doing (like waiting for locks or externals components).
+1
My money is on a database connection pool with maxActive="20".
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkx2qmkACgkQ9CaO5/Lv0PAE3gCeInJO94S/ZIOjyt+qjKAOPwR1
LiIAoKzELwrWGTMmG5ZwWtBkcn1kg1Tz
=aHsu
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: pick load
Posted by Alexandre Chapellon <al...@mana.pf>.
Le jeudi 26 août 2010 à 22:04 -0500, Caldarale, Charles R a écrit :
> > From: Alexandre Chapellon [mailto:alexandre.chapellon@mana.pf]
> > Subject: Re: pick load
>
> > To me this sounds like a pool of database connection
> > that is full
>
> It's not a full connection pool - you're not using one. It's also likely that your database is a bit sluggish in establishing connections. That version of the JVM sets a global lock while creating a connection, with the expectation that it will occur fairly quickly, allowing the next connection to be established. Note that this has nothing to do with Tomcat.
>
> > I don't really know how/where to deal with it.
>
> Fix your DB, and configure a connection pool rather than calling com.mana.oc.DBConnection.getConnectionCRM directly, so each request doesn't have to acquire a new connection. Also upgrade to supported versions of things, since both the level of Tomcat you're on and the JVM have been unsupported for quite some time. The most recent JVMs do not keep that particular global lock up on the connection request to the DB, so moving to Java 6 might help.
>
> > If someone has the kind willing to take a look at the
> > full dump here it is: http://pastebin.com/2v3PVTDm
>
> A brief look showed only one thread having an active DB connection, with all of the rest stuck on the global lock trying to get a connection in order to authenticate the client or else just waiting for something to do. Since you're not using a connection pool for the authentications, you're serializing the requests, so it's no wonder your throughput is terrible. Even the greatly improved performance of newer Tomcat and JVM versions won't overcome bad webapp design.
>
> - Chuck
>
Thank you Chuck I'm sooooooooooooo glad to hear this!
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
>
Re: pick load
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Alexandre,
On 9/2/2010 1:49 PM, Alexandre Chapellon wrote:
> Yes to me too, but what's weired is that thoose errors enver appears
> under normal load... this really drives me crazy!
I would bet that there's an exception being caught and re-thrown without
the original exception being logged. Check the code around where that
exception is being thrown (Client.java:676) and see if it's in a catch
block. If so, try logging the original exception, or chaining the root
cause to the new exception like this:
throw new Exception("Erreur: impossible de recuperer la partie CHAP0712A
: " + whatever, originalException);
This will help you figure out what's really happening.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkyAGMcACgkQ9CaO5/Lv0PBMqgCgove+TKLYTcsd9JWQy3gTff4b
qgMAn0F7jknSIJ1+xrtseRkRZtpMmKKp
=9CDH
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: pick load
Posted by Alexandre Chapellon <al...@mana.pf>.
Le jeudi 02 septembre 2010 à 11:22 -0400, Christopher Schultz a écrit :
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Alexandre,
>
> On 9/2/2010 12:08 AM, Alexandre Chapellon wrote:
> > Forget about it this doesn't seems to be related to the jk connector.
>
> Right: the jk connector can be tweaked separately. You appear to have
> other problems.
>
> > I have the same problems when direct sending requests to tomcat (errors
> > which do not appear under nomal load):
> >
> > 2010-09-01 18:06:53.459 - FAILURE - [tracing] - MANA - Exception dans
> > [Authentification::doAuthentification(String,String)] :
> > [Search::searchPartyByAddInfo(String,object)] Erreur : impossible de
> > recuperer les parties avec l'info additionnel accessCode = CHAP0712A :
> > null
> > 2010-09-01 18:06:53.459 - FAILURE - [tracing] - MANA -
> > [Authentification::performTask(HttpServletRequest,HttpServletResponse)]
> > Exception rencontr?e pendant l'authentification de CHAP0712A.
> > 2010-09-01 18:06:53.467 - FAILURE - [tracing] - MANA - Exception dans
> > [Search::searchParty(String)] : null
> > 2010-09-01 18:06:53.471 - FAILURE - [tracing] - MANA - Exception dans
> > [Client.fetchInformation(String)] :[Search::searchParty(String)]
> > Erreur : impossible de recuperer la partie CHAP0712A : null
> > java.lang.Exception: [Search::searchParty(String)] Erreur : impossible
> > de recuperer la partie CHAP0712A : null
> > at com.mana.om.Client.fetchInformation(Client.java:676)
> > at com.mana.selfcare.Authenticate.performTask(Authenticate.java:207)
> > at com.mana.selfcare.Authenticate.doPost(Authenticate.java:84)
>
> That looks like an application error to me.
Yes to me too, but what's weired is that thoose errors enver appears
under normal load... this really drives me crazy!
>
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkx/wTMACgkQ9CaO5/Lv0PDXFgCgrMYix3jPszsSdOotB2qyZ9+i
> DBIAnjm44KkSTGLwRtl6GswN/njUC5bD
> =ZDt3
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: pick load
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Alexandre,
On 9/2/2010 12:08 AM, Alexandre Chapellon wrote:
> Forget about it this doesn't seems to be related to the jk connector.
Right: the jk connector can be tweaked separately. You appear to have
other problems.
> I have the same problems when direct sending requests to tomcat (errors
> which do not appear under nomal load):
>
> 2010-09-01 18:06:53.459 - FAILURE - [tracing] - MANA - Exception dans
> [Authentification::doAuthentification(String,String)] :
> [Search::searchPartyByAddInfo(String,object)] Erreur : impossible de
> recuperer les parties avec l'info additionnel accessCode = CHAP0712A :
> null
> 2010-09-01 18:06:53.459 - FAILURE - [tracing] - MANA -
> [Authentification::performTask(HttpServletRequest,HttpServletResponse)]
> Exception rencontr?e pendant l'authentification de CHAP0712A.
> 2010-09-01 18:06:53.467 - FAILURE - [tracing] - MANA - Exception dans
> [Search::searchParty(String)] : null
> 2010-09-01 18:06:53.471 - FAILURE - [tracing] - MANA - Exception dans
> [Client.fetchInformation(String)] :[Search::searchParty(String)]
> Erreur : impossible de recuperer la partie CHAP0712A : null
> java.lang.Exception: [Search::searchParty(String)] Erreur : impossible
> de recuperer la partie CHAP0712A : null
> at com.mana.om.Client.fetchInformation(Client.java:676)
> at com.mana.selfcare.Authenticate.performTask(Authenticate.java:207)
> at com.mana.selfcare.Authenticate.doPost(Authenticate.java:84)
That looks like an application error to me.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkx/wTMACgkQ9CaO5/Lv0PDXFgCgrMYix3jPszsSdOotB2qyZ9+i
DBIAnjm44KkSTGLwRtl6GswN/njUC5bD
=ZDt3
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: pick load
Posted by Alexandre Chapellon <al...@mana.pf>.
Forget about it this doesn't seems to be related to the jk connector.
I have the same problems when direct sending requests to tomcat (errors
which do not appear under nomal load):
2010-09-01 18:06:53.459 - FAILURE - [tracing] - MANA - Exception dans
[Authentification::doAuthentification(String,String)] :
[Search::searchPartyByAddInfo(String,object)] Erreur : impossible de
recuperer les parties avec l'info additionnel accessCode = CHAP0712A :
null
2010-09-01 18:06:53.459 - FAILURE - [tracing] - MANA -
[Authentification::performTask(HttpServletRequest,HttpServletResponse)]
Exception rencontr?e pendant l'authentification de CHAP0712A.
2010-09-01 18:06:53.467 - FAILURE - [tracing] - MANA - Exception dans
[Search::searchParty(String)] : null
2010-09-01 18:06:53.471 - FAILURE - [tracing] - MANA - Exception dans
[Client.fetchInformation(String)] :[Search::searchParty(String)]
Erreur : impossible de recuperer la partie CHAP0712A : null
java.lang.Exception: [Search::searchParty(String)] Erreur : impossible
de recuperer la partie CHAP0712A : null
at com.mana.om.Client.fetchInformation(Client.java:676)
at com.mana.selfcare.Authenticate.performTask(Authenticate.java:207)
at com.mana.selfcare.Authenticate.doPost(Authenticate.java:84)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:638)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:720)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:199)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:145)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
at org.apache.catalina.core.StandardPipeline
$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:955)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:139)
at org.apache.catalina.core.StandardPipeline
$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:446)
at org.apache.catalina.core.StandardPipeline
$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:545)
at org.apache.catalina.core.StandardPipeline
$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:955)
at
org.apache.catalina.core.StandardContext.invoke(StandardContext.java:2460)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:133)
at org.apache.catalina.core.StandardPipeline
$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at
org.apache.catalina.valves.ErrorDispatcherValve.invoke(ErrorDispatcherValve.java:119)
at org.apache.catalina.core.StandardPipeline
$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:116)
at org.apache.catalina.core.StandardPipeline
$StandardPipelineValveContext.invokeNext(StandardPipeline.java:594)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:955)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:127)
at org.apache.catalina.core.StandardPipeline
$StandardPipelineValveContext.invokeNext(StandardPipeline.java:596)
at
org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:433)
at
org.apache.catalina.core.ContainerBase.invoke(ContainerBase.java:955)
at
org.apache.coyote.tomcat4.CoyoteAdapter.service(CoyoteAdapter.java:157)
at
org.apache.jk.server.JkCoyoteHandler.invoke(JkCoyoteHandler.java:200)
at org.apache.jk.common.HandlerRequest.invoke(HandlerRequest.java:291)
at org.apache.jk.common.ChannelSocket.invoke(ChannelSocket.java:775)
at
org.apache.jk.common.ChannelSocket.processConnection(ChannelSocket.java:704)
at org.apache.jk.common.ChannelSocket
$SocketConnection.runIt(ChannelSocket.java:897)
at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:689)
at java.lang.Thread.run(Thread.java:534)
Le mercredi 01 septembre 2010 à 17:28 -1000, Alexandre Chapellon a
écrit :
> With load growing I have thoose errors that appear in mod_jk logs:
>
> [Sun Aug 29 08:28:27.976 2010] [26242:4106161040] [info]
> jk_handler::mod_jk.c (2611): Aborting connection for worker=selfcare
> [Sun Aug 29 12:15:45.829 2010] [27162:4097772432] [info]
> ajp_service::jk_ajp_common.c (2540): (selfcare) sending request to
> tomcat failed (recoverable), because of server error (attempt=1)
> [Sun Aug 29 12:15:45.929 2010] [27162:4097772432] [info]
> ajp_send_request::jk_ajp_common.c (1490): (selfcare) did not receive
> END_RESPONSE, closing socket -1
> [Sun Aug 29 12:15:45.933 2010] [27162:4097772432] [info]
> ajp_service::jk_ajp_common.c (2540): (selfcare) sending request to
> tomcat failed (recoverable), because of server error (attempt=2)
> [Sun Aug 29 12:15:45.933 2010] [27162:4097772432] [error]
> ajp_service::jk_ajp_common.c (2559): (selfcare) connecting to tomcat
> failed.
> [Sun Aug 29 12:15:45.933 2010] [27162:4097772432] [info]
> jk_handler::mod_jk.c (2618): Service error=-3 for worker=selfcare
> [Sun Aug 29 15:04:46.507 2010] [27104:4039052176] [info]
> ajp_process_callback::jk_ajp_common.c (1882): Writing to client aborted
> or client network problems
> ...skipping...
>
> I really don't know what to tune....
>
> Le mardi 31 août 2010 à 17:49 -0500, Caldarale, Charles R a écrit :
>
> > > From: Alexandre Chapellon [mailto:alexandre.chapellon@mana.pf]
> > > Subject: Re: pick load
> >
> > > We're using an old version of highdeal billing system which
> > > apparently (tat's what the support says) doens't support java6.
> >
> > Chris suggested *Tomcat* 6, not Java 6. You can run Tomcat 6 on JRE 5.
> >
> > - Chuck
> >
> >
> > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
> >
>
>
RE: pick load
Posted by Alexandre Chapellon <al...@mana.pf>.
With load growing I have thoose errors that appear in mod_jk logs:
[Sun Aug 29 08:28:27.976 2010] [26242:4106161040] [info]
jk_handler::mod_jk.c (2611): Aborting connection for worker=selfcare
[Sun Aug 29 12:15:45.829 2010] [27162:4097772432] [info]
ajp_service::jk_ajp_common.c (2540): (selfcare) sending request to
tomcat failed (recoverable), because of server error (attempt=1)
[Sun Aug 29 12:15:45.929 2010] [27162:4097772432] [info]
ajp_send_request::jk_ajp_common.c (1490): (selfcare) did not receive
END_RESPONSE, closing socket -1
[Sun Aug 29 12:15:45.933 2010] [27162:4097772432] [info]
ajp_service::jk_ajp_common.c (2540): (selfcare) sending request to
tomcat failed (recoverable), because of server error (attempt=2)
[Sun Aug 29 12:15:45.933 2010] [27162:4097772432] [error]
ajp_service::jk_ajp_common.c (2559): (selfcare) connecting to tomcat
failed.
[Sun Aug 29 12:15:45.933 2010] [27162:4097772432] [info]
jk_handler::mod_jk.c (2618): Service error=-3 for worker=selfcare
[Sun Aug 29 15:04:46.507 2010] [27104:4039052176] [info]
ajp_process_callback::jk_ajp_common.c (1882): Writing to client aborted
or client network problems
...skipping...
I really don't know what to tune....
Le mardi 31 août 2010 à 17:49 -0500, Caldarale, Charles R a écrit :
> > From: Alexandre Chapellon [mailto:alexandre.chapellon@mana.pf]
> > Subject: Re: pick load
>
> > We're using an old version of highdeal billing system which
> > apparently (tat's what the support says) doens't support java6.
>
> Chris suggested *Tomcat* 6, not Java 6. You can run Tomcat 6 on JRE 5.
>
> - Chuck
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
>
RE: pick load
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Alexandre Chapellon [mailto:alexandre.chapellon@mana.pf]
> Subject: Re: pick load
> We're using an old version of highdeal billing system which
> apparently (tat's what the support says) doens't support java6.
Chris suggested *Tomcat* 6, not Java 6. You can run Tomcat 6 on JRE 5.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
Re: pick load
Posted by Alexandre Chapellon <al...@mana.pf>.
Le mardi 31 août 2010 à 14:39 -0400, Christopher Schultz a écrit :
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Alexandre,
>
> On 8/30/2010 10:49 PM, Alexandre Chapellon wrote:
> > [Considering] the fair analysis of Charles, we decided to move applications
> > to Poolable connection factory
>
> Excellent.
>
> > To do so I created a factory iun server.xml as follow:
> >
> > <Resource name="jdbc/crm" auth="Container" type="javax.sql.DataSource"/>
>
> I just realized that you're using Tomcat 4.1. Any possibility of moving
> up to Tomcat 6.0? I've recently moved several webapps from 4.1 up to 6.0
> with very little headache. You'll get the benefit of security and
> stability updates, as well as performance improvements in the new version.
>
We're using an old version of highdeal billing system which apparently
(tat's what the support says) doens't support java6.
Migration is planned for next year.
> > <ResourceParams name="jdbc/crm">
> > <parameter>
> > <name>url</name>
> > <value>jdbc:oracle:thin:@1.2.3.4:1521:SID</value>
>
> Double-check that this driver URL is correct. If you copied it from your
> old configuration, it's probably fine.
>
Indeed the url what good for one of the pool (oracle9i) but not for the
other (Oracle10g)
I Changed it, and so get rid of error messages.
4
> > <parameter>
> > <name>factory</name>
> >
> > <value>org.apache.commons.dbcp.BasicDataSourceFactory</value>
> > </parameter>
>
> This parameter should not be necessary. Once you have things working,
> try removing this parameter. If things are still working fine, then
> leave this out of your configuration.
>
> > When trying to connect to the application. Login succeed but I get the
> > following errors in catalina.out:
> >
> > org.apache.commons.dbcp.SQLNestedException: Cannot create
> > PoolableConnectionFactory (invalid argument in call)
>
> Can you post the code that fetches a connection from the pool?
>
> > Did i miss something in xml definitions or is it a problem with the way
> > the app uses the connection Pool?
>
> My guess is the location of the driver .jar file. The file oracle.jar
> (or whatever it is called) should /only/ be in the server's server/lib
> directory, and /not/ in the webapp's WEB-INF/lib directory. If you have
> the driver .jar in both places (or only in WEB-INF/lib), you'll get
> errors like this.
>
I removed the factory param and redundant oracle jar files.
Thanks for your help
> - -chris
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.10 (MingW32)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAkx9TEwACgkQ9CaO5/Lv0PCgHQCfcGRWdjQ8UHYpUizRn49GbD4p
> D/kAn0V5rZ2rjtOjyfBZcQH0UcmZsXdq
> =7a6L
> -----END PGP SIGNATURE-----
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: pick load
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Alexandre,
On 8/30/2010 10:49 PM, Alexandre Chapellon wrote:
> [Considering] the fair analysis of Charles, we decided to move applications
> to Poolable connection factory
Excellent.
> To do so I created a factory iun server.xml as follow:
>
> <Resource name="jdbc/crm" auth="Container" type="javax.sql.DataSource"/>
I just realized that you're using Tomcat 4.1. Any possibility of moving
up to Tomcat 6.0? I've recently moved several webapps from 4.1 up to 6.0
with very little headache. You'll get the benefit of security and
stability updates, as well as performance improvements in the new version.
> <ResourceParams name="jdbc/crm">
> <parameter>
> <name>url</name>
> <value>jdbc:oracle:thin:@1.2.3.4:1521:SID</value>
Double-check that this driver URL is correct. If you copied it from your
old configuration, it's probably fine.
> <parameter>
> <name>factory</name>
>
> <value>org.apache.commons.dbcp.BasicDataSourceFactory</value>
> </parameter>
This parameter should not be necessary. Once you have things working,
try removing this parameter. If things are still working fine, then
leave this out of your configuration.
> When trying to connect to the application. Login succeed but I get the
> following errors in catalina.out:
>
> org.apache.commons.dbcp.SQLNestedException: Cannot create
> PoolableConnectionFactory (invalid argument in call)
Can you post the code that fetches a connection from the pool?
> Did i miss something in xml definitions or is it a problem with the way
> the app uses the connection Pool?
My guess is the location of the driver .jar file. The file oracle.jar
(or whatever it is called) should /only/ be in the server's server/lib
directory, and /not/ in the webapp's WEB-INF/lib directory. If you have
the driver .jar in both places (or only in WEB-INF/lib), you'll get
errors like this.
- -chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAkx9TEwACgkQ9CaO5/Lv0PCgHQCfcGRWdjQ8UHYpUizRn49GbD4p
D/kAn0V5rZ2rjtOjyfBZcQH0UcmZsXdq
=7a6L
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: pick load
Posted by Alexandre Chapellon <al...@mana.pf>.
Consering the fair analisys of Charles, we decided to move applications
to Poolable connection factory
To do so I created a factory iun server.xml as follow:
<Resource name="jdbc/crm" auth="Container" type="javax.sql.DataSource"/>
<ResourceParams name="jdbc/crm">
<parameter>
<name>url</name>
<value>jdbc:oracle:thin:@1.2.3.4:1521:SID</value>
</parameter>
<parameter>
<name>user</name>
<value>scott</value>
</parameter>
<parameter>
<name>password</name>
<value>tiger</value>
</parameter>
<parameter>
<name>maxActive</name>
<value>60</value>
</parameter>
<parameter>
<name>factory</name>
<value>org.apache.commons.dbcp.BasicDataSourceFactory</value>
</parameter>
<parameter>
<name>driverClassName</name>
<value>oracle.jdbc.OracleDriver</value>
</parameter>
</ResourceParams>
and in web.xml of the app, I added at the end:
<resource-ref>
<description>
Database connection pool
</description>
<res-ref-name>jdbc/crm</res-ref-name>
<res-type>javax.sql.DataSource</res-type>
<res-auth>Container</res-auth>
</resource-ref>
When trying to connect to the application. Login succeed but I get the
following errors in catalina.out:
org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (invalid argument in call)
Did i miss something in xml definitions or is it a problem with the way
the app uses the connection Pool?
regards
Le jeudi 26 août 2010 à 22:04 -0500, Caldarale, Charles R a écrit :
> > From: Alexandre Chapellon [mailto:alexandre.chapellon@mana.pf]
> > Subject: Re: pick load
>
> > To me this sounds like a pool of database connection
> > that is full
>
> It's not a full connection pool - you're not using one. It's also likely that your database is a bit sluggish in establishing connections. That version of the JVM sets a global lock while creating a connection, with the expectation that it will occur fairly quickly, allowing the next connection to be established. Note that this has nothing to do with Tomcat.
>
> > I don't really know how/where to deal with it.
>
> Fix your DB, and configure a connection pool rather than calling com.mana.oc.DBConnection.getConnectionCRM directly, so each request doesn't have to acquire a new connection. Also upgrade to supported versions of things, since both the level of Tomcat you're on and the JVM have been unsupported for quite some time. The most recent JVMs do not keep that particular global lock up on the connection request to the DB, so moving to Java 6 might help.
>
> > If someone has the kind willing to take a look at the
> > full dump here it is: http://pastebin.com/2v3PVTDm
>
> A brief look showed only one thread having an active DB connection, with all of the rest stuck on the global lock trying to get a connection in order to authenticate the client or else just waiting for something to do. Since you're not using a connection pool for the authentications, you're serializing the requests, so it's no wonder your throughput is terrible. Even the greatly improved performance of newer Tomcat and JVM versions won't overcome bad webapp design.
>
> - Chuck
>
>
> THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
>
RE: pick load
Posted by "Caldarale, Charles R" <Ch...@unisys.com>.
> From: Alexandre Chapellon [mailto:alexandre.chapellon@mana.pf]
> Subject: Re: pick load
> To me this sounds like a pool of database connection
> that is full
It's not a full connection pool - you're not using one. It's also likely that your database is a bit sluggish in establishing connections. That version of the JVM sets a global lock while creating a connection, with the expectation that it will occur fairly quickly, allowing the next connection to be established. Note that this has nothing to do with Tomcat.
> I don't really know how/where to deal with it.
Fix your DB, and configure a connection pool rather than calling com.mana.oc.DBConnection.getConnectionCRM directly, so each request doesn't have to acquire a new connection. Also upgrade to supported versions of things, since both the level of Tomcat you're on and the JVM have been unsupported for quite some time. The most recent JVMs do not keep that particular global lock up on the connection request to the DB, so moving to Java 6 might help.
> If someone has the kind willing to take a look at the
> full dump here it is: http://pastebin.com/2v3PVTDm
A brief look showed only one thread having an active DB connection, with all of the rest stuck on the global lock trying to get a connection in order to authenticate the client or else just waiting for something to do. Since you're not using a connection pool for the authentications, you're serializing the requests, so it's no wonder your throughput is terrible. Even the greatly improved performance of newer Tomcat and JVM versions won't overcome bad webapp design.
- Chuck
THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY MATERIAL and is thus for use only by the intended recipient. If you received this in error, please contact the sender and delete the e-mail and its attachments from all computers.
Re: pick load
Posted by Alexandre Chapellon <al...@mana.pf>.
Thanks.
I got the dump of the running JVM (only the benchmark is running at dump
time).
I tried to take a look at it, but as am not familiar with java, I prefer
talk about it here.
-first I noticed the number of TP-Processor is twice the number of
concurrent resquests send by ab (why twice? i don't know).
About half of them are in Object.wait state
"TP-Processor40" daemon prio=1 tid=0xdef53168 nid=0x646f in
Object.wait() [0xda729000..0xda7291b8]
at java.lang.Object.wait(Native Method)
- waiting on <0xe17bc808> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Object.wait(Object.java:429)
at org.apache.tomcat.util.threads.ThreadPool
$ControlRunnable.run(ThreadPool.java:661)
- locked <0xe17bc808> (a
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Thread.run(Thread.java:534)
and the other half is waiting for a monitor entry.
"TP-Processor34" daemon prio=1 tid=0xdef3edc0 nid=0x6469 waiting for
monitor entry [0xdaa28000..0xdaa291b8]
at java.sql.DriverManager.getConnection(DriverManager.java:158)
- waiting to lock <0xeff0b350> (a java.lang.Class)
at
com.mana.oc.DBConnection.getConnectionCRM(DBConnection.java:155)
at org.apache.jsp.Login_jsp._jspService(Login_jsp.java:192)
......
And only 5 runnable processors.
To me this sounds like a pool of database connection that is full but
I'm not sure of it and I don't really know how/where to deal with it.
If someone has the kind willing to take a look at the full dump here it
is: http://pastebin.com/2v3PVTDm
Regards
Le jeudi 26 août 2010 à 21:36 +0200, Rainer Jung a écrit :
> On 26.08.2010 21:00, Alexandre Chapellon wrote:
> > Le jeudi 26 août 2010 à 09:25 +0200, Rainer Jung a écrit :
> >
> >> On 26.08.2010 03:28, Alexandre Chapellon wrote:
> >>> Hello,
> >>>
> >>> I'm quite new to tomcat and have an old webapps running on tomcat 4.1
> >>> and jvm 1.4.2 with apach2.2 in front ofthem (using modjk).
> >>> I'm trying to get ready for a comming pick load I will have to face.
> >>> I Try to do some benchmark using ab and the jkstatus worker.
> >>> Whatever the configuration of my connecter (both on the apache or tomcat
> >>> side) I never go upper than 20 requests / second.
> >>> Here are few parameters I changed in order to get better performances:
> >>>
> >>> -Apache2 (worker):
> >>> increased ServerLimit (64), ThreadLimit (256), MaxClients (2048),
> >>> ThreadsPerChild (128)
> >>> set to a non zero value MaxRequestsPerChild (500)
> >>>
> >>> - modjk (1.2.30):
> >>> set to non-zero value worker.selfcare.connection_pool_timeout=60
> >>>
> >>> -Tomcat AJP13 Connector:
> >>> acceptCount="50" enableLookups="false" maxProcessors="500"
> >>> bufferSize="4096" socketBuffer="20000"
> >>>
> >>> Unfortunately this doesn't help and am still stuck with 20req/s when the
> >>> machines' load is not that high and 60% of CPU at most is used during
> >>> stress test.
> >>> I've googled around but can't find anything else about increasing
> >>> performances of apache/tomcat... Help much appreciated
> >>>
> >>> Regards
> >>>
> >>> P.S: right now am using ab to send 2000 request with 50 concurrents.
> >>
> >> Take thread dumps of the Tomcat JVM and check what your applicaion is
> >> actually doing (like waiting for locks or externals components).
> >>
> >
> >
> > This sounds an excellent idea indeed, and it's surely what I would have
> > done if I new it was possible and how I could do it :)
> > What's the way to do it?
>
> http://wiki.apache.org/tomcat/HowTo#How_do_I_obtain_a_thread_dump_of_my_running_webapp_.3F
>
> Regards,
>
> Rainer
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: pick load
Posted by Rainer Jung <ra...@kippdata.de>.
On 26.08.2010 21:00, Alexandre Chapellon wrote:
> Le jeudi 26 août 2010 à 09:25 +0200, Rainer Jung a écrit :
>
>> On 26.08.2010 03:28, Alexandre Chapellon wrote:
>>> Hello,
>>>
>>> I'm quite new to tomcat and have an old webapps running on tomcat 4.1
>>> and jvm 1.4.2 with apach2.2 in front ofthem (using modjk).
>>> I'm trying to get ready for a comming pick load I will have to face.
>>> I Try to do some benchmark using ab and the jkstatus worker.
>>> Whatever the configuration of my connecter (both on the apache or tomcat
>>> side) I never go upper than 20 requests / second.
>>> Here are few parameters I changed in order to get better performances:
>>>
>>> -Apache2 (worker):
>>> increased ServerLimit (64), ThreadLimit (256), MaxClients (2048),
>>> ThreadsPerChild (128)
>>> set to a non zero value MaxRequestsPerChild (500)
>>>
>>> - modjk (1.2.30):
>>> set to non-zero value worker.selfcare.connection_pool_timeout=60
>>>
>>> -Tomcat AJP13 Connector:
>>> acceptCount="50" enableLookups="false" maxProcessors="500"
>>> bufferSize="4096" socketBuffer="20000"
>>>
>>> Unfortunately this doesn't help and am still stuck with 20req/s when the
>>> machines' load is not that high and 60% of CPU at most is used during
>>> stress test.
>>> I've googled around but can't find anything else about increasing
>>> performances of apache/tomcat... Help much appreciated
>>>
>>> Regards
>>>
>>> P.S: right now am using ab to send 2000 request with 50 concurrents.
>>
>> Take thread dumps of the Tomcat JVM and check what your applicaion is
>> actually doing (like waiting for locks or externals components).
>>
>
>
> This sounds an excellent idea indeed, and it's surely what I would have
> done if I new it was possible and how I could do it :)
> What's the way to do it?
http://wiki.apache.org/tomcat/HowTo#How_do_I_obtain_a_thread_dump_of_my_running_webapp_.3F
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: pick load
Posted by Alexandre Chapellon <al...@mana.pf>.
Le jeudi 26 août 2010 à 09:25 +0200, Rainer Jung a écrit :
> On 26.08.2010 03:28, Alexandre Chapellon wrote:
> > Hello,
> >
> > I'm quite new to tomcat and have an old webapps running on tomcat 4.1
> > and jvm 1.4.2 with apach2.2 in front ofthem (using modjk).
> > I'm trying to get ready for a comming pick load I will have to face.
> > I Try to do some benchmark using ab and the jkstatus worker.
> > Whatever the configuration of my connecter (both on the apache or tomcat
> > side) I never go upper than 20 requests / second.
> > Here are few parameters I changed in order to get better performances:
> >
> > -Apache2 (worker):
> > increased ServerLimit (64), ThreadLimit (256), MaxClients (2048),
> > ThreadsPerChild (128)
> > set to a non zero value MaxRequestsPerChild (500)
> >
> > - modjk (1.2.30):
> > set to non-zero value worker.selfcare.connection_pool_timeout=60
> >
> > -Tomcat AJP13 Connector:
> > acceptCount="50" enableLookups="false" maxProcessors="500"
> > bufferSize="4096" socketBuffer="20000"
> >
> > Unfortunately this doesn't help and am still stuck with 20req/s when the
> > machines' load is not that high and 60% of CPU at most is used during
> > stress test.
> > I've googled around but can't find anything else about increasing
> > performances of apache/tomcat... Help much appreciated
> >
> > Regards
> >
> > P.S: right now am using ab to send 2000 request with 50 concurrents.
>
> Take thread dumps of the Tomcat JVM and check what your applicaion is
> actually doing (like waiting for locks or externals components).
>
This sounds an excellent idea indeed, and it's surely what I would have
done if I new it was possible and how I could do it :)
What's the way to do it?
> Regards,
>
> Rainer
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: pick load
Posted by Rainer Jung <ra...@kippdata.de>.
On 26.08.2010 03:28, Alexandre Chapellon wrote:
> Hello,
>
> I'm quite new to tomcat and have an old webapps running on tomcat 4.1
> and jvm 1.4.2 with apach2.2 in front ofthem (using modjk).
> I'm trying to get ready for a comming pick load I will have to face.
> I Try to do some benchmark using ab and the jkstatus worker.
> Whatever the configuration of my connecter (both on the apache or tomcat
> side) I never go upper than 20 requests / second.
> Here are few parameters I changed in order to get better performances:
>
> -Apache2 (worker):
> increased ServerLimit (64), ThreadLimit (256), MaxClients (2048),
> ThreadsPerChild (128)
> set to a non zero value MaxRequestsPerChild (500)
>
> - modjk (1.2.30):
> set to non-zero value worker.selfcare.connection_pool_timeout=60
>
> -Tomcat AJP13 Connector:
> acceptCount="50" enableLookups="false" maxProcessors="500"
> bufferSize="4096" socketBuffer="20000"
>
> Unfortunately this doesn't help and am still stuck with 20req/s when the
> machines' load is not that high and 60% of CPU at most is used during
> stress test.
> I've googled around but can't find anything else about increasing
> performances of apache/tomcat... Help much appreciated
>
> Regards
>
> P.S: right now am using ab to send 2000 request with 50 concurrents.
Take thread dumps of the Tomcat JVM and check what your applicaion is
actually doing (like waiting for locks or externals components).
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: pick load
Posted by Alexandre Chapellon <al...@mana.pf>.
ab doesn't says much... except the requests completed and most of them
did in less than 5 seconds:
###########################
Server Software: Apache
Server Hostname: blablablabla.hostname
Server Port: 443
SSL/TLS Protocol: TLSv1/SSLv3,DHE-RSA-AES256-SHA,1024,256
Document Path: /path/to/Login.jsp
Document Length: 5576 bytes
Concurrency Level: 50
Time taken for tests: 81.949 seconds
Complete requests: 1648
Failed requests: 0
Write errors: 0
Total transferred: 9752864 bytes
HTML transferred: 9189248 bytes
Requests per second: 20.11 [#/sec] (mean)
Time per request: 2486.315 [ms] (mean)
Time per request: 49.726 [ms] (mean, across all concurrent
requests)
Transfer rate: 116.22 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 29 55 114.8 35 1425
Processing: 74 2392 377.9 2289 3589
Waiting: 73 2390 377.8 2286 3587
Total: 128 2447 363.8 2330 4256
Percentage of the requests served within a certain time (ms)
50% 2330
66% 2498
75% 2596
80% 2688
90% 2892
95% 3075
98% 3421
99% 3520
100% 4256 (longest request)
Not sure it helps... It's the jkmanger that shows 20req/s as max
thx
Le jeudi 26 août 2010 à 07:56 +0200, Domenico Briganti a écrit :
> Il giorno mer, 25/08/2010 alle 15.28 -1000, Alexandre Chapellon ha
> scritto:
> > P.S: right now am using ab to send 2000 request with 50 concurrents.
>
> What is the report of ab?
>
>
>
> Domenico
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Re: pick load
Posted by Domenico Briganti <do...@gmail.com>.
Il giorno mer, 25/08/2010 alle 15.28 -1000, Alexandre Chapellon ha
scritto:
> P.S: right now am using ab to send 2000 request with 50 concurrents.
What is the report of ab?
Domenico
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org