You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Yasser <ya...@gmail.com> on 2012/06/28 16:43:39 UTC
Tomcat 7.0.28 connection pool issue
What we are doing?
We are in the process of migrating from Resin 2.0.3 to Tomcat 7.0.28.
The codebase running on Resin when subjected to a stress test (using
resin's connection pool) with maxactive of 50 perform fine. It meets out
non functionality requirements including hits/sec, throughtput, passed
transactions etc...
We added two new elements in the migration effort - Tomcat 7.0.28 and CAS
3.4.11.
What is the issue?
When we run a stress test on the same codebase deployed to Tomcat 7.0.28,
at about 2hr45min into the test with 530 virtual users logged in (at peak
load), I get a lot of connection pool empty errors. The maxactive attribute
(using tomcat connection pool) has been set to 100.
Other information:
CAS runs on a different server with a dedicated tomcat home and base. I
dont see any errors on this box.
7.0.26 was found to have a bug in the way connection count is determined,
which got fixed in 7.0.28, hence we switched to the latest version.
Here is one of the connection pool config from server.xml. We do a resource
link to this pool in the context.xml
<Resource name="jdbc/global-wl" auth="Container"
type="javax.sql.DataSource" username="webconnect" password="xxxxxxx"
driverClassName="oracle.jdbc.driver.OracleDriver"
url="jdbc:oracle:thin:@&&dbs.ip&&:&&oracle.db.port&&:&&dts.dbname&&"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
maxActive="100" />
Any help is appreciated. Please let me know if you need more information or
code snippets.
Thanks,
Yasser
Re: Tomcat 7.0.28 connection pool issue
Posted by Yasser <ya...@gmail.com>.
We use Oracle 11.2.0 and it can handle more than 1000 connections.
Here is the stacktrace of the connection pool empty error that I receive in
tomcat logs
Caused by: org.apache.tomcat.jdbc.pool.PoolExhaustedException:
[http-apr-30690-exec-743] Timeout: Pool empty. Unable to fetch a connection
in 30 seconds, none available[size:100; busy:72; idle:0; lastwait:30000].
at
org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:667)
at
org.apache.tomcat.jdbc.pool.ConnectionPool.getConnection(ConnectionPool.java:182)
at
org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:128)
at
org.springframework.orm.hibernate3.LocalDataSourceConnectionProvider.getConnection(LocalDataSourceConnectionProvider.java:82)
at
org.hibernate.jdbc.ConnectionManager.openConnection(ConnectionManager.java:417)
On Thu, Jun 28, 2012 at 11:46 AM, chris derham <ch...@derham.me.uk> wrote:
> On Thu, Jun 28, 2012 at 12:38 PM, Yasser <ya...@gmail.com> wrote:
>
> > Yes. It does show that maxactive has reached 100. I also use splunk to
> get
> > the connection status at the oracle side.
> > What I dont understand is that Resin needs just 50 connections to handle
> > the same load. I am in the process of increasing the count to 300 and see
> > if that makes a difference. Oracle has the capacity to handle that many
> > connections.
> >
> > We hit an odd oracle exception that may be related. Can't explain it
> really well, but the essence of the bug in Oracle was that the
> tnslistener's connection count wasn't updated "regularly enough" when
> connections closed, so it could believe that the connection limit had
> exceeded when the db instance was still happy to accept more connections.
> As I recall this was a known Oracle issue, and was fixed by increasing the
> acceptable connection count in Oracle. Can you provide the exact exception
> from the logs that you hit?
>
> Also have you tried turning on connection pool logging to diagnose when
> actual physical connections are created?
>
> Chris
>
RE: Tomcat 7.0.28 connection pool issue
Posted by Jeffrey Janner <Je...@PolyDyne.com>.
> -----Original Message-----
> From: cjderham@gmail.com [mailto:cjderham@gmail.com] On Behalf Of chris
> derham
> Sent: Thursday, June 28, 2012 10:47 AM
> To: Tomcat Users List
> Subject: Re: Tomcat 7.0.28 connection pool issue
>
> On Thu, Jun 28, 2012 at 12:38 PM, Yasser <ya...@gmail.com> wrote:
>
> > Yes. It does show that maxactive has reached 100. I also use splunk
> to
> > get the connection status at the oracle side.
> > What I dont understand is that Resin needs just 50 connections to
> > handle the same load. I am in the process of increasing the count to
> > 300 and see if that makes a difference. Oracle has the capacity to
> > handle that many connections.
> >
> > We hit an odd oracle exception that may be related. Can't explain it
> really well, but the essence of the bug in Oracle was that the
> tnslistener's connection count wasn't updated "regularly enough" when
> connections closed, so it could believe that the connection limit had
> exceeded when the db instance was still happy to accept more
> connections.
> As I recall this was a known Oracle issue, and was fixed by increasing
> the acceptable connection count in Oracle. Can you provide the exact
> exception from the logs that you hit?
Chris, this is one of the reasons for using a pool. Since the physical DB connection between Tomcat & Oracle is never actually closed in a pool scenario, you don't run into this problem (unless, of course, you have some other app also connection to the DB and not using a connection pool).
It's a function of how Oracle is designed, the Listener adds one to the "open" count for each new connection, and then hands the connection off to another process. At this point, the Listener no longer cares about the connection, and is not involved in "closing" the connection. There is another Oracle process that periodically updates the Listener with the actual current count of connections. I'm not sure what the "periodically" is set to, something like 10/15 seconds. Think of it as analogous to the TIMED-WAIT status of TCP connections (something that also easy to get caught up in without a connection pool).
Jeff
__________________________________________________________________________
Confidentiality Notice: This Transmission (including any attachments) may contain information that is privileged, confidential, and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient you are hereby notified that any dissemination, distribution, or copying of this communication is strictly prohibited.
If you have received this transmission in error, please immediately reply to the sender or telephone (512) 343-9100 and delete this transmission from your system.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 7.0.28 connection pool issue
Posted by chris derham <ch...@derham.me.uk>.
On Thu, Jun 28, 2012 at 12:38 PM, Yasser <ya...@gmail.com> wrote:
> Yes. It does show that maxactive has reached 100. I also use splunk to get
> the connection status at the oracle side.
> What I dont understand is that Resin needs just 50 connections to handle
> the same load. I am in the process of increasing the count to 300 and see
> if that makes a difference. Oracle has the capacity to handle that many
> connections.
>
> We hit an odd oracle exception that may be related. Can't explain it
really well, but the essence of the bug in Oracle was that the
tnslistener's connection count wasn't updated "regularly enough" when
connections closed, so it could believe that the connection limit had
exceeded when the db instance was still happy to accept more connections.
As I recall this was a known Oracle issue, and was fixed by increasing the
acceptable connection count in Oracle. Can you provide the exact exception
from the logs that you hit?
Also have you tried turning on connection pool logging to diagnose when
actual physical connections are created?
Chris
Re: Tomcat 7.0.28 connection pool issue
Posted by Yasser <ya...@gmail.com>.
By default fairQueue is set to true and the value of maxWait is 30 seconds.
I will try increasing the maxwait to 60 seconds and see if that buys tomcat
enough time to return connections to the pool and have it available for the
next connection request. Will let you know.
On Thu, Jun 28, 2012 at 1:36 PM, Filip Hanik (mailing lists) <
devlists@hanik.com> wrote:
> Then the issue you may be running into is that your Tomcat configuration
> supports a higher concurrency level than what your Resin configuration is
> setup to do.
> With higher concurrency, there will be a need for more data base
> connections. If you still want to run with a lower number of connections,
> what you can do is set
>
> fairQueue="true"
> maxWait="time in milliseconds to wait for next available connection"
>
> and what this effectively will do, is lower your concurrency. Recommended
> is
> of course to increase maxActive if the database supports it.
>
> Filip
>
>
> > -----Original Message-----
> > From: Yasser [mailto:yarafatin@gmail.com]
> > Sent: Thursday, June 28, 2012 11:33 AM
> > To: Tomcat Users List
> > Subject: Re: Tomcat 7.0.28 connection pool issue
> >
> > That was the issue with Tomcat 7.0.26 and they fixed it in 7.0.28
> >
> >
> > On Thu, Jun 28, 2012 at 11:54 AM, Filip Hanik (mailing lists) <
> > devlists@hanik.com> wrote:
> >
> > > Could you have run into
> > > https://issues.apache.org/bugzilla/show_bug.cgi?id=53367
> > >
> > > ?
> > >
> > > You could try out
> > > http://people.apache.org/~fhanik/jdbc-pool/bz53367-jdbc-pool.jar
> > >
> > >
> > > > -----Original Message-----
> > > > From: Yasser [mailto:yarafatin@gmail.com]
> > > > Sent: Thursday, June 28, 2012 9:39 AM
> > > > To: Tomcat Users List
> > > > Subject: Re: Tomcat 7.0.28 connection pool issue
> > > >
> > > > Yes. It does show that maxactive has reached 100. I also use splunk
> > to
> > > > get
> > > > the connection status at the oracle side.
> > > > What I dont understand is that Resin needs just 50 connections to
> > handle
> > > > the same load. I am in the process of increasing the count to 300
> > and
> > > > see
> > > > if that makes a difference. Oracle has the capacity to handle that
> > many
> > > > connections.
> > > >
> > > >
> > > > On Thu, Jun 28, 2012 at 11:01 AM, Hedrick, Brooke - 43 <
> > > > brooke.hedrick@rainhail.com> wrote:
> > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Yasser [mailto:yarafatin@gmail.com]
> > > > > > Sent: Thursday, June 28, 2012 9:44 AM
> > > > > > To: users@tomcat.apache.org
> > > > > > Subject: Tomcat 7.0.28 connection pool issue
> > > > > >
> > > > > ...
> > > > > > What is the issue?
> > > > > > When we run a stress test on the same codebase deployed to
> > Tomcat
> > > > 7.0.28,
> > > > > > at about 2hr45min into the test with 530 virtual users logged in
> > (at
> > > > > peak load),
> > > > > > I get a lot of connection pool empty errors. The maxactive
> > attribute
> > > > > (using
> > > > > > tomcat connection pool) has been set to 100.
> > > > > >
> > > > >
> > > > > Have you used jconsole to monitor the pool usage?
> > > > >
> > > > > -Brooke
> > > > >
> > > > > > Other information:
> > > > > > CAS runs on a different server with a dedicated tomcat home and
> > > > base. I
> > > > > > dont see any errors on this box.
> > > > > > 7.0.26 was found to have a bug in the way connection count is
> > > > determined,
> > > > > > which got fixed in 7.0.28, hence we switched to the latest
> > version.
> > > > > >
> > > > > > Here is one of the connection pool config from server.xml. We do
> > a
> > > > > resource
> > > > > > link to this pool in the context.xml
> > > > > >
> > > > > > <Resource name="jdbc/global-wl" auth="Container"
> > > > > > type="javax.sql.DataSource" username="webconnect"
> > > > > > password="xxxxxxx"
> > > > > > driverClassName="oracle.jdbc.driver.OracleDriver"
> > > > > >
> > > > > >
> > url="jdbc:oracle:thin:@&&dbs.ip&&:&&oracle.db.port&&:&&dts.dbname&&
> > > > > > "
> > > > > >
> > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
> > > > > > maxActive="100" />
> > > > > >
> > > > > > Any help is appreciated. Please let me know if you need more
> > > > information
> > > > > or
> > > > > > code snippets.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Yasser
> > > > >
> > > > > ------------------------------------------------------------------
> > ---
> > > > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > > > For additional commands, e-mail: users-help@tomcat.apache.org
> > > > >
> > > > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > For additional commands, e-mail: users-help@tomcat.apache.org
> > >
> > >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
RE: Tomcat 7.0.28 connection pool issue
Posted by "Filip Hanik (mailing lists)" <de...@hanik.com>.
An update on this issue. Yassir tested with a JAR I built against trunk.
What this could mean is that the bug
https://issues.apache.org/bugzilla/show_bug.cgi?id=53367
somehow didn't make it into the build of 7.0.28
I will double check it.
Filip
> -----Original Message-----
> From: Filip Hanik (mailing lists) [mailto:devlists@hanik.com]
> Sent: Thursday, June 28, 2012 2:42 PM
> To: 'Tomcat Users List'
> Subject: RE: Tomcat 7.0.28 connection pool issue
>
> Martin, generally I would run with fairQueue="false" - this is the
> default.
> The only time I would change to fairQueue="true" is if we see threads
> being
> starved, and not getting connections. However, this scenario is very
> unlikely unless there is extreme concurrency going on.
>
> Filip
>
> > -----Original Message-----
> > From: Martin Gainty [mailto:mgainty@hotmail.com]
> > Sent: Thursday, June 28, 2012 1:38 PM
> > To: Tomcat Users List
> > Subject: RE: Tomcat 7.0.28 connection pool issue
> >
> >
> > Hi Filip
> >
> > Is there an algorithm we can use to determine if the op should
> configure
> > concurrent db connections (fairQueue=false) vs config non-concurrent
> db
> > connections (fairQueue=true)
> > e.g. if 50%+ of database cursors are 'read-only' then concurrent
> > connections *should be used* and TC attribute of fairQueue should be
> set
> > to false
> > Thanks!
> > Martin
> > ______________________________________________
> > Place legal disclaimer here
> >
> > > From: devlists@hanik.com
> > > To: users@tomcat.apache.org
> > > Subject: RE: Tomcat 7.0.28 connection pool issue
> > > Date: Thu, 28 Jun 2012 11:36:49 -0600
> > >
> > > Then the issue you may be running into is that your Tomcat
> > configuration
> > > supports a higher concurrency level than what your Resin
> configuration
> > is
> > > setup to do.
> > > With higher concurrency, there will be a need for more data base
> > > connections. If you still want to run with a lower number of
> > connections,
> > > what you can do is set
> > >
> > > fairQueue="true"
> > > maxWait="time in milliseconds to wait for next available connection"
> > >
> > > and what this effectively will do, is lower your concurrency.
> > Recommended is
> > > of course to increase maxActive if the database supports it.
> > >
> > > Filip
> > >
> > >
> > > > -----Original Message-----
> > > > From: Yasser [mailto:yarafatin@gmail.com]
> > > > Sent: Thursday, June 28, 2012 11:33 AM
> > > > To: Tomcat Users List
> > > > Subject: Re: Tomcat 7.0.28 connection pool issue
> > > >
> > > > That was the issue with Tomcat 7.0.26 and they fixed it in 7.0.28
> > > >
> > > >
> > > > On Thu, Jun 28, 2012 at 11:54 AM, Filip Hanik (mailing lists) <
> > > > devlists@hanik.com> wrote:
> > > >
> > > > > Could you have run into
> > > > > https://issues.apache.org/bugzilla/show_bug.cgi?id=53367
> > > > >
> > > > > ?
> > > > >
> > > > > You could try out
> > > > > http://people.apache.org/~fhanik/jdbc-pool/bz53367-jdbc-pool.jar
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Yasser [mailto:yarafatin@gmail.com]
> > > > > > Sent: Thursday, June 28, 2012 9:39 AM
> > > > > > To: Tomcat Users List
> > > > > > Subject: Re: Tomcat 7.0.28 connection pool issue
> > > > > >
> > > > > > Yes. It does show that maxactive has reached 100. I also use
> > splunk
> > > > to
> > > > > > get
> > > > > > the connection status at the oracle side.
> > > > > > What I dont understand is that Resin needs just 50 connections
> > to
> > > > handle
> > > > > > the same load. I am in the process of increasing the count to
> > 300
> > > > and
> > > > > > see
> > > > > > if that makes a difference. Oracle has the capacity to handle
> > that
> > > > many
> > > > > > connections.
> > > > > >
> > > > > >
> > > > > > On Thu, Jun 28, 2012 at 11:01 AM, Hedrick, Brooke - 43 <
> > > > > > brooke.hedrick@rainhail.com> wrote:
> > > > > >
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Yasser [mailto:yarafatin@gmail.com]
> > > > > > > > Sent: Thursday, June 28, 2012 9:44 AM
> > > > > > > > To: users@tomcat.apache.org
> > > > > > > > Subject: Tomcat 7.0.28 connection pool issue
> > > > > > > >
> > > > > > > ...
> > > > > > > > What is the issue?
> > > > > > > > When we run a stress test on the same codebase deployed to
> > > > Tomcat
> > > > > > 7.0.28,
> > > > > > > > at about 2hr45min into the test with 530 virtual users
> > logged in
> > > > (at
> > > > > > > peak load),
> > > > > > > > I get a lot of connection pool empty errors. The maxactive
> > > > attribute
> > > > > > > (using
> > > > > > > > tomcat connection pool) has been set to 100.
> > > > > > > >
> > > > > > >
> > > > > > > Have you used jconsole to monitor the pool usage?
> > > > > > >
> > > > > > > -Brooke
> > > > > > >
> > > > > > > > Other information:
> > > > > > > > CAS runs on a different server with a dedicated tomcat
> home
> > and
> > > > > > base. I
> > > > > > > > dont see any errors on this box.
> > > > > > > > 7.0.26 was found to have a bug in the way connection count
> > is
> > > > > > determined,
> > > > > > > > which got fixed in 7.0.28, hence we switched to the latest
> > > > version.
> > > > > > > >
> > > > > > > > Here is one of the connection pool config from server.xml.
> > We do
> > > > a
> > > > > > > resource
> > > > > > > > link to this pool in the context.xml
> > > > > > > >
> > > > > > > > <Resource name="jdbc/global-wl" auth="Container"
> > > > > > > > type="javax.sql.DataSource" username="webconnect"
> > > > > > > > password="xxxxxxx"
> > > > > > > > driverClassName="oracle.jdbc.driver.OracleDriver"
> > > > > > > >
> > > > > > > >
> > > >
> url="jdbc:oracle:thin:@&&dbs.ip&&:&&oracle.db.port&&:&&dts.dbname&&
> > > > > > > > "
> > > > > > > >
> > > > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
> > > > > > > > maxActive="100" />
> > > > > > > >
> > > > > > > > Any help is appreciated. Please let me know if you need
> more
> > > > > > information
> > > > > > > or
> > > > > > > > code snippets.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Yasser
> > > > > > >
> > > > > > > ------------------------------------------------------------
> --
> > ----
> > > > ---
> > > > > > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > > > > > For additional commands, e-mail: users-
> help@tomcat.apache.org
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > > ----------------------------------------------------------------
> --
> > ---
> > > > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > > > For additional commands, e-mail: users-help@tomcat.apache.org
> > > > >
> > > > >
> > >
> > >
> > > --------------------------------------------------------------------
> -
> > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > For additional commands, e-mail: users-help@tomcat.apache.org
> > >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat 7.0.28 connection pool issue
Posted by "Filip Hanik (mailing lists)" <de...@hanik.com>.
Martin, generally I would run with fairQueue="false" - this is the default.
The only time I would change to fairQueue="true" is if we see threads being
starved, and not getting connections. However, this scenario is very
unlikely unless there is extreme concurrency going on.
Filip
> -----Original Message-----
> From: Martin Gainty [mailto:mgainty@hotmail.com]
> Sent: Thursday, June 28, 2012 1:38 PM
> To: Tomcat Users List
> Subject: RE: Tomcat 7.0.28 connection pool issue
>
>
> Hi Filip
>
> Is there an algorithm we can use to determine if the op should configure
> concurrent db connections (fairQueue=false) vs config non-concurrent db
> connections (fairQueue=true)
> e.g. if 50%+ of database cursors are 'read-only' then concurrent
> connections *should be used* and TC attribute of fairQueue should be set
> to false
> Thanks!
> Martin
> ______________________________________________
> Place legal disclaimer here
>
> > From: devlists@hanik.com
> > To: users@tomcat.apache.org
> > Subject: RE: Tomcat 7.0.28 connection pool issue
> > Date: Thu, 28 Jun 2012 11:36:49 -0600
> >
> > Then the issue you may be running into is that your Tomcat
> configuration
> > supports a higher concurrency level than what your Resin configuration
> is
> > setup to do.
> > With higher concurrency, there will be a need for more data base
> > connections. If you still want to run with a lower number of
> connections,
> > what you can do is set
> >
> > fairQueue="true"
> > maxWait="time in milliseconds to wait for next available connection"
> >
> > and what this effectively will do, is lower your concurrency.
> Recommended is
> > of course to increase maxActive if the database supports it.
> >
> > Filip
> >
> >
> > > -----Original Message-----
> > > From: Yasser [mailto:yarafatin@gmail.com]
> > > Sent: Thursday, June 28, 2012 11:33 AM
> > > To: Tomcat Users List
> > > Subject: Re: Tomcat 7.0.28 connection pool issue
> > >
> > > That was the issue with Tomcat 7.0.26 and they fixed it in 7.0.28
> > >
> > >
> > > On Thu, Jun 28, 2012 at 11:54 AM, Filip Hanik (mailing lists) <
> > > devlists@hanik.com> wrote:
> > >
> > > > Could you have run into
> > > > https://issues.apache.org/bugzilla/show_bug.cgi?id=53367
> > > >
> > > > ?
> > > >
> > > > You could try out
> > > > http://people.apache.org/~fhanik/jdbc-pool/bz53367-jdbc-pool.jar
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Yasser [mailto:yarafatin@gmail.com]
> > > > > Sent: Thursday, June 28, 2012 9:39 AM
> > > > > To: Tomcat Users List
> > > > > Subject: Re: Tomcat 7.0.28 connection pool issue
> > > > >
> > > > > Yes. It does show that maxactive has reached 100. I also use
> splunk
> > > to
> > > > > get
> > > > > the connection status at the oracle side.
> > > > > What I dont understand is that Resin needs just 50 connections
> to
> > > handle
> > > > > the same load. I am in the process of increasing the count to
> 300
> > > and
> > > > > see
> > > > > if that makes a difference. Oracle has the capacity to handle
> that
> > > many
> > > > > connections.
> > > > >
> > > > >
> > > > > On Thu, Jun 28, 2012 at 11:01 AM, Hedrick, Brooke - 43 <
> > > > > brooke.hedrick@rainhail.com> wrote:
> > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Yasser [mailto:yarafatin@gmail.com]
> > > > > > > Sent: Thursday, June 28, 2012 9:44 AM
> > > > > > > To: users@tomcat.apache.org
> > > > > > > Subject: Tomcat 7.0.28 connection pool issue
> > > > > > >
> > > > > > ...
> > > > > > > What is the issue?
> > > > > > > When we run a stress test on the same codebase deployed to
> > > Tomcat
> > > > > 7.0.28,
> > > > > > > at about 2hr45min into the test with 530 virtual users
> logged in
> > > (at
> > > > > > peak load),
> > > > > > > I get a lot of connection pool empty errors. The maxactive
> > > attribute
> > > > > > (using
> > > > > > > tomcat connection pool) has been set to 100.
> > > > > > >
> > > > > >
> > > > > > Have you used jconsole to monitor the pool usage?
> > > > > >
> > > > > > -Brooke
> > > > > >
> > > > > > > Other information:
> > > > > > > CAS runs on a different server with a dedicated tomcat home
> and
> > > > > base. I
> > > > > > > dont see any errors on this box.
> > > > > > > 7.0.26 was found to have a bug in the way connection count
> is
> > > > > determined,
> > > > > > > which got fixed in 7.0.28, hence we switched to the latest
> > > version.
> > > > > > >
> > > > > > > Here is one of the connection pool config from server.xml.
> We do
> > > a
> > > > > > resource
> > > > > > > link to this pool in the context.xml
> > > > > > >
> > > > > > > <Resource name="jdbc/global-wl" auth="Container"
> > > > > > > type="javax.sql.DataSource" username="webconnect"
> > > > > > > password="xxxxxxx"
> > > > > > > driverClassName="oracle.jdbc.driver.OracleDriver"
> > > > > > >
> > > > > > >
> > > url="jdbc:oracle:thin:@&&dbs.ip&&:&&oracle.db.port&&:&&dts.dbname&&
> > > > > > > "
> > > > > > >
> > > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
> > > > > > > maxActive="100" />
> > > > > > >
> > > > > > > Any help is appreciated. Please let me know if you need more
> > > > > information
> > > > > > or
> > > > > > > code snippets.
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Yasser
> > > > > >
> > > > > > --------------------------------------------------------------
> ----
> > > ---
> > > > > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > > > > For additional commands, e-mail: users-help@tomcat.apache.org
> > > > > >
> > > > > >
> > > >
> > > >
> > > > ------------------------------------------------------------------
> ---
> > > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > > For additional commands, e-mail: users-help@tomcat.apache.org
> > > >
> > > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> >
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
RE: Tomcat 7.0.28 connection pool issue
Posted by Martin Gainty <mg...@hotmail.com>.
Hi Filip
Is there an algorithm we can use to determine if the op should configure concurrent db connections (fairQueue=false) vs config non-concurrent db connections (fairQueue=true)
e.g. if 50%+ of database cursors are 'read-only' then concurrent connections *should be used* and TC attribute of fairQueue should be set to false
Thanks!
Martin
______________________________________________
Place legal disclaimer here
> From: devlists@hanik.com
> To: users@tomcat.apache.org
> Subject: RE: Tomcat 7.0.28 connection pool issue
> Date: Thu, 28 Jun 2012 11:36:49 -0600
>
> Then the issue you may be running into is that your Tomcat configuration
> supports a higher concurrency level than what your Resin configuration is
> setup to do.
> With higher concurrency, there will be a need for more data base
> connections. If you still want to run with a lower number of connections,
> what you can do is set
>
> fairQueue="true"
> maxWait="time in milliseconds to wait for next available connection"
>
> and what this effectively will do, is lower your concurrency. Recommended is
> of course to increase maxActive if the database supports it.
>
> Filip
>
>
> > -----Original Message-----
> > From: Yasser [mailto:yarafatin@gmail.com]
> > Sent: Thursday, June 28, 2012 11:33 AM
> > To: Tomcat Users List
> > Subject: Re: Tomcat 7.0.28 connection pool issue
> >
> > That was the issue with Tomcat 7.0.26 and they fixed it in 7.0.28
> >
> >
> > On Thu, Jun 28, 2012 at 11:54 AM, Filip Hanik (mailing lists) <
> > devlists@hanik.com> wrote:
> >
> > > Could you have run into
> > > https://issues.apache.org/bugzilla/show_bug.cgi?id=53367
> > >
> > > ?
> > >
> > > You could try out
> > > http://people.apache.org/~fhanik/jdbc-pool/bz53367-jdbc-pool.jar
> > >
> > >
> > > > -----Original Message-----
> > > > From: Yasser [mailto:yarafatin@gmail.com]
> > > > Sent: Thursday, June 28, 2012 9:39 AM
> > > > To: Tomcat Users List
> > > > Subject: Re: Tomcat 7.0.28 connection pool issue
> > > >
> > > > Yes. It does show that maxactive has reached 100. I also use splunk
> > to
> > > > get
> > > > the connection status at the oracle side.
> > > > What I dont understand is that Resin needs just 50 connections to
> > handle
> > > > the same load. I am in the process of increasing the count to 300
> > and
> > > > see
> > > > if that makes a difference. Oracle has the capacity to handle that
> > many
> > > > connections.
> > > >
> > > >
> > > > On Thu, Jun 28, 2012 at 11:01 AM, Hedrick, Brooke - 43 <
> > > > brooke.hedrick@rainhail.com> wrote:
> > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Yasser [mailto:yarafatin@gmail.com]
> > > > > > Sent: Thursday, June 28, 2012 9:44 AM
> > > > > > To: users@tomcat.apache.org
> > > > > > Subject: Tomcat 7.0.28 connection pool issue
> > > > > >
> > > > > ...
> > > > > > What is the issue?
> > > > > > When we run a stress test on the same codebase deployed to
> > Tomcat
> > > > 7.0.28,
> > > > > > at about 2hr45min into the test with 530 virtual users logged in
> > (at
> > > > > peak load),
> > > > > > I get a lot of connection pool empty errors. The maxactive
> > attribute
> > > > > (using
> > > > > > tomcat connection pool) has been set to 100.
> > > > > >
> > > > >
> > > > > Have you used jconsole to monitor the pool usage?
> > > > >
> > > > > -Brooke
> > > > >
> > > > > > Other information:
> > > > > > CAS runs on a different server with a dedicated tomcat home and
> > > > base. I
> > > > > > dont see any errors on this box.
> > > > > > 7.0.26 was found to have a bug in the way connection count is
> > > > determined,
> > > > > > which got fixed in 7.0.28, hence we switched to the latest
> > version.
> > > > > >
> > > > > > Here is one of the connection pool config from server.xml. We do
> > a
> > > > > resource
> > > > > > link to this pool in the context.xml
> > > > > >
> > > > > > <Resource name="jdbc/global-wl" auth="Container"
> > > > > > type="javax.sql.DataSource" username="webconnect"
> > > > > > password="xxxxxxx"
> > > > > > driverClassName="oracle.jdbc.driver.OracleDriver"
> > > > > >
> > > > > >
> > url="jdbc:oracle:thin:@&&dbs.ip&&:&&oracle.db.port&&:&&dts.dbname&&
> > > > > > "
> > > > > >
> > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
> > > > > > maxActive="100" />
> > > > > >
> > > > > > Any help is appreciated. Please let me know if you need more
> > > > information
> > > > > or
> > > > > > code snippets.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Yasser
> > > > >
> > > > > ------------------------------------------------------------------
> > ---
> > > > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > > > For additional commands, e-mail: users-help@tomcat.apache.org
> > > > >
> > > > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > For additional commands, e-mail: users-help@tomcat.apache.org
> > >
> > >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
RE: Tomcat 7.0.28 connection pool issue
Posted by "Filip Hanik (mailing lists)" <de...@hanik.com>.
Then the issue you may be running into is that your Tomcat configuration
supports a higher concurrency level than what your Resin configuration is
setup to do.
With higher concurrency, there will be a need for more data base
connections. If you still want to run with a lower number of connections,
what you can do is set
fairQueue="true"
maxWait="time in milliseconds to wait for next available connection"
and what this effectively will do, is lower your concurrency. Recommended is
of course to increase maxActive if the database supports it.
Filip
> -----Original Message-----
> From: Yasser [mailto:yarafatin@gmail.com]
> Sent: Thursday, June 28, 2012 11:33 AM
> To: Tomcat Users List
> Subject: Re: Tomcat 7.0.28 connection pool issue
>
> That was the issue with Tomcat 7.0.26 and they fixed it in 7.0.28
>
>
> On Thu, Jun 28, 2012 at 11:54 AM, Filip Hanik (mailing lists) <
> devlists@hanik.com> wrote:
>
> > Could you have run into
> > https://issues.apache.org/bugzilla/show_bug.cgi?id=53367
> >
> > ?
> >
> > You could try out
> > http://people.apache.org/~fhanik/jdbc-pool/bz53367-jdbc-pool.jar
> >
> >
> > > -----Original Message-----
> > > From: Yasser [mailto:yarafatin@gmail.com]
> > > Sent: Thursday, June 28, 2012 9:39 AM
> > > To: Tomcat Users List
> > > Subject: Re: Tomcat 7.0.28 connection pool issue
> > >
> > > Yes. It does show that maxactive has reached 100. I also use splunk
> to
> > > get
> > > the connection status at the oracle side.
> > > What I dont understand is that Resin needs just 50 connections to
> handle
> > > the same load. I am in the process of increasing the count to 300
> and
> > > see
> > > if that makes a difference. Oracle has the capacity to handle that
> many
> > > connections.
> > >
> > >
> > > On Thu, Jun 28, 2012 at 11:01 AM, Hedrick, Brooke - 43 <
> > > brooke.hedrick@rainhail.com> wrote:
> > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Yasser [mailto:yarafatin@gmail.com]
> > > > > Sent: Thursday, June 28, 2012 9:44 AM
> > > > > To: users@tomcat.apache.org
> > > > > Subject: Tomcat 7.0.28 connection pool issue
> > > > >
> > > > ...
> > > > > What is the issue?
> > > > > When we run a stress test on the same codebase deployed to
> Tomcat
> > > 7.0.28,
> > > > > at about 2hr45min into the test with 530 virtual users logged in
> (at
> > > > peak load),
> > > > > I get a lot of connection pool empty errors. The maxactive
> attribute
> > > > (using
> > > > > tomcat connection pool) has been set to 100.
> > > > >
> > > >
> > > > Have you used jconsole to monitor the pool usage?
> > > >
> > > > -Brooke
> > > >
> > > > > Other information:
> > > > > CAS runs on a different server with a dedicated tomcat home and
> > > base. I
> > > > > dont see any errors on this box.
> > > > > 7.0.26 was found to have a bug in the way connection count is
> > > determined,
> > > > > which got fixed in 7.0.28, hence we switched to the latest
> version.
> > > > >
> > > > > Here is one of the connection pool config from server.xml. We do
> a
> > > > resource
> > > > > link to this pool in the context.xml
> > > > >
> > > > > <Resource name="jdbc/global-wl" auth="Container"
> > > > > type="javax.sql.DataSource" username="webconnect"
> > > > > password="xxxxxxx"
> > > > > driverClassName="oracle.jdbc.driver.OracleDriver"
> > > > >
> > > > >
> url="jdbc:oracle:thin:@&&dbs.ip&&:&&oracle.db.port&&:&&dts.dbname&&
> > > > > "
> > > > >
> factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
> > > > > maxActive="100" />
> > > > >
> > > > > Any help is appreciated. Please let me know if you need more
> > > information
> > > > or
> > > > > code snippets.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Yasser
> > > >
> > > > ------------------------------------------------------------------
> ---
> > > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > > For additional commands, e-mail: users-help@tomcat.apache.org
> > > >
> > > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 7.0.28 connection pool issue
Posted by Yasser <ya...@gmail.com>.
That was the issue with Tomcat 7.0.26 and they fixed it in 7.0.28
On Thu, Jun 28, 2012 at 11:54 AM, Filip Hanik (mailing lists) <
devlists@hanik.com> wrote:
> Could you have run into
> https://issues.apache.org/bugzilla/show_bug.cgi?id=53367
>
> ?
>
> You could try out
> http://people.apache.org/~fhanik/jdbc-pool/bz53367-jdbc-pool.jar
>
>
> > -----Original Message-----
> > From: Yasser [mailto:yarafatin@gmail.com]
> > Sent: Thursday, June 28, 2012 9:39 AM
> > To: Tomcat Users List
> > Subject: Re: Tomcat 7.0.28 connection pool issue
> >
> > Yes. It does show that maxactive has reached 100. I also use splunk to
> > get
> > the connection status at the oracle side.
> > What I dont understand is that Resin needs just 50 connections to handle
> > the same load. I am in the process of increasing the count to 300 and
> > see
> > if that makes a difference. Oracle has the capacity to handle that many
> > connections.
> >
> >
> > On Thu, Jun 28, 2012 at 11:01 AM, Hedrick, Brooke - 43 <
> > brooke.hedrick@rainhail.com> wrote:
> >
> > >
> > > > -----Original Message-----
> > > > From: Yasser [mailto:yarafatin@gmail.com]
> > > > Sent: Thursday, June 28, 2012 9:44 AM
> > > > To: users@tomcat.apache.org
> > > > Subject: Tomcat 7.0.28 connection pool issue
> > > >
> > > ...
> > > > What is the issue?
> > > > When we run a stress test on the same codebase deployed to Tomcat
> > 7.0.28,
> > > > at about 2hr45min into the test with 530 virtual users logged in (at
> > > peak load),
> > > > I get a lot of connection pool empty errors. The maxactive attribute
> > > (using
> > > > tomcat connection pool) has been set to 100.
> > > >
> > >
> > > Have you used jconsole to monitor the pool usage?
> > >
> > > -Brooke
> > >
> > > > Other information:
> > > > CAS runs on a different server with a dedicated tomcat home and
> > base. I
> > > > dont see any errors on this box.
> > > > 7.0.26 was found to have a bug in the way connection count is
> > determined,
> > > > which got fixed in 7.0.28, hence we switched to the latest version.
> > > >
> > > > Here is one of the connection pool config from server.xml. We do a
> > > resource
> > > > link to this pool in the context.xml
> > > >
> > > > <Resource name="jdbc/global-wl" auth="Container"
> > > > type="javax.sql.DataSource" username="webconnect"
> > > > password="xxxxxxx"
> > > > driverClassName="oracle.jdbc.driver.OracleDriver"
> > > >
> > > > url="jdbc:oracle:thin:@&&dbs.ip&&:&&oracle.db.port&&:&&dts.dbname&&
> > > > "
> > > > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
> > > > maxActive="100" />
> > > >
> > > > Any help is appreciated. Please let me know if you need more
> > information
> > > or
> > > > code snippets.
> > > >
> > > > Thanks,
> > > >
> > > > Yasser
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > > For additional commands, e-mail: users-help@tomcat.apache.org
> > >
> > >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
RE: Tomcat 7.0.28 connection pool issue
Posted by "Filip Hanik (mailing lists)" <de...@hanik.com>.
Could you have run into
https://issues.apache.org/bugzilla/show_bug.cgi?id=53367
?
You could try out
http://people.apache.org/~fhanik/jdbc-pool/bz53367-jdbc-pool.jar
> -----Original Message-----
> From: Yasser [mailto:yarafatin@gmail.com]
> Sent: Thursday, June 28, 2012 9:39 AM
> To: Tomcat Users List
> Subject: Re: Tomcat 7.0.28 connection pool issue
>
> Yes. It does show that maxactive has reached 100. I also use splunk to
> get
> the connection status at the oracle side.
> What I dont understand is that Resin needs just 50 connections to handle
> the same load. I am in the process of increasing the count to 300 and
> see
> if that makes a difference. Oracle has the capacity to handle that many
> connections.
>
>
> On Thu, Jun 28, 2012 at 11:01 AM, Hedrick, Brooke - 43 <
> brooke.hedrick@rainhail.com> wrote:
>
> >
> > > -----Original Message-----
> > > From: Yasser [mailto:yarafatin@gmail.com]
> > > Sent: Thursday, June 28, 2012 9:44 AM
> > > To: users@tomcat.apache.org
> > > Subject: Tomcat 7.0.28 connection pool issue
> > >
> > ...
> > > What is the issue?
> > > When we run a stress test on the same codebase deployed to Tomcat
> 7.0.28,
> > > at about 2hr45min into the test with 530 virtual users logged in (at
> > peak load),
> > > I get a lot of connection pool empty errors. The maxactive attribute
> > (using
> > > tomcat connection pool) has been set to 100.
> > >
> >
> > Have you used jconsole to monitor the pool usage?
> >
> > -Brooke
> >
> > > Other information:
> > > CAS runs on a different server with a dedicated tomcat home and
> base. I
> > > dont see any errors on this box.
> > > 7.0.26 was found to have a bug in the way connection count is
> determined,
> > > which got fixed in 7.0.28, hence we switched to the latest version.
> > >
> > > Here is one of the connection pool config from server.xml. We do a
> > resource
> > > link to this pool in the context.xml
> > >
> > > <Resource name="jdbc/global-wl" auth="Container"
> > > type="javax.sql.DataSource" username="webconnect"
> > > password="xxxxxxx"
> > > driverClassName="oracle.jdbc.driver.OracleDriver"
> > >
> > > url="jdbc:oracle:thin:@&&dbs.ip&&:&&oracle.db.port&&:&&dts.dbname&&
> > > "
> > > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
> > > maxActive="100" />
> > >
> > > Any help is appreciated. Please let me know if you need more
> information
> > or
> > > code snippets.
> > >
> > > Thanks,
> > >
> > > Yasser
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> > For additional commands, e-mail: users-help@tomcat.apache.org
> >
> >
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: Tomcat 7.0.28 connection pool issue
Posted by Yasser <ya...@gmail.com>.
Yes. It does show that maxactive has reached 100. I also use splunk to get
the connection status at the oracle side.
What I dont understand is that Resin needs just 50 connections to handle
the same load. I am in the process of increasing the count to 300 and see
if that makes a difference. Oracle has the capacity to handle that many
connections.
On Thu, Jun 28, 2012 at 11:01 AM, Hedrick, Brooke - 43 <
brooke.hedrick@rainhail.com> wrote:
>
> > -----Original Message-----
> > From: Yasser [mailto:yarafatin@gmail.com]
> > Sent: Thursday, June 28, 2012 9:44 AM
> > To: users@tomcat.apache.org
> > Subject: Tomcat 7.0.28 connection pool issue
> >
> ...
> > What is the issue?
> > When we run a stress test on the same codebase deployed to Tomcat 7.0.28,
> > at about 2hr45min into the test with 530 virtual users logged in (at
> peak load),
> > I get a lot of connection pool empty errors. The maxactive attribute
> (using
> > tomcat connection pool) has been set to 100.
> >
>
> Have you used jconsole to monitor the pool usage?
>
> -Brooke
>
> > Other information:
> > CAS runs on a different server with a dedicated tomcat home and base. I
> > dont see any errors on this box.
> > 7.0.26 was found to have a bug in the way connection count is determined,
> > which got fixed in 7.0.28, hence we switched to the latest version.
> >
> > Here is one of the connection pool config from server.xml. We do a
> resource
> > link to this pool in the context.xml
> >
> > <Resource name="jdbc/global-wl" auth="Container"
> > type="javax.sql.DataSource" username="webconnect"
> > password="xxxxxxx"
> > driverClassName="oracle.jdbc.driver.OracleDriver"
> >
> > url="jdbc:oracle:thin:@&&dbs.ip&&:&&oracle.db.port&&:&&dts.dbname&&
> > "
> > factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
> > maxActive="100" />
> >
> > Any help is appreciated. Please let me know if you need more information
> or
> > code snippets.
> >
> > Thanks,
> >
> > Yasser
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
>
RE: Tomcat 7.0.28 connection pool issue
Posted by "Hedrick, Brooke - 43" <br...@rainhail.com>.
> -----Original Message-----
> From: Yasser [mailto:yarafatin@gmail.com]
> Sent: Thursday, June 28, 2012 9:44 AM
> To: users@tomcat.apache.org
> Subject: Tomcat 7.0.28 connection pool issue
>
...
> What is the issue?
> When we run a stress test on the same codebase deployed to Tomcat 7.0.28,
> at about 2hr45min into the test with 530 virtual users logged in (at peak load),
> I get a lot of connection pool empty errors. The maxactive attribute (using
> tomcat connection pool) has been set to 100.
>
Have you used jconsole to monitor the pool usage?
-Brooke
> Other information:
> CAS runs on a different server with a dedicated tomcat home and base. I
> dont see any errors on this box.
> 7.0.26 was found to have a bug in the way connection count is determined,
> which got fixed in 7.0.28, hence we switched to the latest version.
>
> Here is one of the connection pool config from server.xml. We do a resource
> link to this pool in the context.xml
>
> <Resource name="jdbc/global-wl" auth="Container"
> type="javax.sql.DataSource" username="webconnect"
> password="xxxxxxx"
> driverClassName="oracle.jdbc.driver.OracleDriver"
>
> url="jdbc:oracle:thin:@&&dbs.ip&&:&&oracle.db.port&&:&&dts.dbname&&
> "
> factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
> maxActive="100" />
>
> Any help is appreciated. Please let me know if you need more information or
> code snippets.
>
> Thanks,
>
> Yasser
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org