You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@commons.apache.org by Todd Carmichael <to...@concur.com> on 2009/02/24 01:28:17 UTC
[DBCP] performance and scalability enhancements
Our site is using DBCP in several different applications. Our site has
high scalability requirements and some areas of DBCP and Pooling have
needed tuning in order to scale. I am soliciting feedback on these
changes. If the projects DBCP/POOL would/could make use of these
changes and to present a problem that we still having.
1) A single eviction thread for cleanup of stale connections. Our site
can have hundreds of database connection pools. An evictor thread per
connection pool puts a strain on the OS.
2) testOnBorrow causes far too much overhead on our site and limits
performance. We have implemented a testOnBorrow after a X milliseconds.
Meaning, it will only test the connection if the connection has NOT been
tested for X milliseconds.
The problem we are having is due in part to not using the out of the box
testOnBorrow functionality but can happen other times as well. Our JDBC
driver is throwing a SocketException when network connectivity to the
database server is lost (using MSSQL, issue a kill spid or the Activity
Monitor tool to kill a connection). Look at the following stack.
Feb 10, 2009 5:52:40 PM
com.concur.cup.webservices.xmlhttp.util.ErrorHandler logException
SEVERE:
com.inet.tds.be: java.net.SocketException: Connection reset by peer:
socket write error
at com.inet.tds.al.a(Unknown Source)
at com.inet.tds.n.a(Unknown Source)
at com.inet.tds.n.a(Unknown Source)
at com.inet.tds.n.setAutoCommit(Unknown Source)
at
org.apache.commons.dbcp.cpdsadapter.ConnectionImpl.setAutoCommit(Connect
ionImpl.java:325)
at
com.concur.cup.services.QueueProcessor.ProcessStepPoller.getProcessStepI
nstToProcess(ProcessStepPoller.java:344)
at
com.concur.cup.services.QueueProcessor.ProcessStepPoller.run(ProcessStep
Poller.java:290)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.net.SocketException: Connection reset by peer: socket
write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at
java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at com.inet.tds.g.a(Unknown Source)
... 6 more
The method signature for setAutoCommit does not state that it will throw
a SocketException. And, if I return this connection to the pool (in a
finally block), another bit of code will get a connection in a bad state
that cannot be used.
Any suggestions? Should Every single call to JDBC be wrapped with
something like:
catch(Exception e) { if (e instanceof SocketException) { don't return
connection to pool }}
Has anyone run into something like this?
Todd
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
RE: [DBCP] performance and scalability enhancements
Posted by Wes Clark <wc...@guidewire.com>.
We've overridden certain parts of DBCP.
We don't run testOnBorrow at all for performance reasons because we
return the connection to the pool if all we're doing is a SELECT, and
only keep the connection if we are planning on doing updates.
If during a rollback, if we catch an exception, we
"markConnectionForDeath". We then override
PoolableConnectionFactory,passivateObject(), removing all the clean-up
that DBCP does every time we return a connection to the pool, and check
to see if the connection has been marked for death. If so, we though an
exception at that point to discard the connection.
We an evictor thread as well, and that should find and evict any bad
connections.
Does this help at all?
-----Original Message-----
From: Todd Carmichael [mailto:toddc@concur.com]
Sent: Monday, February 23, 2009 4:28 PM
To: Commons Users List
Subject: [DBCP] performance and scalability enhancements
Our site is using DBCP in several different applications. Our site has
high scalability requirements and some areas of DBCP and Pooling have
needed tuning in order to scale. I am soliciting feedback on these
changes. If the projects DBCP/POOL would/could make use of these
changes and to present a problem that we still having.
1) A single eviction thread for cleanup of stale connections. Our site
can have hundreds of database connection pools. An evictor thread per
connection pool puts a strain on the OS.
2) testOnBorrow causes far too much overhead on our site and limits
performance. We have implemented a testOnBorrow after a X milliseconds.
Meaning, it will only test the connection if the connection has NOT been
tested for X milliseconds.
The problem we are having is due in part to not using the out of the box
testOnBorrow functionality but can happen other times as well. Our JDBC
driver is throwing a SocketException when network connectivity to the
database server is lost (using MSSQL, issue a kill spid or the Activity
Monitor tool to kill a connection). Look at the following stack.
Feb 10, 2009 5:52:40 PM
com.concur.cup.webservices.xmlhttp.util.ErrorHandler logException
SEVERE:
com.inet.tds.be: java.net.SocketException: Connection reset by peer:
socket write error
at com.inet.tds.al.a(Unknown Source)
at com.inet.tds.n.a(Unknown Source)
at com.inet.tds.n.a(Unknown Source)
at com.inet.tds.n.setAutoCommit(Unknown Source)
at
org.apache.commons.dbcp.cpdsadapter.ConnectionImpl.setAutoCommit(Connect
ionImpl.java:325)
at
com.concur.cup.services.QueueProcessor.ProcessStepPoller.getProcessStepI
nstToProcess(ProcessStepPoller.java:344)
at
com.concur.cup.services.QueueProcessor.ProcessStepPoller.run(ProcessStep
Poller.java:290)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.net.SocketException: Connection reset by peer: socket
write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at
java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at com.inet.tds.g.a(Unknown Source)
... 6 more
The method signature for setAutoCommit does not state that it will throw
a SocketException. And, if I return this connection to the pool (in a
finally block), another bit of code will get a connection in a bad state
that cannot be used.
Any suggestions? Should Every single call to JDBC be wrapped with
something like:
catch(Exception e) { if (e instanceof SocketException) { don't return
connection to pool }}
Has anyone run into something like this?
Todd
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
RE: [DBCP] performance and scalability enhancements
Posted by Todd Carmichael <to...@concur.com>.
I will look at writing up a patch for the single evictor thread.
The other reply indicating that a team wrote an addon for marking
connections dead indicates a need for a higher throughput mechanism for
removing bad connections from the pool. Ideally since the DBCP library
has a connection wrapper class, code could be added to all methods in
the wrapper class to mark a connection as dead and not put it back into
the pool.
-----Original Message-----
From: Mark Thomas [mailto:markt@apache.org]
Sent: Tuesday, February 24, 2009 1:43 AM
To: Commons Users List
Subject: Re: [DBCP] performance and scalability enhancements
Todd Carmichael wrote:
> Our site is using DBCP in several different applications. Our site
has
> high scalability requirements and some areas of DBCP and Pooling have
> needed tuning in order to scale. I am soliciting feedback on these
> changes. If the projects DBCP/POOL would/could make use of these
> changes and to present a problem that we still having.
>
> 1) A single eviction thread for cleanup of stale connections. Our
site
> can have hundreds of database connection pools. An evictor thread per
> connection pool puts a strain on the OS.
> 2) testOnBorrow causes far too much overhead on our site and limits
> performance. We have implemented a testOnBorrow after a X
milliseconds.
> Meaning, it will only test the connection if the connection has NOT
been
> tested for X milliseconds.
Both enhancements sound useful. Can you create a Jira entry and attach a
patch?
Cheers,
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org
Re: [DBCP] performance and scalability enhancements
Posted by Mark Thomas <ma...@apache.org>.
Todd Carmichael wrote:
> Our site is using DBCP in several different applications. Our site has
> high scalability requirements and some areas of DBCP and Pooling have
> needed tuning in order to scale. I am soliciting feedback on these
> changes. If the projects DBCP/POOL would/could make use of these
> changes and to present a problem that we still having.
>
> 1) A single eviction thread for cleanup of stale connections. Our site
> can have hundreds of database connection pools. An evictor thread per
> connection pool puts a strain on the OS.
> 2) testOnBorrow causes far too much overhead on our site and limits
> performance. We have implemented a testOnBorrow after a X milliseconds.
> Meaning, it will only test the connection if the connection has NOT been
> tested for X milliseconds.
Both enhancements sound useful. Can you create a Jira entry and attach a patch?
Cheers,
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org