[dbcp][pool] Robust pooling

[Phil Steitz <ph...@gmail.com>]

Despite lots of user requests over the years, DBCP has always backed
off the challenge of providing really robust connection pooling -
i.e., seamlessly handling server or network failures.  The reason
for this is that from the vantage point of DBCP, doing more than
just validating connections and destroying / attempting to replace
broken ones ends up getting very complicated and ultimately hopeless
given the information available to DBCP itself.

There are three things that have changed that could lead to big
improvements in DBCP robustnesss:

0) Memory is now plentiful enough that the elegant but crippling
decision in the original design of GenericObjectPool to not hold
onto references to objects checked out from the pool can safely be
reversed.  DBCP 2 should hold references to and listen for events
from checked out connections.
1) JDBC 4 API provides better communication with connections
2) Coordination services are emerging

It is really 2) that takes us out of the "hopeless" realm.  Systems
like Zookeeper [1], Doozer [2] and other cloud management /
distributed coordination frameworks make it possible for DBCP to
skirt the responsibility of deciding *what* to do (the hopeless
part) and just decide instead on a protocol and algorithm for
executing failover / rerouting given instructions from a
coordinator.  It should be possible to tell DBCP to "stop using
server A and start using server B" in such a way that applications
using pooled connections are minimally impacted and, depending on
the state of server A and instructions from the coordinator,
transactions in progress are handled optimally.

There are lots of things to consider in making DBCP manageable in
the sense described above.  I am starting this discussion now
because I want to make sure that we build whatever we need to build
into pool 2 to make these features possible in DBCP 2.  Minimally,
we are going to need a GOP that holds references to checked out
instances and the "maintenance thread" concept now embodied in the
evictor needs to be extended to active instances.

Phil

[1] http://zookeeper.apache.org/
[2] *http://s.apache.org/uzQ*

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [dbcp][pool] Robust pooling

[sebb <se...@gmail.com>]

On 6 May 2011 18:16, Mark Thomas <ma...@apache.org> wrote:
> On 06/05/2011 17:55, Phil Steitz wrote:
>> There are lots of things to consider in making DBCP manageable in
>> the sense described above.  I am starting this discussion now
>> because I want to make sure that we build whatever we need to build
>> into pool 2 to make these features possible in DBCP 2.  Minimally,
>> we are going to need a GOP that holds references to checked out
>> instances and the "maintenance thread" concept now embodied in the
>> evictor needs to be extended to active instances.
>
> Good points. I need to reflect on what this might mean for the pool2 design.

Might be worth adding some kind of "owner" flag to pool entries.
When entries are checked out, the owner is set; when entries are
returned, the owner is checked against the current owner and set back
to the pool.
This can be used to check for double de-allocation and (perhaps) use
of entries by the "wrong" owner.

Just a thought.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [dbcp][pool] Robust pooling

[Mark Thomas <ma...@apache.org>]

On 06/05/2011 17:55, Phil Steitz wrote:
> There are lots of things to consider in making DBCP manageable in
> the sense described above.  I am starting this discussion now
> because I want to make sure that we build whatever we need to build
> into pool 2 to make these features possible in DBCP 2.  Minimally,
> we are going to need a GOP that holds references to checked out
> instances and the "maintenance thread" concept now embodied in the
> evictor needs to be extended to active instances.

Good points. I need to reflect on what this might mean for the pool2 design.

Mark



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org