Problem on DBCP with exhausted pool

[ITAGAKI Takahiro <it...@oss.ntt.co.jp>]

Hello,

I found DBCP does not work well when the connection pool is exhausted:

  1. There are many context switches in the server.
  2. Client-to-client response times are not stable.

If the issues are fiexed, we can use DBCP as a better connection multiplexer
(i.e, server threads > max database connections).

As far as I can see, their cause is same; the combination of wait() and
notifyAll() is used for the thread management in pool.impl.GenericObjectPool.
When the connection pool is exhausted and a connection is freed, all of
waiters are waken up. One of the threads will win the race, and other threads
will sleep again. It's the reason of 1. At the same time, we cannot control
the order of threads acquiring the free connection. FIFO ordering is needed
to tackle with 2.

Is it possible to replace wait-and-notifyAll to waking-up one-by-one
and FIFO ordering? For example, to use ReentrantLock(fair=true).
If I'll try to fix the issues, which class should I change?
I only need the connection pool used in DBCP, but GenericObjectPool
might affect other modules.

Comments and advices welcome.

Regards,
---
ITAGAKI Takahiro
NTT Open Source Software Center


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org

Re: Problem on DBCP with exhausted pool

["Wm.A.Stafford" <st...@marine.rutgers.edu>]

I would like to add something to this request.  I have spend a two 
frustrating weeks trying to figure out what was happening with our 
application only to eventually discover what Itagaki described .  I 
think the JavaDocs for BasicDataSource at least and probably anything 
that might use the pooling support should mention something about the 
behavior when the pool is  exhausted.  Specifically a description of 
the  situation, what the symptoms of pool exhaustion problems are and 
what some possible solutions might be.  Also, the JavaDoc for 
BasicDataSource in dbcp-1.2.2 has virtually nothing about abandoned 
connections. I think some discussion of the situation and how dbcp  
deals with it might help users of dbcp or BasicDataSource quickly figure 
out what is going wrong when they see usual behavior. 

 I would be glad to write this JavaDoc, just let me know.

-=bill stafford

ITAGAKI Takahiro wrote:
> Hello,
>
> I found DBCP does not work well when the connection pool is exhausted:
>
>   1. There are many context switches in the server.
>   2. Client-to-client response times are not stable.
>
> If the issues are fiexed, we can use DBCP as a better connection multiplexer
> (i.e, server threads > max database connections).
>
> As far as I can see, their cause is same; the combination of wait() and
> notifyAll() is used for the thread management in pool.impl.GenericObjectPool.
> When the connection pool is exhausted and a connection is freed, all of
> waiters are waken up. One of the threads will win the race, and other threads
> will sleep again. It's the reason of 1. At the same time, we cannot control
> the order of threads acquiring the free connection. FIFO ordering is needed
> to tackle with 2.
>
> Is it possible to replace wait-and-notifyAll to waking-up one-by-one
> and FIFO ordering? For example, to use ReentrantLock(fair=true).
> If I'll try to fix the issues, which class should I change?
> I only need the connection pool used in DBCP, but GenericObjectPool
> might affect other modules.
>
> Comments and advices welcome.
>
> Regards,
> ---
> ITAGAKI Takahiro
> NTT Open Source Software Center
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
> For additional commands, e-mail: user-help@commons.apache.org
>
>   


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org