[DBCP][POOL] Patches to support LRU and configurability of prepar ed statement pool

[Todd Carmichael <to...@concur.com>]

Here are a couple patches that you might find useful.  

Patch 1: jakarta-commons/pool (implementation of TODO in
GenericKeyedObjectPool) 
If maxTotal is specified for genericKeyedObjectPool, then clearOldest method
is called which removes the oldest 15% of the objects from the pool:
basically the cache now implements an LRU for pruning.  At some point it
would be nice if the percentage was configurable.  I will add to bugzilla as
an attachemnt if it will let me.  


Patch 2: jakarta-commons/dbcp
Currently there is not much configurability for the prepared statement
pooling.  This patch defines a new property on the driveradaptercpds class:
maxPreparedStatements.  If this value is set to something other than -1,
then the underlying statement pool will not use an evictor thread and will
use the mechanism in patch 1 to remove the oldest objects.  If set to -1,
then the old behavior is used.  You may find this patch useful.  In our
environment we are concerned about creating a separate thread for EVERY
connection.  

ToddC


-----Original Message-----
From: Dirk Verbeeck [mailto:dirk.verbeeck@pandora.be] 
Sent: Tuesday, February 24, 2004 1:51 PM
To: Todd Carmichael
Subject: Re: Prepared Statement pooling


Todd

The code looks just fine, building a TreeMap is needed if you want to 
remove multiple items at once. A small optimalization would be to 
limit the number of items you put in the map.
(if the size>itemsToRemove remove the youngest item)
But I don't see a way to avoid the sorted map.

If you can make some JUnit tests then I will commit this weekend.

Cheers
Dirk


Todd Carmichael wrote:

> Dirk,
> Here is the code that will clear the oldest items in a pool (basically 
> an LRU).  I would appreciate any thoughts you have concerning this 
> code.  It removes the oldest 25% of the items in the cache.  The 
> number of items to remove could definitely be a configurable option.  
> Is building the TreeMap (sorting the items) necessary (i.e. is there a 
> better way to determine the oldest items)?
> 
> Thanks.
> 
>     class ObjectTimeStampComparable implements Comparator
>     {
>         public int compare(Object o1, Object o2)
>         {
>             return (int)(((ObjectTimestampPair)o1).tstamp - 
> ((ObjectTimestampPair)o2).tstamp);
>         }
>     }
>     public synchronized void clearOldest() {
> 
>         TreeMap map = new TreeMap(new ObjectTimeStampComparable());
>         for(Iterator keyiter = _poolList.iterator(); keyiter.hasNext(); )
{
>             Object key = keyiter.next();
>             CursorableLinkedList list = 
> (CursorableLinkedList)(_poolMap.get(key));
>             for(Iterator it = list.iterator(); it.hasNext(); ) {
>                 try {
>                     // each item into the map uses the 
> objectimestamppair object
>                     // as the key.  It then gets sorted based on the 
> timstamp field
>                     // each value in the map is the parent list it 
> belongs in.
>                     ObjectTimestampPair pair = 
> (ObjectTimestampPair)(it.next());
>                     map.put(pair,key);
>                 } catch(Exception e) {
>                     // ignore error, keep destroying the rest
>                 }
>             }
>         }
>         Set setPairKeys = map.entrySet();
>         int itemsToRemove = (int)(_maxTotal*.25) + 1; // 25% of items 
> removed plus one to account for zero
>         Iterator iter = setPairKeys.iterator();
>         while(iter.hasNext() && itemsToRemove>0)
>         {
>             Map.Entry entry = (Map.Entry)iter.next();
>             // kind of backwards on naming.  In the map, each key is 
> the objecttimestamppair
>             // because it has the ordering with the timestamp value.  
> Each value that the
>             // key references is the key of the list it belongs to.
>             CursorableLinkedList list = 
> (CursorableLinkedList)(_poolMap.get(entry.getValue()));
>             list.remove(entry.getKey());
>             try {
>                 _factory.destroyObject(entry.getValue(),entry.getKey());
>             }
>             catch(Exception e) {
>                     // ignore error, keep destroying the rest
>             }
>             _totalIdle--;
>             itemsToRemove--;
>         }
>         notifyAll();
>     }
> 
> 
> ToddC