[pool] GenericKeyedObjectPool liveness

[Phil Steitz <ph...@gmail.com>]

Looking at POOL-350, I realized that we don't really have a coherent 
strategy for handling liveness issues in GKOP.  We have been playing 
whack-a-mole with problems resulting from two facts about how GKOP works:

 1. There are two capacity constraints that bind on individual keyed
    pools at different times:  total instance count and total per key.
 2. Borrowers waiting on keyed pools have no way to know when capacity
    exists to create new instances.  They have to wait for new instances
    to be added to the LinkedBlockingDeques that they are waiting on.

To (partially) work around 2, we added two methods.  First, clearOldest, 
called by borrowObject when no instances are available for the given 
key, creates make capacity by clearing the oldest idle instances across 
all of the keyed pools.  This makes it less likely that a borrower will 
park waiting when overall capacity exists to create an instance under 
the desired key. Second, reuseCapacity, called by returnObject and 
invalidateObject, tries to create an instance in the most heavily loaded 
pool that can add an instance.  These both help reduce the incidence of 
borrowers waiting when instances could be created to serve them, but 
they don't handle all cases and as reported in POOL-350, when there are 
a lot of pools, they can cause performance problems.

I think we need to agree on some principles before continuing to patch 
liveness issues in GKOP.  I hope we can lazily agree with:

0) Users should be able to turn off all optimizations that look across 
keyed pools (e.g. reuseCapacity, clearOdest).  With these things turned 
off, liveness at the individual keyed pool level should work like GOP 
(failed validations, invalidate, etc trigger creates under the same key).
1) New borrowers should not take precedence over waiting threads when 
capacity exists to create new instances.
2) When configured to globally optimize resources (new config option), 
GKOP should decide on each return, invalidate, evict (anything that 
passivates or destroys an instance) where to allocate the newly 
available capacity.  How exactly this works should be configurable.

None of these are true in the code today.  Note that 2) means that when 
instances are returned, they may not be added to their returning idle 
instance pools.  That is basically what clearOldest is doing ad hoc only 
for new borrowers.

Please anyone interested - especially users - let me know what you think 
of 0)-2).  If we can agree on them, we can refactor to make the code 
consistent with the principles.

Phil

Re: [pool] GenericKeyedObjectPool liveness

[Phil Steitz <ph...@gmail.com>]

On 11/16/21 6:34 AM, Gary Gregory wrote:
> Hi Phil,
>
> My only concern is what exactly is (my emphasis)  "When configured to
> *globally* optimize resources (new config option),"?
>
> Is this a static variable somewhere? I hope it would not because I do not
> want to debug a customer set up one day where where one corner of an app needs
> one setting and another a different one.

This would be a suitably named config property - possibly several or an 
enum-valued thing.  GKOP half does this now in an inconsistent and not 
client-fair way.  What I mean by "globally optimize resources" is the 
kind of thing that clearOldest does - trying to use "idle" capacity in 
keyed pools to meet the needs of borrowers waiting on other keys.  The 
situation you describe above is present now and is the cause of a lot of 
the liveness issues reported and hacks like clearOldest and 
reuseCapacity.  I am not sure how best to set up explicit configs, but 
what we have now is neither consistent nor even well-documented.  I 
would at least like to allow users to turn off attempts at global 
optimization entirely.  That would certainly improve throughput in GKOPs 
that don't care about optimizing across pools.

Phil
>
> TY!
> Gary
>
>
> On Sun, Nov 14, 2021 at 1:29 PM Phil Steitz <ph...@gmail.com> wrote:
>
>> Looking at POOL-350, I realized that we don't really have a coherent
>> strategy for handling liveness issues in GKOP.  We have been playing
>> whack-a-mole with problems resulting from two facts about how GKOP works:
>>
>>   1. There are two capacity constraints that bind on individual keyed
>>      pools at different times:  total instance count and total per key.
>>   2. Borrowers waiting on keyed pools have no way to know when capacity
>>      exists to create new instances.  They have to wait for new instances
>>      to be added to the LinkedBlockingDeques that they are waiting on.
>>
>> To (partially) work around 2, we added two methods.  First, clearOldest,
>> called by borrowObject when no instances are available for the given
>> key, creates make capacity by clearing the oldest idle instances across
>> all of the keyed pools.  This makes it less likely that a borrower will
>> park waiting when overall capacity exists to create an instance under
>> the desired key. Second, reuseCapacity, called by returnObject and
>> invalidateObject, tries to create an instance in the most heavily loaded
>> pool that can add an instance.  These both help reduce the incidence of
>> borrowers waiting when instances could be created to serve them, but
>> they don't handle all cases and as reported in POOL-350, when there are
>> a lot of pools, they can cause performance problems.
>>
>> I think we need to agree on some principles before continuing to patch
>> liveness issues in GKOP.  I hope we can lazily agree with:
>>
>> 0) Users should be able to turn off all optimizations that look across
>> keyed pools (e.g. reuseCapacity, clearOdest).  With these things turned
>> off, liveness at the individual keyed pool level should work like GOP
>> (failed validations, invalidate, etc trigger creates under the same key).
>> 1) New borrowers should not take precedence over waiting threads when
>> capacity exists to create new instances.
>> 2) When configured to globally optimize resources (new config option),
>> GKOP should decide on each return, invalidate, evict (anything that
>> passivates or destroys an instance) where to allocate the newly
>> available capacity.  How exactly this works should be configurable.
>>
>> None of these are true in the code today.  Note that 2) means that when
>> instances are returned, they may not be added to their returning idle
>> instance pools.  That is basically what clearOldest is doing ad hoc only
>> for new borrowers.
>>
>> Please anyone interested - especially users - let me know what you think
>> of 0)-2).  If we can agree on them, we can refactor to make the code
>> consistent with the principles.
>>
>> Phil
>>
>>
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org

Re: [pool] GenericKeyedObjectPool liveness

[Gary Gregory <ga...@gmail.com>]

Hi Phil,

My only concern is what exactly is (my emphasis)  "When configured to
*globally* optimize resources (new config option),"?

Is this a static variable somewhere? I hope it would not because I do not
want to debug a customer set up one day where one corner of an app needs
one setting and another a different one.

TY!
Gary


On Sun, Nov 14, 2021 at 1:29 PM Phil Steitz <ph...@gmail.com> wrote:

> Looking at POOL-350, I realized that we don't really have a coherent
> strategy for handling liveness issues in GKOP.  We have been playing
> whack-a-mole with problems resulting from two facts about how GKOP works:
>
>  1. There are two capacity constraints that bind on individual keyed
>     pools at different times:  total instance count and total per key.
>  2. Borrowers waiting on keyed pools have no way to know when capacity
>     exists to create new instances.  They have to wait for new instances
>     to be added to the LinkedBlockingDeques that they are waiting on.
>
> To (partially) work around 2, we added two methods.  First, clearOldest,
> called by borrowObject when no instances are available for the given
> key, creates make capacity by clearing the oldest idle instances across
> all of the keyed pools.  This makes it less likely that a borrower will
> park waiting when overall capacity exists to create an instance under
> the desired key. Second, reuseCapacity, called by returnObject and
> invalidateObject, tries to create an instance in the most heavily loaded
> pool that can add an instance.  These both help reduce the incidence of
> borrowers waiting when instances could be created to serve them, but
> they don't handle all cases and as reported in POOL-350, when there are
> a lot of pools, they can cause performance problems.
>
> I think we need to agree on some principles before continuing to patch
> liveness issues in GKOP.  I hope we can lazily agree with:
>
> 0) Users should be able to turn off all optimizations that look across
> keyed pools (e.g. reuseCapacity, clearOdest).  With these things turned
> off, liveness at the individual keyed pool level should work like GOP
> (failed validations, invalidate, etc trigger creates under the same key).
> 1) New borrowers should not take precedence over waiting threads when
> capacity exists to create new instances.
> 2) When configured to globally optimize resources (new config option),
> GKOP should decide on each return, invalidate, evict (anything that
> passivates or destroys an instance) where to allocate the newly
> available capacity.  How exactly this works should be configurable.
>
> None of these are true in the code today.  Note that 2) means that when
> instances are returned, they may not be added to their returning idle
> instance pools.  That is basically what clearOldest is doing ad hoc only
> for new borrowers.
>
> Please anyone interested - especially users - let me know what you think
> of 0)-2).  If we can agree on them, we can refactor to make the code
> consistent with the principles.
>
> Phil
>
>
>