[jira] [Updated] (GEODE-6594) Simplify client connection pooling

["Jacob S. Barrett (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/GEODE-6594?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacob S. Barrett updated GEODE-6594:
------------------------------------
    Description: 
Current client connection pooling is multi-layered. A {{Connection}} is wrapped in a pooled {PooledConnection}}. A {{PooledConnection}} is kept in primary pool in an {allConnections}} set and when unused it is also in the {{availableConnections}} deque. A {{PooledConnection}} can also be held in a thread local pool (secondary pool). If in a thread local pool then it is not in the primary pool's {{availableConnections}}. This arrangement between the secondary and primary pools results in complex relationships with expirations and rebalancing actions. Since GEODE-6515 thread local connections does not provide a performance boost over the primary pool anymore we should remove it entirely, thus simplifying these complexities.

Idle connection timeouts must scan the {{allConnections}} set since the secondary may hold idle connections that won't be in the {{availableConnections}}. This requires scanning the entire set of connections with an exclusive lock, which can be really expensive for large connection pools. If there were no secondary pools we could scan the {{availableConnections}} deque in reverse order stopping at the first idle connection not exceeding the timeout limit. This scan would be lock free and would not touch every connection. The {{availableConnections}} deque is used as a LIFO queue so connections at the front are hot and connection at the end are cold.

Connection lifetime expiration currently requires, again scanning the {{allConnections}} set with exclusive lock looking for connections that are over the lifetime limit. If the connection needs to be expired then the {{PooledConnection}} as its {{Connection}} swapped out for a new one to the same server. If we remove the secondary pool then expired connections could simply be removed from the {{availableConnections}} and {{allConnections}} and destroyed. A new connection would be created either on demand of from pre-fill. No complicated logic is required to support swapping out the underlying {{Connection}}.

Currently load conditioning requires scanning the {{allConnections}} for N number of idle connections to specific servers. Each matching {{PooledConnection}} then has it's underlying {{Connection}} swapped out for one to a different server. This requires extra logic in the {{borrowConnection(forServer)}} method to check that the associated server hasn't changed between to matching and activation, if it has it must be placed back in the queue and the scan must be repeated. Removing the secondary pool will removed the need to swap out connections. Any matching idle connection from the {{availableConnections}} queue will just be destroyed. New connections will be created to the less loaded server either on demand or via pre-fill. No complicated logic for swapping connections is required and no recheck for chaining servers is required.



  was:
Current client connection pooling is multi-layered. A {{Connection}} is wrapped in a pooled {PooledConnection}}. A {{PooledConnection}} is kept in primary pool in an {allConnections}} set and when unused it is also in the {{availableConnections}} deque. A {{PooledConnection}} can also be held in a thread local pool (secondary pool). If in a thread local pool then it is not in the primary pool's {{availableConnections}}. This arrangement between the secondary and primary pools results in complex relationships with expirations and rebalancing actions. Since thread local connections does not provide a performance boost over the primary pool anymore we should remove it entirely, thus simplifying these complexities.

Idle connection timeouts must scan the {{allConnections}} set since the secondary may hold idle connections that won't be in the {{availableConnections}}. This requires scanning the entire set of connections with an exclusive lock, which can be really expensive for large connection pools. If there were no secondary pools we could scan the {{availableConnections}} deque in reverse order stopping at the first idle connection not exceeding the timeout limit. This scan would be lock free and would not touch every connection. The {{availableConnections}} deque is used as a LIFO queue so connections at the front are hot and connection at the end are cold.

Connection lifetime expiration currently requires, again scanning the {{allConnections}} set with exclusive lock looking for connections that are over the lifetime limit. If the connection needs to be expired then the {{PooledConnection}} as its {{Connection}} swapped out for a new one to the same server. If we remove the secondary pool then expired connections could simply be removed from the {{availableConnections}} and {{allConnections}} and destroyed. A new connection would be created either on demand of from pre-fill. No complicated logic is required to support swapping out the underlying {{Connection}}.

Currently load conditioning requires scanning the {{allConnections}} for N number of idle connections to specific servers. Each matching {{PooledConnection}} then has it's underlying {{Connection}} swapped out for one to a different server. This requires extra logic in the {{borrowConnection(forServer)}} method to check that the associated server hasn't changed between to matching and activation, if it has it must be placed back in the queue and the scan must be repeated. Removing the secondary pool will removed the need to swap out connections. Any matching idle connection from the {{availableConnections}} queue will just be destroyed. New connections will be created to the less loaded server either on demand or via pre-fill. No complicated logic for swapping connections is required and no recheck for chaining servers is required.




> Simplify client connection pooling
> ----------------------------------
>
>                 Key: GEODE-6594
>                 URL: https://issues.apache.org/jira/browse/GEODE-6594
>             Project: Geode
>          Issue Type: Improvement
>          Components: client/server
>            Reporter: Jacob S. Barrett
>            Priority: Major
>
> Current client connection pooling is multi-layered. A {{Connection}} is wrapped in a pooled {PooledConnection}}. A {{PooledConnection}} is kept in primary pool in an {allConnections}} set and when unused it is also in the {{availableConnections}} deque. A {{PooledConnection}} can also be held in a thread local pool (secondary pool). If in a thread local pool then it is not in the primary pool's {{availableConnections}}. This arrangement between the secondary and primary pools results in complex relationships with expirations and rebalancing actions. Since GEODE-6515 thread local connections does not provide a performance boost over the primary pool anymore we should remove it entirely, thus simplifying these complexities.
> Idle connection timeouts must scan the {{allConnections}} set since the secondary may hold idle connections that won't be in the {{availableConnections}}. This requires scanning the entire set of connections with an exclusive lock, which can be really expensive for large connection pools. If there were no secondary pools we could scan the {{availableConnections}} deque in reverse order stopping at the first idle connection not exceeding the timeout limit. This scan would be lock free and would not touch every connection. The {{availableConnections}} deque is used as a LIFO queue so connections at the front are hot and connection at the end are cold.
> Connection lifetime expiration currently requires, again scanning the {{allConnections}} set with exclusive lock looking for connections that are over the lifetime limit. If the connection needs to be expired then the {{PooledConnection}} as its {{Connection}} swapped out for a new one to the same server. If we remove the secondary pool then expired connections could simply be removed from the {{availableConnections}} and {{allConnections}} and destroyed. A new connection would be created either on demand of from pre-fill. No complicated logic is required to support swapping out the underlying {{Connection}}.
> Currently load conditioning requires scanning the {{allConnections}} for N number of idle connections to specific servers. Each matching {{PooledConnection}} then has it's underlying {{Connection}} swapped out for one to a different server. This requires extra logic in the {{borrowConnection(forServer)}} method to check that the associated server hasn't changed between to matching and activation, if it has it must be placed back in the queue and the scan must be repeated. Removing the secondary pool will removed the need to swap out connections. Any matching idle connection from the {{availableConnections}} queue will just be destroyed. New connections will be created to the less loaded server either on demand or via pre-fill. No complicated logic for swapping connections is required and no recheck for chaining servers is required.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)