[LDAP API] Connection pool issue

[Emmanuel Lécharny <el...@gmail.com>]

Hi guys,

in the past 5 days, I fought with some serious issues with the LDAP
Connection pool. What I saw is that at some point, the pool was stopping
to work. The pb was only visible when you had many threads, as it was
associated with connections being released and closed because the number
of idle connections was above the maximum allowed by the pool (you can
configure the pool so that it creates connections on demand, but delete
them when the number o finactive connection is above a given number.
This is a way to mitigate the pool growth).

The pb was that we were not checking the result of the
connection.close() operation : it returns a CloseFuture that has to be
used to wait for the terminaison, otheriwse the connection might be in a
transient state for a bit of time, and under heavy load, that will
really be a pb.

I will commit the change, create a JIRA and most certainly cut a new
release for this.

I have created three tests that are in @Ignore mode, because they are
quite long to run (around 2 minutes each). Note that in order to have
one of those tests to pass, you will have to tune your system -
specifically the TIME_WAIT delay - to let the test create new sockets,
as it creates hundred of thousands of them, when you can only have
64k-1024 sockets created with <Address, X, adress, port>n where X is an
ephemeral port (on Mac, the number is even lower : 16K, per
http://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml).
Settung a delay of 500ms or even 1s should work, but please, don't do
that on a normal server or client.

Re: [LDAP API] Connection pool issue

[Colm O hEigeartaigh <co...@apache.org>]

Yep sounds good.

Colm.

On Wed, Jan 7, 2015 at 10:45 AM, Emmanuel Lécharny <el...@gmail.com>
wrote:

> Le 07/01/15 11:31, Colm O hEigeartaigh a écrit :
> > Hi Emmanuel,
> >
> > Do you have any plans for a new Apache DS release?
>
> I will first cut an API release, followed by a ApacheDS release as soon
> as the API release is done.
>
> I'd like to fix a few low hanging fruits on ApacheDS.
>
> We can expect the release to be done early next week.
>
> Is that ok ?
>
>


-- 
Colm O hEigeartaigh

Talend Community Coder
http://coders.talend.com

Re: [LDAP API] Connection pool issue

[Emmanuel Lécharny <el...@gmail.com>]

Le 07/01/15 11:31, Colm O hEigeartaigh a écrit :
> Hi Emmanuel,
>
> Do you have any plans for a new Apache DS release?

I will first cut an API release, followed by a ApacheDS release as soon
as the API release is done.

I'd like to fix a few low hanging fruits on ApacheDS.

We can expect the release to be done early next week.

Is that ok ?

Re: [LDAP API] Connection pool issue

[Colm O hEigeartaigh <co...@apache.org>]

Hi Emmanuel,

Do you have any plans for a new Apache DS release?

Colm.

On Wed, Jan 7, 2015 at 10:07 AM, Emmanuel Lécharny <el...@gmail.com>
wrote:

> Hi guys,
>
> in the past 5 days, I fought with some serious issues with the LDAP
> Connection pool. What I saw is that at some point, the pool was stopping
> to work. The pb was only visible when you had many threads, as it was
> associated with connections being released and closed because the number
> of idle connections was above the maximum allowed by the pool (you can
> configure the pool so that it creates connections on demand, but delete
> them when the number o finactive connection is above a given number.
> This is a way to mitigate the pool growth).
>
> The pb was that we were not checking the result of the
> connection.close() operation : it returns a CloseFuture that has to be
> used to wait for the terminaison, otheriwse the connection might be in a
> transient state for a bit of time, and under heavy load, that will
> really be a pb.
>
> I will commit the change, create a JIRA and most certainly cut a new
> release for this.
>
> I have created three tests that are in @Ignore mode, because they are
> quite long to run (around 2 minutes each). Note that in order to have
> one of those tests to pass, you will have to tune your system -
> specifically the TIME_WAIT delay - to let the test create new sockets,
> as it creates hundred of thousands of them, when you can only have
> 64k-1024 sockets created with <Address, X, adress, port>n where X is an
> ephemeral port (on Mac, the number is even lower : 16K, per
>
> http://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.xhtml
> ).
> Settung a delay of 500ms or even 1s should work, but please, don't do
> that on a normal server or client.
>
>


-- 
Colm O hEigeartaigh

Talend Community Coder
http://coders.talend.com