You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Joe Orton <jo...@redhat.com> on 2017/03/15 09:29:18 UTC

listener buckets regressions

There seems to be some weird regressions in this.  A mis-configuration 
like:

Listen 127.0.0.1:8025
Listen [::1]:8025
Listen 127.0.0.1:8025
Listen [::1]:8025
ListenCoresBucketsRatio 8

... no longer triggers a startup failure - is that expected?  I guess it 
should be documented if so.

Such a config then leaks fds across reloads:

$ ss -ntlp | grep :8025 | wc -l
8
$ ./bin/apachectl graceful
$ ss -ntlp | grep :8025 | wc -l
10
$ ./bin/apachectl graceful
$ ss -ntlp | grep :8025 | wc -l
12

Maybe somebody familiar with the code can understand why.

These issues are a regression in 2.4.x without ListenCoresBucketsRatio 
set as well, which the below seems to fix (proposed for backport):

http://svn.apache.org/viewvc?view=revision&revision=1777923

Regards, Joe

Re: listener buckets regressions

Posted by Yann Ylavic <yl...@gmail.com>.
On Tue, Mar 28, 2017 at 11:21 AM, Yann Ylavic <yl...@gmail.com> wrote:
>
> Any special configuration (or directive change before graceful
> restart) for the above test?
> Could you please share the same output w/o "| wc -l"?

Please ignore this, I should have read the first message in the
thread, all is there :p
I can reproduce with it...

>
>>
>> Regards,
>> Yann.

Re: listener buckets regressions

Posted by Yann Ylavic <yl...@gmail.com>.
On Tue, Mar 28, 2017 at 9:48 AM, Yann Ylavic <yl...@gmail.com> wrote:
> On Wed, Mar 22, 2017 at 2:38 PM, Yann Ylavic <yl...@gmail.com> wrote:
>> On Wed, Mar 15, 2017 at 10:29 AM, Joe Orton <jo...@redhat.com> wrote:
>>>
>>> Such a config then leaks fds across reloads:
>>>
>>> $ ss -ntlp | grep :8025 | wc -l
>>> 8
>>> $ ./bin/apachectl graceful
>>> $ ss -ntlp | grep :8025 | wc -l
>>> 10
>>> $ ./bin/apachectl graceful
>>> $ ss -ntlp | grep :8025 | wc -l
>>> 12
>>>
>>> Maybe somebody familiar with the code can understand why.
>>
>> Will try...
>
> Fixed in r1789061, and proposed for backport to 2.4.x.

Actually I've just reverted r1789061 because it can't fix anything.
I can't reproduce the leak in 2.4.x (even before r1777923), I thought
it was the duplicated listeners that were not cleaned up on restart
but in fact they are created on pconf so they ought to be...

Any special configuration (or directive change before graceful
restart) for the above test?
Could you please share the same output w/o "| wc -l"?

>
> Regards,
> Yann.

Re: listener buckets regressions

Posted by Yann Ylavic <yl...@gmail.com>.
On Wed, Mar 22, 2017 at 2:38 PM, Yann Ylavic <yl...@gmail.com> wrote:
> On Wed, Mar 15, 2017 at 10:29 AM, Joe Orton <jo...@redhat.com> wrote:
>>
>> Such a config then leaks fds across reloads:
>>
>> $ ss -ntlp | grep :8025 | wc -l
>> 8
>> $ ./bin/apachectl graceful
>> $ ss -ntlp | grep :8025 | wc -l
>> 10
>> $ ./bin/apachectl graceful
>> $ ss -ntlp | grep :8025 | wc -l
>> 12
>>
>> Maybe somebody familiar with the code can understand why.
>
> Will try...

Fixed in r1789061, and proposed for backport to 2.4.x.

Thanks for finding this Joe.

Regards,
Yann.

Re: listener buckets regressions

Posted by Yann Ylavic <yl...@gmail.com>.
On Wed, Mar 15, 2017 at 10:29 AM, Joe Orton <jo...@redhat.com> wrote:
> There seems to be some weird regressions in this.  A mis-configuration
> like:
>
> Listen 127.0.0.1:8025
> Listen [::1]:8025
> Listen 127.0.0.1:8025
> Listen [::1]:8025
> ListenCoresBucketsRatio 8
>
> ... no longer triggers a startup failure - is that expected?  I guess it
> should be documented if so.

Yes, SO_REUSEPORT allows multiple processes with the same EUID to bind
the same port.
I guess it's more a side effect than a feature from httpd's
perspective (albeit it possibly could have an application...), in any
case I agree that it should be documented (will have a look at it).

>
> Such a config then leaks fds across reloads:
>
> $ ss -ntlp | grep :8025 | wc -l
> 8
> $ ./bin/apachectl graceful
> $ ss -ntlp | grep :8025 | wc -l
> 10
> $ ./bin/apachectl graceful
> $ ss -ntlp | grep :8025 | wc -l
> 12
>
> Maybe somebody familiar with the code can understand why.

Will try...

>
> These issues are a regression in 2.4.x without ListenCoresBucketsRatio
> set as well, which the below seems to fix (proposed for backport):
>
> http://svn.apache.org/viewvc?view=revision&revision=1777923

Seems to be backported already, good thing.


Regards,
Yann.

Re: listener buckets regressions

Posted by Stefan Eissing <st...@greenbytes.de>.
> Am 28.03.2017 um 19:50 schrieb Ruediger Pluem <rp...@apache.org>:
> 
> 
> 
> On 03/28/2017 05:47 PM, Yann Ylavic wrote:
>> On Wed, Mar 15, 2017 at 10:29 AM, Joe Orton <jo...@redhat.com> wrote:
>>> There seems to be some weird regressions in this.  A mis-configuration
>>> like:
>>> 
>>> Listen 127.0.0.1:8025
>>> Listen [::1]:8025
>>> Listen 127.0.0.1:8025
>>> Listen [::1]:8025
>>> ListenCoresBucketsRatio 8
>>> 
>>> ... no longer triggers a startup failure - is that expected?
>> 
>> Is the failure really expected w/o listener buckets?
>> It happens on bind ("Address already in use: AH00072: make_sock"),
>> which is indeed not an issue with SO_REUSEPORT.
>> 
>> I want to fix the leak of course but I wonder if we should:
>> 1. ignore duplicated Listen directives for the ListenCoresBucketsRatio
>>> 0 case only,
>> 2. ignore duplicated Listen directives in any case,
>> 3. fail in any case.
> 
> I would vote for 3.

3 on overlap
2 on completely the same

> 
> Regards
> 
> Rüdiger

Stefan Eissing

<green/>bytes GmbH
Hafenstrasse 16
48155 Münster
www.greenbytes.de

Re: listener buckets regressions

Posted by Ruediger Pluem <rp...@apache.org>.

On 03/28/2017 05:47 PM, Yann Ylavic wrote:
> On Wed, Mar 15, 2017 at 10:29 AM, Joe Orton <jo...@redhat.com> wrote:
>> There seems to be some weird regressions in this.  A mis-configuration
>> like:
>>
>> Listen 127.0.0.1:8025
>> Listen [::1]:8025
>> Listen 127.0.0.1:8025
>> Listen [::1]:8025
>> ListenCoresBucketsRatio 8
>>
>> ... no longer triggers a startup failure - is that expected?
> 
> Is the failure really expected w/o listener buckets?
> It happens on bind ("Address already in use: AH00072: make_sock"),
> which is indeed not an issue with SO_REUSEPORT.
> 
> I want to fix the leak of course but I wonder if we should:
> 1. ignore duplicated Listen directives for the ListenCoresBucketsRatio
>> 0 case only,
> 2. ignore duplicated Listen directives in any case,
> 3. fail in any case.

I would vote for 3.

Regards

R�diger

Re: listener buckets regressions

Posted by Yann Ylavic <yl...@gmail.com>.
On Mon, Apr 10, 2017 at 2:20 PM, Joe Orton <jo...@redhat.com> wrote:
> On Tue, Mar 28, 2017 at 05:47:53PM +0200, Yann Ylavic wrote:
>> Is the failure really expected w/o listener buckets?
>> It happens on bind ("Address already in use: AH00072: make_sock"),
>> which is indeed not an issue with SO_REUSEPORT.
>>
>> I want to fix the leak of course but I wonder if we should:
>> 1. ignore duplicated Listen directives for the ListenCoresBucketsRatio
>> > 0 case only,
>> 2. ignore duplicated Listen directives in any case,
>> 3. fail in any case.
>
> I like your r1789220 because it makes a startup time failure a config
> time failure, which is good... but things like:
>
> Listen localhost:8025
> Listen 127.0.0.1:8025
>
> do not appear to get caught by that test.  I wonder if accurately
> duplicating the bind() conflict detection is going to feasible/sensible,
> and maybe we should still warn in the docs that SO_REUSEPORT is special
> here.

I've just committed something in r1792675 (and proposed for backport
with r1789220), does it sound appropriate?


Regards,
Yann.

Re: listener buckets regressions

Posted by Joe Orton <jo...@redhat.com>.
On Tue, Mar 28, 2017 at 05:47:53PM +0200, Yann Ylavic wrote:
> Is the failure really expected w/o listener buckets?
> It happens on bind ("Address already in use: AH00072: make_sock"),
> which is indeed not an issue with SO_REUSEPORT.
> 
> I want to fix the leak of course but I wonder if we should:
> 1. ignore duplicated Listen directives for the ListenCoresBucketsRatio
> > 0 case only,
> 2. ignore duplicated Listen directives in any case,
> 3. fail in any case.

I like your r1789220 because it makes a startup time failure a config 
time failure, which is good... but things like:

Listen localhost:8025
Listen 127.0.0.1:8025

do not appear to get caught by that test.  I wonder if accurately 
duplicating the bind() conflict detection is going to feasible/sensible, 
and maybe we should still warn in the docs that SO_REUSEPORT is special 
here.

Regards, Joe

Re: listener buckets regressions

Posted by Yann Ylavic <yl...@gmail.com>.
On Tue, Mar 28, 2017 at 5:47 PM, Yann Ylavic <yl...@gmail.com> wrote:
> On Wed, Mar 15, 2017 at 10:29 AM, Joe Orton <jo...@redhat.com> wrote:
>> There seems to be some weird regressions in this.  A mis-configuration
>> like:
>>
>> Listen 127.0.0.1:8025
>> Listen [::1]:8025
>> Listen 127.0.0.1:8025
>> Listen [::1]:8025
>> ListenCoresBucketsRatio 8
>>
>> ... no longer triggers a startup failure - is that expected?
>
> Is the failure really expected w/o listener buckets?
> It happens on bind ("Address already in use: AH00072: make_sock"),
> which is indeed not an issue with SO_REUSEPORT.
>
> I want to fix the leak of course but I wonder if we should:
> 1. ignore duplicated Listen directives for the ListenCoresBucketsRatio
>> 0 case only,
> 2. ignore duplicated Listen directives in any case,
> 3. fail in any case.

4. create an entry for each duplicate in the buckets, which would be a
kind of accept factor (uglily configurable)...

>
> Thoughts?

Re: listener buckets regressions

Posted by Yann Ylavic <yl...@gmail.com>.
On Wed, Mar 15, 2017 at 10:29 AM, Joe Orton <jo...@redhat.com> wrote:
> There seems to be some weird regressions in this.  A mis-configuration
> like:
>
> Listen 127.0.0.1:8025
> Listen [::1]:8025
> Listen 127.0.0.1:8025
> Listen [::1]:8025
> ListenCoresBucketsRatio 8
>
> ... no longer triggers a startup failure - is that expected?

Is the failure really expected w/o listener buckets?
It happens on bind ("Address already in use: AH00072: make_sock"),
which is indeed not an issue with SO_REUSEPORT.

I want to fix the leak of course but I wonder if we should:
1. ignore duplicated Listen directives for the ListenCoresBucketsRatio
> 0 case only,
2. ignore duplicated Listen directives in any case,
3. fail in any case.

Thoughts?