You are viewing a plain text version of this content. The canonical link for it is here.
Posted to bugs@httpd.apache.org by bu...@apache.org on 2021/11/19 14:39:28 UTC

[Bug 65689] New: proxy connections to target left in CLOSE_WAIT state

https://bz.apache.org/bugzilla/show_bug.cgi?id=65689

            Bug ID: 65689
           Summary: proxy connections to target left in CLOSE_WAIT state
           Product: Apache httpd-2
           Version: 2.4.51
          Hardware: PC
            Status: NEW
          Severity: normal
          Priority: P2
         Component: mod_proxy
          Assignee: bugs@httpd.apache.org
          Reporter: lothar.webmin@gmail.com
  Target Milestone: ---

Hi all,
since we updated apache from 2.4.46 to 2.4.51 we have a lot of open connections
to destination hanging in "CLOSE_WAIT" state.

The connection is closed by target after a 10 secs. with a TCP RST.
Which causes alerts in our monitoring system.
I tried to disable keepalive with
      SetEnv proxy-nokeepalive 1
We are using this config:
    <IfModule mod_proxy.c>
        ProxyRequests On
        AllowCONNECT 443
        SetEnv proxy-nokeepalive 1
        <Proxy "*">
                Require all granted
        </Proxy>
    </IfModule>
The Client is initiation a connection via method CONNECT:
CONNECT www.example.com:443 HTTP/1.1
Somehow disable keepalive is not working.
Also the behavior has changed, with apache 2.4.46 the connection was closed
smoothly:
apache -> target ACK
target -> apache FIN,ACK
apache -> target ACK
apache -> target FIN,ACK

With apache 2.4.51 the last FIN,ACK is not sent anymore but we receive 
target -> apache RST, ACK after 10 sec.

Could this be fixed, or is there any workaround ( like disable keepalive)

Thanks a lot!
Kind Regards, Lothar

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 65689] proxy connections to target left in CLOSE_WAIT state

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65689

Piotr Staszeski <te...@axit.pl> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 OS|                            |All

--- Comment #1 from Piotr Staszeski <te...@axit.pl> ---
pstack 27036
#0  0x00007f4e0ac060c3 in __epoll_wait_nocancel () from /lib64/libc.so.6
#1  0x00007f4e0b758a59 in ?? ()
#2  0x00007ffcdd3b13fc in ?? ()
#3  0x00007ffcdd3b14a4 in ?? ()
#4  0x00007ffcdd3b14a8 in ?? ()
#5  0x0000000001bb9b98 in ?? ()
#6  0x0000000001bb9cd8 in ?? ()
#7  0x0000000001bb9b80 in ?? ()
#8  0xffffffffffffffff in ?? ()
#9  0x00007ffcdd3b14a4 in ?? ()
#10 0x0000000001bb9ca8 in ?? ()
#11 0x0000000001bb9cd8 in ?? ()
#12 0x00007ffcdd3b14a8 in ?? ()
#13 0x00007f4e092b1721 in ?? ()
#14 0x0000000001b99a98 in ?? ()
#15 0x0000000001bace78 in ?? ()
#16 0x0000000001b994c0 in ?? ()
#17 0x00007ffcdd3b1540 in ?? ()
#18 0x0000000001bb9cd8 in ?? ()
#19 0x00007f4e0b753283 in ?? ()
#20 0x0000000001bb7e20 in ?? ()
#21 0x0000000001b994c0 in ?? ()
#22 0x0000000001bb9ca0 in ?? ()
#23 0x0000000001bb9b80 in ?? ()
#24 0x0000000001bb9b38 in ?? ()
#25 0xffffffffffffffff in ?? ()
#26 0x00007ffcdd3b14a4 in ?? ()
#27 0x00007ffcdd3b14a8 in ?? ()
#28 0x0000000001b99040 in ?? ()
#29 0x0000000001bb9c40 in ?? ()
#30 0x0000000001b99040 in ?? ()
#31 0x0000000001bb9af8 in ?? ()
#32 0x0000000001b994c0 in ?? ()
#33 0x0000000001b99040 in ?? ()
#34 0x00007ffcdd3b15b0 in ?? ()
#35 0x0000000001bb7e20 in ?? ()
#36 0x0000000000440910 in ?? () at util_filter.c:593
#37 0x00007f4e09094f35 in ?? ()
#38 0x0000000001b7bc68 in ?? ()
#39 0x00007f4e0b7477d7 in ?? ()
#40 0x0000000000000000 in ?? ()

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 65689] proxy connections to target left in CLOSE_WAIT state

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65689

Yann Ylavic <yl...@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|---                         |DUPLICATE

--- Comment #8 from Yann Ylavic <yl...@gmail.com> ---


*** This bug has been marked as a duplicate of bug 65662 ***

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 65689] proxy connections to target left in CLOSE_WAIT state

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65689

--- Comment #5 from Lothar <lo...@gmail.com> ---
Created attachment 38100
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=38100&action=edit
Trace8 Log filtered by one PID, serving the Request IP-Addresses/hostnames
anonymised (

Trace8 Log filtered by one PID, which was serving the Request
IP-Addresses/hostnames anonymised

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 65689] proxy connections to target left in CLOSE_WAIT state

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65689

--- Comment #9 from Graham Leggett <mi...@apache.org> ---
Backported to v2.4.52.

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 65689] proxy connections to target left in CLOSE_WAIT state

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65689

Lothar <lo...@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|DUPLICATE                   |---
             Status|RESOLVED                    |REOPENED

--- Comment #7 from Lothar <lo...@gmail.com> ---
(In reply to Yann Ylavic from comment #3)
> (In reply to Lothar from comment #0)
> > Also the behavior has changed, with apache 2.4.46 the connection was closed
> > smoothly:
> > apache -> target ACK
> > target -> apache FIN,ACK
> > apache -> target ACK
> > apache -> target FIN,ACK
> > 
> > With apache 2.4.51 the last FIN,ACK is not sent anymore but we receive 
> > target -> apache RST, ACK after 10 sec.
> 
> What happens to the connection on the client side when the target/backend
> server initiates connection close like the above?

The Client receives a 200 OK and everything on client side is OK

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 65689] proxy connections to target left in CLOSE_WAIT state

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65689

Piotr Staszeski <te...@axit.pl> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tech@axit.pl

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 65689] proxy connections to target left in CLOSE_WAIT state

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65689

Yann Ylavic <yl...@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|---                         |DUPLICATE
             Status|NEW                         |RESOLVED

--- Comment #4 from Yann Ylavic <yl...@gmail.com> ---
Let's track this at a single place (bug 65662) since the reports look very
similar.

*** This bug has been marked as a duplicate of bug 65662 ***

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 65689] proxy connections to target left in CLOSE_WAIT state

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65689

--- Comment #6 from Lothar <lo...@gmail.com> ---
(In reply to Piotr Staszeski from comment #2)
> (In reply to Piotr Staszeski from comment #1)
> > pstack 27036
> 
> missclick - @Lothar do You also see related FIN_WAIT2 connections? Maybe You
> observe same issues as I in Bug 65662

The Connection switches immediately from ESTABLISHED into CLOSE_WAIT state

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 65689] proxy connections to target left in CLOSE_WAIT state

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65689

--- Comment #2 from Piotr Staszeski <te...@axit.pl> ---
(In reply to Piotr Staszeski from comment #1)
> pstack 27036

missclick - @Lothar do You also see related FIN_WAIT2 connections? Maybe You
observe same issues as I in Bug 65662

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 65689] proxy connections to target left in CLOSE_WAIT state

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=65689

--- Comment #3 from Yann Ylavic <yl...@gmail.com> ---
(In reply to Lothar from comment #0)
> Also the behavior has changed, with apache 2.4.46 the connection was closed
> smoothly:
> apache -> target ACK
> target -> apache FIN,ACK
> apache -> target ACK
> apache -> target FIN,ACK
> 
> With apache 2.4.51 the last FIN,ACK is not sent anymore but we receive 
> target -> apache RST, ACK after 10 sec.

What happens to the connection on the client side when the target/backend
server initiates connection close like the above?

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org