You are viewing a plain text version of this content. The canonical link for it is here.
Posted to bugs@httpd.apache.org by bu...@apache.org on 2009/04/24 23:02:49 UTC

DO NOT REPLY [Bug 47095] New: Problems with pipelined HEAD requests...

https://issues.apache.org/bugzilla/show_bug.cgi?id=47095

           Summary: Problems with pipelined HEAD requests...
           Product: Apache httpd-2
           Version: 2.3-HEAD
          Platform: PC
        OS/Version: Mac OS X 10.4
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Core
        AssignedTo: bugs@httpd.apache.org
        ReportedBy: douglas@openplans.org


I'm gonna apologize in advance for this bug report, because while I'll give you
detailed reproduction steps, it's not a simple test case.  It will, however,
ensure you reproduce the problem.

I am running Apache 2.2 (every instance of 2.2 I try exhibits the same problem)
with mod_php running wordpress.  Inside of wordpress, I have the Xinha4WP
plugin installed (from http://oss.openplans.org/xinhatools).  In the admin
screen, Safari fails to load Xinha one out of every three times.  After much
testing, packet sniffing, etc., it seems that Xinha's new 'ping' causes Apache
to unexpectedly close connections.

Whenever the web browser is making large numbers of HEAD requests over a
pipelined connection, Apache will occasionally shut down a socket without
responding to all requests in the pipeline.  I've sniffed the traffic with
Wireshark, and the requests are all sent, most of them are responded to, and
the last three or four are completely ignored.  These ignored requests never
show up in neither access_log nor error_log, they are simply ignored.

I've just found another Apache instance that does not exhibit the problem, it's
running itk, I'm going to test to see if that's the only variable difference
that causes the problem.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #12 from Douglas Mayle <do...@openplans.org>  2009-04-27 13:03:57 PST ---
Also of note is that the tests I'm doing today are on OS X (hence the ::1 of
IPv6) and running connected to localhost.  Last week I was also connecting to a
remote linux server (on the same local network) and experiencing the same
problems.  The local network was IPv4 only.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095


Ruediger Pluem <rp...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEEDINFO                    |RESOLVED
         Resolution|                            |INVALID




--- Comment #17 from Ruediger Pluem <rp...@apache.org>  2009-04-27 14:42:11 PST ---
Same thing as with 5s keepalive: The client does not sent a request for 50
seconds and *after* the webserver closed down the connection the client starts
sending again although it confirmed the FIN received by the server (frame 442).
The client  should not sent any more requests over this connection and if it
does it must be prepared to handle this situation gracefully. Have you tried
with a different browser? IMHO this is a bug in Safari.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095


Douglas Mayle <do...@openplans.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |REOPENED
         Resolution|INVALID                     |




--- Comment #18 from Douglas Mayle <do...@openplans.org>  2009-05-05 08:32:09 PST ---
Sorry I didn't get back until sooner.  This is the exact same behavior that
Opera exhibits, and if you examine the TCP activity, you'll see that Apache
never responds to the last TCP ack, which is why Safari (and Opera) never
continue with the requests until after teardown.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #2 from Ruediger Pluem <rp...@apache.org>  2009-04-24 14:40:56 PST ---
Whats your setting for MaxKeepAliveRequests?
Can you share a network sniff that shows this behaviour?

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #14 from Douglas Mayle <do...@openplans.org>  2009-04-27 14:18:25 PST ---
Created an attachment (id=23555)
 --> (https://issues.apache.org/bugzilla/attachment.cgi?id=23555)
Retest with KeepAliveTimeout at 50

(ipv6.addr eq ::1 and ipv6.addr eq ::1) and (tcp.port eq 54100 and tcp.port eq
80)

The above expression will show you the affected tcp stream.  I notice that the
Apache Keep-Alive header says 5, even though I've specified 50 in httpd.conf. 
Is there any way to be sure it takes effect?

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #9 from Eric Covener <co...@gmail.com>  2009-04-27 12:51:05 PST ---
(In reply to comment #8)
> Setting EnableSendfile off has no effect, the bug is still present.  I should
> clarify that if I configure the client to send GET requests instead of HEAD
> requests, there is no problem.

Sorry, I missed that the CharacterMap.js was a HEAD request.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #11 from Douglas Mayle <do...@openplans.org>  2009-04-27 13:01:33 PST ---
The timing must be coincidental.  I just set the KeepAliveTimeout to 50, and
the behavior is exactly the same.  I don't know if you saw above, but the exact
same configuration with itk doesn't have problems.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #20 from Douglas Mayle <do...@openplans.org>  2009-05-05 11:48:26 PST ---
(In reply to comment #19)
> 
> If you're talking about the ACK in frame 344, what kind of response is expected
> from Apache? 
> 
> he ACK in frame 344 is the ACK from the client of the last byte of the response
> to the only outstanding HEAD request.
> 
> After Apache sends the full response, and the client has ackowledged it, no
> subsequent request is issued until A) after KeepAliveTimeout elapses and B)
> Apache has sent a FIN to terminate the connection.

I'm sorry, I gave you one filter, and was using another to reexamine the data
(as there are two separate streams that go wrong).  You're absolutely right
about the ACK in 344.  I'm not entirely sure why Opera has the same problem,
and itk solves the problem...  Still... I'm kind of baffled...

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #8 from Douglas Mayle <do...@openplans.org>  2009-04-27 12:39:17 PST ---
Setting EnableSendfile off has no effect, the bug is still present.  I should
clarify that if I configure the client to send GET requests instead of HEAD
requests, there is no problem.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #5 from Douglas Mayle <do...@openplans.org>  2009-04-27 08:26:44 PST ---
More fun info, replaying this request specifically, with the magic of telnet,
and it triggers the shutdown every time:

HEAD
/~douglas/wordpress/wp-content/plugins/xinha4wp//xinha_core/plugins/CharacterMap/CharacterMap.js
HTTP/1.1
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us)
AppleWebKit/525.27.1 (KHTML, like Gecko) Version/3.2.1 Safari/525.27.1
Referer:
http://localhost/~douglas/wordpress/wp-admin/post.php?action=edit&post=21
Cache-Control: max-age=0
Accept: */*
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Cookie:
wordpress_c7d9d1984711466c714230dfd3d92045=admin%7C1241631202%7C02f9f92f26df75dfe21a468a80afaa37;
wordpress_logged_in_c7d9d1984711466c714230dfd3d92045=admin%7C1241631202%7Cc0d1a0ccf9c1d871b4386d7e170b9780;
wp-settings-1=editor%3Dtinymce%26m0%3Do%26m1%3Do%26m2%3Do%26m3%3Dc%26m4%3Dc%26m5%3Do%26m6%3Do%26m7%3Do%26m8%3Do;
wp-settings-time-1=1240421602; PHPSESSID=48671964231edf4f60a2c24bcb931b21
Content-Length: 0
Connection: keep-alive
Host: localhost

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095


Eric Covener <co...@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |NEEDINFO




--- Comment #7 from Eric Covener <co...@gmail.com>  2009-04-27 12:27:25 PST ---
Looks like Apache sends an empty response to CharacterMap.js in frame 355. 

The client is waiting for the advertised body, and Apache is waiting on
KeepAliveTimeout (5s).  Apache closes the connection while the client still
tries to slam some requests in.

Tried EnableSendfile off?

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #3 from Douglas Mayle <do...@openplans.org>  2009-04-27 08:14:58 PST ---
I'm attaching a libpcap dump file captured with Wireshark on OS X.  In this
case, I'm just testing against the default installed Apache.  If you find the
TCP stream that includes the GET request:

GET /~douglas/wordpress/wp-admin/wp-admin.css?ver=20081210 HTTP/1.1

You'll see the problem.  As soon as there are HEAD requests in the pipeline,
Apache handles the first, and shuts down on all subsequent requests.

Also, it appears that MaxKeepAliveRequests is set to 100 on this instance.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #16 from Douglas Mayle <do...@openplans.org>  2009-04-27 14:24:59 PST ---
I'll try to provide an itk sniff on Thursday, when I have access to the testing
rig again...

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095


inisayaaulia@yahoo.co.id changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |inisayaaulia@yahoo.co.id




-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095


Douglas Mayle <do...@openplans.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #23555|0                           |1
        is obsolete|                            |




--- Comment #15 from Douglas Mayle <do...@openplans.org>  2009-04-27 14:23:45 PST ---
Created an attachment (id=23556)
 --> (https://issues.apache.org/bugzilla/attachment.cgi?id=23556)
Supersedes previous test

In this test, we see keepalive at 50, but with the same error condition

Filter:

(ipv6.addr eq ::1 and ipv6.addr eq ::1) and (tcp.port eq 54184 and tcp.port eq
80)

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #13 from Ruediger Pluem <rp...@apache.org>  2009-04-27 13:55:28 PST ---
I agree with Erics findings. Please provide a network dump that shows the same
behaviour with 50s keepalive timeout. The example with 5s keepalive timeout
seems to indicate a race condition that the browser / the javascript sending
the request must be prepared to handle. itk may not use keepalive connections.
Can you please provide an itk sniff as well?

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #6 from Douglas Mayle <do...@openplans.org>  2009-04-27 08:28:38 PST ---
I'm sorry, please ignore the last comment about the request, I was pasting in
the wrong content to telnet...

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095


Eric Covener <co...@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|REOPENED                    |RESOLVED
         Resolution|                            |INVALID




--- Comment #19 from Eric Covener <co...@gmail.com>  2009-05-05 10:11:42 PST ---
(In reply to comment #18)
> Sorry I didn't get back until sooner.  This is the exact same behavior that
> Opera exhibits, and if you examine the TCP activity, you'll see that Apache
> never responds to the last TCP ack, which is why Safari (and Opera) never
> continue with the requests until after teardown.

If you're talking about the ACK in frame 344, what kind of response is expected
from Apache? 

he ACK in frame 344 is the ACK from the client of the last byte of the response
to the only outstanding HEAD request.

After Apache sends the full response, and the client has ackowledged it, no
subsequent request is issued until A) after KeepAliveTimeout elapses and B)
Apache has sent a FIN to terminate the connection.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #10 from Eric Covener <co...@gmail.com>  2009-04-27 12:54:16 PST ---
(ipv6.addr eq ::1 and ipv6.addr eq ::1) and (tcp.port eq 80 and tcp.port eq
50088)

It still seems to me that the KeepAliveTimeout is 5s and the client waits just
over 5 seconds between two requests.  This is why the connection is dropped.

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #1 from Douglas Mayle <do...@openplans.org>  2009-04-24 14:09:57 PST ---
Just an update, an identical Apache where the only difference is itk as opposed
to prefork, and it does NOT exhibit the problem...

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org

DO NOT REPLY [Bug 47095] Problems with pipelined HEAD requests...

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=47095





--- Comment #4 from Douglas Mayle <do...@openplans.org>  2009-04-27 08:16:16 PST ---
Created an attachment (id=23554)
 --> (https://issues.apache.org/bugzilla/attachment.cgi?id=23554)
Wireshark dump showing problem

Look for the TCP stream starting with this request:

GET /~douglas/wordpress/wp-admin/wp-admin.css?ver=20081210 HTTP/1.1

-- 
Configure bugmail: https://issues.apache.org/bugzilla/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org