You are viewing a plain text version of this content. The canonical link for it is here.
Posted to bugs@httpd.apache.org by bu...@apache.org on 2014/02/18 13:11:02 UTC
[Bug 56151] New: mod_proxy sometimes returns 503 errors
https://issues.apache.org/bugzilla/show_bug.cgi?id=56151
Bug ID: 56151
Summary: mod_proxy sometimes returns 503 errors
Product: Apache httpd-2
Version: 2.2.15
Hardware: PC
OS: Windows NT
Status: NEW
Severity: normal
Priority: P2
Component: mod_proxy
Assignee: bugs@httpd.apache.org
Reporter: renaud.vanderhagen@gmail.com
Hi there,
We are running a high volume web application that uses Apache 2.2.15 mod_proxy
to reverse proxy content from apache to JBoss 6.
We found 503 errors which happen sporadically throughout the day on random
requests (perhaps 1/2% of daily requests).
Proxy config:
<IfModule mod_ssl.c>
<IfModule mod_proxy_http.c>
ProxyTimeout 600
SetEnv force-proxy-request-1.0 1
SetEnv proxy-nokeepalive 1
ProxyPass /unproxied/ !
ProxyPass /error/ !
ProxyPass / http://127.0.0.1:8080/ retry=0 ttl=600
ProxyPassReverse / http://127.0.0.1:8080/
</IfModule>
</IfModule>
JBoss config:
<Connector
address = "${jboss.bind.address}"
port = "8080"
maxThreads = "200"
acceptCount = "100"
scheme = "https"
secure = "true"
proxyName = "xxx"
proxyPort = "443"
xpoweredBy = "true"
connectionTimeout = "1200000"
enableLookups = "false"
maxKeepAliveRequests = "-1"
maxHttpHeaderSize = "50000"
compression = "force"
compressionMinSize = "512"
compressableMimeType =
"text/html,text/xml,text/css,text/javascript,application/x-javascript,application/javascript,image/svg+xml,application/json,text/json"
/>
What we can see in the error log:
[Tue Feb 18 10:11:05 2014] [error] (110)Connection timed out: proxy: HTTP:
attempt to connect to 127.0.0.1:9355 (127.0.0.1) failed
[Tue Feb 18 10:11:05 2014] [error] ap_proxy_connect_backend disabling worker
for (127.0.0.1)
What se can see in the access log:
[18/Feb/2014:10:11:07 +0100] "HEAD /xxx HTTP/1.1" 503 - "https://xxx"
"Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
Nothing appears in the JBoss log.
In the kernel log, we see that every error coincide with an invalid tcp packet:
kernel: invalid:IN=lo OUT= MAC=00:00:00:00:00:00:00:00:00:00:00:00:08:00
SRC=127.0.0.1 DST=127.0.0.1 LEN=60 TOS=0x00 PREC=0x00 TTL=64 ID=33082 DF
PROTO=TCP SPT=48340 DPT=8080 WINDOW=32792 RES=0x00 SYN URGP=0
After some investigations, this SYN packet is not acknowledged by JBoss in
order to perform the TCP 3-Way Handshake.
I don't know if this is related.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 56151] mod_proxy sometimes returns 503 errors
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56151
bob.sauvage@gmx.fr changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEEDINFO |NEW
Hardware|PC |Other
OS|Windows NT |Linux
--- Comment #4 from bob.sauvage@gmx.fr ---
The OS is RedHat 6.5
Some additional info:
Invalid packets were rejected by iptables. So the error in the error log was :
"(111)Connection refused: proxy".
The tcpdump indicated that JBoss didn't reply to this TCP SYN. Quite normal, it
was not received by JBoss due to iptables.
I've added a rule in order to authorize invalid packet coming from Apache to
JBoss. The error is now "(110)Connection timed out: proxy".
In the tcpdump, I can see the SYN packet sent by Apache, the SYN ACK sent by
JBoss but the handshake is stopped by Apache which sends a RST packet.
Why Apache sends an invalid packet ? Why Apache sents a RST packet ?
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 56151] mod_proxy sometimes returns 503 errors
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56151
Eric Covener <co...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |NEEDINFO
--- Comment #2 from Eric Covener <co...@gmail.com> ---
Would seem to be an OS, HW, or network problem if the connection fails in this
way. Can you fill the OS field appropriately and try on a different platform?
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 56151] mod_proxy sometimes returns 503 errors
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56151
--- Comment #3 from Jeff Trawick <tr...@apache.org> ---
or the back-end is just temporarily overloaded... maybe increasing the listen
queue depth on the back-end would be enough to get over the occasional issue...
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 56151] mod_proxy sometimes returns 503 errors
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56151
--- Comment #1 from bob.sauvage@gmx.fr ---
The error in the error log is not "(110)Connection timed out: proxy:" but
"(111)Connection refused: proxy"
Sorry.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 56151] mod_proxy sometimes returns 503 errors
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56151
bob.sauvage@gmx.fr changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |INVALID
--- Comment #5 from bob.sauvage@gmx.fr ---
This issue was related to iptables. Some problems with rules on SYN packets.
I decided to add a rule in order to ACCEPT all packets from 127.0.0.1 to or
from port 8080.
Since this update, I don't see this kind of errors anymore.
Thanks.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org