You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Lazy <la...@gmail.com> on 2012/11/15 23:43:18 UTC
[users@httpd] mod_proxy_fcgi + php-fpm + connection reset by peer
Hi,
we are using httpd 2.4.3 with php 5.4 running fpm mode.
requests are passed to fpm using ProxyPassMatch. Everything works fine except,
Sometimes database data gets corrupted. All of the data was visible in
the form, but it gets truncated after ajax request commiting changes
gets aborted (shows as aborted in firebug).
in the access log:
80.5.x.x - - [15/Nov/2012:14:03:43 +0100] "POST
/xxx,action,pollb,id,958.html HTTP/1.1" 503 299
"https://xxx,action,pollb,id,958.html" "Mozilla/5.0 (Windows NT 6.1;
WOW64; rv:14.0) Gecko/20100101 Firefox/14.0.1"
in the error log:
[Thu Nov 15 14:04:03.140026 2012] [proxy_fcgi:error] [pid 15633:tid
140459279939328] (104)Connection reset by peer: [client
80.5.x.x:34541] AH01075: Error dispatching request to :, referer:
https://xxx,action,pollb,id,958.html
There is haproxy and varnish before httpd, it is also doing ssl.
80.5.x.x is remote clients ip (we are using mod_remoteip).
What does the error code exactly mean ? Our client (haproxy) reset ed
the connection ? Or was it php ?
Is it possible that somehow php processed the begining of the aborted
POST truncating the data in the database ?
OpenSSL support in haproxy is new, and aborted connection directs
suspicion on it (in case of httpd or varnish error there should be
some kind of 5xx error not an aborted connection), but still I don't
understand how the data got lost.
Thanks for Your time,
Regards,
Michal Grzedzicki
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] mod_proxy_fcgi + php-fpm + connection reset by peer
Posted by Lazy <la...@gmail.com>.
2012/11/16 Jeff Trawick <tr...@gmail.com>:
> On Fri, Nov 16, 2012 at 9:19 AM, Jeff Trawick <tr...@gmail.com> wrote:
>> On Thu, Nov 15, 2012 at 5:43 PM, Lazy <la...@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> we are using httpd 2.4.3 with php 5.4 running fpm mode.
>>>
>>> requests are passed to fpm using ProxyPassMatch. Everything works fine
>>> except,
>>>
>>> Sometimes database data gets corrupted. All of the data was visible in
>>> the form, but it gets truncated after ajax request commiting changes
>>> gets aborted (shows as aborted in firebug).
>>>
>>> in the access log:
>>> 80.5.x.x - - [15/Nov/2012:14:03:43 +0100] "POST
>>> /xxx,action,pollb,id,958.html HTTP/1.1" 503 299
>>> "https://xxx,action,pollb,id,958.html" "Mozilla/5.0 (Windows NT 6.1;
>>> WOW64; rv:14.0) Gecko/20100101 Firefox/14.0.1"
>>>
>>> in the error log:
>>> [Thu Nov 15 14:04:03.140026 2012] [proxy_fcgi:error] [pid 15633:tid
>>> 140459279939328] (104)Connection reset by peer: [client
>>> 80.5.x.x:34541] AH01075: Error dispatching request to :, referer:
>>> https://xxx,action,pollb,id,958.html
>>>
>>> There is haproxy and varnish before httpd, it is also doing ssl.
>>> 80.5.x.x is remote clients ip (we are using mod_remoteip).
>>>
>>> What does the error code exactly mean ? Our client (haproxy) reset ed
>>> the connection ? Or was it php ?
>>
>>
>> PHP dropped the connection.
>>
>> By chance is PHP_FCGI_MAX_REQUESTS set somewhere for the PHP process? In
>> the old days it defaulted to 500 (exit after 500 requests), which led to
>> exactly this scenario, but with php-fpm it defaults to 0 (don't exit).
>
>
> whoops, the analogous php-fpm setting is pm.max_requests
this is set to 0, and It happens everytime when connection is aborted
this is the communication betwean apache and php-fpm, when client
aborted the connection (haporxy or telnet)
00:44:35.298623 IP 127.0.0.1.55765 > 127.0.0.1.9000: Flags [S], seq
1371227382, win 32792, options [mss 16396,sackOK,TS val 74917062 ecr
0,nop,wscale 7], length 0
00:44:35.298632 IP 127.0.0.1.9000 > 127.0.0.1.55765: Flags [S.], seq
2312853950, ack 1371227383, win 32768, options [mss 16396,sackOK,TS
val 74917062 ecr 74917062,nop,wscale 7], length 0
00:44:35.298637 IP 127.0.0.1.55765 > 127.0.0.1.9000: Flags [.], ack 1,
win 257, options [nop,nop,TS val 74917062 ecr 74917062], length 0
00:44:35.298655 IP 127.0.0.1.55765 > 127.0.0.1.9000: Flags [P.], seq
1:17, ack 1, win 257, options [nop,nop,TS val 74917062 ecr 74917062],
length 16
00:44:35.298661 IP 127.0.0.1.9000 > 127.0.0.1.55765: Flags [.], ack
17, win 256, options [nop,nop,TS val 74917062 ecr 74917062], length 0
00:44:35.298696 IP 127.0.0.1.55765 > 127.0.0.1.9000: Flags [P.], seq
17:1706, ack 1, win 257, options [nop,nop,TS val 74917062 ecr
74917062], length 1689
00:44:35.298704 IP 127.0.0.1.9000 > 127.0.0.1.55765: Flags [.], ack
1706, win 256, options [nop,nop,TS val 74917062 ecr 74917062], length
0
00:44:35.298711 IP 127.0.0.1.55765 > 127.0.0.1.9000: Flags [P.], seq
1706:1714, ack 1, win 257, options [nop,nop,TS val 74917062 ecr
74917062], length 8
00:44:35.298716 IP 127.0.0.1.9000 > 127.0.0.1.55765: Flags [.], ack
1714, win 256, options [nop,nop,TS val 74917062 ecr 74917062], length
0
00:44:35.298737 IP 127.0.0.1.55765 > 127.0.0.1.9000: Flags [P.], seq
1714:9028, ack 1, win 257, options [nop,nop,TS val 74917062 ecr
74917062], length 7314
00:44:35.298745 IP 127.0.0.1.9000 > 127.0.0.1.55765: Flags [.], ack
9028, win 256, options [nop,nop,TS val 74917062 ecr 74917062], length
0
00:44:55.428420 IP 127.0.0.1.55765 > 127.0.0.1.9000: Flags [F.], seq
9028, ack 1, win 257, options [nop,nop,TS val 74922094 ecr 74917062],
length 0
00:44:55.466463 IP 127.0.0.1.9000 > 127.0.0.1.55765: Flags [.], ack
9029, win 256, options [nop,nop,TS val 74922104 ecr 74922094], length
0
00:44:55.623301 IP 127.0.0.1.9000 > 127.0.0.1.55765: Flags [P.], seq
1:233, ack 9029, win 256, options [nop,nop,TS val 74922143 ecr
74922094], length 232
00:44:55.623322 IP 127.0.0.1.55765 > 127.0.0.1.9000: Flags [R], seq
1371236411, win 0, length 0
when I send
POST /p.php HTTP/1.1
Host: test
User-Agent: HTTPTool/1.0
Content-Type: application/x-www-form-urlencoded
Content-Length: 500
home=C&favoriteaflavor=fli
and close the connection
on mod_fcgid php 5.2 the php script is not executed, ond
mod_proxy_fcgi with php 5.4, script is executed with empty post data
Regards,
Michal Grzedzicki
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] mod_proxy_fcgi + php-fpm + connection reset by peer
Posted by Jeff Trawick <tr...@gmail.com>.
On Fri, Nov 16, 2012 at 9:19 AM, Jeff Trawick <tr...@gmail.com> wrote:
>
>
>
> On Thu, Nov 15, 2012 at 5:43 PM, Lazy <la...@gmail.com> wrote:
>
>> Hi,
>>
>> we are using httpd 2.4.3 with php 5.4 running fpm mode.
>>
>> requests are passed to fpm using ProxyPassMatch. Everything works fine
>> except,
>>
>> Sometimes database data gets corrupted. All of the data was visible in
>> the form, but it gets truncated after ajax request commiting changes
>> gets aborted (shows as aborted in firebug).
>>
>> in the access log:
>> 80.5.x.x - - [15/Nov/2012:14:03:43 +0100] "POST
>> /xxx,action,pollb,id,958.html HTTP/1.1" 503 299
>> "https://xxx,action,pollb,id,958.html" "Mozilla/5.0 (Windows NT 6.1;
>> WOW64; rv:14.0) Gecko/20100101 Firefox/14.0.1"
>>
>> in the error log:
>> [Thu Nov 15 14:04:03.140026 2012] [proxy_fcgi:error] [pid 15633:tid
>> 140459279939328] (104)Connection reset by peer: [client
>> 80.5.x.x:34541] AH01075: Error dispatching request to :, referer:
>> https://xxx,action,pollb,id,958.html
>>
>> There is haproxy and varnish before httpd, it is also doing ssl.
>> 80.5.x.x is remote clients ip (we are using mod_remoteip).
>>
>> What does the error code exactly mean ? Our client (haproxy) reset ed
>> the connection ? Or was it php ?
>>
>
> PHP dropped the connection.
>
> By chance is PHP_FCGI_MAX_REQUESTS set somewhere for the PHP process? In
> the old days it defaulted to 500 (exit after 500 requests), which led to
> exactly this scenario, but with php-fpm it defaults to 0 (don't exit).
>
whoops, the analogous php-fpm setting is pm.max_requests
*
*
>
>
>
>>
>> Is it possible that somehow php processed the begining of the aborted
>> POST truncating the data in the database ?
>>
>
> Unclear... Do you have a way of tracking of crashes of php-fpm? (Does it
> have a log, or is there anything in the system log?)
>
>
>>
>> OpenSSL support in haproxy is new, and aborted connection directs
>> suspicion on it (in case of httpd or varnish error there should be
>> some kind of 5xx error not an aborted connection), but still I don't
>> understand how the data got lost.
>>
>>
>> Thanks for Your time,
>>
>>
>> Regards,
>> Michal Grzedzicki
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
>> For additional commands, e-mail: users-help@httpd.apache.org
>>
>>
>
>
> --
> Born in Roswell... married an alien...
> http://emptyhammock.com/
>
>
--
Born in Roswell... married an alien...
http://emptyhammock.com/
Re: [users@httpd] mod_proxy_fcgi + php-fpm + connection reset by peer
Posted by Jeff Trawick <tr...@gmail.com>.
On Thu, Nov 15, 2012 at 5:43 PM, Lazy <la...@gmail.com> wrote:
> Hi,
>
> we are using httpd 2.4.3 with php 5.4 running fpm mode.
>
> requests are passed to fpm using ProxyPassMatch. Everything works fine
> except,
>
> Sometimes database data gets corrupted. All of the data was visible in
> the form, but it gets truncated after ajax request commiting changes
> gets aborted (shows as aborted in firebug).
>
> in the access log:
> 80.5.x.x - - [15/Nov/2012:14:03:43 +0100] "POST
> /xxx,action,pollb,id,958.html HTTP/1.1" 503 299
> "https://xxx,action,pollb,id,958.html" "Mozilla/5.0 (Windows NT 6.1;
> WOW64; rv:14.0) Gecko/20100101 Firefox/14.0.1"
>
> in the error log:
> [Thu Nov 15 14:04:03.140026 2012] [proxy_fcgi:error] [pid 15633:tid
> 140459279939328] (104)Connection reset by peer: [client
> 80.5.x.x:34541] AH01075: Error dispatching request to :, referer:
> https://xxx,action,pollb,id,958.html
>
> There is haproxy and varnish before httpd, it is also doing ssl.
> 80.5.x.x is remote clients ip (we are using mod_remoteip).
>
> What does the error code exactly mean ? Our client (haproxy) reset ed
> the connection ? Or was it php ?
>
PHP dropped the connection.
By chance is PHP_FCGI_MAX_REQUESTS set somewhere for the PHP process? In
the old days it defaulted to 500 (exit after 500 requests), which led to
exactly this scenario, but with php-fpm it defaults to 0 (don't exit).
>
> Is it possible that somehow php processed the begining of the aborted
> POST truncating the data in the database ?
>
Unclear... Do you have a way of tracking of crashes of php-fpm? (Does it
have a log, or is there anything in the system log?)
>
> OpenSSL support in haproxy is new, and aborted connection directs
> suspicion on it (in case of httpd or varnish error there should be
> some kind of 5xx error not an aborted connection), but still I don't
> understand how the data got lost.
>
>
> Thanks for Your time,
>
>
> Regards,
> Michal Grzedzicki
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
>
--
Born in Roswell... married an alien...
http://emptyhammock.com/