You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Chirouze Olivier <ol...@volvo.com> on 2007/01/26 16:56:46 UTC
[users@httpd] Apache 2.0.58 + Solaris 5.9: status "...reading..." & TCP state "FIN_WAIT_2"
Hi all,
I'm facing a quite tricky situation with Apache 2.0.58 running on
Solaris 5.9.
Apache is running as a reverse proxy (mod_proxy + mod_rewrite).
The maximum concurrent connections is set to 150.
Because we reached the maximum a few times and got the reverse proxy
saturated, we started monitoring the Apache status page (/status).
We noticed that many requests were in the "..reading.." state (up to
40!), and they block a lot of slots.
At first, we upgraded from 2.0.47 to 2.0.58 because it seemed there was
a security hole in the earlier, fixed in 2.0.48.
I found some explanation here:
http://www.monkeybrains.net/~rudy/example/server_busy_state.html.
The thing is, the situation is starting to appear again with 2.0.58.
We've gone down to Unix and found that most of these requests were in
"FIN_WAIT_2" TCP state, and for a while (approx. 8min!!).
We found this: http://httpd.apache.org/docs/2.0/misc/fin_wait_2.html.
What it says, in a word, is that these things can happen and are
"normal": the connection stays in "FIN_WAIT_2" state until the timeout,
if clients do not close it properly. They just say it can be a problem
on the Unix point of view because.
I don't know if this is still true for 2.0 because the article was just
copied from 1.3. The thing is, it says that "The connections in
FIN_WAIT_2 do not tie up an httpd process". For us, IT DOES! Every
"..reading.." request happend to be in the "FIN_WAIT_2" state.
We have contacted Sun to get their opinion. The short answer is "you can
change the FIN_WAIT_2 timeout but be careful because wrong tuning will
have negative impact. Maybe you should wonder why these connections stay
alive". As far as I understood, the connection is not closed by the
client. The server (Apache) does nothing wrong. But maybe it does, as it
doesn't leave the process free?
My questions are:
Does anyone have heard about similar problems?
Why do these connections hold a process of Apache while the
documentation says it doesn't?
Do you recon tuning the Unix timeout would help? (current value of
tcp_fin_wait_2_flush_interval: 675000 ms - 11min!! This looks just
huge!)
Thanks in advance,
Olivier
Olivier CHIROUZE
I&0 Infrastructure
Volvo Information Technology
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] Apache 2.0.58 + Solaris 5.9: status "...reading..." & TCP state "FIN_WAIT_2"
Posted by Chirouze Olivier <ol...@volvo.com>.
Your different experiences are very interesting.
I think we will soon play with these params too :-)
Thanks a lot!
Olivier
Olivier CHIROUZE
I&0 Infrastructure
Volvo Information Technology
> -----Original Message-----
> From: Krist van Besien [mailto:krist.vanbesien@gmail.com]
> Sent: 31 January 2007 17:10
> To: users@httpd.apache.org
> Subject: Re: [users@httpd] Apache 2.0.58 + Solaris 5.9:
> status "...reading..." & TCP state "FIN_WAIT_2"
>
> On 1/31/07, Chirouze Olivier <ol...@volvo.com> wrote:
> > Hi John,
> >
> > Thanks a lot for your response. 30 seconds sounds so much
> more normal than 8 minutes to me...
> > I guess if you have that and didn't change back, it means
> you didn't get any side effects?
>
> We are currently running several production solaris servers with
> tcp_fin_wait_2_flush_interval set to one minute (60000) and have no
> issues. In fact, lowering it solved quite a few issues we had that
> were similar to yours.
>
> Krist
>
>
> --
> krist.vanbesien@gmail.com
> Bremgarten b. Bern, Switzerland
> --
> "...what you don't realize is that in the future Google WILL reach
> sentience, will [have had] invent[ed] a time machine, and will [have
> had] travel[ed] back in time to prevent Bill Gates... only to become
> Bill Gates by accident because of a search engine optimization
> miscalculation." (Comment on the Dilbert Blog)
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP
> Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Apache 2.0.58 + Solaris 5.9: status "...reading..." & TCP state "FIN_WAIT_2"
Posted by Krist van Besien <kr...@gmail.com>.
On 1/31/07, Chirouze Olivier <ol...@volvo.com> wrote:
> Hi John,
>
> Thanks a lot for your response. 30 seconds sounds so much more normal than 8 minutes to me...
> I guess if you have that and didn't change back, it means you didn't get any side effects?
We are currently running several production solaris servers with
tcp_fin_wait_2_flush_interval set to one minute (60000) and have no
issues. In fact, lowering it solved quite a few issues we had that
were similar to yours.
Krist
--
krist.vanbesien@gmail.com
Bremgarten b. Bern, Switzerland
--
"...what you don't realize is that in the future Google WILL reach
sentience, will [have had] invent[ed] a time machine, and will [have
had] travel[ed] back in time to prevent Bill Gates... only to become
Bill Gates by accident because of a search engine optimization
miscalculation." (Comment on the Dilbert Blog)
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] Apache 2.0.58 + Solaris 5.9: status "...reading..." & TCP state "FIN_WAIT_2"
Posted by Chirouze Olivier <ol...@volvo.com>.
Hi John,
Thanks a lot for your response. 30 seconds sounds so much more normal than 8 minutes to me...
I guess if you have that and didn't change back, it means you didn't get any side effects?
Olivier
Olivier CHIROUZE
I&0 Infrastructure
Volvo Information Technology
> -----Original Message-----
> From: John P. Dodge [mailto:dodge@cruciate.ca.boeing.com]
> Sent: 30 January 2007 21:24
> To: users@httpd.apache.org
> Subject: Re: [users@httpd] Apache 2.0.58 + Solaris 5.9:
> status "...reading..." & TCP state "FIN_WAIT_2"
>
> On Fri, 26 Jan 2007, Chirouze Olivier wrote:
>
> > Hi all,
> >
> > I'm facing a quite tricky situation with Apache 2.0.58 running on
> > Solaris 5.9.
> >
> > Apache is running as a reverse proxy (mod_proxy + mod_rewrite).
> > The maximum concurrent connections is set to 150.
> >
> >
> > We've gone down to Unix and found that most of these
> requests were in
> > "FIN_WAIT_2" TCP state, and for a while (approx. 8min!!).
> >
> > Do you recon tuning the Unix timeout would help? (current value of
> > tcp_fin_wait_2_flush_interval: 675000 ms - 11min!! This looks just
> > huge!)
> >
> > Thanks in advance,
> >
> My setting way less than that:
> tcp_fin_wait_2_flush_interval 30000
>
> Here are some more of my settings:
>
> tcp_old_urp_interpretation 1
> tcp_rexmit_interval_min 1000
> tcp_rexmit_interval_initial 500
> tcp_recv_hiwat 24576
> tcp_fin_wait_2_flush_interval 30000
> tcp_mss_def 1460
> tcp_ip_ttl 60
> tcp_ip_abort_interval 60000
> tcp_ip_abort_cinterval 75000
> tcp_close_wait_interval 60000
> tcp_rexmit_interval_max 10000
> tcp_keepalive_interval 30000
>
>
> ----------------------------------------
> "Mon aéroglisseur est plein d'anguilles"
> John P. Dodge
> Boeing Shared Services
>
>
> ---------------------------------------------------------------------
> The official User-To-User support forum of the Apache HTTP
> Server Project.
> See <URL:http://httpd.apache.org/userslist.html> for more info.
> To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
> " from the digest: users-digest-unsubscribe@httpd.apache.org
> For additional commands, e-mail: users-help@httpd.apache.org
>
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] Apache 2.0.58 + Solaris 5.9: status "...reading..."
& TCP state "FIN_WAIT_2"
Posted by "John P. Dodge" <do...@cruciate.ca.boeing.com>.
On Fri, 26 Jan 2007, Chirouze Olivier wrote:
> Hi all,
>
> I'm facing a quite tricky situation with Apache 2.0.58 running on
> Solaris 5.9.
>
> Apache is running as a reverse proxy (mod_proxy + mod_rewrite).
> The maximum concurrent connections is set to 150.
>
>
> We've gone down to Unix and found that most of these requests were in
> "FIN_WAIT_2" TCP state, and for a while (approx. 8min!!).
>
> Do you recon tuning the Unix timeout would help? (current value of
> tcp_fin_wait_2_flush_interval: 675000 ms - 11min!! This looks just
> huge!)
>
> Thanks in advance,
>
My setting way less than that:
tcp_fin_wait_2_flush_interval 30000
Here are some more of my settings:
tcp_old_urp_interpretation 1
tcp_rexmit_interval_min 1000
tcp_rexmit_interval_initial 500
tcp_recv_hiwat 24576
tcp_fin_wait_2_flush_interval 30000
tcp_mss_def 1460
tcp_ip_ttl 60
tcp_ip_abort_interval 60000
tcp_ip_abort_cinterval 75000
tcp_close_wait_interval 60000
tcp_rexmit_interval_max 10000
tcp_keepalive_interval 30000
----------------------------------------
"Mon aéroglisseur est plein d'anguilles"
John P. Dodge
Boeing Shared Services
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org