You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "Asankha C. Perera" <as...@apache.org> on 2010/02/03 05:13:36 UTC
Controlling "acceptance" of connections
Hi All
With some recent load testing in an environment where requests are
queued and handled by a worker pool, I've seen that the NIO listener
will accept any number of connections - possibly even more than what the
worker pool (or the backend server to which they are forwarded) could
handle.
Thus I would like to impose a limit on the connections handled in a
graceful way - e.g. without refusing connections with a TCP RST - but by
not accepting the connection - so that the client can retransmit the SYN
again and try to connect. Is this the correct way to handle such load,
or should the server reply with a HTTP 503 (Service Unavilable) with a
"Retry-After" header? Any pointers and comments are very much appreciated
Tomcat 6 seems to refuse connections after the "acceptCount" is reached
[1] - but I think that might cause a load balancer in front to assume
the node is down - am I wrong?
thanks
asankha
[1] http://tomcat.apache.org/tomcat-6.0-doc/config/http.html
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Controlling "acceptance" of connections
Posted by Ortwin Glück <od...@odi.ch>.
On 03.02.2010 10:05, Asankha C. Perera wrote:
> However, the NIO reactor will process events over threads that are
> different from the worker threads processing the request.
I see what you mean... Once the reactor gets an OP_ACCEPT you have no way to
reject that event again, so the you will effectively always accept a new
connection. But since that op is the only one of ServerSocketChannel, the
reactor should simply not select on the channel when the number of concurrent
connections is reached (unregister the channel from the selector). This way the
connections will queue in the OS and will be subject to the backlog limit. Right?
Ortwin
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Controlling "acceptance" of connections
Posted by Ortwin Glück <od...@odi.ch>.
On 03.02.2010 11:34, Asankha C. Perera wrote:
> I think turning off interest in OP_ACCEPT will prevent a busy loop, and
> be the ideal solution.. This takes me back to HTTPCORE-127 where I
> wanted the client to see a connection refused.. so this new requirement
> is slightly different, as now I simply want to delay acceptance of
> connections..
If you set backlog to 1, new connections will be refused on TCP level. However
if you want a HTTP error response, this doesn't help you.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Controlling "acceptance" of connections
Posted by "Asankha C. Perera" <as...@apache.org>.
Hi Oleg / Odi
> On Wed, 2010-02-03 at 11:04 +0100, Ortwin Glück wrote:
>
>> Answering myself again :-
>>> In theory yes. But you get the ACCEPT ready *once* only. So if you ignore it,
>>> you forget that a channel was ready and you can end up never accepting it. So
>>> not registering for OP_ACCEPT in the first place when you can't handle new
>>> sockets is a must.
>>>
>> I may be wrong here, and you get ACCEPT ready over and over again until you
>> actually accept. That's equally bad because you end up with a busy wait.
>>
> I guess it should be possible to turn off interest in OP_ACCEPT
> notifications on that channel and turn it on again once the server is
> ready to accept more connections. Control logic might get quite complex,
> though.
I think turning off interest in OP_ACCEPT will prevent a busy loop, and
be the ideal solution.. This takes me back to HTTPCORE-127 where I
wanted the client to see a connection refused.. so this new requirement
is slightly different, as now I simply want to delay acceptance of
connections..
I will try to see how complex it would be to expose this.. and if the
effort is worth it
thanks
asankha
Re: Controlling "acceptance" of connections
Posted by Ortwin Glück <od...@odi.ch>.
On 03.02.2010 11:16, Oleg Kalnichevski wrote:
> I guess it should be possible to turn off interest in OP_ACCEPT
> notifications on that channel and turn it on again once the server is
> ready to accept more connections. Control logic might get quite complex,
> though.
exactly
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Controlling "acceptance" of connections
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2010-02-03 at 11:04 +0100, Ortwin Glück wrote:
> Answering myself again :-)
>
> > In theory yes. But you get the ACCEPT ready *once* only. So if you ignore it,
> > you forget that a channel was ready and you can end up never accepting it. So
> > not registering for OP_ACCEPT in the first place when you can't handle new
> > sockets is a must.
>
> I may be wrong here, and you get ACCEPT ready over and over again until you
> actually accept. That's equally bad because you end up with a busy wait.
>
I guess it should be possible to turn off interest in OP_ACCEPT
notifications on that channel and turn it on again once the server is
ready to accept more connections. Control logic might get quite complex,
though.
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Controlling "acceptance" of connections
Posted by Ortwin Glück <od...@odi.ch>.
Answering myself again :-)
> In theory yes. But you get the ACCEPT ready *once* only. So if you ignore it,
> you forget that a channel was ready and you can end up never accepting it. So
> not registering for OP_ACCEPT in the first place when you can't handle new
> sockets is a must.
I may be wrong here, and you get ACCEPT ready over and over again until you
actually accept. That's equally bad because you end up with a busy wait.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Controlling "acceptance" of connections
Posted by Ortwin Glück <od...@odi.ch>.
On 03.02.2010 10:58, Oleg Kalnichevski wrote:
> I think NIO should work quite similarly. A selectable channel can be
> marked as OP_ACCEPT ready, but that does not mean the connection
> listener must accept that connection or accept it immediately.
In theory yes. But you get the ACCEPT ready *once* only. So if you ignore it,
you forget that a channel was ready and you can end up never accepting it. So
not registering for OP_ACCEPT in the first place when you can't handle new
sockets is a must.
Odi
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Controlling "acceptance" of connections
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Wed, 2010-02-03 at 14:35 +0530, Asankha C. Perera wrote:
> Hi Ortwin
> > What you are referring to is the "backlog" of a Socket. That is the number of
> > half-open / unaccepted connections that is queued by the OS before sending RSTs
> > for any more SYNs.
> >
> I thought so too.. but I think in the NIO model its a bit different -
> especially when a request is handled by a separate thread pool. In the
> classic model of a thread per socket - an implementation will not be
> doing a ServerSocket.accept() once the thread pool is exhausted.
> However, the NIO reactor will process events over threads that are
> different from the worker threads processing the request. Thus I believe
> all connections gets accepted by default - without being queued at TCP
> level, and this is what I would like to control if possible.. provided
> that my understanding on this is correct :D
>
Asankha
I think NIO should work quite similarly. A selectable channel can be
marked as OP_ACCEPT ready, but that does not mean the connection
listener must accept that connection or accept it immediately.
Hope this helps
Oleg
> thanks
> asankha
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Controlling "acceptance" of connections
Posted by "Asankha C. Perera" <as...@apache.org>.
Hi Ortwin
> What you are referring to is the "backlog" of a Socket. That is the number of
> half-open / unaccepted connections that is queued by the OS before sending RSTs
> for any more SYNs.
>
I thought so too.. but I think in the NIO model its a bit different -
especially when a request is handled by a separate thread pool. In the
classic model of a thread per socket - an implementation will not be
doing a ServerSocket.accept() once the thread pool is exhausted.
However, the NIO reactor will process events over threads that are
different from the worker threads processing the request. Thus I believe
all connections gets accepted by default - without being queued at TCP
level, and this is what I would like to control if possible.. provided
that my understanding on this is correct :D
thanks
asankha
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: Controlling "acceptance" of connections
Posted by Ortwin Glück <od...@odi.ch>.
Asankha,
What you are referring to is the "backlog" of a Socket. That is the number of
half-open / unaccepted connections that is queued by the OS before sending RSTs
for any more SYNs.
See java.net.Socket bind() method that accept the backlog parameter.
Unfortunately you can not change this property after binding. I suggest that we
should provide the backlog as a parameter.
But actually a more flexible (and more intrusive) way is to make a
ServerSocketChannel factory or at least a factory method, and do away with the
prepareSocket call. That way even more aspects of the underlying implementation
are tunable to the caller and the HttpNIO code doesn't need to be changed for
them. Think of a SCTP implementation for instance :-)
Thanks.
Ortwin
On 03.02.2010 05:13, Asankha C. Perera wrote:
> Hi All
>
> With some recent load testing in an environment where requests are
> queued and handled by a worker pool, I've seen that the NIO listener
> will accept any number of connections - possibly even more than what the
> worker pool (or the backend server to which they are forwarded) could
> handle.
>
> Thus I would like to impose a limit on the connections handled in a
> graceful way - e.g. without refusing connections with a TCP RST - but by
> not accepting the connection - so that the client can retransmit the SYN
> again and try to connect. Is this the correct way to handle such load,
> or should the server reply with a HTTP 503 (Service Unavilable) with a
> "Retry-After" header? Any pointers and comments are very much appreciated
>
> Tomcat 6 seems to refuse connections after the "acceptCount" is reached
> [1] - but I think that might cause a load balancer in front to assume
> the node is down - am I wrong?
>
> thanks
> asankha
>
> [1] http://tomcat.apache.org/tomcat-6.0-doc/config/http.html
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
> For additional commands, e-mail: dev-help@hc.apache.org
>
--
[web] http://www.odi.ch/
[blog] http://www.odi.ch/weblog/
[pgp] key 0x81CF3416
finger print F2B1 B21F F056 D53E 5D79 A5AF 02BE 70F5 81CF 3416
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org