You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@avro.apache.org by Yang <te...@gmail.com> on 2011/06/08 03:58:46 UTC

thread pool for Socket server?

currently SocketServer basically does this:

      while (true) {
          new Connection(channel.accept());

      }



so it faithfully accepts any incoming connection and creates a worker thread
processing the request.

I think this is prone to DOS attacks, or ungraceful failure in the case of
heavy traffic.

if we use a limited thread pool inside Connection() implementation, we can
reject connections if we are too heavily loaded.

thought it's basically a trivial thing to implement, but considering this is
a very common use case, is it worthwhile to put
such an implementation into the avro source?

Thanks
Yang

Re: thread pool for Socket server?

Posted by Yang <te...@gmail.com>.

Thanks Doug
I'll try betty and contribute the result
Yang
On Jun 8, 2011 7:40 AM, "Doug Cutting" <cu...@apache.org> wrote:
> A thread pool might indeed provide an improvement. However the Netty
> and Jetty-based servers are probably better starting points for reliable
> performance.
>
> Also, please use SaslSocketServer instead of SocketSerer, with the
> anonymous mechanism if no security is required. I'd like to standardize
> on SASL for non-HTTP, high-performance Avro RPC.
>
> It would be great if someone could contribute a Netty-based
> implementation of Avro's SASL profile. That would provide a
> high-performance client and server that can support authentication
> and/or encryption.
>
> Doug
>
> On 06/08/2011 03:58 AM, Yang wrote:
>> currently SocketServer basically does this:
>>
>> while (true) {
>> new Connection(channel.accept());
>>
>> }
>>
>>
>>
>> so it faithfully accepts any incoming connection and creates a worker
>> thread processing the request.
>>
>> I think this is prone to DOS attacks, or ungraceful failure in the case
>> of heavy traffic.
>>
>> if we use a limited thread pool inside Connection() implementation, we
>> can reject connections if we are too heavily loaded.
>>
>> thought it's basically a trivial thing to implement, but considering
>> this is a very common use case, is it worthwhile to put
>> such an implementation into the avro source?
>>
>> Thanks
>> Yang

Re: thread pool for Socket server?

Posted by Philip Zeyliger <ph...@cloudera.com>.

I'd say please do contribute a thread pool implementation.  Even if the
eventual answer is Netty, I've definitely run into issue.

-- Philip

On Wed, Jun 8, 2011 at 7:39 AM, Doug Cutting <cu...@apache.org> wrote:

> A thread pool might indeed provide an improvement.  However the Netty
> and Jetty-based servers are probably better starting points for reliable
> performance.
>
> Also, please use SaslSocketServer instead of SocketSerer, with the
> anonymous mechanism if no security is required.  I'd like to standardize
> on SASL for non-HTTP, high-performance Avro RPC.
>
> It would be great if someone could contribute a Netty-based
> implementation of Avro's SASL profile.  That would provide a
> high-performance client and server that can support authentication
> and/or encryption.
>
> Doug
>
> On 06/08/2011 03:58 AM, Yang wrote:
> > currently SocketServer basically does this:
> >
> >       while (true) {
> >           new Connection(channel.accept());
> >
> >       }
> >
> >
> >
> > so it faithfully accepts any incoming connection and creates a worker
> > thread processing the request.
> >
> > I think this is prone to DOS attacks, or ungraceful failure in the case
> > of heavy traffic.
> >
> > if we use a limited thread pool inside Connection() implementation, we
> > can reject connections if we are too heavily loaded.
> >
> > thought it's basically a trivial thing to implement, but considering
> > this is a very common use case, is it worthwhile to put
> > such an implementation into the avro source?
> >
> > Thanks
> > Yang
>

Re: thread pool for Socket server?

Posted by Doug Cutting <cu...@apache.org>.

A thread pool might indeed provide an improvement.  However the Netty
and Jetty-based servers are probably better starting points for reliable
performance.

Also, please use SaslSocketServer instead of SocketSerer, with the
anonymous mechanism if no security is required.  I'd like to standardize
on SASL for non-HTTP, high-performance Avro RPC.

It would be great if someone could contribute a Netty-based
implementation of Avro's SASL profile.  That would provide a
high-performance client and server that can support authentication
and/or encryption.

Doug

On 06/08/2011 03:58 AM, Yang wrote:
> currently SocketServer basically does this:
> 
>       while (true) {
>           new Connection(channel.accept());
> 
>       }
> 
> 
> 
> so it faithfully accepts any incoming connection and creates a worker
> thread processing the request.
> 
> I think this is prone to DOS attacks, or ungraceful failure in the case
> of heavy traffic.
> 
> if we use a limited thread pool inside Connection() implementation, we
> can reject connections if we are too heavily loaded.
> 
> thought it's basically a trivial thing to implement, but considering
> this is a very common use case, is it worthwhile to put
> such an implementation into the avro source?
> 
> Thanks
> Yang