You are viewing a plain text version of this content. The canonical link for it is here.
Posted to httpclient-users@hc.apache.org by Sachin Nikumbh <sa...@gmail.com> on 2012/12/02 06:12:36 UTC

HTTP Async client and SO_SNDBUF

Hi,

Is there a way to set the SO_SNDBUF while working with the async HTTP
client? CoreConnectionPNames class does not expose this property. When I
google'ed on this topic, I found that
org.apache.commons.httpclient.params.HttpConnectionParams* *class exposes
it.

I am comparing the throughput of the HTTP async client with Sun's
HttpURLConnection. The test case is a single threaded application that's
sending ~1Mb of data around 1000 number of times. With Wireshark, I see
that apache HTTP client's request content-length never exceeds 4096 bytes
whereas HttpURLConnectioncan have content-length of much higher size. This
is resulting in the Sun's implementation showing faster results for sending
the request. The response is received at almost the same speed as the
response content-length is same for both.

I was wondering if tweaking SO_SNDBUF would allow me to get HTTP async
client to perform faster than Sun's.

Any feedback would be highly appreciated.

Thanks
Sachin

Re: HTTP Async client and SO_SNDBUF

Posted by Oleg Kalnichevski <ol...@apache.org>.
On Mon, 2012-12-03 at 10:28 -0500, Sachin Nikumbh wrote:
> Hi Oleg,
> 
> Thanks for the response. I have read about NIO being slower than classic
> I/O. Will you be able to make any comments on why this is the case? I have
> failed to find documentation that would say more about the "why" behind the
> NIO being slower than classic I/O.
> 
> Thanks
> Sachin
> 

I think mainly due to the fact that modern OS / JVMs got more efficient
at context switching. Modern JVMs are faster switching between several
hundred threads than a I/O multiplexer switching between non-blocking
channels.

Oleg

> On Mon, Dec 3, 2012 at 8:55 AM, Oleg Kalnichevski <ol...@apache.org> wrote:
> 
> > On Sun, 2012-12-02 at 00:12 -0500, Sachin Nikumbh wrote:
> > > Hi,
> > >
> > > Is there a way to set the SO_SNDBUF while working with the async HTTP
> > > client? CoreConnectionPNames class does not expose this property. When I
> > > google'ed on this topic, I found that
> > > org.apache.commons.httpclient.params.HttpConnectionParams* *class exposes
> > > it.
> > >
> > > I am comparing the throughput of the HTTP async client with Sun's
> > > HttpURLConnection. The test case is a single threaded application that's
> > > sending ~1Mb of data around 1000 number of times. With Wireshark, I see
> > > that apache HTTP client's request content-length never exceeds 4096 bytes
> > > whereas HttpURLConnectioncan have content-length of much higher size.
> > This
> > > is resulting in the Sun's implementation showing faster results for
> > sending
> > > the request. The response is received at almost the same speed as the
> > > response content-length is same for both.
> > >
> > > I was wondering if tweaking SO_SNDBUF would allow me to get HTTP async
> > > client to perform faster than Sun's.
> > >
> > > Any feedback would be highly appreciated.
> > >
> > > Thanks
> > > Sachin
> >
> > Sachin
> >
> > Generally NIO based HTTP transports can be expected to be as much as 25
> > to 50% slower that those based on the classic (blocking) I/O (as long as
> > the number of concurrent connections is below 1000). I would say HUC
> > being faster than HttpAsyncClient is perfectly reasonable, especially
> > for a single threaded test case. Try that with 2000 concurrent threads
> > and results might turn out to be totally different.
> >
> > I also do not think this has anything to do with the SO_SNDBUF
> > parameter. The reason for having 4096 long TCP frames is probably your
> > using the default request producer which in its turn serializes content
> > entity using EntityAsyncContentProducer [1]. I would recommend using a
> > custom request producer / response consumer implementations for optimal
> > results.
> >
> > Oleg
> >
> > [1]
> >
> > http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/xref/org/apache/http/nio/entity/EntityAsyncContentProducer.html#54
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> > For additional commands, e-mail: httpclient-users-help@hc.apache.org
> >
> >



---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org

Re: HTTP Async client and SO_SNDBUF

Posted by Sachin Nikumbh <sa...@gmail.com>.
Hi Oleg,

Thanks for the response. I have read about NIO being slower than classic
I/O. Will you be able to make any comments on why this is the case? I have
failed to find documentation that would say more about the "why" behind the
NIO being slower than classic I/O.

Thanks
Sachin

On Mon, Dec 3, 2012 at 8:55 AM, Oleg Kalnichevski <ol...@apache.org> wrote:

> On Sun, 2012-12-02 at 00:12 -0500, Sachin Nikumbh wrote:
> > Hi,
> >
> > Is there a way to set the SO_SNDBUF while working with the async HTTP
> > client? CoreConnectionPNames class does not expose this property. When I
> > google'ed on this topic, I found that
> > org.apache.commons.httpclient.params.HttpConnectionParams* *class exposes
> > it.
> >
> > I am comparing the throughput of the HTTP async client with Sun's
> > HttpURLConnection. The test case is a single threaded application that's
> > sending ~1Mb of data around 1000 number of times. With Wireshark, I see
> > that apache HTTP client's request content-length never exceeds 4096 bytes
> > whereas HttpURLConnectioncan have content-length of much higher size.
> This
> > is resulting in the Sun's implementation showing faster results for
> sending
> > the request. The response is received at almost the same speed as the
> > response content-length is same for both.
> >
> > I was wondering if tweaking SO_SNDBUF would allow me to get HTTP async
> > client to perform faster than Sun's.
> >
> > Any feedback would be highly appreciated.
> >
> > Thanks
> > Sachin
>
> Sachin
>
> Generally NIO based HTTP transports can be expected to be as much as 25
> to 50% slower that those based on the classic (blocking) I/O (as long as
> the number of concurrent connections is below 1000). I would say HUC
> being faster than HttpAsyncClient is perfectly reasonable, especially
> for a single threaded test case. Try that with 2000 concurrent threads
> and results might turn out to be totally different.
>
> I also do not think this has anything to do with the SO_SNDBUF
> parameter. The reason for having 4096 long TCP frames is probably your
> using the default request producer which in its turn serializes content
> entity using EntityAsyncContentProducer [1]. I would recommend using a
> custom request producer / response consumer implementations for optimal
> results.
>
> Oleg
>
> [1]
>
> http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/xref/org/apache/http/nio/entity/EntityAsyncContentProducer.html#54
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
> For additional commands, e-mail: httpclient-users-help@hc.apache.org
>
>

Re: HTTP Async client and SO_SNDBUF

Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, 2012-12-02 at 00:12 -0500, Sachin Nikumbh wrote:
> Hi,
> 
> Is there a way to set the SO_SNDBUF while working with the async HTTP
> client? CoreConnectionPNames class does not expose this property. When I
> google'ed on this topic, I found that
> org.apache.commons.httpclient.params.HttpConnectionParams* *class exposes
> it.
> 
> I am comparing the throughput of the HTTP async client with Sun's
> HttpURLConnection. The test case is a single threaded application that's
> sending ~1Mb of data around 1000 number of times. With Wireshark, I see
> that apache HTTP client's request content-length never exceeds 4096 bytes
> whereas HttpURLConnectioncan have content-length of much higher size. This
> is resulting in the Sun's implementation showing faster results for sending
> the request. The response is received at almost the same speed as the
> response content-length is same for both.
> 
> I was wondering if tweaking SO_SNDBUF would allow me to get HTTP async
> client to perform faster than Sun's.
> 
> Any feedback would be highly appreciated.
> 
> Thanks
> Sachin

Sachin

Generally NIO based HTTP transports can be expected to be as much as 25
to 50% slower that those based on the classic (blocking) I/O (as long as
the number of concurrent connections is below 1000). I would say HUC
being faster than HttpAsyncClient is perfectly reasonable, especially
for a single threaded test case. Try that with 2000 concurrent threads
and results might turn out to be totally different. 

I also do not think this has anything to do with the SO_SNDBUF
parameter. The reason for having 4096 long TCP frames is probably your
using the default request producer which in its turn serializes content
entity using EntityAsyncContentProducer [1]. I would recommend using a
custom request producer / response consumer implementations for optimal
results.

Oleg

[1]
http://hc.apache.org/httpcomponents-core-ga/httpcore-nio/xref/org/apache/http/nio/entity/EntityAsyncContentProducer.html#54 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@hc.apache.org
For additional commands, e-mail: httpclient-users-help@hc.apache.org