You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by Oleg Kalnichevski <ol...@apache.org> on 2005/08/20 22:01:19 UTC
[HttpCommon] Poor performance of chunked streams
Folks,
I have just identified another problem with HttpCommon code, which
causes massive performance degradation when streaming in or out chunked
data. I also suspect HttpDataReceiver and HttpDataTransmitter interfaces
will have to be completely rethought. Since it is very rainy and cold
here, and the weekend looks all but ruined, I guess this is just the
perfect opportunity to get some code fixed ;-)
Oleg
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [HttpCommon] Poor performance of chunked streams
Posted by Oleg Kalnichevski <ol...@apache.org>.
On Sun, Aug 21, 2005 at 03:08:33PM +0200, Oleg Kalnichevski wrote:
> I have also compiled another bunch of test cases intended to test
> performance of various read operations. Interestingly enough, but not
> entirely unexpectedly, NIO handsomely outperforms classic IO when
> reading single bytes and lines of text:
>
> ==============================
> Old IO byte array read average time (ms): 246
> NIO byte array read average time (ms): 216
> Old IO one byte read average time (ms): 537
> NIO one byte read average time (ms): 178
> Old IO one line read average time (ms): 340
> NIO one line read average time (ms): 77
> ==============================
> Source [2]
>
What a bloody mess. I ran the test on my WinXP box at work and this is
what I got:
Java 1.4.2.3
==============================================
Old IO byte array read average time (ms): 1031
NIO byte array read average time (ms): 1094
Old IO one byte read average time (ms): 281
NIO one byte read average time (ms): 296
Old IO one line read average time (ms): 157
NIO one line read average time (ms): 157
Java 1.5.0.3
=============================================
Old IO byte array read average time (ms): 955
NIO byte array read average time (ms): 1017
Old IO one byte read average time (ms): 250
NIO one byte read average time (ms): 281
Old IO one line read average time (ms): 141
NIO one line read average time (ms): 125
Classic IO seems to outperform NIO on Windows, whereas the same test run
on my Linux box produced the opposite result
Anyone observing the same phenomenon?
Oleg
> Please let me know if you can spot any flaws in the test cases
>
> Cheers,
>
> Oleg
>
> [1]
> http://svn.apache.org/repos/asf/jakarta/httpclient/trunk/http-common/src/test/tests/performance/RawNIOvsRawOIOTest.java
> [2]
> http://svn.apache.org/repos/asf/jakarta/httpclient/trunk/http-common/src/test/tests/performance/HttpNIOvsHttpOIOTest.java
>
> On Sat, 2005-08-20 at 22:01 +0200, Oleg Kalnichevski wrote:
> > Folks,
> >
> > I have just identified another problem with HttpCommon code, which
> > causes massive performance degradation when streaming in or out chunked
> > data. I also suspect HttpDataReceiver and HttpDataTransmitter interfaces
> > will have to be completely rethought. Since it is very rainy and cold
> > here, and the weekend looks all but ruined, I guess this is just the
> > perfect opportunity to get some code fixed ;-)
> >
> > Oleg
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
Re: [HttpCommon] Poor performance of chunked streams
Posted by Oleg Kalnichevski <ol...@apache.org>.
Folks,
Once again NIO proves full of surprises. After having experimented with
chunked streams a little more I discovered that the problem is fact once
again lied with NIO. It turned out that NIO performs absolutely dismally
when the socket receive and send buffers are relatively small. With 2048
byte receive and send buffers NIO tends to perform several THOUSAND
times slower than with 20480 buffers.
Check this out:
==============================
RCV_BUFFER_SIZE: 2048
Old IO average time (ms): 3301
Blocking NIO average time (ms): 6515
NIO with Select average time (ms): 15395
RCV_BUFFER_SIZE: 20480
Old IO average time (ms): 2
Blocking NIO average time (ms): 2
NIO with Select average time (ms): 3
==============================
Source [1]
I have also compiled another bunch of test cases intended to test
performance of various read operations. Interestingly enough, but not
entirely unexpectedly, NIO handsomely outperforms classic IO when
reading single bytes and lines of text:
==============================
Old IO byte array read average time (ms): 246
NIO byte array read average time (ms): 216
Old IO one byte read average time (ms): 537
NIO one byte read average time (ms): 178
Old IO one line read average time (ms): 340
NIO one line read average time (ms): 77
==============================
Source [2]
Please let me know if you can spot any flaws in the test cases
Cheers,
Oleg
[1]
http://svn.apache.org/repos/asf/jakarta/httpclient/trunk/http-common/src/test/tests/performance/RawNIOvsRawOIOTest.java
[2]
http://svn.apache.org/repos/asf/jakarta/httpclient/trunk/http-common/src/test/tests/performance/HttpNIOvsHttpOIOTest.java
On Sat, 2005-08-20 at 22:01 +0200, Oleg Kalnichevski wrote:
> Folks,
>
> I have just identified another problem with HttpCommon code, which
> causes massive performance degradation when streaming in or out chunked
> data. I also suspect HttpDataReceiver and HttpDataTransmitter interfaces
> will have to be completely rethought. Since it is very rainy and cold
> here, and the weekend looks all but ruined, I guess this is just the
> perfect opportunity to get some code fixed ;-)
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpclient-dev-help@jakarta.apache.org