Re: [HttpCore] Tuning HttpCore / NIO

[Oleg Kalnichevski <ol...@apache.org>]

On Fri, 2007-06-01 at 17:11 +0530, Asankha C. Perera wrote:
> Hi All
> 
> I was doing some performance tests for Synapse (which uses HttpCore/NIO) 
> underneath, and am glad to say that we have great performance for 
> messages of around ~1K (request and response). However, when the message 
> size increases to around ~5K (request and response) there is a slight 
> degrading of performance, and so I would like to learn from you all on 
> how I could get the best performance out of HttpCore/NIO
> 
> My configuration by default starts up a http sender and listener, and 
> the same for https - creating a total of 4 IO Reactors. I am using 2 
> workers per reactor, and use 2K byte buffers [each] to read and write 
> messages.
> 
> For the client/sender side I am using 60s SO_TIMEOUT, 10s 
> CONNECTION_TIMEOUT, 8K SOCKET_BUFFER_SIZE,
> STALE_CONNECTION_CHECK off and TCP_NODELAY off
> 
> For the server side I am using 60s SO_TIMEOUT, 8K SOCKET_BUFFER_SIZE, 
> STALE_CONNECTION_CHECK off and TCP_NODELAY off
> 
> My configuration is expected to make maximum use of keepalives and 
> connection reuse. In addition, I am setting the following Linux parameters:
> 
> echo "1024 65535" > /proc/sys/net/ipv4/ip_local_port_range
> echo "30" > /proc/sys/net/ipv4/tcp_fin_timeout
> echo 2097152 > /proc/sys/fs/file-max
> echo 1 > /proc/sys/net/ipv4/tcp_tw_recycle
> echo 1 > /proc/sys/net/ipv4/tcp_tw_reuse
> ulimit unlimited
> 
> Any help on this matter is very much appreciated and I believe would 
> help us create a valuable resource of advice that would help other users 
> of HttpCore esp in the future
> 
> thanks
> asankha
> 

Hi Asankha

You probably _really_ want to set TCP_NODELAY on. This will disable
Nagle algorithm and should result in a considerable performance increase
for persistent connections [1]. 

You may also want to experiment with SO_RCVBUF and SO_SNDBUF TCP/IP
parameters. These parameters tend to have a huge (order of magnitude)
impact on performance. Usually the larger the buffer the higher the data
through-put. HttpCore does not allow you to modify the SO_RCVBUF and
SO_SNDBUF values using HttpParams, as generally it is a good idea to
leave these parameters set to the OS defaults. You may still want to
tune them on the OS level (net.core.rmem_default, net.core.wmem_default,
net.core.rmem_max, net.core.wmem_max parameters in Linux)

I also think Synapse could benefit from a custom NIO pipe implementation
capable of throttling I/O rate under load using IOControl interface. If
the underlying pipe implementation were capable of temporarily
suspending I/O events when unable to process them instead of just
ignoring them, it would result in some performance improvement, because
the I/O reactors would not fire I/O events unnecessarily. But this is
probably something for Synapse 1.1

Hope this helps somewhat

Oleg

[1] http://en.wikipedia.org/wiki/Nagle's_algorithm



> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org
> 
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org

Re: [HttpCore] Tuning HttpCore / NIO

["Asankha C. Perera" <as...@wso2.com>]

Hi Oleg
> probably _really_ want to set TCP_NODELAY on. This will disable
> Nagle algorithm and should result in a considerable performance increase
> for persistent connections [1]. 
>
> You may also want to experiment with SO_RCVBUF and SO_SNDBUF TCP/IP
> parameters. These parameters tend to have a huge (order of magnitude)
> impact on performance. Usually the larger the buffer the higher the data
> through-put. HttpCore does not allow you to modify the SO_RCVBUF and
> SO_SNDBUF values using HttpParams, as generally it is a good idea to
> leave these parameters set to the OS defaults. You may still want to
> tune them on the OS level (net.core.rmem_default, net.core.wmem_default,
> net.core.rmem_max, net.core.wmem_max parameters in Linux)
>   
Thanks for this excellent information, and I would try it early next 
week in my performance test environment.
> I also think Synapse could benefit from a custom NIO pipe implementation
> capable of throttling I/O rate under load using IOControl interface. If
> the underlying pipe implementation were capable of temporarily
> suspending I/O events when unable to process them instead of just
> ignoring them, it would result in some performance improvement, because
> the I/O reactors would not fire I/O events unnecessarily. But this is
> probably something for Synapse 1.1
>   
Yes, I have seen a few issues with the current implementation - where I 
use native Pipe implementation for Unix and a simulated Pipe for Windows 
by default. I hope to revisit this code to improve it, when I get some 
time to look at it in detail.

many thanks
asankha

---------------------------------------------------------------------
To unsubscribe, e-mail: httpcomponents-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: httpcomponents-dev-help@jakarta.apache.org