Client http performance

[Stefan Magnus Landrø <st...@gmail.com>]

Hi,

It says in the docs that cxf can use the commons asynchronous client. The docs also say: "However, the non-blocking model does not perform quite as well as the blocking model for pure synchronous request/response transactions."  What does this mean in practical terms? Why is there no support for the standard commons http client? 

Cheers Stefan

Re: Client http performance

[Daniel Kulp <dk...@apache.org>]

On Nov 21, 2012, at 1:24 AM, Stefan Magnus Landrø <st...@gmail.com> wrote:
> It says in the docs that cxf can use the commons asynchronous client. The docs also say: "However, the non-blocking model does not perform quite as well as the blocking model for pure synchronous request/response transactions."  What does this mean in practical terms? Why is there no support for the standard commons http client? 

There are two types of "commons http client", the non-blocking NIO based version (httpcore-nio + httpasyncclient) and the blocking version (httpcore + httpclient).  

For the most part, the blocking version provides us very little benefit, but would greatly complicate things and likely yield significant performance problems as compared to using the HttpURLConnection built into the JDK.   Much of CXF is designed to stream stuff out as it's produced by writing directly to an output stream that the HttpURLConnection returns.   All this code is perfectly re-used on the outgoing side of the servers by using the OutputStream form the ServletResponse (or other transports).    The CXF interceptors each may write out a little bit of data to the stream, the next interceptor may write more, etc….    The blocking httpclients don't fit that model very well.   They are very good for blasting out fully rendered content (like a File object or byte[] or similar) or some other content object that can render itself on one call (such as Axiom or SAAJ that can take an output stream).   Getting the blocking clients to work well with CXF would either involve rendering the entire message to a buffer to then stream out (performance would suck) or using Pipes and a secondary thread (performance issue) or would require major re-architecting of the entire CXF interceptor model and call stack (not something we're likely to do).     The commons-http client also would complicate some of the lifecycle things in CXF.   Basically, a lot of work.   There are very few features that the blocking client can provide that the HttpURLConnection doesn't support (although the two that come to mind are NTLMv2 and advanced control of the keep-alives).  It would also add dependencies and such that we've pretty much been able to avoid.     For the most part, if the version in the JDK "works", we tend to use it when possible.

The Async version, on the other hand, does provide some value.   The HTTP commons folks will readily admit that the Async stuff is a good 10-20% (or more) slower than using blocking IO.   Thus,  raw performance is certainly not it's value.  The main value it has with CXF is scalability with the JAX-WS Async client calls (and now the JAX-RS AsyncResponse things).  With the HttpURLConnect (and the blocking http client if we had it), we need to maintain a thread per outstanding request (some what).  We do maintain a thread pool for that, but that still limits things.    With the non-blocking IO, we don't have that issue at all.   The Async client has a very small thread pool (default is # of processors in the machine) and that can handle thousands of outstanding requests.   The IO api's actually work OK with CXF's interceptor model as well.  Since the writing has to happen on those threads, the "Pipe" kind of IO works acceptably.  (we don't use Pipes, but same basic idea.  The "main" thread can write to an OutputStream that fills a buffer that the IO threads write out when they can.)   

In any case, the main answer really comes down to the value to work ratio, but also with the extra dependencies requirement providing influence.   If someone would like a blocking http client transport, they are certainly welcome to contribute one.   However, at this point, it doesn't provide enough value to those of us working on things right now.   I think my time is better spent on other things.   That said, the work done to the HTTPConduit pull out all the hard HTTPUrlConnection stuff into a subclass to allow it to support the async version could certainly make it easier to write a blocking client version.    

-- 
Daniel Kulp
dkulp@apache.org - http://dankulp.com/blog
Talend Community Coder - http://coders.talend.com