Re: Understanding asynchrounous request processing in Apache Synapse/Axis2/Http Core NIO

["Asankha C. Perera" <as...@wso2.com>]

Hi Eric

Oleg Kalnichevski wrote:
> On Sun, 2008-07-20 at 16:39 +0200, Hubert, Eric wrote:
>   
>> Hi devs,
>>
>> we are facing an issue in Apache Synapse where HttpClientWorkers are in
>> the process of writing a response back to the client using
>> HttpCoreNIOSender.sendAsyncResponse(), but are then waiting at
>> org.apache.http.nio.util.SharedOutputBuffer.flushContent() to get
>> notified by someone...
>>
>> I'd like to take this issue as a start to gather more knowledge about
>> the internal working of Apache Synapse and http core nio in general.
>> Could someone please point me to some documentation, which describes,
>> how the request/response processing is working. 
>>     
>
> Eric,
>
> I am very sorry but we currently have almost nothing in terms of
> documentation for HttpCore. I am planning to start working on an
> HttpCore tutorial in August. 
>
> Asankha knows best about the Synapse NIO transport. I'll happily chip
> in information about HttpCore internal stuff
>
> ...
>   
I know that ideally I should have made more documentation available on 
the NIO transport implementation, but due to other work I am involved 
with, this has gone down in my TODO list.. I will try to come back to 
this in the near future.. However, HttpCore/NIO is where the really cool 
code exists, and Synapse/Axis2 is merely using it..
>> The response is processed by the ClientWorker who delegates the response
>> processing to Axis2 (again the blackbox of AxisEngine.receive,
>> AxisEngine.send) and then the response shall be send back to the client
>> over the MessageFormatter which again uses some NIO buffer. Here we have
>> the trouble that the code in
>> org.apache.http.nio.util.SharedOutputBuffer.flushContent() is waiting
>> for some notification. Who is responsible for that notification? 
>>     
>
> It expects a notification from the I/O reactor it is ready to accept
> more output.
This is because we recently enhanced the way we converted from NIO 
channels to Java streams, using the shared buffers available in 
HttpCore/NIO. In addition, we now throttle the connection, and suspend 
IO on the socket when the corresponding buffer is filled up - and 
prevents out of memory errors, and allows the transport to operate with 
constant memory.
>> The
>> HttpCoreNIOSender.reactor? I couldn't find any useful API-doc on this.
>> I'm also looking for some httpcore-nio-4.0-beta1-sources.jar from any
>> Maven Repo to attach to my synapse-IDE project.
>>
>> Does anybody know something about possible causes for not getting such a
>> notification?
>>     
I was able to reproduce a 'simulated' case of what you experienced, and 
this was caused by a connection close while the worker thread was 
writing data and waiting on the flushContent() for a large response. 
Once you confirm the temporary fix I provided you, I will check it into 
the trunk with a JIRA so that it goes into the next release

asankha

-- 
Asankha C. Perera

WSO2 - http://wso2.org
http://esbmagic.blogspot.com