TCP socket transport

[Bob Arthur <cr...@gmail.com>]

Hi,

We are currently investigating options for migrating our existing
software solution towards an ESB, with a particular emphasis on
asynchronous IO.

Our system is essentially a switching router, accepting requests over
HTTP(S), and routing them to an appropriate service-provider.
Communication to these third-party providers is via TCP socket
request/response.

We are looking to maximise concurrency; by far the bulk of our
round-trip time is currently spent waiting for responses from these
3rd parties, so we are keen to replace our current process-per-child
model with one based on an asynchronous core, and Synapse is one of
the frameworks being considered.

However, my initial investigations suggests that there is not
currently a plain old TCP socket transport available.  Can somebody
please confirm/deny this?

Many thanks,

Bob.

Re: TCP socket transport

[Rajith Attapattu <ra...@gmail.com>]

On Tue, May 27, 2008 at 3:29 PM, Andreas Veithen <an...@skynet.be>
wrote:

>
> On 27 May 2008, at 20:27, Asankha C. Perera wrote:
>
>  Hi Bob
>>
>>> We are currently investigating options for migrating our existing
>>> software solution towards an ESB, with a particular emphasis on
>>> asynchronous IO.
>>>
>>> Our system is essentially a switching router, accepting requests over
>>> HTTP(S), and routing them to an appropriate service-provider.
>>> Communication to these third-party providers is via TCP socket
>>> request/response.
>>>
>>> We are looking to maximise concurrency; by far the bulk of our
>>> round-trip time is currently spent waiting for responses from these
>>> 3rd parties, so we are keen to replace our current process-per-child
>>> model with one based on an asynchronous core, and Synapse is one of
>>> the frameworks being considered.
>>>
>>>  Synapse already has a non blocking http/s implementation using Apache
>> HttpComponents-HttpCore/NIO, which gives it great performance.
>>
>>> However, my initial investigations suggests that there is not
>>> currently a plain old TCP socket transport available.  Can somebody
>>> please confirm/deny this?
>>>
>>>  There was an initial raw TCP transport available with Apache Axis2, but
>> I have seen Andreas committing a better TCP/UDP implementation using NIO
>> being checked into the Synapse trunk in the last few weeks. Thus, the
>> combination of these two non-blocking transports will get you what you are
>> looking for..
>>
>>
> This is more credit than I deserve :-)
>
> Actually I added two new transports: one for UDP and one to read from a
> UNIX pipe (aka named pipe, aka FIFO). The UDP transport indeed uses a non
> blocking approach, but it is a datagram transport in the sense that it reads
> the entire message into memory before starting to process it. The pipe
> transport uses file I/O and is blocking (there is no non blocking file I/O
> in Java). For various reasons it is actually also built as a datagram
> transport, i.e. it doesn't support streaming. Currently the only TCP
> transport implementation is the one from Axis2. Neither the UDP and nor the
> pipe transport implementation is useful as a starting point to build a TCP
> transport. Indeed, a good TCP transport should be non blocking and support
> streaming. Implementing this is a non trivial task as we learned from the
> problems with the HTTP transport (SYNAPSE-321).
>
> Andreas
>
> Writing such a transport is indeed a non trivial excercise. There are
frameworks such as MINA, Grizzly for nio.
For Apache Qpid we used MINA but we are reevaluating it.
I did quite a bit of perf testing on the client side with MINA and see that
memory is definitely an issue. Balancing throughput vs latency was
difficult.
I finally wrote a blocking transport on the client side (using java.io )
that gave much much better latency, throughput and memory.
However this model is only good where you can devote a thread per
connection. For a client this usually is the case, but not for a server in
general.

Even for a server a blocking transport may not be an issue if the number of
connections are low and you have enough CPU.
In such a case the overhead of context switching (if any) will be less than
the overhead of select/poll or epoll system calls you get with non blocking
io multiplexing with java.nio.
However for a server with a large no of connections non blocking is the way
to go.

There is also something called async io written by Tim (Apache Harmony) and
Mike (SCA). I think it didnt' go much further, but they had some interesting
points in some articles. From the looks of it they even had asynchronous
write operations and completion was notified via a callback.
This was written after nio came in to place and I wonder why they abandoned
the project
http://www.alphaworks.ibm.com/tech/aio4j

Anyways I am keen on this area as Qpid can benefit from what you guys
discover (and vice versa).
Andreas I am looking forward to the work you are doing in this space.

Regards,

Rajith Attapattu
Red Hat
http://rajith.2rlabs.com/

Re: TCP socket transport

[Paul Fremantle <pz...@gmail.com>]

Yes that was exactly what I was asking.

I guess I'm thinking maybe the right thing to do here is to write a
"sample" transport that deals with the heavy lifting and can then be
modified to handle the specifics of different endpoints as described
below. That is, unless you believe that these settings and different
approaches can all be captured in config information in some more
general way?

Paul

On Wed, May 28, 2008 at 3:35 PM, Bob Arthur <cr...@gmail.com> wrote:
> On Wed, May 28, 2008 at 3:08 PM, Paul Fremantle <pz...@gmail.com> wrote:
>> Bob
>>
>> Can I ask if there is any framing going on with your TCP transport?
>> And what is flowing over it? It certainly would make sense for us to
>> have a better non-blocking/streaming TCP transport and it would help
>> if we could collect the requirements.
>
> Hi,
>
> I'm not 100% sure I follow the question - would I be right in thinking
> that by "framing" you are asking how we recognise that a complete
> message has been sent/received?  Unfortunately, as a gateway providing
> access to a number of 3rd-party services, this is variable across our
> service base; in some cases messages are length-prefixed, in others we
> perform a half-close after writing and read until the connection is
> fully closed, while in others there are "end-of-data" marker
> characters.
>
> All of which is, of course, not very helpful from the point of view of
> adding support to a general-purpose framework!
>
> Cheers,
>
> Bob.
>
>>
>> Paul
>>
>> On Wed, May 28, 2008 at 9:49 AM, Bob Arthur <cr...@gmail.com> wrote:
>>> On Wed, May 28, 2008 at 3:19 AM, Asankha C. Perera <as...@wso2.com> wrote:
>>>> Hi Bob / Andreas
>>>>>
>>>>> Actually I added two new transports: one for UDP and one to read from a
>>>>> UNIX pipe (aka named pipe, aka FIFO). The UDP transport indeed uses a non
>>>>> blocking approach, but it is a datagram transport in the sense that it reads
>>>>> the entire message into memory before starting to process it. The pipe
>>>>> transport uses file I/O and is blocking (there is no non blocking file I/O
>>>>> in Java). For various reasons it is actually also built as a datagram
>>>>> transport, i.e. it doesn't support streaming. Currently the only TCP
>>>>> transport implementation is the one from Axis2. Neither the UDP and nor the
>>>>> pipe transport implementation is useful as a starting point to build a TCP
>>>>> transport. Indeed, a good TCP transport should be non blocking and support
>>>>> streaming. Implementing this is a non trivial task as we learned from the
>>>>> problems with the HTTP transport (SYNAPSE-321).
>>>>
>>>> Andreas, I am sorry I was not following up in detail the changes on the
>>>> trunk due to the work on the release lately :)..
>>>>
>>>> Bob, if you are willing to support the development of a new non-blocking TCP
>>>> transport, we could understand your requirements/expectations and look into
>>>> the possibilities for implementation.
>>>
>>> Hi,
>>>
>>> Thanks for the quick responses on this; currently we are still in the
>>> process of setting out the precise goals of the project, so it would
>>> be a bit premature for us to commit to such support.  However, this
>>> has certainly given us some food for thought.
>>>
>>> Cheers,
>>>
>>> Bob.
>>>
>>
>>
>>
>> --
>> Paul Fremantle
>> Co-Founder and CTO, WSO2
>> Apache Synapse PMC Chair
>> OASIS WS-RX TC Co-chair
>>
>> blog: http://pzf.fremantle.org
>> paul@wso2.com
>>
>> "Oxygenating the Web Service Platform", www.wso2.com
>>
>



-- 
Paul Fremantle
Co-Founder and CTO, WSO2
Apache Synapse PMC Chair
OASIS WS-RX TC Co-chair

blog: http://pzf.fremantle.org
paul@wso2.com

"Oxygenating the Web Service Platform", www.wso2.com

Re: TCP socket transport

[Bob Arthur <cr...@gmail.com>]

On Wed, May 28, 2008 at 3:08 PM, Paul Fremantle <pz...@gmail.com> wrote:
> Bob
>
> Can I ask if there is any framing going on with your TCP transport?
> And what is flowing over it? It certainly would make sense for us to
> have a better non-blocking/streaming TCP transport and it would help
> if we could collect the requirements.

Hi,

I'm not 100% sure I follow the question - would I be right in thinking
that by "framing" you are asking how we recognise that a complete
message has been sent/received?  Unfortunately, as a gateway providing
access to a number of 3rd-party services, this is variable across our
service base; in some cases messages are length-prefixed, in others we
perform a half-close after writing and read until the connection is
fully closed, while in others there are "end-of-data" marker
characters.

All of which is, of course, not very helpful from the point of view of
adding support to a general-purpose framework!

Cheers,

Bob.

>
> Paul
>
> On Wed, May 28, 2008 at 9:49 AM, Bob Arthur <cr...@gmail.com> wrote:
>> On Wed, May 28, 2008 at 3:19 AM, Asankha C. Perera <as...@wso2.com> wrote:
>>> Hi Bob / Andreas
>>>>
>>>> Actually I added two new transports: one for UDP and one to read from a
>>>> UNIX pipe (aka named pipe, aka FIFO). The UDP transport indeed uses a non
>>>> blocking approach, but it is a datagram transport in the sense that it reads
>>>> the entire message into memory before starting to process it. The pipe
>>>> transport uses file I/O and is blocking (there is no non blocking file I/O
>>>> in Java). For various reasons it is actually also built as a datagram
>>>> transport, i.e. it doesn't support streaming. Currently the only TCP
>>>> transport implementation is the one from Axis2. Neither the UDP and nor the
>>>> pipe transport implementation is useful as a starting point to build a TCP
>>>> transport. Indeed, a good TCP transport should be non blocking and support
>>>> streaming. Implementing this is a non trivial task as we learned from the
>>>> problems with the HTTP transport (SYNAPSE-321).
>>>
>>> Andreas, I am sorry I was not following up in detail the changes on the
>>> trunk due to the work on the release lately :)..
>>>
>>> Bob, if you are willing to support the development of a new non-blocking TCP
>>> transport, we could understand your requirements/expectations and look into
>>> the possibilities for implementation.
>>
>> Hi,
>>
>> Thanks for the quick responses on this; currently we are still in the
>> process of setting out the precise goals of the project, so it would
>> be a bit premature for us to commit to such support.  However, this
>> has certainly given us some food for thought.
>>
>> Cheers,
>>
>> Bob.
>>
>
>
>
> --
> Paul Fremantle
> Co-Founder and CTO, WSO2
> Apache Synapse PMC Chair
> OASIS WS-RX TC Co-chair
>
> blog: http://pzf.fremantle.org
> paul@wso2.com
>
> "Oxygenating the Web Service Platform", www.wso2.com
>

Re: TCP socket transport

[Paul Fremantle <pz...@gmail.com>]

Bob

Can I ask if there is any framing going on with your TCP transport?
And what is flowing over it? It certainly would make sense for us to
have a better non-blocking/streaming TCP transport and it would help
if we could collect the requirements.

Paul

On Wed, May 28, 2008 at 9:49 AM, Bob Arthur <cr...@gmail.com> wrote:
> On Wed, May 28, 2008 at 3:19 AM, Asankha C. Perera <as...@wso2.com> wrote:
>> Hi Bob / Andreas
>>>
>>> Actually I added two new transports: one for UDP and one to read from a
>>> UNIX pipe (aka named pipe, aka FIFO). The UDP transport indeed uses a non
>>> blocking approach, but it is a datagram transport in the sense that it reads
>>> the entire message into memory before starting to process it. The pipe
>>> transport uses file I/O and is blocking (there is no non blocking file I/O
>>> in Java). For various reasons it is actually also built as a datagram
>>> transport, i.e. it doesn't support streaming. Currently the only TCP
>>> transport implementation is the one from Axis2. Neither the UDP and nor the
>>> pipe transport implementation is useful as a starting point to build a TCP
>>> transport. Indeed, a good TCP transport should be non blocking and support
>>> streaming. Implementing this is a non trivial task as we learned from the
>>> problems with the HTTP transport (SYNAPSE-321).
>>
>> Andreas, I am sorry I was not following up in detail the changes on the
>> trunk due to the work on the release lately :)..
>>
>> Bob, if you are willing to support the development of a new non-blocking TCP
>> transport, we could understand your requirements/expectations and look into
>> the possibilities for implementation.
>
> Hi,
>
> Thanks for the quick responses on this; currently we are still in the
> process of setting out the precise goals of the project, so it would
> be a bit premature for us to commit to such support.  However, this
> has certainly given us some food for thought.
>
> Cheers,
>
> Bob.
>



-- 
Paul Fremantle
Co-Founder and CTO, WSO2
Apache Synapse PMC Chair
OASIS WS-RX TC Co-chair

blog: http://pzf.fremantle.org
paul@wso2.com

"Oxygenating the Web Service Platform", www.wso2.com

Re: TCP socket transport

[Bob Arthur <cr...@gmail.com>]

On Wed, May 28, 2008 at 3:19 AM, Asankha C. Perera <as...@wso2.com> wrote:
> Hi Bob / Andreas
>>
>> Actually I added two new transports: one for UDP and one to read from a
>> UNIX pipe (aka named pipe, aka FIFO). The UDP transport indeed uses a non
>> blocking approach, but it is a datagram transport in the sense that it reads
>> the entire message into memory before starting to process it. The pipe
>> transport uses file I/O and is blocking (there is no non blocking file I/O
>> in Java). For various reasons it is actually also built as a datagram
>> transport, i.e. it doesn't support streaming. Currently the only TCP
>> transport implementation is the one from Axis2. Neither the UDP and nor the
>> pipe transport implementation is useful as a starting point to build a TCP
>> transport. Indeed, a good TCP transport should be non blocking and support
>> streaming. Implementing this is a non trivial task as we learned from the
>> problems with the HTTP transport (SYNAPSE-321).
>
> Andreas, I am sorry I was not following up in detail the changes on the
> trunk due to the work on the release lately :)..
>
> Bob, if you are willing to support the development of a new non-blocking TCP
> transport, we could understand your requirements/expectations and look into
> the possibilities for implementation.

Hi,

Thanks for the quick responses on this; currently we are still in the
process of setting out the precise goals of the project, so it would
be a bit premature for us to commit to such support.  However, this
has certainly given us some food for thought.

Cheers,

Bob.

Re: TCP socket transport

["Asankha C. Perera" <as...@wso2.com>]

Hi Bob / Andreas
> Actually I added two new transports: one for UDP and one to read from 
> a UNIX pipe (aka named pipe, aka FIFO). The UDP transport indeed uses 
> a non blocking approach, but it is a datagram transport in the sense 
> that it reads the entire message into memory before starting to 
> process it. The pipe transport uses file I/O and is blocking (there is 
> no non blocking file I/O in Java). For various reasons it is actually 
> also built as a datagram transport, i.e. it doesn't support streaming. 
> Currently the only TCP transport implementation is the one from Axis2. 
> Neither the UDP and nor the pipe transport implementation is useful as 
> a starting point to build a TCP transport. Indeed, a good TCP 
> transport should be non blocking and support streaming. Implementing 
> this is a non trivial task as we learned from the problems with the 
> HTTP transport (SYNAPSE-321).
Andreas, I am sorry I was not following up in detail the changes on the 
trunk due to the work on the release lately :)..

Bob, if you are willing to support the development of a new non-blocking 
TCP transport, we could understand your requirements/expectations and 
look into the possibilities for implementation.

asankha

Re: TCP socket transport

[Andreas Veithen <an...@skynet.be>]

On 27 May 2008, at 20:27, Asankha C. Perera wrote:

> Hi Bob
>> We are currently investigating options for migrating our existing
>> software solution towards an ESB, with a particular emphasis on
>> asynchronous IO.
>>
>> Our system is essentially a switching router, accepting requests over
>> HTTP(S), and routing them to an appropriate service-provider.
>> Communication to these third-party providers is via TCP socket
>> request/response.
>>
>> We are looking to maximise concurrency; by far the bulk of our
>> round-trip time is currently spent waiting for responses from these
>> 3rd parties, so we are keen to replace our current process-per-child
>> model with one based on an asynchronous core, and Synapse is one of
>> the frameworks being considered.
>>
> Synapse already has a non blocking http/s implementation using  
> Apache HttpComponents-HttpCore/NIO, which gives it great performance.
>> However, my initial investigations suggests that there is not
>> currently a plain old TCP socket transport available.  Can somebody
>> please confirm/deny this?
>>
> There was an initial raw TCP transport available with Apache Axis2,  
> but I have seen Andreas committing a better TCP/UDP implementation  
> using NIO being checked into the Synapse trunk in the last few  
> weeks. Thus, the combination of these two non-blocking transports  
> will get you what you are looking for..
>

This is more credit than I deserve :-)

Actually I added two new transports: one for UDP and one to read from  
a UNIX pipe (aka named pipe, aka FIFO). The UDP transport indeed uses  
a non blocking approach, but it is a datagram transport in the sense  
that it reads the entire message into memory before starting to  
process it. The pipe transport uses file I/O and is blocking (there is  
no non blocking file I/O in Java). For various reasons it is actually  
also built as a datagram transport, i.e. it doesn't support streaming.  
Currently the only TCP transport implementation is the one from Axis2.  
Neither the UDP and nor the pipe transport implementation is useful as  
a starting point to build a TCP transport. Indeed, a good TCP  
transport should be non blocking and support streaming. Implementing  
this is a non trivial task as we learned from the problems with the  
HTTP transport (SYNAPSE-321).

Andreas

Re: TCP socket transport

["Asankha C. Perera" <as...@wso2.com>]

Hi Bob
> We are currently investigating options for migrating our existing
> software solution towards an ESB, with a particular emphasis on
> asynchronous IO.
>
> Our system is essentially a switching router, accepting requests over
> HTTP(S), and routing them to an appropriate service-provider.
> Communication to these third-party providers is via TCP socket
> request/response.
>
> We are looking to maximise concurrency; by far the bulk of our
> round-trip time is currently spent waiting for responses from these
> 3rd parties, so we are keen to replace our current process-per-child
> model with one based on an asynchronous core, and Synapse is one of
> the frameworks being considered.
>   
Synapse already has a non blocking http/s implementation using Apache 
HttpComponents-HttpCore/NIO, which gives it great performance.
> However, my initial investigations suggests that there is not
> currently a plain old TCP socket transport available.  Can somebody
> please confirm/deny this?
>   
There was an initial raw TCP transport available with Apache Axis2, but 
I have seen Andreas committing a better TCP/UDP implementation using NIO 
being checked into the Synapse trunk in the last few weeks. Thus, the 
combination of these two non-blocking transports will get you what you 
are looking for..

asankha