You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Mark <el...@gmail.com> on 2007/09/10 19:41:24 UTC
slightly OT: how to write an NIO provider
I am interested in writing an NIO provider and wondered if anyone here has
experience doing this.
Thanks!
--
..Cheers
Mark
Re: slightly OT: how to write an NIO provider
Posted by Jeanfrancois Arcand <jf...@apache.org>.
Hi Mark,
Mark wrote:
> I am looking to write a provider that performs NIO on a Unix Domain Socket.
> This would be used to communicate between JVMs on the same box and do it in
> a way that is faster than TCP.
The GlassFish project has one if you want to take a look at:
https://glassfish.dev.java.net/source/browse/glassfish/appserv-core/src/java/com/sun/enterprise/server/ss/
This implementation is used and explained here:
http://weblogs.java.net/blog/binod/archive/2005/09/lazy_initializa.html
Have fun.
-- Jeanfrancois
>
Re: slightly OT: how to write an NIO provider
Posted by Mark <el...@gmail.com>.
Interesting. Never thought about that.
Thanks for the info.
--
..Cheers
Mark
On 9/14/07, Julien Vermillard <jv...@archean.fr> wrote:
>
> On loopback you won't drop UDP packets, that's the trick ;)
>
> On Fri, 14 Sep 2007 11:49:14 -0400
> Mark <el...@gmail.com> wrote:
>
> > I think even I am getting confused. I have a group of processes on a
> > box that will always be on the same box, so I was looking for a fast
> > way for them to communicate. My options are:
> >
> > TCP over loopback
> > Unix Domain Sockets
> > Pipes
> >
> > Pipes will not fulfill my needs, so I was looking into Unix Domain
> > Sockets(UDS). Since there are no NIO providers for UDS, I was
> > interested in writing my own and that is when I posed the question to
> > the list. But if TCP over loopback is not going to be any faster
> > than UDS, then I will stick with TCP over loopback. UDP is not an
> > option because I cannot drop packets.
> >
> > Sorry for the confusion. I hope this clears things up.
> > Thank you.
> >
>
Re: slightly OT: how to write an NIO provider
Posted by Julien Vermillard <jv...@archean.fr>.
On loopback you won't drop UDP packets, that's the trick ;)
On Fri, 14 Sep 2007 11:49:14 -0400
Mark <el...@gmail.com> wrote:
> I think even I am getting confused. I have a group of processes on a
> box that will always be on the same box, so I was looking for a fast
> way for them to communicate. My options are:
>
> TCP over loopback
> Unix Domain Sockets
> Pipes
>
> Pipes will not fulfill my needs, so I was looking into Unix Domain
> Sockets(UDS). Since there are no NIO providers for UDS, I was
> interested in writing my own and that is when I posed the question to
> the list. But if TCP over loopback is not going to be any faster
> than UDS, then I will stick with TCP over loopback. UDP is not an
> option because I cannot drop packets.
>
> Sorry for the confusion. I hope this clears things up.
> Thank you.
>
Re: slightly OT: how to write an NIO provider
Posted by Mark <el...@gmail.com>.
I think even I am getting confused. I have a group of processes on a box
that will always be on the same box, so I was looking for a fast way for
them to communicate. My options are:
TCP over loopback
Unix Domain Sockets
Pipes
Pipes will not fulfill my needs, so I was looking into Unix Domain
Sockets(UDS). Since there are no NIO providers for UDS, I was interested in
writing my own and that is when I posed the question to the list. But if
TCP over loopback is not going to be any faster than UDS, then I will stick
with TCP over loopback. UDP is not an option because I cannot drop packets.
Sorry for the confusion. I hope this clears things up.
Thank you.
--
..Cheers
Mark
On 9/14/07, Maarten Bosteels <mb...@gmail.com> wrote:
>
> Mark,
>
> What do you mean by "TCP over Unix Domain Sockets" ??
> I think Unix Domain Sockets could be an alternative for TCP (or UDP) if
> you
> don't need inter-machine connectivity.
> But TCP 'over' Unix Domain Sockets ?
>
> Julien is just saying that UDS might be faster than loopback TCP, but
> probably not much faster than loopback UDP.
>
>
> http://lists.freebsd.org/pipermail/freebsd-performance/2005-February/001143.html
>
> regards,
> Maarten
>
>
>
> On 9/14/07, Mark <el...@gmail.com> wrote:
> >
> > Additionally, I want to use TCP over Unix Domain Sockets and not UDP.
> >
> > --
> > ..Cheers
> > Mark
> >
> > On 9/14/07, Mark <el...@gmail.com> wrote:
> > >
> > > I think you are confusing User Datagram Protocol with Unix Domain
> > > Sockets. UDP are the datagram packets, UDS communicates via a virtual
> > > socket represented by a file on POSIX compliant operating systems.
> > >
> > > --
> > > ..Cheers
> > > Mark
> > >
> > > On 9/14/07, Julien Vermillard <jv...@archean.fr> wrote:
> > > >
> > > > On Fri, 14 Sep 2007 09:18:50 -0400
> > > > Mark <el...@gmail.com> wrote:
> > > >
> > > > > Domain
> > > > > Sockets are no faster than TCP over loopback I would love to see
> > > > > them. I would hate to spend much more time on this project only
> to
> > > > > find out that TCP would work just as well.
> > > > >
> > > > > Thanks again!
> > > > >
> > > > > --
> > > >
> > > > Unix domain sockets are supposed to be faster due to the fact it's
> not
> > > > passing thru the operating system TCP/IP stack (ex : ACK provoquing
> > > > context switching) but using UDP I'm pretty sure the overhead is
> small
> > > > (perhaps the CRC?). It prolly worth a benchmark ;)
> > > >
> > > > Julien
> > > >
> > >
> > >
> >
>
Re: slightly OT: how to write an NIO provider
Posted by Maarten Bosteels <mb...@gmail.com>.
Mark,
What do you mean by "TCP over Unix Domain Sockets" ??
I think Unix Domain Sockets could be an alternative for TCP (or UDP) if you
don't need inter-machine connectivity.
But TCP 'over' Unix Domain Sockets ?
Julien is just saying that UDS might be faster than loopback TCP, but
probably not much faster than loopback UDP.
http://lists.freebsd.org/pipermail/freebsd-performance/2005-February/001143.html
regards,
Maarten
On 9/14/07, Mark <el...@gmail.com> wrote:
>
> Additionally, I want to use TCP over Unix Domain Sockets and not UDP.
>
> --
> ..Cheers
> Mark
>
> On 9/14/07, Mark <el...@gmail.com> wrote:
> >
> > I think you are confusing User Datagram Protocol with Unix Domain
> > Sockets. UDP are the datagram packets, UDS communicates via a virtual
> > socket represented by a file on POSIX compliant operating systems.
> >
> > --
> > ..Cheers
> > Mark
> >
> > On 9/14/07, Julien Vermillard <jv...@archean.fr> wrote:
> > >
> > > On Fri, 14 Sep 2007 09:18:50 -0400
> > > Mark <el...@gmail.com> wrote:
> > >
> > > > Domain
> > > > Sockets are no faster than TCP over loopback I would love to see
> > > > them. I would hate to spend much more time on this project only to
> > > > find out that TCP would work just as well.
> > > >
> > > > Thanks again!
> > > >
> > > > --
> > >
> > > Unix domain sockets are supposed to be faster due to the fact it's not
> > > passing thru the operating system TCP/IP stack (ex : ACK provoquing
> > > context switching) but using UDP I'm pretty sure the overhead is small
> > > (perhaps the CRC?). It prolly worth a benchmark ;)
> > >
> > > Julien
> > >
> >
> >
>
Re: slightly OT: how to write an NIO provider
Posted by Mark <el...@gmail.com>.
Additionally, I want to use TCP over Unix Domain Sockets and not UDP.
--
..Cheers
Mark
On 9/14/07, Mark <el...@gmail.com> wrote:
>
> I think you are confusing User Datagram Protocol with Unix Domain
> Sockets. UDP are the datagram packets, UDS communicates via a virtual
> socket represented by a file on POSIX compliant operating systems.
>
> --
> ..Cheers
> Mark
>
> On 9/14/07, Julien Vermillard <jv...@archean.fr> wrote:
> >
> > On Fri, 14 Sep 2007 09:18:50 -0400
> > Mark <el...@gmail.com> wrote:
> >
> > > Domain
> > > Sockets are no faster than TCP over loopback I would love to see
> > > them. I would hate to spend much more time on this project only to
> > > find out that TCP would work just as well.
> > >
> > > Thanks again!
> > >
> > > --
> >
> > Unix domain sockets are supposed to be faster due to the fact it's not
> > passing thru the operating system TCP/IP stack (ex : ACK provoquing
> > context switching) but using UDP I'm pretty sure the overhead is small
> > (perhaps the CRC?). It prolly worth a benchmark ;)
> >
> > Julien
> >
>
>
Re: slightly OT: how to write an NIO provider
Posted by Mark <el...@gmail.com>.
I think you are confusing User Datagram Protocol with Unix Domain Sockets.
UDP are the datagram packets, UDS communicates via a virtual socket
represented by a file on POSIX compliant operating systems.
--
..Cheers
Mark
On 9/14/07, Julien Vermillard <jv...@archean.fr> wrote:
>
> On Fri, 14 Sep 2007 09:18:50 -0400
> Mark <el...@gmail.com> wrote:
>
> > Domain
> > Sockets are no faster than TCP over loopback I would love to see
> > them. I would hate to spend much more time on this project only to
> > find out that TCP would work just as well.
> >
> > Thanks again!
> >
> > --
>
> Unix domain sockets are supposed to be faster due to the fact it's not
> passing thru the operating system TCP/IP stack (ex : ACK provoquing
> context switching) but using UDP I'm pretty sure the overhead is small
> (perhaps the CRC?). It prolly worth a benchmark ;)
>
> Julien
>
Re: slightly OT: how to write an NIO provider
Posted by Julien Vermillard <jv...@archean.fr>.
On Fri, 14 Sep 2007 09:18:50 -0400
Mark <el...@gmail.com> wrote:
> Domain
> Sockets are no faster than TCP over loopback I would love to see
> them. I would hate to spend much more time on this project only to
> find out that TCP would work just as well.
>
> Thanks again!
>
> --
Unix domain sockets are supposed to be faster due to the fact it's not
passing thru the operating system TCP/IP stack (ex : ACK provoquing
context switching) but using UDP I'm pretty sure the overhead is small
(perhaps the CRC?). It prolly worth a benchmark ;)
Julien
Re: slightly OT: how to write an NIO provider
Posted by Mark <el...@gmail.com>.
Oh, and if you have any resources that support the notion that Unix Domain
Sockets are no faster than TCP over loopback I would love to see them. I
would hate to spend much more time on this project only to find out that TCP
would work just as well.
Thanks again!
--
..Cheers
Mark
On 9/14/07, Julien Vermillard <jv...@archean.fr> wrote:
>
> On Thu, 13 Sep 2007 13:02:25 -0400
> Mark <el...@gmail.com> wrote:
>
> > I am looking to write a provider that performs NIO on a Unix Domain
> > Socket. This would be used to communicate between JVMs on the same
> > box and do it in a way that is faster than TCP.
> >
>
> Hi Mark,
>
> My APR based connector is starting purring. TCP & UDP are supported
> (I think SCTP too but I made no test). Perhaps I can force/modify APR
> for supporting Unix domain socket. Unix socket can be very usefull too
> for communication between a C prog and a Java one, but I'm not sure
> nowadays loopback UDP is really slower than Unix sockets.
>
> Julien
>
Re: slightly OT: how to write an NIO provider
Posted by Mark <el...@gmail.com>.
sounds great!
I will keep my eye on this project for future updates and will revisit this
soon.
--
..Cheers
Mark
On 9/19/07, Julien Vermillard <jv...@archean.fr> wrote:
>
> Hi,
>
> I just committed it, it's compiling, a bit working, but it's not really
> for production usage (or any usage actually :D).Trustin is going to
> refactor AbstractIoProcessor
>
> The APR goal is to be portable on Unix and non Unix OSes, so there is
> no Unix Domain socket :(
>
> Perhaps a great idea whould be a to produce IOPrcessor working with a
> homemade library using epoll for achieving great perfs on linux for
> TCP, UDP, files, serial, Unix Domain Socket, Audio, pipes , whatever
> I/O using a file descriptor :)
>
> Or if APR perfs are goods, adding Unix Domain support to APR.
>
> Julien
>
> On Wed, 19 Sep 2007 10:51:31
> -0400 Mark <el...@gmail.com> wrote:
>
> > Julien,
> >
> > I just saw that the APR code you wrote is in the trunk. Let me know
> > if you want to try and get it to support Unix Domain Sockets if it
> > does not already.
> >
> > Thanks.
> >
> >
> > On 9/14/07, Julien Vermillard <jv...@archean.fr> wrote:
> > >
> > > On Thu, 13 Sep 2007 13:02:25 -0400
> > > Mark <el...@gmail.com> wrote:
> > >
> > > > I am looking to write a provider that performs NIO on a Unix
> > > > Domain Socket. This would be used to communicate between JVMs on
> > > > the same box and do it in a way that is faster than TCP.
> > > >
> > >
> > > Hi Mark,
> > >
> > > My APR based connector is starting purring. TCP & UDP are supported
> > > (I think SCTP too but I made no test). Perhaps I can force/modify
> > > APR for supporting Unix domain socket. Unix socket can be very
> > > usefull too for communication between a C prog and a Java one, but
> > > I'm not sure nowadays loopback UDP is really slower than Unix
> > > sockets.
> > >
> > > Julien
> > >
>
Re: slightly OT: how to write an NIO provider
Posted by Julien Vermillard <jv...@archean.fr>.
Hi,
I just committed it, it's compiling, a bit working, but it's not really
for production usage (or any usage actually :D).Trustin is going to
refactor AbstractIoProcessor
The APR goal is to be portable on Unix and non Unix OSes, so there is
no Unix Domain socket :(
Perhaps a great idea whould be a to produce IOPrcessor working with a
homemade library using epoll for achieving great perfs on linux for
TCP, UDP, files, serial, Unix Domain Socket, Audio, pipes , whatever
I/O using a file descriptor :)
Or if APR perfs are goods, adding Unix Domain support to APR.
Julien
On Wed, 19 Sep 2007 10:51:31
-0400 Mark <el...@gmail.com> wrote:
> Julien,
>
> I just saw that the APR code you wrote is in the trunk. Let me know
> if you want to try and get it to support Unix Domain Sockets if it
> does not already.
>
> Thanks.
>
>
> On 9/14/07, Julien Vermillard <jv...@archean.fr> wrote:
> >
> > On Thu, 13 Sep 2007 13:02:25 -0400
> > Mark <el...@gmail.com> wrote:
> >
> > > I am looking to write a provider that performs NIO on a Unix
> > > Domain Socket. This would be used to communicate between JVMs on
> > > the same box and do it in a way that is faster than TCP.
> > >
> >
> > Hi Mark,
> >
> > My APR based connector is starting purring. TCP & UDP are supported
> > (I think SCTP too but I made no test). Perhaps I can force/modify
> > APR for supporting Unix domain socket. Unix socket can be very
> > usefull too for communication between a C prog and a Java one, but
> > I'm not sure nowadays loopback UDP is really slower than Unix
> > sockets.
> >
> > Julien
> >
Re: slightly OT: how to write an NIO provider
Posted by Mark <el...@gmail.com>.
Julien,
I just saw that the APR code you wrote is in the trunk. Let me know if you
want to try and get it to support Unix Domain Sockets if it does not
already.
Thanks.
On 9/14/07, Julien Vermillard <jv...@archean.fr> wrote:
>
> On Thu, 13 Sep 2007 13:02:25 -0400
> Mark <el...@gmail.com> wrote:
>
> > I am looking to write a provider that performs NIO on a Unix Domain
> > Socket. This would be used to communicate between JVMs on the same
> > box and do it in a way that is faster than TCP.
> >
>
> Hi Mark,
>
> My APR based connector is starting purring. TCP & UDP are supported
> (I think SCTP too but I made no test). Perhaps I can force/modify APR
> for supporting Unix domain socket. Unix socket can be very usefull too
> for communication between a C prog and a Java one, but I'm not sure
> nowadays loopback UDP is really slower than Unix sockets.
>
> Julien
>
Re: slightly OT: how to write an NIO provider
Posted by Mark <el...@gmail.com>.
Thanks for the information. If you would like any help, please let me know
as I have been prototyping all different variations of UnixDomainSockets and
Java.
--
..Cheers
Mark
On 9/14/07, Julien Vermillard <jv...@archean.fr> wrote:
>
> On Thu, 13 Sep 2007 13:02:25 -0400
> Mark <el...@gmail.com> wrote:
>
> > I am looking to write a provider that performs NIO on a Unix Domain
> > Socket. This would be used to communicate between JVMs on the same
> > box and do it in a way that is faster than TCP.
> >
>
> Hi Mark,
>
> My APR based connector is starting purring. TCP & UDP are supported
> (I think SCTP too but I made no test). Perhaps I can force/modify APR
> for supporting Unix domain socket. Unix socket can be very usefull too
> for communication between a C prog and a Java one, but I'm not sure
> nowadays loopback UDP is really slower than Unix sockets.
>
> Julien
>
Re: slightly OT: how to write an NIO provider
Posted by Julien Vermillard <jv...@archean.fr>.
On Thu, 13 Sep 2007 13:02:25 -0400
Mark <el...@gmail.com> wrote:
> I am looking to write a provider that performs NIO on a Unix Domain
> Socket. This would be used to communicate between JVMs on the same
> box and do it in a way that is faster than TCP.
>
Hi Mark,
My APR based connector is starting purring. TCP & UDP are supported
(I think SCTP too but I made no test). Perhaps I can force/modify APR
for supporting Unix domain socket. Unix socket can be very usefull too
for communication between a C prog and a Java one, but I'm not sure
nowadays loopback UDP is really slower than Unix sockets.
Julien
Re: slightly OT: how to write an NIO provider
Posted by Mark <el...@gmail.com>.
I am looking to write a provider that performs NIO on a Unix Domain Socket.
This would be used to communicate between JVMs on the same box and do it in
a way that is faster than TCP.
--
..Cheers
Mark
On 9/13/07, mat <fo...@gmail.com> wrote:
>
> What does that mean? Mina's NIO performance is not good enough?
>
> On 9/13/07, Trustin Lee <tr...@gmail.com> wrote:
> >
> > On 9/11/07, Mark <el...@gmail.com> wrote:
> > > I am interested in writing an NIO provider and wondered if anyone here
> > has
> > > experience doing this.
> >
> > It would be great if we can provide an alternative NIO provider for
> > better performance, aside from Julien's APR transport effort. I'd
> > love to see what is realized sooner. :D
> >
> > Trustin
> > --
> > what we call human nature is actually human habit
> > --
> > http://gleamynode.net/
> > --
> > PGP Key ID: 0x0255ECA6
> >
>
Re: slightly OT: how to write an NIO provider
Posted by mat <fo...@gmail.com>.
What does that mean? Mina's NIO performance is not good enough?
On 9/13/07, Trustin Lee <tr...@gmail.com> wrote:
>
> On 9/11/07, Mark <el...@gmail.com> wrote:
> > I am interested in writing an NIO provider and wondered if anyone here
> has
> > experience doing this.
>
> It would be great if we can provide an alternative NIO provider for
> better performance, aside from Julien's APR transport effort. I'd
> love to see what is realized sooner. :D
>
> Trustin
> --
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
> --
> PGP Key ID: 0x0255ECA6
>
Re: slightly OT: how to write an NIO provider
Posted by Trustin Lee <tr...@gmail.com>.
On 9/11/07, Mark <el...@gmail.com> wrote:
> I am interested in writing an NIO provider and wondered if anyone here has
> experience doing this.
It would be great if we can provide an alternative NIO provider for
better performance, aside from Julien's APR transport effort. I'd
love to see what is realized sooner. :D
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6