You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Emmanuel Lécharny <el...@gmail.com> on 2013/03/02 09:43:15 UTC
Re: Interesting post about AIO
Le 3/2/13 9:10 AM, Julien Vermillard a écrit :
> Hi,
> Agreeing with jeff, you can't complain of the lack of an accessor, when you
> just voided the attachment ;)
Again, this is not the interesting part n this article...
>
> I really wonder what JDK guys have in mind with AIO, it's far of being a
> slick API, it doesn't support UDP (you need to mix AIO and NIO if you want
> to support TCP and UDP) and I can't see how it could be a performance gain ?
Java AIO implementation is supposed to benefit from OS taht has a
specific way to deal with AIO (through completion queues, like what is
done on Windows, some Linux versions and probably other OS).
This page is providing some information :
http://www.kegel.com/c10k.html#aio
Another very complete description :
http://stackoverflow.com/questions/4093185/whats-the-difference-between-epoll-poll-threadpool
Regarding AIO/ePoll on linux, an interesting paper :
https://docs.google.com/viewer?a=v&q=cache:CwPj9geCCTkJ:www.linuxsymposium.org/archives/OLS/Reprints-2002/lahaise-reprint.pdf+&hl=en&gl=fr&pid=bl&srcid=ADGEESj6bzG_kAkqH8KQGWL5Evt4yDbIEhrsaaNMD4eD7p-bhJHoAjjQeT3Ys8fXWY_dn88WY98kW0Hg2-sBIPrvFpBpniMrKK6wY07lY80x1ZT9tEX_TERnxwosL6WXjokFs_huSdX6&sig=AHIEtbRAgWiO-HR0BOjkxh3JcYetJpTXnA
>
> After few year of usage I start to think the whole java.nio and
> java.concurency are weirdly designed API.
That's probably why API like MINA are necessary :)
>
> Anyway we can't expect more performance for AIO since it's probably just an
> API built on top of NIO.
Not so sure.
> If we want more performance, we need to zero copy
> interface with the linux epoll() sys call. But I'm not sure we want to do
> that :)
AIO does no copy the data, compared to NIO. It's probably one of the
aspect that leads to better performances.
I'm still wondering if we can see any gain, but the more I read about
AIO, the more I'm convinced we need to spend some time on a prototype to
get valid numbers.
In any case, would we implement a layer on top of AIO, the upper API
should not be different, and that's were we have to be careful !
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Interesting post about AIO
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 3/2/13 1:23 PM, Jeff MAURY a écrit :
> On Sat, Mar 2, 2013 at 9:43 AM, Emmanuel Lécharny <el...@gmail.com>wrote:
>
>> Le 3/2/13 9:10 AM, Julien Vermillard a écrit :
>>> Hi,
>>> Agreeing with jeff, you can't complain of the lack of an accessor, when
>> you
>>> just voided the attachment ;)
>> Again, this is not the interesting part n this article...
>>> I really wonder what JDK guys have in mind with AIO, it's far of being a
>>> slick API, it doesn't support UDP (you need to mix AIO and NIO if you
>> want
>>> to support TCP and UDP) and I can't see how it could be a performance
>> gain ?
>>
>> Java AIO implementation is supposed to benefit from OS taht has a
>> specific way to deal with AIO (through completion queues, like what is
>> done on Windows, some Linux versions and probably other OS).
>>
> This is not yet done in the current implementation of the JDK. Maybe JDK8 ?
We have to check if JDK8 is any different.
>
>> This page is providing some information :
>> http://www.kegel.com/c10k.html#aio
>>
>> Another very complete description :
>>
>> http://stackoverflow.com/questions/4093185/whats-the-difference-between-epoll-poll-threadpool
>>
>> Regarding AIO/ePoll on linux, an interesting paper :
>>
>> https://docs.google.com/viewer?a=v&q=cache:CwPj9geCCTkJ:www.linuxsymposium.org/archives/OLS/Reprints-2002/lahaise-reprint.pdf+&hl=en&gl=fr&pid=bl&srcid=ADGEESj6bzG_kAkqH8KQGWL5Evt4yDbIEhrsaaNMD4eD7p-bhJHoAjjQeT3Ys8fXWY_dn88WY98kW0Hg2-sBIPrvFpBpniMrKK6wY07lY80x1ZT9tEX_TERnxwosL6WXjokFs_huSdX6&sig=AHIEtbRAgWiO-HR0BOjkxh3JcYetJpTXnA
>>> After few year of usage I start to think the whole java.nio and
>>> java.concurency are weirdly designed API.
>> That's probably why API like MINA are necessary :)
>>> Anyway we can't expect more performance for AIO since it's probably just
>> an
>>> API built on top of NIO.
>> Not so sure.
>>
> It's not done on top of NIO but the current implementation (JDK7) does not
> use OS based NIO but does more or less what MINA NIO does
That's not exactly true.
If you look at OpenJDK 7 code, at least on Windows, AIO uses
CompletionPort (the WindowsAsynchronousChannelProvider class depends on
Iocp class).
On MacOSX, I'm not sure if it's not using a wrapper on top of NIO (seems
likely)
We can ignore Microsoft implementation, but MO, that would be a mistake...
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Interesting post about AIO
Posted by Jeff MAURY <je...@jeffmaury.com>.
On Sat, Mar 2, 2013 at 9:43 AM, Emmanuel Lécharny <el...@gmail.com>wrote:
> Le 3/2/13 9:10 AM, Julien Vermillard a écrit :
> > Hi,
> > Agreeing with jeff, you can't complain of the lack of an accessor, when
> you
> > just voided the attachment ;)
> Again, this is not the interesting part n this article...
> >
> > I really wonder what JDK guys have in mind with AIO, it's far of being a
> > slick API, it doesn't support UDP (you need to mix AIO and NIO if you
> want
> > to support TCP and UDP) and I can't see how it could be a performance
> gain ?
>
> Java AIO implementation is supposed to benefit from OS taht has a
> specific way to deal with AIO (through completion queues, like what is
> done on Windows, some Linux versions and probably other OS).
>
This is not yet done in the current implementation of the JDK. Maybe JDK8 ?
>
> This page is providing some information :
> http://www.kegel.com/c10k.html#aio
>
> Another very complete description :
>
> http://stackoverflow.com/questions/4093185/whats-the-difference-between-epoll-poll-threadpool
>
> Regarding AIO/ePoll on linux, an interesting paper :
>
> https://docs.google.com/viewer?a=v&q=cache:CwPj9geCCTkJ:www.linuxsymposium.org/archives/OLS/Reprints-2002/lahaise-reprint.pdf+&hl=en&gl=fr&pid=bl&srcid=ADGEESj6bzG_kAkqH8KQGWL5Evt4yDbIEhrsaaNMD4eD7p-bhJHoAjjQeT3Ys8fXWY_dn88WY98kW0Hg2-sBIPrvFpBpniMrKK6wY07lY80x1ZT9tEX_TERnxwosL6WXjokFs_huSdX6&sig=AHIEtbRAgWiO-HR0BOjkxh3JcYetJpTXnA
> >
> > After few year of usage I start to think the whole java.nio and
> > java.concurency are weirdly designed API.
> That's probably why API like MINA are necessary :)
> >
> > Anyway we can't expect more performance for AIO since it's probably just
> an
> > API built on top of NIO.
> Not so sure.
>
It's not done on top of NIO but the current implementation (JDK7) does not
use OS based NIO but does more or less what MINA NIO does
>
> > If we want more performance, we need to zero copy
> > interface with the linux epoll() sys call. But I'm not sure we want to do
> > that :)
> AIO does no copy the data, compared to NIO. It's probably one of the
> aspect that leads to better performances.
>
I don't see the difference between NIO and AIO on this topic as both API
use the ByteBuffer for exchanging data.
>
> I'm still wondering if we can see any gain, but the more I read about
> AIO, the more I'm convinced we need to spend some time on a prototype to
> get valid numbers.
>
I don't expect good numbers given the current implementation but at least
we've got something when the JDK implementation will be better.
>
> In any case, would we implement a layer on top of AIO, the upper API
> should not be different, and that's were we have to be careful !
>
Yes, that what I have in my POC
Jeff
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
>
--
Jeff MAURY
"Legacy code" often differs from its suggested alternative by actually
working and scaling.
- Bjarne Stroustrup
http://www.jeffmaury.com
http://riadiscuss.jeffmaury.com
http://www.twitter.com/jeffmaury
Re: Fwd: Re: Interesting post about AIO
Posted by Emmanuel Lécharny <el...@gmail.com>.
Le 3/2/13 7:01 PM, Christian Schwarz a écrit :
> Maybe this is an interesting read about AIO too. Its written by one of the
> jetty developers.
>
> http://webtide.intalio.com/2013/02/on-jdk-7-asynchronous-io/
Hmmm, this is the exact link that started this discussion :-)
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
Re: Fwd: Re: Interesting post about AIO
Posted by Christian Schwarz <ch...@gmail.com>.
Maybe this is an interesting read about AIO too. Its written by one of the
jetty developers.
http://webtide.intalio.com/2013/02/on-jdk-7-asynchronous-io/
On Sat, Mar 2, 2013 at 5:42 PM, Julien Vermillard <jv...@gmail.com>wrote:
> Thanks for the pointers Kiran I m going to read it.
> Julien
> Le 2 mars 2013 12:46, "Kiran Ayyagari" <ka...@apache.org> a écrit :
>
> > forwarding to the correct ML
> > ---------- Forwarded message ----------
> > From: "Emmanuel Lécharny" <el...@gmail.com>
> > Date: Mar 2, 2013 2:13 PM
> > Subject: Re: Interesting post about AIO
> > To: "Apache Directory Developers List" <de...@directory.apache.org>
> >
> > Le 3/2/13 9:10 AM, Julien Vermillard a écrit :
> > > Hi,
> > > Agreeing with jeff, you can't complain of the lack of an accessor, when
> > you
> > > just voided the attachment ;)
> > Again, this is not the interesting part n this article...
> > >
> > > I really wonder what JDK guys have in mind with AIO, it's far of being
> a
> > > slick API, it doesn't support UDP (you need to mix AIO and NIO if you
> > want
> > > to support TCP and UDP) and I can't see how it could be a performance
> > gain ?
> >
> > Java AIO implementation is supposed to benefit from OS taht has a
> > specific way to deal with AIO (through completion queues, like what is
> > done on Windows, some Linux versions and probably other OS).
> >
> > This page is providing some information :
> > http://www.kegel.com/c10k.html#aio
> >
> > Another very complete description :
> >
> >
> http://stackoverflow.com/questions/4093185/whats-the-difference-between-epoll-poll-threadpool
> >
> > Regarding AIO/ePoll on linux, an interesting paper :
> >
> >
> https://docs.google.com/viewer?a=v&q=cache:CwPj9geCCTkJ:www.linuxsymposium.org/archives/OLS/Reprints-2002/lahaise-reprint.pdf+&hl=en&gl=fr&pid=bl&srcid=ADGEESj6bzG_kAkqH8KQGWL5Evt4yDbIEhrsaaNMD4eD7p-bhJHoAjjQeT3Ys8fXWY_dn88WY98kW0Hg2-sBIPrvFpBpniMrKK6wY07lY80x1ZT9tEX_TERnxwosL6WXjokFs_huSdX6&sig=AHIEtbRAgWiO-HR0BOjkxh3JcYetJpTXnA
> > >
> > > After few year of usage I start to think the whole java.nio and
> > > java.concurency are weirdly designed API.
> > That's probably why API like MINA are necessary :)
> > >
> > > Anyway we can't expect more performance for AIO since it's probably
> just
> > an
> > > API built on top of NIO.
> > Not so sure.
> >
> > > If we want more performance, we need to zero copy
> > > interface with the linux epoll() sys call. But I'm not sure we want to
> do
> > > that :)
> > AIO does no copy the data, compared to NIO. It's probably one of the
> > aspect that leads to better performances.
> >
> > I'm still wondering if we can see any gain, but the more I read about
> > AIO, the more I'm convinced we need to spend some time on a prototype to
> > get valid numbers.
> >
> > In any case, would we implement a layer on top of AIO, the upper API
> > should not be different, and that's were we have to be careful !
> >
> >
> > --
> > Regards,
> > Cordialement,
> > Emmanuel Lécharny
> > www.iktek.com
> >
>
Re: Fwd: Re: Interesting post about AIO
Posted by Julien Vermillard <jv...@gmail.com>.
Thanks for the pointers Kiran I m going to read it.
Julien
Le 2 mars 2013 12:46, "Kiran Ayyagari" <ka...@apache.org> a écrit :
> forwarding to the correct ML
> ---------- Forwarded message ----------
> From: "Emmanuel Lécharny" <el...@gmail.com>
> Date: Mar 2, 2013 2:13 PM
> Subject: Re: Interesting post about AIO
> To: "Apache Directory Developers List" <de...@directory.apache.org>
>
> Le 3/2/13 9:10 AM, Julien Vermillard a écrit :
> > Hi,
> > Agreeing with jeff, you can't complain of the lack of an accessor, when
> you
> > just voided the attachment ;)
> Again, this is not the interesting part n this article...
> >
> > I really wonder what JDK guys have in mind with AIO, it's far of being a
> > slick API, it doesn't support UDP (you need to mix AIO and NIO if you
> want
> > to support TCP and UDP) and I can't see how it could be a performance
> gain ?
>
> Java AIO implementation is supposed to benefit from OS taht has a
> specific way to deal with AIO (through completion queues, like what is
> done on Windows, some Linux versions and probably other OS).
>
> This page is providing some information :
> http://www.kegel.com/c10k.html#aio
>
> Another very complete description :
>
> http://stackoverflow.com/questions/4093185/whats-the-difference-between-epoll-poll-threadpool
>
> Regarding AIO/ePoll on linux, an interesting paper :
>
> https://docs.google.com/viewer?a=v&q=cache:CwPj9geCCTkJ:www.linuxsymposium.org/archives/OLS/Reprints-2002/lahaise-reprint.pdf+&hl=en&gl=fr&pid=bl&srcid=ADGEESj6bzG_kAkqH8KQGWL5Evt4yDbIEhrsaaNMD4eD7p-bhJHoAjjQeT3Ys8fXWY_dn88WY98kW0Hg2-sBIPrvFpBpniMrKK6wY07lY80x1ZT9tEX_TERnxwosL6WXjokFs_huSdX6&sig=AHIEtbRAgWiO-HR0BOjkxh3JcYetJpTXnA
> >
> > After few year of usage I start to think the whole java.nio and
> > java.concurency are weirdly designed API.
> That's probably why API like MINA are necessary :)
> >
> > Anyway we can't expect more performance for AIO since it's probably just
> an
> > API built on top of NIO.
> Not so sure.
>
> > If we want more performance, we need to zero copy
> > interface with the linux epoll() sys call. But I'm not sure we want to do
> > that :)
> AIO does no copy the data, compared to NIO. It's probably one of the
> aspect that leads to better performances.
>
> I'm still wondering if we can see any gain, but the more I read about
> AIO, the more I'm convinced we need to spend some time on a prototype to
> get valid numbers.
>
> In any case, would we implement a layer on top of AIO, the upper API
> should not be different, and that's were we have to be careful !
>
>
> --
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
>
Fwd: Re: Interesting post about AIO
Posted by Kiran Ayyagari <ka...@apache.org>.
forwarding to the correct ML
---------- Forwarded message ----------
From: "Emmanuel Lécharny" <el...@gmail.com>
Date: Mar 2, 2013 2:13 PM
Subject: Re: Interesting post about AIO
To: "Apache Directory Developers List" <de...@directory.apache.org>
Le 3/2/13 9:10 AM, Julien Vermillard a écrit :
> Hi,
> Agreeing with jeff, you can't complain of the lack of an accessor, when
you
> just voided the attachment ;)
Again, this is not the interesting part n this article...
>
> I really wonder what JDK guys have in mind with AIO, it's far of being a
> slick API, it doesn't support UDP (you need to mix AIO and NIO if you want
> to support TCP and UDP) and I can't see how it could be a performance
gain ?
Java AIO implementation is supposed to benefit from OS taht has a
specific way to deal with AIO (through completion queues, like what is
done on Windows, some Linux versions and probably other OS).
This page is providing some information :
http://www.kegel.com/c10k.html#aio
Another very complete description :
http://stackoverflow.com/questions/4093185/whats-the-difference-between-epoll-poll-threadpool
Regarding AIO/ePoll on linux, an interesting paper :
https://docs.google.com/viewer?a=v&q=cache:CwPj9geCCTkJ:www.linuxsymposium.org/archives/OLS/Reprints-2002/lahaise-reprint.pdf+&hl=en&gl=fr&pid=bl&srcid=ADGEESj6bzG_kAkqH8KQGWL5Evt4yDbIEhrsaaNMD4eD7p-bhJHoAjjQeT3Ys8fXWY_dn88WY98kW0Hg2-sBIPrvFpBpniMrKK6wY07lY80x1ZT9tEX_TERnxwosL6WXjokFs_huSdX6&sig=AHIEtbRAgWiO-HR0BOjkxh3JcYetJpTXnA
>
> After few year of usage I start to think the whole java.nio and
> java.concurency are weirdly designed API.
That's probably why API like MINA are necessary :)
>
> Anyway we can't expect more performance for AIO since it's probably just
an
> API built on top of NIO.
Not so sure.
> If we want more performance, we need to zero copy
> interface with the linux epoll() sys call. But I'm not sure we want to do
> that :)
AIO does no copy the data, compared to NIO. It's probably one of the
aspect that leads to better performances.
I'm still wondering if we can see any gain, but the more I read about
AIO, the more I'm convinced we need to spend some time on a prototype to
get valid numbers.
In any case, would we implement a layer on top of AIO, the upper API
should not be different, and that's were we have to be careful !
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com