You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@directory.apache.org by Wes McKean <wm...@logictrends.com> on 2003/11/25 13:56:33 UTC
Serverside NIO: Not ready for Prime Time
Hi Folks!
I was running ldapd on a Windows XP box with the new server side NIO stuff
included, and I was getting all kinds of weird messages inside the sun
implementation classes. The same exact code works fine under Linux. It is
my professional opinion that there are some issues with the Windows XP
implementation of non-blocking sockets, and we couldn't put this stuff into
the server until these issues are addressed.
I fully expect the issues to be my fault :-)
Wes
Re: Serverside NIO: Not ready for Prime Time
Posted by Peter Donald <pe...@realityforge.org>.
On Wed, 26 Nov 2003 12:37 pm, Geir Magnusson Jr wrote:
> Ye gads. Why can't you multithread the reads? What possible reason
> could one come up with that the reads have to happen on the same thread
> as the select?
Because it doesn't work ;)
I could offer a reasonable explanation of why that is so but ultimately it
comes down to the fact that the underlying OSes are inconsistent -
particularly when there is multiple CPUs. The java layers on top proably
introduce more ways in which sligt differences can occur.
Consider the scenario
CPU1 returns Select with FD3
> > See attached files for the sort of thing I mean. Associated with each
> > connection object I have a TcpTransport object. The selector threads
> > does all
> > the nbio and then sends event to "parsing" stage. This seems to work
> > like a
> > charm.
>
> Don't that serialize the I/O?
Not sure what you mean. It is a little bit "unfair" because usually the
SelectorKeys are returned in order of underlying FDs and thus channels early
in the set are always early in the set. But as long as you randomizethe
processing order this should not be an issue.
As long as you dont do memory allocation in selector thread then all that
thread does is select() and memcpy() which can scale relatively well even if
you batch it and randomize events when you pass them o0nto next stage.
--
Cheers,
Peter Donald
*-----------------------------------------------------*
| Never argue with an idiot, they'll drag you down to |
| their level, and beat you with experience |
*-----------------------------------------------------*
Re: Serverside NIO: Not ready for Prime Time
Posted by Geir Magnusson Jr <ge...@4quarters.com>.
On Nov 25, 2003, at 8:25 PM, Peter Donald wrote:
> On Wed, 26 Nov 2003 03:13 am, Wes McKean wrote:
>> The CancelledKeyException seems to be happing because I am removing
>> the
>> SelectionKey from the iterator. The other error seems to occur when
>> I have
>> multiple reads and writes going on with different SocketChannels.
>> Here is
>> the exception:
>
> I think some of problems will be that you are processing the channels
> in
> different threads from the select. I can not see a good reason for
> this.
> Would it not be better to do the reads in the same thread as the
> selector,
> buffer the data and then send an event that is processed in another
> thread.
Ye gads. Why can't you multithread the reads? What possible reason
could one come up with that the reads have to happen on the same thread
as the select?
>
> See attached files for the sort of thing I mean. Associated with each
> connection object I have a TcpTransport object. The selector threads
> does all
> the nbio and then sends event to "parsing" stage. This seems to work
> like a
> charm.
Don't that serialize the I/O?
>
> Anyways you may want to check out the spice/sandbox/sca project at
> cvs.codehaus.org:/scm/cvs as it contains code that simplifies working
> with
> selectors etc.
>
> --
> Cheers,
>
> Peter Donald
> I think the next best thing to solving a
> problem is finding some humor in it.
> - Frank A. Clark
> <TcpTransport.java>
--
Geir Magnusson Jr 203-247-1713(m)
geir@4quarters.com
Re: Serverside NIO: Not ready for Prime Time
Posted by Peter Donald <pe...@realityforge.org>.
On Wed, 26 Nov 2003 03:13 am, Wes McKean wrote:
> The CancelledKeyException seems to be happing because I am removing the
> SelectionKey from the iterator. The other error seems to occur when I have
> multiple reads and writes going on with different SocketChannels. Here is
> the exception:
I think some of problems will be that you are processing the channels in
different threads from the select. I can not see a good reason for this.
Would it not be better to do the reads in the same thread as the selector,
buffer the data and then send an event that is processed in another thread.
See attached files for the sort of thing I mean. Associated with each
connection object I have a TcpTransport object. The selector threads does all
the nbio and then sends event to "parsing" stage. This seems to work like a
charm.
Anyways you may want to check out the spice/sandbox/sca project at
cvs.codehaus.org:/scm/cvs as it contains code that simplifies working with
selectors etc.
--
Cheers,
Peter Donald
I think the next best thing to solving a
problem is finding some humor in it.
- Frank A. Clark
Re: Serverside NIO: Not ready for Prime Time
Posted by Wes McKean <wm...@logictrends.com>.
If you want to look at my code, check out the nio sub project under sandbox0
on the sf.net site. To recap at a high level here, I am:
1. Creating a non blocking server socket which waits for connect events.
Once a connect event occurs, the resulting SocketChannel is accepted and
passed off to the input manager.
2. The InputManager makes the SocketChannel non-blocking, then registers it
for READ events with its Selector. This occurs in two different threads
( the registration and the selecting ).
3. When a read event occurs, I do the usual. I get the selected keys, get an
iterator, process them one at a time, removing them as I do, then loop back
around to select again.
I am getting two errors, a NullPointerException, and a CancelledKeyException.
The CancelledKeyException seems to be happing because I am removing the
SelectionKey from the iterator. The other error seems to occur when I have
multiple reads and writes going on with different SocketChannels. Here is
the exception:
java.lang.NullPointerException
at sun.nio.ch.WindowsSelectorImpl
$SubSelector.processFDSet(WindowsSelect
orImpl.java:309)
at sun.nio.ch.WindowsSelectorImpl
$SubSelector.processSelectedKeys(Window
sSelectorImpl.java:284)
at sun.nio.ch.WindowsSelectorImpl$SubSelector.access$2600
(WindowsSelecto
rImpl.java:245)
at
sun.nio.ch.WindowsSelectorImpl.updateSelectedKeys(WindowsSelectorImpl
.java:427)
at
sun.nio.ch.WindowsSelectorImpl.doSelect(WindowsSelectorImpl.java:142)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:59)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:70)
at ldapd.server.input.InputModule.run(InputModule.java:131)
at java.lang.Thread.run(Thread.java:534)
Wes
On Tuesday 25 November 2003 08:38 am, Peter Donald wrote:
> On Tue, 25 Nov 2003 11:56 pm, Wes McKean wrote:
> > I was running ldapd on a Windows XP box with the new server side NIO
> > stuff included, and I was getting all kinds of weird messages inside the
> > sun implementation classes. The same exact code works fine under Linux.
> > It is my professional opinion that there are some issues with the Windows
> > XP implementation of non-blocking sockets, and we couldn't put this stuff
> > into the server until these issues are addressed.
>
> There are a large number of platform specific issues - but that is no
> different from differences between pre-nio sockets on linux and windows. I
> have been building a NIO based network layer recently so if you describe
> some of the exact issues then I may be able to show how to avoid or give
> code to do just that.
Re: Serverside NIO: Not ready for Prime Time
Posted by Peter Donald <pe...@realityforge.org>.
On Tue, 25 Nov 2003 11:56 pm, Wes McKean wrote:
> I was running ldapd on a Windows XP box with the new server side NIO stuff
> included, and I was getting all kinds of weird messages inside the sun
> implementation classes. The same exact code works fine under Linux. It is
> my professional opinion that there are some issues with the Windows XP
> implementation of non-blocking sockets, and we couldn't put this stuff into
> the server until these issues are addressed.
There are a large number of platform specific issues - but that is no
different from differences between pre-nio sockets on linux and windows. I
have been building a NIO based network layer recently so if you describe some
of the exact issues then I may be able to show how to avoid or give code to
do just that.
--
Cheers,
Peter Donald
"Education is a progressive discovery of
our own ignorance." - Will Durant