[MINA 3.0] Thoughts on TCP server

[Emmanuel Lécharny <el...@gmail.com>]

Hi guys, some thoughts about the TCP server. feel free to comment.

TCP server MINA 3
-----------------

As we are reworking the server part of MINA, we can review the current 
architecture. There are a few problems we can address.

1) One vs Many selectors
In MINA 2, we do have at least 2 selectors used :
- one for the OP_ACCEPT event
- One or many for the OP_READ/OP_WRITE

I don't think that it makes a lot of sense to force such a choice. IMO, 
we could perfectly start with one single selector to handle all the 
events, assuming we are fast enough to process them. Otherwise, we can 
also configure the server to use more selectors, if needed.

Typically, the acceptor selector will just deal with incoming new 
connections, and the created channel will be registred on an IoProcessor 
selector on MINA 2. We could register this channel on the acceptor selector.

In any case, if we do create more than one selector, I suggest that the 
first one would always handle OP_ACCEPT events, and that's all.

The general algorithm will look likes :

signal started

while true
   nbSelect = select( delay )

   if nbSelect != 0
     for each selectionKey do
       case isAccept // Only if the selector is the first one, otherwise 
we don't need to heck this case
         create session

       case isRead
         process read

       case isWrite
         process write
     done

   if dispose // In order to stop the server
     process dispose
     break
done

The key is to start all the selector workers *before* accepting any 
connection, otherwise we may lose some messages. One more thing : each 
selector should signal that there have started before entering in the 
loop, so that the caller don't have to wait a random period of time for 
the selectors to be started.

2) Events processing
Now, in order to limit the number of selectors to use, we need to limit 
the time it takes to process the read/write/accept events. But even if 
we have many selectors, we should try to minimize the risk that one 
selector is blocked by a single session blocked somewhere while doing 
some heavy prcoessing, as it will block all the other sessions.

Having more than a selector is one way to mitigate this issue : as we 
have many threads (one per selector), we spread the loads on as many 
threads.
Another solution would be to use an executor in charge of processing the 
events, with a queue between the selector and the executor, queue that 
is used to process the events as fast as possible on the selector (this 
is really important for UDP, as we don't want to lose messages simply 
because the OS buffer is full).

The problem is that we are just not solving the problem of a rogue 
service that block a thread for a long time (if we use a limited size 
executor), or we may end with so many threads that it may kill the 
server. But anyway, it sounds like a better solution, as incoming events 
that won't require a long processing will be favored in the long term.

3) Write processing
This is a complex issue too : we may not be able to push all the data we 
want into the socket, if it becoms full (or was already full). In this 
case, we will have to store the data in a queue. The following algorithm 
describe this situation and a proposal to solve it

     if there are some data in the writeQueue
       then
         // We need to enqueue the data, and write the head of the queue
         enqueue data

         // Now, we shoudl try to write as much of the queue as we can
         while ( queue not empty)
           do
               poll the data from the queue
             nbWritten = channel.write( remaining data ) // We may have 
already written some part of the head data

             if nbWritten < data.remaining
               then
                 // the socket is already full, set its OP_WRITE 
interest, and don't touch the queue
                 selectionKey.ops = OP_WRITE
                 break // We can stop, the socket is full anyway
               else
                 pop the data from the queue // We just remove the head, 
and continue with the next data
           done

           if queue is empry
             then
               selectionKeys.ops = ! OP_WRITE // We remoe this flag, 
it's ueless now
       else
         nbWritten = channel.write( remaining data ) // We may have 
already written some part of the head data

         if nbWritten < data.remaining
           then
             // the socket is already full, set its OP_WRITE interest, 
and add the data in the queue
             selectionKey.ops = OP_WRITE
             writeQueue.add( remaining data )

4) Read/Write order
MINA 2 was processing all the reads first, then all the writes. This is 
not necessarilly a good idea. We may rather process read and write on a 
per session basis. It's perfectly possible that a session has to process 
some reads and some write at the same time. Waiting for all the reads to 
be processed create some memory load, as we will have to wait until all 
the reads are done, storing all the data to be written in the mean time, 
until we are done with the last session.

5) MINA events
Atm, we are processing the following events :
- messageReceived (when we have read some data from the channel)
- messageSent (when a user message has been sentcompletely)
- exceptionCaugth (when an exception has been caught)
- sessionIdle (if the session has not received or sent anything for a 
period of time)
- sessionCreated (when a new session is created)
- sessionOpened
- sessionClosed (when the session has been closed)
- filterWrite
- filterClose

The sessionOpened, filterWrite and filterClose are a bit mysterious. I 
don't see why we need a special event SessionOpened when we alreayd have 
the sessionCreated event. That may seems we *may* create a session, but 
not accept any incoming or outgoing messages for this session, until 
it's initialized. I'm not sure this is necessary.

The two other events are probably some YAGNY iplementation...

-- 
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com

Re: [MINA 3.0] Thoughts on TCP server

[Chad Beaulac <ca...@gmail.com>]

Hi Emmanuel,

1) One Reactor is preferable. Easier to manage, easier to code, meets all the requirements.

2) Event Processing: Use non-blocking sockets. Accept, connect, read and write won't hang each other up.

I have some code that does this. I'll get it and attach to a subsequent email. It might take a couple days.

Regards,
Chad Beaulac
Objective Solutions, Inc.
www.objectivesolutions.com
chad@objectivesolutions.com



On Oct 1, 2012, at 6:45 AM, Emmanuel Lécharny wrote:

> Hi guys, some thoughts about the TCP server. feel free to comment.
> 
> TCP server MINA 3
> -----------------
> 
> As we are reworking the server part of MINA, we can review the current architecture. There are a few problems we can address.
> 
> 1) One vs Many selectors
> In MINA 2, we do have at least 2 selectors used :
> - one for the OP_ACCEPT event
> - One or many for the OP_READ/OP_WRITE
> 
> I don't think that it makes a lot of sense to force such a choice. IMO, we could perfectly start with one single selector to handle all the events, assuming we are fast enough to process them. Otherwise, we can also configure the server to use more selectors, if needed.
> 
> Typically, the acceptor selector will just deal with incoming new connections, and the created channel will be registred on an IoProcessor selector on MINA 2. We could register this channel on the acceptor selector.
> 
> In any case, if we do create more than one selector, I suggest that the first one would always handle OP_ACCEPT events, and that's all.
> 
> The general algorithm will look likes :
> 
> signal started
> 
> while true
>  nbSelect = select( delay )
> 
>  if nbSelect != 0
>    for each selectionKey do
>      case isAccept // Only if the selector is the first one, otherwise we don't need to heck this case
>        create session
> 
>      case isRead
>        process read
> 
>      case isWrite
>        process write
>    done
> 
>  if dispose // In order to stop the server
>    process dispose
>    break
> done
> 
> The key is to start all the selector workers *before* accepting any connection, otherwise we may lose some messages. One more thing : each selector should signal that there have started before entering in the loop, so that the caller don't have to wait a random period of time for the selectors to be started.
> 
> 2) Events processing
> Now, in order to limit the number of selectors to use, we need to limit the time it takes to process the read/write/accept events. But even if we have many selectors, we should try to minimize the risk that one selector is blocked by a single session blocked somewhere while doing some heavy prcoessing, as it will block all the other sessions.
> 
> Having more than a selector is one way to mitigate this issue : as we have many threads (one per selector), we spread the loads on as many threads.
> Another solution would be to use an executor in charge of processing the events, with a queue between the selector and the executor, queue that is used to process the events as fast as possible on the selector (this is really important for UDP, as we don't want to lose messages simply because the OS buffer is full).
> 
> The problem is that we are just not solving the problem of a rogue service that block a thread for a long time (if we use a limited size executor), or we may end with so many threads that it may kill the server. But anyway, it sounds like a better solution, as incoming events that won't require a long processing will be favored in the long term.
> 
> 3) Write processing
> This is a complex issue too : we may not be able to push all the data we want into the socket, if it becoms full (or was already full). In this case, we will have to store the data in a queue. The following algorithm describe this situation and a proposal to solve it
> 
>    if there are some data in the writeQueue
>      then
>        // We need to enqueue the data, and write the head of the queue
>        enqueue data
> 
>        // Now, we shoudl try to write as much of the queue as we can
>        while ( queue not empty)
>          do
>              poll the data from the queue
>            nbWritten = channel.write( remaining data ) // We may have already written some part of the head data
> 
>            if nbWritten < data.remaining
>              then
>                // the socket is already full, set its OP_WRITE interest, and don't touch the queue
>                selectionKey.ops = OP_WRITE
>                break // We can stop, the socket is full anyway
>              else
>                pop the data from the queue // We just remove the head, and continue with the next data
>          done
> 
>          if queue is empry
>            then
>              selectionKeys.ops = ! OP_WRITE // We remoe this flag, it's ueless now
>      else
>        nbWritten = channel.write( remaining data ) // We may have already written some part of the head data
> 
>        if nbWritten < data.remaining
>          then
>            // the socket is already full, set its OP_WRITE interest, and add the data in the queue
>            selectionKey.ops = OP_WRITE
>            writeQueue.add( remaining data )
> 
> 4) Read/Write order
> MINA 2 was processing all the reads first, then all the writes. This is not necessarilly a good idea. We may rather process read and write on a per session basis. It's perfectly possible that a session has to process some reads and some write at the same time. Waiting for all the reads to be processed create some memory load, as we will have to wait until all the reads are done, storing all the data to be written in the mean time, until we are done with the last session.
> 
> 5) MINA events
> Atm, we are processing the following events :
> - messageReceived (when we have read some data from the channel)
> - messageSent (when a user message has been sentcompletely)
> - exceptionCaugth (when an exception has been caught)
> - sessionIdle (if the session has not received or sent anything for a period of time)
> - sessionCreated (when a new session is created)
> - sessionOpened
> - sessionClosed (when the session has been closed)
> - filterWrite
> - filterClose
> 
> The sessionOpened, filterWrite and filterClose are a bit mysterious. I don't see why we need a special event SessionOpened when we alreayd have the sessionCreated event. That may seems we *may* create a session, but not accept any incoming or outgoing messages for this session, until it's initialized. I'm not sure this is necessary.
> 
> The two other events are probably some YAGNY iplementation...
> 
> -- 
> Regards,
> Cordialement,
> Emmanuel Lécharny
> www.iktek.com
> 

Re: [MINA 3.0] Thoughts on TCP server

[Mike van Goor <mi...@probie.nl>]

Hello Emmanuel,

I think your ideas are golden, I just want to comment on one thing.
 > The sessionOpened, filterWrite and filterClose are a bit mysterious. I
 > don't see why we need a special event SessionOpened when we alreayd have
 > the sessionCreated event. That may seems we *may* create a session, but
 > not accept any incoming or outgoing messages for this session, until
 > it's initialized. I'm not sure this is necessary.
The SessionOpened event tells us if a connection has been opened.
If you connect to a http server your hand the channel to a selector and 
wait for the socket connected notice to start your program. This is 
especially important of the side you are connecting to does not send a 
message to indicate the server is ready (220 in smtp, 200 in ftp, etc).

I created a lot of servers that don't tell you anything and wait for the 
client to send the first command. In which case I wait for the 
socketconnected event to send this command.

So in short, I see a very valuable reason to have the socketconnected event.
I agree on the fact that the socketconnected event has no value for a 
server socket.

Just my 2 cents.

Regards,
Mike

Op 1-10-2012 12:45, Emmanuel Lécharny schreef:
> Hi guys, some thoughts about the TCP server. feel free to comment.
>
> TCP server MINA 3
> -----------------
>
> As we are reworking the server part of MINA, we can review the current
> architecture. There are a few problems we can address.
>
> 1) One vs Many selectors
> In MINA 2, we do have at least 2 selectors used :
> - one for the OP_ACCEPT event
> - One or many for the OP_READ/OP_WRITE
>
> I don't think that it makes a lot of sense to force such a choice. IMO,
> we could perfectly start with one single selector to handle all the
> events, assuming we are fast enough to process them. Otherwise, we can
> also configure the server to use more selectors, if needed.
>
> Typically, the acceptor selector will just deal with incoming new
> connections, and the created channel will be registred on an IoProcessor
> selector on MINA 2. We could register this channel on the acceptor
> selector.
>
> In any case, if we do create more than one selector, I suggest that the
> first one would always handle OP_ACCEPT events, and that's all.
>
> The general algorithm will look likes :
>
> signal started
>
> while true
>    nbSelect = select( delay )
>
>    if nbSelect != 0
>      for each selectionKey do
>        case isAccept // Only if the selector is the first one, otherwise
> we don't need to heck this case
>          create session
>
>        case isRead
>          process read
>
>        case isWrite
>          process write
>      done
>
>    if dispose // In order to stop the server
>      process dispose
>      break
> done
>
> The key is to start all the selector workers *before* accepting any
> connection, otherwise we may lose some messages. One more thing : each
> selector should signal that there have started before entering in the
> loop, so that the caller don't have to wait a random period of time for
> the selectors to be started.
>
> 2) Events processing
> Now, in order to limit the number of selectors to use, we need to limit
> the time it takes to process the read/write/accept events. But even if
> we have many selectors, we should try to minimize the risk that one
> selector is blocked by a single session blocked somewhere while doing
> some heavy prcoessing, as it will block all the other sessions.
>
> Having more than a selector is one way to mitigate this issue : as we
> have many threads (one per selector), we spread the loads on as many
> threads.
> Another solution would be to use an executor in charge of processing the
> events, with a queue between the selector and the executor, queue that
> is used to process the events as fast as possible on the selector (this
> is really important for UDP, as we don't want to lose messages simply
> because the OS buffer is full).
>
> The problem is that we are just not solving the problem of a rogue
> service that block a thread for a long time (if we use a limited size
> executor), or we may end with so many threads that it may kill the
> server. But anyway, it sounds like a better solution, as incoming events
> that won't require a long processing will be favored in the long term.
>
> 3) Write processing
> This is a complex issue too : we may not be able to push all the data we
> want into the socket, if it becoms full (or was already full). In this
> case, we will have to store the data in a queue. The following algorithm
> describe this situation and a proposal to solve it
>
>      if there are some data in the writeQueue
>        then
>          // We need to enqueue the data, and write the head of the queue
>          enqueue data
>
>          // Now, we shoudl try to write as much of the queue as we can
>          while ( queue not empty)
>            do
>                poll the data from the queue
>              nbWritten = channel.write( remaining data ) // We may have
> already written some part of the head data
>
>              if nbWritten < data.remaining
>                then
>                  // the socket is already full, set its OP_WRITE
> interest, and don't touch the queue
>                  selectionKey.ops = OP_WRITE
>                  break // We can stop, the socket is full anyway
>                else
>                  pop the data from the queue // We just remove the head,
> and continue with the next data
>            done
>
>            if queue is empry
>              then
>                selectionKeys.ops = ! OP_WRITE // We remoe this flag,
> it's ueless now
>        else
>          nbWritten = channel.write( remaining data ) // We may have
> already written some part of the head data
>
>          if nbWritten < data.remaining
>            then
>              // the socket is already full, set its OP_WRITE interest,
> and add the data in the queue
>              selectionKey.ops = OP_WRITE
>              writeQueue.add( remaining data )
>
> 4) Read/Write order
> MINA 2 was processing all the reads first, then all the writes. This is
> not necessarilly a good idea. We may rather process read and write on a
> per session basis. It's perfectly possible that a session has to process
> some reads and some write at the same time. Waiting for all the reads to
> be processed create some memory load, as we will have to wait until all
> the reads are done, storing all the data to be written in the mean time,
> until we are done with the last session.
>
> 5) MINA events
> Atm, we are processing the following events :
> - messageReceived (when we have read some data from the channel)
> - messageSent (when a user message has been sentcompletely)
> - exceptionCaugth (when an exception has been caught)
> - sessionIdle (if the session has not received or sent anything for a
> period of time)
> - sessionCreated (when a new session is created)
> - sessionOpened
> - sessionClosed (when the session has been closed)
> - filterWrite
> - filterClose
>
> The sessionOpened, filterWrite and filterClose are a bit mysterious. I
> don't see why we need a special event SessionOpened when we alreayd have
> the sessionCreated event. That may seems we *may* create a session, but
> not accept any incoming or outgoing messages for this session, until
> it's initialized. I'm not sure this is necessary.
>
> The two other events are probably some YAGNY iplementation...
>