Slow transfer rate with mina

[Steve Gury <st...@mimesis-republic.com>]

Hi,

I'm currently experiencing some issue with MINA, I'm using it from the
scala language (which is definitely not the source of the problem).
When I send big amount of data through the network, the upload rate is
first high and then it slows down dramatically (I'm running the server
and the client on the same computer).

I believe it is due to bad usage of the library.

Did anyone experience a similar issue ? Does anyone can correct me if
I'm doing it wrong.

Here is how I encode message (this is scala, but it is still readable
for a java developer)

>       val bufferSize = 200 * 1024 * 1024
>       var buffer = IoBuffer.allocate( bufferSize , false )
>       buffer.putInt( bufferSize - 4 )
>       [...]
>
>       buffer.flip()
>       _session.write( buffer )

Here is how I decode messages

>  def doDecode(session : IoSession, in : IoBuffer, out : ProtocolDecoderOutput) : Boolean = {
>    log.info( "Decoding current buffer (size:{}ko)" , in.remaining()/1024 )
>    if( in.prefixedDataAvailable(4) ) {
>      log.debug( "Decoding started" )
>      val remainingBytesToRead = in.getInt()
>      val buffer = new Array[Byte]( _cmdSize )
>      in.get( buffer )
>      val cmd = new String( buffer , _utf8Charset )
>
>      [...]
>
>      true
>    }
>    else {
>      false
>    }
>  }

Thanks,
Steve

AW: Slow transfer rate with mina

[Steve Ulrich <st...@proemion.com>]

Hi!

I don't know if this will fix your problem, but your messages are rather
large (200MB as I can see). Maybe you should consider a stateful decoder.
Your current decoder forces mina to work like this:
1: Some bytes received -> push buffer through the chain
2: Decoder reads 4 bytes, checks length
3: If not enough: leave buffer alone, wait for more bytes and goto 1

Every time, the processor gets some bytes, he pushes the whole buffer through
the chain, again and again. IIRC a typical TCP packet transfers up to 1.5 KB,
so your message might go through the chain up to 130.000 times! If the brandwidth
is big enough, the TCP packets should come in fast enough so that this isn't 
a problem, especially if you connect over loopback. But the lower the brandwidth,
the higher the amount of runs through the chain.

hth

Steve

btw: The decoder looks somewhat strange:
val remainingBytesToRead = in.getInt()
val buffer = new Array[Byte]( _cmdSize )
why do you use _cmdSize, not remainingBytesToRead?


> Steve Gury [mailto:steve.gury@mimesis-republic.com] wrote:
> 
> Hi,
> 
> I'm currently experiencing some issue with MINA, I'm using it from the
> scala language (which is definitely not the source of the problem).
> When I send big amount of data through the network, the upload rate is
> first high and then it slows down dramatically (I'm running the server
> and the client on the same computer).
> 
> I believe it is due to bad usage of the library.
> 
> Did anyone experience a similar issue ? Does anyone can correct me if
> I'm doing it wrong.
> 
> Here is how I encode message (this is scala, but it is still readable
> for a java developer)
> 
> >       val bufferSize = 200 * 1024 * 1024
> >       var buffer = IoBuffer.allocate( bufferSize , false )
> >       buffer.putInt( bufferSize - 4 )
> >       [...]
> >
> >       buffer.flip()
> >       _session.write( buffer )
> 
> Here is how I decode messages
> 
> >  def doDecode(session : IoSession, in : IoBuffer, out :
> ProtocolDecoderOutput) : Boolean = {
> >    log.info( "Decoding current buffer (size:{}ko)" ,
> in.remaining()/1024 )
> >    if( in.prefixedDataAvailable(4) ) {
> >      log.debug( "Decoding started" )
> >      val remainingBytesToRead = in.getInt()
> >      val buffer = new Array[Byte]( _cmdSize )
> >      in.get( buffer )
> >      val cmd = new String( buffer , _utf8Charset )
> >
> >      [...]
> >
> >      true
> >    }
> >    else {
> >      false
> >    }
> >  }
> 
> Thanks,
> Steve