You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by mrGibby <he...@gmail.com> on 2007/03/13 10:15:57 UTC
Re: [VOTE] optimal configuration for "diameter-like" protocols
Thanks for the quick response.. You do realize that now I wont bother to read
any documentation and previous posts, but just post a new question every
time I have a problem =)
Anyway, I did what you instructed and indeed I got around 6000 tps (a little
less). With one cpu windows laptop machine, that is.
However, I got another problem.. Now, if I establish two client connections
the other client gets something like 5500 tps, but the other only around
1000 tps? The setup is the same as described below, with your modifications
included. I do not expect it to scale in linear fashion, but I expect that
the service level is about the same on both clients.
Trustin Lee wrote:
>
> Oh, and please don't forget to get back here and tell use how MINA
> performs
> for you. ;)
>
> Trustin
>
> On 3/13/07, Trustin Lee <tr...@gmail.com> wrote:
>>
>> Hi,
>>
>> On 3/13/07, mrGibby <he...@gmail.com> wrote:
>> >
>> >
>> > Yesterday I did a quick test to check how MINA could be used to
>> > implement a
>> > diameter stack. I also did some initial benchmarks. So far the tests
>> > have
>> > shown that our own IO framework out performs mina quite easily (~4000
>> > tps vs
>> > >6000 tps, one connected client, 10 io threads). Also, the MINA version
>> > of
>> > diameter stack is missing the protocol state machine that exists the
>> > other
>> > framework, that surely adds some overhead to the system. There has
>> > already
>> > been some posts in this forum about diameter protocol. The main
>> > difference
>> > to protocols normally discussed here is that peers use only few
>> physical
>> > connections to serve multiple user sessions (compared e.g. to hundreds
>> > http
>> > connections). Increasing connections to boost performance is not always
>> > possible.
>> >
>> > So here is some of my test code. Basically just used the http example
>> as
>> > basis. Could someone give me some pointers how to do improve
>> > performance?
>> > The actual diameter message parsing is not a bottle neck since both
>> > benchmarked frameworks used the same code (well, I did need to change
>> > the
>> > bytebuffer to mina version). I did read the thread model configuration
>> > and
>> > it did not seem to improve the perforrmance. There has also been
>> > discussion
>> > that the events for each iosession are invoked on handler serially and
>> > the
>> > is no built in support in mina to have parallel processing of messages
>> > in
>> > iohandler (I would need to implement own threadpoolfilter). Is this
>> > still
>> > true or was this old news?
>> >
>> > // main
>> > IoAcceptor acceptor = new SocketAcceptor(10,
>> > Executors.newCachedThreadPool());
>> > // Create a service configuration
>> > SocketAcceptorConfig cfg = new SocketAcceptorConfig();
>> > cfg.setReuseAddress( true );
>> > cfg.getFilterChain().addLast(
>> > "protocolFilter",
>> > new ProtocolCodecFilter( new
>> > DiameterProtocolCodecFactory() ) );
>> > //cfg.getFilterChain().addLast( "logger", new LoggingFilter() );
>> >
>> > acceptor.bind( new InetSocketAddress( port ), new ServerHandler(), cfg
>> > );
>> >
>> >
>> > public MessageDecoderResult decodable( IoSession session, ByteBuffer in
>> > )
>> > {
>> > // Return NEED_DATA if the whole header is not read yet.
>> > try
>> > {
>> > return messageComplete( in ) ? MessageDecoderResult.OK
>> > : MessageDecoderResult.NEED_DATA ;
>> > }
>> > catch( Exception ex )
>> > {
>> > ex.printStackTrace();
>> > }
>> >
>> > return MessageDecoderResult.NOT_OK;
>> > }
>> >
>> > public MessageDecoderResult decode( IoSession session, ByteBuffer
>> > in,
>> > ProtocolDecoderOutput out ) throws Exception
>> > {
>> > // Try to decode body
>> > DiameterMessage m = decodeBody( in );
>> >
>> > // Return NEED_DATA if the body is not fully read.
>> > if( m == null )
>> > return MessageDecoderResult.NEED_DATA;
>> >
>> > out.write( m );
>> >
>> > return MessageDecoderResult.OK;
>> > }
>> > .
>> > .
>> > .
>> > }
>> >
>> > // iohandler, just changes the request to answer and echoes it back
>> > public class ServerHandler extends IoHandlerAdapter
>> > {
>> > .
>> > .
>> > public void messageReceived( IoSession session, Object message )
>> > {
>> > DiameterMessage m = (DiameterMessage) message;
>> > m.setRequest(false);
>> > if( m != null )
>> > session.write(m).join();
>> > }
>> > .
>> > .
>> > }
>>
>>
>> 1. You don't need to call join() after calling write().
>> 2. Please try to disable the thread model and don't add any
>> ExecutorFilter
>> because your business logic uses CPU only. To find out more about
>> configuring your thread model, please refer here:
>> http://mina.apache.org/configuring-thread-model.html
>> 3. Additionally, insert thr following two lines in your main class:
>>
>> ByteBuffer.setUseDirectBuffers (false);
>> ByteBuffer.setAllocator(new SimpleByteBufferAllocator());
>>
>> HTH,
>> Trustin
>> --
>> what we call human nature is actually human habit
>> --
>> http://gleamynode.net/
>> --
>> PGP Key ID: 0x0255ECA6
>
>
>
>
> --
> what we call human nature is actually human habit
> --
> http://gleamynode.net/
> --
> PGP Key ID: 0x0255ECA6
>
>
--
View this message in context: http://www.nabble.com/optimal-configuration-for-%22diameter-like%22-protocols-tf3394204.html#a9450713
Sent from the mina dev mailing list archive at Nabble.com.
Re: [VOTE] optimal configuration for "diameter-like" protocols
Posted by mrGibby <he...@gmail.com>.
Just wanted to inform you that I run a quick test with following hardware:
SunFire 6900, 20*CPU core 1.050 GHz, 40 GB
.. and using 20 I/O processor threads. scaled really nicely (nearly linear),
and did not see any unfair performance figures between connected clients.
E.g. with four connected clients all of them got something around 5000 tps.
But again, this was a quick test so if there is a random effect in that
"feature" it could be easily missed in my test.
Anyway, I need to look this more closely.. I just don't want to have this
biased throughput problem in a production environment.
Trustin Lee wrote:
>
> The code looks OK to me. It seems like unfair I/O problem in MINA is
> worse than I thought initially. Let me investigate this issue further
> when I have more free time.
>
--
View this message in context: http://www.nabble.com/optimal-configuration-for-%22diameter-like%22-protocols-tf3394204.html#a9470119
Sent from the mina dev mailing list archive at Nabble.com.
Re: [VOTE] optimal configuration for "diameter-like" protocols
Posted by Trustin Lee <tr...@gmail.com>.
On 3/13/07, mrGibby <he...@gmail.com> wrote:
> 330k tps is a very large number.. honestly it is hard for me to believe that
> you can get such results with single jvm (or did the test have multiple
> hosts and jvm's). But if it can I am really impressed. Would be nice to see
> some hardware specs and details on the actual test. I have to run some tests
> with our hosts that have a bit more juice.. Anyway, 10k tps with single jvm
> is sufficient for me.
Oops, I made a mistake. It was 220k msgs/sec. Please refer to the
page 7 of the following presentation material:
http://mina.apache.org/documentation.data/ACAsia2006.pdf
It was single JVM AFAIK. The performance test has been performed by
Robert Greg, a committer of Qpid project (formerly AMQP, currently
under incubation).
Interestingly, the Java implementation based on MINA outperformed the
C++ implementation by twice IIRC.
> Anyway, here is the code:
>
> // Create an acceptor
> IoAcceptor acceptor = new SocketAcceptor(20,
> Executors.newCachedThreadPool());
>
> // Create a service configuration
> SocketAcceptorConfig cfg = new SocketAcceptorConfig();
> cfg.setReuseAddress( true );
> cfg.setThreadModel(ThreadModel.MANUAL);
> cfg.getFilterChain().addLast(
> "protocolFilter",
> new ProtocolCodecFilter( new
> DiameterProtocolCodecFactory() ) );
> //cfg.getFilterChain().addLast( "logger", new LoggingFilter() );
> //cfg.getFilterChain().addLast("threadPool", new
> ExecutorFilter(Executors.newFixedThreadPool(10)));
> ByteBuffer.setUseDirectBuffers (false);
> ByteBuffer.setAllocator(new SimpleByteBufferAllocator());
>
> acceptor.bind( new InetSocketAddress( port ), new
> ServerHandler(), cfg );
The code looks OK to me. It seems like unfair I/O problem in MINA is
worse than I thought initially. Let me investigate this issue further
when I have more free time.
HTH,
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: [VOTE] optimal configuration for "diameter-like" protocols
Posted by mrGibby <he...@gmail.com>.
330k tps is a very large number.. honestly it is hard for me to believe that
you can get such results with single jvm (or did the test have multiple
hosts and jvm's). But if it can I am really impressed. Would be nice to see
some hardware specs and details on the actual test. I have to run some tests
with our hosts that have a bit more juice.. Anyway, 10k tps with single jvm
is sufficient for me.
I may also take a look at the mina 2.0.
Anyway, here is the code:
// Create an acceptor
IoAcceptor acceptor = new SocketAcceptor(20,
Executors.newCachedThreadPool());
// Create a service configuration
SocketAcceptorConfig cfg = new SocketAcceptorConfig();
cfg.setReuseAddress( true );
cfg.setThreadModel(ThreadModel.MANUAL);
cfg.getFilterChain().addLast(
"protocolFilter",
new ProtocolCodecFilter( new
DiameterProtocolCodecFactory() ) );
//cfg.getFilterChain().addLast( "logger", new LoggingFilter() );
//cfg.getFilterChain().addLast("threadPool", new
ExecutorFilter(Executors.newFixedThreadPool(10)));
ByteBuffer.setUseDirectBuffers (false);
ByteBuffer.setAllocator(new SimpleByteBufferAllocator());
acceptor.bind( new InetSocketAddress( port ), new
ServerHandler(), cfg );
Trustin Lee wrote:
>
>
> Can I see your code where an IoAcceptor is created and configured?
> There's
> a known issue with unfair I/O as I mentioned in the previous post, so it
> can
> happen somehow. There's a known patch here, but we didn't review it yet.
> The patch is also known to improve performance at some cases, but it
> didn't
> in my case.
>
> http://issues.apache.org/jira/browse/DIRMINA-301
>
> To be more accurate, it depends on the machine you run on and the size of
> a
> message. If your machine is powerful enough, it can perform that good.
> The
> biggest throughput number I heard from a MINA users was around 330k
> msgs/sec, though the number of clients was not 500.
>
>
--
View this message in context: http://www.nabble.com/optimal-configuration-for-%22diameter-like%22-protocols-tf3394204.html#a9453496
Sent from the mina dev mailing list archive at Nabble.com.
Re: [VOTE] optimal configuration for "diameter-like" protocols
Posted by Trustin Lee <tr...@gmail.com>.
On 3/13/07, mrGibby <he...@gmail.com> wrote:
>
> I used 10 I/O processor threads. Tried to decrease the amount to 4 and
> increase it to 20. I really could not see any pattern there. The results
> were mixed. On some runs the performance was quite even (2500 vs 3500),
> but
> in most cases it was something like 1000 and 4000. Thread count did not
> seem
> to make the difference. Also, the situation does not seem to even out in
> time. If I leave the spam on for ten or so minutes, the problem exists.
Can I see your code where an IoAcceptor is created and configured? There's
a known issue with unfair I/O as I mentioned in the previous post, so it can
happen somehow. There's a known patch here, but we didn't review it yet.
The patch is also known to improve performance at some cases, but it didn't
in my case.
http://issues.apache.org/jira/browse/DIRMINA-301
I also found this detail from previous posts:
>
> > 3) My app has to support minimum of 500 users with
> > each user getting 1000 messages per second.
> >
> >MINA can handle this amount of concurrency definitely.
>
> This can not be true, right? In my calculations this would mean 500000
> tps/second.
To be more accurate, it depends on the machine you run on and the size of a
message. If your machine is powerful enough, it can perform that good. The
biggest throughput number I heard from a MINA users was around 330k
msgs/sec, though the number of clients was not 500.
If you want to squeeze out more performance from MINA, you could try
2.0.0-M1-SNAPSHOT, which can be retrieved from our SVN repository (trunk),
which requires migration.
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6
Re: [VOTE] optimal configuration for "diameter-like" protocols
Posted by mrGibby <he...@gmail.com>.
I used 10 I/O processor threads. Tried to decrease the amount to 4 and
increase it to 20. I really could not see any pattern there. The results
were mixed. On some runs the performance was quite even (2500 vs 3500), but
in most cases it was something like 1000 and 4000. Thread count did not seem
to make the difference. Also, the situation does not seem to even out in
time. If I leave the spam on for ten or so minutes, the problem exists.
I also found this detail from previous posts:
> 3) My app has to support minimum of 500 users with
> each user getting 1000 messages per second.
>
>MINA can handle this amount of concurrency definitely.
This can not be true, right? In my calculations this would mean 500000
tps/second.
Trustin Lee wrote:
>
>
> How many I/O processor threads did you create? Current I/O processor
> implementation can cause biased throughput, but it's too much in this
> case.
> Please try to increase/decrease the number of I/O processor threads.
>
>
--
View this message in context: http://www.nabble.com/optimal-configuration-for-%22diameter-like%22-protocols-tf3394204.html#a9452781
Sent from the mina dev mailing list archive at Nabble.com.
Re: [VOTE] optimal configuration for "diameter-like" protocols
Posted by Trustin Lee <tr...@gmail.com>.
On 3/13/07, mrGibby <he...@gmail.com> wrote:
>
>
> Thanks for the quick response.. You do realize that now I wont bother to
> read
> any documentation and previous posts, but just post a new question every
> time I have a problem =)
>
> Anyway, I did what you instructed and indeed I got around 6000 tps (a
> little
> less). With one cpu windows laptop machine, that is.
>
> However, I got another problem.. Now, if I establish two client
> connections
> the other client gets something like 5500 tps, but the other only around
> 1000 tps? The setup is the same as described below, with your
> modifications
> included. I do not expect it to scale in linear fashion, but I expect that
> the service level is about the same on both clients.
How many I/O processor threads did you create? Current I/O processor
implementation can cause biased throughput, but it's too much in this case.
Please try to increase/decrease the number of I/O processor threads.
Trustin
--
what we call human nature is actually human habit
--
http://gleamynode.net/
--
PGP Key ID: 0x0255ECA6