You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by Frédéric Brégier <fr...@free.fr> on 2006/09/28 08:10:52 UTC
ThreadPool in Mina 1.0
Sorry to come back and to bother you with this question,
but I would like to understand this kind of points since it can
changed the performance aspect of my project.
I will resume my questions as follow :
1) I understand before (MINA < 0.9.5) there were only 2
level of poolthread in Mina. But when I looked
in the API (MINA 1.0), I found 3 parts :
a) IoAcceptor acceptor = new SocketAcceptor(nbThread, executor1);
b) IoAccaptorConfig config = new SocketAcceptorConfig();
config.setThreadModel(executor2);
c) IoFilterChain chain = IoConfig.getFilterChain();
chain.addLast( "myPool",
new ExecutorFilter( new ExecutorExecutor( executor3 ) ) );
I suppose that one of these threee is a double of another or that
I don't understand how to use them.
2) In the mailing list, I saw that (2) could be called with
IoConfig.setThreadModel( ThreadModel.MANUAL );
But I don't understand what is the consequences since it is
nowhere used in the Mina code and why I should not (?)
used a standard executor (threadpool).
3) The first call (1) takes another argument (number of thread).
How should it be relative or not to the number of thread in the
executor ?
When I looked in the code, the number of thread is used to define
an array of ioProcessors but should it be relative to the number
of threads defined in the executor (thread pool) ?
If so (same number of thread defined than the number of ioProcessors),
there is an API to get the number of threads defined in a fixedThreadPool.
Of course, it implies that if someone call this constructor, it should pass
such a fixedThreadPool and not a CachedThreadPool which doesn't
specify a number of threads.
Is it correct ?
4) For me, it looks like the first level of threads should be a
fixedThreadPool,
but other levels (2 or 3 ?) could be either fixed or cache or whatever.
Is this ok ?
Sorry again to bother you...
Frederic
==
In addition of my previous mail,
I have one question :
Why in the interface of new SocketAcceptor(nbThread, executor1)
do we precise the number of Thread ?
Is it relative to the number of thread in the executor or not ?
If yes, should we say in the API that executor should be a
fixedThreadPool (and the number of thread could be determined
from the ThreadPoolExecutor).
Frederic
----- Original Message -----
From: "Frédéric Brégier"
Sent: Monday, September 25, 2006 8:55 PM
Subject: Re: ThreadPool in Mina with concurrent from Java5 Howto ?
Sorry to come back on this subject, but I am a bit
confused with the Mina logic for ThreadPool.
First, what I suppose to understand :
- First IoAcceptor have one ThreadPool to handle
incoming IoSessions.
- Second, the IoConfig use a ThreadPool to handle
process of filters.
- Eventually, in the filter chain, we can insert a new ThreadPool
for specific filters that are supposed to consume a lot of cpu
(for example compression or ssl ?).
For me, the doc says there are two level of ThreadPool management,
but in the API I found three levels. Do I missed something ?
Now from the coding part, I do the following.
Of course, if I am wrong in the model, it should be wrong
also in the code, so don't hesitate to give me your advice.
(exemple is with Socket)
0) I define several ThreadPool :
Executor executor1 = Executors.newFixedThreadPool(nbThread1);
Executor executor2 = Executors.newCachedThreadPool();
Executor executor3 = Executors.newCachedThreadPool();
1) Put a fixed threadPool to IoAcceptor : let me known if someone
can say if it is better here to use a CachedThreadPool or a FixedThreadPool
?
It highly depends I woul say how Mina is using it.
IoAcceptor acceptor = new SocketAcceptor(nbThread, executor1);
2) Put a cached threadPool in IoAcceptorConfig :
I don't know here if it is the same than (1) or not ?
When I look into the code of Mina, it seems not,
but I am really not sure.
IoAccaptorConfig config = new SocketAcceptorConfig();
config.setThreadModel(executor2);
Also I don't understand why Peter (and in the old doc)
I should use :
IoConfig.setThreadModel( ThreadModel.MANUAL );
3) And finally inside the filter chain, for instance for "compression" :
IoFilterChain chain = IoConfig.getFilterChain();
chain.addLast( "compressionPool",
new ExecutorFilter( new ExecutorExecutor( executor3 ) ) );
chain.addLast( "compression", compressionFilter);
Thank you for your help
Frederic
Re: ThreadPool in Mina 1.0
Posted by Niklas Therning <ni...@trillian.se>.
That sounds really cool!!! Let us know how it goes.
/Niklas
Frédéric Brégier wrote:
> Really clear now !
> Thank you Niklas !
> My project will run on 32 or even 64 processors servers,
> so knowing this information will be very useful for me !
>
> Thank you again !
> Frederic
>
Re: ThreadPool in Mina 1.0
Posted by Frédéric Brégier <fr...@free.fr>.
Really clear now !
Thank you Niklas !
My project will run on 32 or even 64 processors servers,
so knowing this information will be very useful for me !
Thank you again !
Frederic
----- Original Message -----
From: "Niklas Therning" <ni...@trillian.se>
To: <mi...@directory.apache.org>
Sent: Thursday, September 28, 2006 8:53 AM
Subject: Re: ThreadPool in Mina 1.0
The processorCount parameter specifies the number of SocketIoProcessors
the SocketAcceptor should use. Each SocketIoProcessor owns a
java.nio.channels.Selector. When the SocketAcceptor accepts a new
connection it will add the new connection to the next SocketIoProcessor
in a round-robin fashion. The default is 1 which is fine if you're
running on single CPU hardware.
The Executor specified in the SocketAcceptor constructor will be used by
the SocketAcceptor to run its internal Worker thread which listens for
new connections. If you don't need 100% control over the threads created
in your application you won't have to bother about this.
If you specify the ThreadModel.MANUAL thread model MINA won't create an
ExecutorFilter for your sessions' filter chains. If you use MANUAL and
you don't add an ExecutorFilter yourself all processing will take place
in a single SocketIoProcessor Worker thread (or processorCount threads
if processorCount > 1).
If you want to add your own ExecutorFilter explicitly (like in c) you
should specify ThreadModel.MANUAL otherwise you will end up with two
ExecutorFilters.
A ThreadModel can be specified for all ports your SocketAcceptor is
listening on by using acceptor.getDefaultConfig().setThreadModel(...) or
on a port by port basis by specifying a SocketAcceptorConfig when
calling acceptor.bind(...).
I hope this makes things a little bit clearer.
/Niklas
Frédéric Brégier wrote:
> Sorry to come back and to bother you with this question,
> but I would like to understand this kind of points since it can
> changed the performance aspect of my project.
> I will resume my questions as follow :
>
> 1) I understand before (MINA < 0.9.5) there were only 2
> level of poolthread in Mina. But when I looked
> in the API (MINA 1.0), I found 3 parts :
> a) IoAcceptor acceptor = new SocketAcceptor(nbThread, executor1);
> b) IoAccaptorConfig config = new SocketAcceptorConfig();
> config.setThreadModel(executor2);
> c) IoFilterChain chain = IoConfig.getFilterChain();
> chain.addLast( "myPool",
> new ExecutorFilter( new ExecutorExecutor( executor3 ) ) );
>
> I suppose that one of these threee is a double of another or that
> I don't understand how to use them.
>
> 2) In the mailing list, I saw that (2) could be called with
> IoConfig.setThreadModel( ThreadModel.MANUAL );
> But I don't understand what is the consequences since it is
> nowhere used in the Mina code and why I should not (?)
> used a standard executor (threadpool).
>
> 3) The first call (1) takes another argument (number of thread).
> How should it be relative or not to the number of thread in the
> executor ?
> When I looked in the code, the number of thread is used to define
> an array of ioProcessors but should it be relative to the number
> of threads defined in the executor (thread pool) ?
> If so (same number of thread defined than the number of ioProcessors),
> there is an API to get the number of threads defined in a fixedThreadPool.
> Of course, it implies that if someone call this constructor, it should
> pass
> such a fixedThreadPool and not a CachedThreadPool which doesn't
> specify a number of threads.
> Is it correct ?
>
> 4) For me, it looks like the first level of threads should be a
> fixedThreadPool,
> but other levels (2 or 3 ?) could be either fixed or cache or whatever.
> Is this ok ?
>
> Sorry again to bother you...
>
> Frederic
>
> ==
>
> In addition of my previous mail,
> I have one question :
> Why in the interface of new SocketAcceptor(nbThread, executor1)
> do we precise the number of Thread ?
> Is it relative to the number of thread in the executor or not ?
> If yes, should we say in the API that executor should be a
> fixedThreadPool (and the number of thread could be determined
> from the ThreadPoolExecutor).
>
> Frederic
> ----- Original Message -----
> From: "Frédéric Brégier"
> Sent: Monday, September 25, 2006 8:55 PM
> Subject: Re: ThreadPool in Mina with concurrent from Java5 Howto ?
>
>
> Sorry to come back on this subject, but I am a bit
> confused with the Mina logic for ThreadPool.
>
> First, what I suppose to understand :
> - First IoAcceptor have one ThreadPool to handle
> incoming IoSessions.
> - Second, the IoConfig use a ThreadPool to handle
> process of filters.
> - Eventually, in the filter chain, we can insert a new ThreadPool
> for specific filters that are supposed to consume a lot of cpu
> (for example compression or ssl ?).
>
> For me, the doc says there are two level of ThreadPool management,
> but in the API I found three levels. Do I missed something ?
>
> Now from the coding part, I do the following.
> Of course, if I am wrong in the model, it should be wrong
> also in the code, so don't hesitate to give me your advice.
>
> (exemple is with Socket)
>
> 0) I define several ThreadPool :
> Executor executor1 = Executors.newFixedThreadPool(nbThread1);
> Executor executor2 = Executors.newCachedThreadPool();
> Executor executor3 = Executors.newCachedThreadPool();
>
> 1) Put a fixed threadPool to IoAcceptor : let me known if someone
> can say if it is better here to use a CachedThreadPool or a
> FixedThreadPool
> ?
> It highly depends I woul say how Mina is using it.
>
> IoAcceptor acceptor = new SocketAcceptor(nbThread, executor1);
>
> 2) Put a cached threadPool in IoAcceptorConfig :
> I don't know here if it is the same than (1) or not ?
> When I look into the code of Mina, it seems not,
> but I am really not sure.
>
> IoAccaptorConfig config = new SocketAcceptorConfig();
> config.setThreadModel(executor2);
>
> Also I don't understand why Peter (and in the old doc)
> I should use :
> IoConfig.setThreadModel( ThreadModel.MANUAL );
>
> 3) And finally inside the filter chain, for instance for "compression" :
>
> IoFilterChain chain = IoConfig.getFilterChain();
> chain.addLast( "compressionPool",
> new ExecutorFilter( new ExecutorExecutor( executor3 ) ) );
> chain.addLast( "compression", compressionFilter);
>
>
> Thank you for your help
> Frederic
>
>
>
Re: ThreadPool in Mina 1.0
Posted by Niklas Therning <ni...@trillian.se>.
The processorCount parameter specifies the number of SocketIoProcessors
the SocketAcceptor should use. Each SocketIoProcessor owns a
java.nio.channels.Selector. When the SocketAcceptor accepts a new
connection it will add the new connection to the next SocketIoProcessor
in a round-robin fashion. The default is 1 which is fine if you're
running on single CPU hardware.
The Executor specified in the SocketAcceptor constructor will be used by
the SocketAcceptor to run its internal Worker thread which listens for
new connections. If you don't need 100% control over the threads created
in your application you won't have to bother about this.
If you specify the ThreadModel.MANUAL thread model MINA won't create an
ExecutorFilter for your sessions' filter chains. If you use MANUAL and
you don't add an ExecutorFilter yourself all processing will take place
in a single SocketIoProcessor Worker thread (or processorCount threads
if processorCount > 1).
If you want to add your own ExecutorFilter explicitly (like in c) you
should specify ThreadModel.MANUAL otherwise you will end up with two
ExecutorFilters.
A ThreadModel can be specified for all ports your SocketAcceptor is
listening on by using acceptor.getDefaultConfig().setThreadModel(...) or
on a port by port basis by specifying a SocketAcceptorConfig when
calling acceptor.bind(...).
I hope this makes things a little bit clearer.
/Niklas
Frédéric Brégier wrote:
> Sorry to come back and to bother you with this question,
> but I would like to understand this kind of points since it can
> changed the performance aspect of my project.
> I will resume my questions as follow :
>
> 1) I understand before (MINA < 0.9.5) there were only 2
> level of poolthread in Mina. But when I looked
> in the API (MINA 1.0), I found 3 parts :
> a) IoAcceptor acceptor = new SocketAcceptor(nbThread, executor1);
> b) IoAccaptorConfig config = new SocketAcceptorConfig();
> config.setThreadModel(executor2);
> c) IoFilterChain chain = IoConfig.getFilterChain();
> chain.addLast( "myPool",
> new ExecutorFilter( new ExecutorExecutor( executor3 ) ) );
>
> I suppose that one of these threee is a double of another or that
> I don't understand how to use them.
>
> 2) In the mailing list, I saw that (2) could be called with
> IoConfig.setThreadModel( ThreadModel.MANUAL );
> But I don't understand what is the consequences since it is
> nowhere used in the Mina code and why I should not (?)
> used a standard executor (threadpool).
>
> 3) The first call (1) takes another argument (number of thread).
> How should it be relative or not to the number of thread in the
> executor ?
> When I looked in the code, the number of thread is used to define
> an array of ioProcessors but should it be relative to the number
> of threads defined in the executor (thread pool) ?
> If so (same number of thread defined than the number of ioProcessors),
> there is an API to get the number of threads defined in a fixedThreadPool.
> Of course, it implies that if someone call this constructor, it should pass
> such a fixedThreadPool and not a CachedThreadPool which doesn't
> specify a number of threads.
> Is it correct ?
>
> 4) For me, it looks like the first level of threads should be a
> fixedThreadPool,
> but other levels (2 or 3 ?) could be either fixed or cache or whatever.
> Is this ok ?
>
> Sorry again to bother you...
>
> Frederic
>
> ==
>
> In addition of my previous mail,
> I have one question :
> Why in the interface of new SocketAcceptor(nbThread, executor1)
> do we precise the number of Thread ?
> Is it relative to the number of thread in the executor or not ?
> If yes, should we say in the API that executor should be a
> fixedThreadPool (and the number of thread could be determined
> from the ThreadPoolExecutor).
>
> Frederic
> ----- Original Message -----
> From: "Frédéric Brégier"
> Sent: Monday, September 25, 2006 8:55 PM
> Subject: Re: ThreadPool in Mina with concurrent from Java5 Howto ?
>
>
> Sorry to come back on this subject, but I am a bit
> confused with the Mina logic for ThreadPool.
>
> First, what I suppose to understand :
> - First IoAcceptor have one ThreadPool to handle
> incoming IoSessions.
> - Second, the IoConfig use a ThreadPool to handle
> process of filters.
> - Eventually, in the filter chain, we can insert a new ThreadPool
> for specific filters that are supposed to consume a lot of cpu
> (for example compression or ssl ?).
>
> For me, the doc says there are two level of ThreadPool management,
> but in the API I found three levels. Do I missed something ?
>
> Now from the coding part, I do the following.
> Of course, if I am wrong in the model, it should be wrong
> also in the code, so don't hesitate to give me your advice.
>
> (exemple is with Socket)
>
> 0) I define several ThreadPool :
> Executor executor1 = Executors.newFixedThreadPool(nbThread1);
> Executor executor2 = Executors.newCachedThreadPool();
> Executor executor3 = Executors.newCachedThreadPool();
>
> 1) Put a fixed threadPool to IoAcceptor : let me known if someone
> can say if it is better here to use a CachedThreadPool or a FixedThreadPool
> ?
> It highly depends I woul say how Mina is using it.
>
> IoAcceptor acceptor = new SocketAcceptor(nbThread, executor1);
>
> 2) Put a cached threadPool in IoAcceptorConfig :
> I don't know here if it is the same than (1) or not ?
> When I look into the code of Mina, it seems not,
> but I am really not sure.
>
> IoAccaptorConfig config = new SocketAcceptorConfig();
> config.setThreadModel(executor2);
>
> Also I don't understand why Peter (and in the old doc)
> I should use :
> IoConfig.setThreadModel( ThreadModel.MANUAL );
>
> 3) And finally inside the filter chain, for instance for "compression" :
>
> IoFilterChain chain = IoConfig.getFilterChain();
> chain.addLast( "compressionPool",
> new ExecutorFilter( new ExecutorExecutor( executor3 ) ) );
> chain.addLast( "compression", compressionFilter);
>
>
> Thank you for your help
> Frederic
>
>
>