You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@mina.apache.org by Patrizio Munzi <pa...@eris4.com> on 2009/02/23 17:35:14 UTC
OrderedThreadPoolExecutor shutdown problem in MINA 2.0.0-M4
Hi everybody,
I'm having the following exception after the shutdown of the
OrderedThreadPoolExecutor.
It looks not an exception I'm responsible for.
I add an OrderedThreadPoolExecutor to my FilterChain as following:
----------------------------
ExecutorFilter executorFilter = new ExecutorFilter();
connector.getFilterChain().addLast("executor", executorFilter);
----------------------------
then I shutdown it in the following way:
----------------------------
CloseFuture closeFuture=session.close(false);
closeFuture.awaitUninterruptibly();
executorFilter.destroy();
----------------------------
with the following exception thrown:
----------------------------
750 [NioProcessor-1] WARN
org.apache.mina.core.filterchain.DefaultIoFilterChain - Unexpected
exception from exceptionCaught handler.
java.util.concurrent.RejectedExecutionException
at
java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1477)
at
org.apache.mina.filter.executor.OrderedThreadPoolExecutor.rejectTask(OrderedThreadPoolExecutor.java:309)
at
org.apache.mina.filter.executor.OrderedThreadPoolExecutor.execute(OrderedThreadPoolExecutor.java:272)
at
org.apache.mina.filter.executor.ExecutorFilter.fireEvent(ExecutorFilter.java:535)
at
org.apache.mina.filter.executor.ExecutorFilter.exceptionCaught(ExecutorFilter.java:593)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:480)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1100(DefaultIoFilterChain.java:48)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.exceptionCaught(DefaultIoFilterChain.java:797)
at
org.apache.mina.core.filterchain.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:112)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:480)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1100(DefaultIoFilterChain.java:48)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.exceptionCaught(DefaultIoFilterChain.java:797)
at
org.apache.mina.core.filterchain.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:112)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:480)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.fireExceptionCaught(DefaultIoFilterChain.java:468)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextSessionClosed(DefaultIoFilterChain.java:397)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.access$900(DefaultIoFilterChain.java:48)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.sessionClosed(DefaultIoFilterChain.java:787)
at
org.apache.mina.filter.codec.ProtocolCodecFilter.sessionClosed(ProtocolCodecFilter.java:363)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextSessionClosed(DefaultIoFilterChain.java:395)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.access$900(DefaultIoFilterChain.java:48)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.sessionClosed(DefaultIoFilterChain.java:787)
at
org.apache.mina.core.filterchain.IoFilterAdapter.sessionClosed(IoFilterAdapter.java:96)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextSessionClosed(DefaultIoFilterChain.java:395)
at
org.apache.mina.core.filterchain.DefaultIoFilterChain.fireSessionClosed(DefaultIoFilterChain.java:388)
at
org.apache.mina.core.service.IoServiceListenerSupport.fireSessionDestroyed(IoServiceListenerSupport.java:211)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor.removeNow(AbstractPollingIoProcessor.java:489)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor.remove(AbstractPollingIoProcessor.java:458)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:57)
at
org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:872)
at
org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
----------------------------
Thanks for reading.
Re: OrderedThreadPoolExecutor shutdown problem in MINA 2.0.0-M4
Posted by Maarten Bosteels <mb...@gmail.com>.
Hi,
And it also depends on how you define "performance".
Are you after lower latency or higher throughput ?
Maarten
On Thu, Feb 26, 2009 at 10:24 AM, Emmanuel Lecharny <el...@apache.org>wrote:
> Patrizio Munzi wrote:
>
>> Doing some performance tests on my client/server application with two
>> 2-CPUs machines I see that adding two thread pools (one for
>> MESSAGE_RECEIVED and one for WRITE) application performances lower.
>> Of course it depends from my application and test machines but it shows
>> that using thread pools doesn't always brings benefit with it.
>>
>
> Can you post the performance results you get in the different cases you
> tested ?
>
>>
>> I wonder if performances reduction in case of thread pools executors
>> depends on some kind of overhead spreading work among threads.
>>
> There are many factors :
> - you now have a pool to manage : that mean some synchronization
> - spreading jobs on threads does not lead to a linear improvement
> - depending on what the threads are doing, you may have a lower performance
> if, for instance, you are sharing data across tasks
> - from the MINA pov, adding a new filter in the chain add some more
> processing (each filter induce 3 more methods calls), but that should be not
> measurable.
>
> Now, it also depends on how many threads you manage in the pool.
>
>
>
>
> --
> --
> cordialement, regards,
> Emmanuel Lécharny
> www.iktek.com
> directory.apache.org
>
>
>
Re: OrderedThreadPoolExecutor shutdown problem in MINA 2.0.0-M4
Posted by Emmanuel Lecharny <el...@apache.org>.
Patrizio Munzi wrote:
> Doing some performance tests on my client/server application with two
> 2-CPUs machines I see that adding two thread pools (one for
> MESSAGE_RECEIVED and one for WRITE) application performances lower.
> Of course it depends from my application and test machines but it
> shows that using thread pools doesn't always brings benefit with it.
Can you post the performance results you get in the different cases you
tested ?
>
> I wonder if performances reduction in case of thread pools executors
> depends on some kind of overhead spreading work among threads.
There are many factors :
- you now have a pool to manage : that mean some synchronization
- spreading jobs on threads does not lead to a linear improvement
- depending on what the threads are doing, you may have a lower
performance if, for instance, you are sharing data across tasks
- from the MINA pov, adding a new filter in the chain add some more
processing (each filter induce 3 more methods calls), but that should be
not measurable.
Now, it also depends on how many threads you manage in the pool.
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
Re: OrderedThreadPoolExecutor shutdown problem in MINA 2.0.0-M4
Posted by Patrizio Munzi <pa...@eris4.com>.
Doing some performance tests on my client/server application with two
2-CPUs machines I see that adding two thread pools (one for
MESSAGE_RECEIVED and one for WRITE) application performances lower.
Of course it depends from my application and test machines but it shows
that using thread pools doesn't always brings benefit with it.
I wonder if performances reduction in case of thread pools executors
depends on some kind of overhead spreading work among threads.
Cheers
Emmanuel Lecharny wrote:
> David Rosenstrauch wrote:
>> Hmmm .... this seemed to solve the problem for me:
>>
>> readerThreadPool = new OrderedThreadPoolExecutor();
>> filterChain.addLast("readExecutor", new
>> ExecutorFilter(readerThreadPool, IoEventType.MESSAGE_RECEIVED));
>> filterChain.addLast("codec", new ProtocolCodecFilter(codecFactory));
>> writerThreadPool = new OrderedThreadPoolExecutor();
>> filterChain.addLast("writeExecutor", new
>> ExecutorFilter(writerThreadPool, IoEventType.WRITE));
>>
>>
>> ... and then on session close:
>>
>> readerThreadPool.shutdown();
>> writerThreadPool.shutdown();
>>
>>
>> I'm guessing that what going on previously is that when I was using
>> the single thread pool I wasn't explicitly specifying event types
>> that the thread pool should be used for, and so it was also being
>> used to handle events like session closed or (in your case) exception
>> caught. And since the thread pool was already shut down, it wasn't
>> able to handle those events.
>>
>> That said, I'm a bit unclear about how this thread pool stuff works
>> in Mina 2.0. Is it the code I wrote above considered good practice?
>> i.e.:
>>
>> * is it sensible to have a thread pool handling MESSAGE_RECEIVED, and
>> WRITE?
> It depends on how expensive those operations are. The best would be to
> do some stress test, and see if yur program is bound by the CPU, by
> the bandwith or by the tasks handled in the filters or handler for
> received or write messages. Typically, if encoding or decoding a
> message is a costly operation, without an executor on a N-way CPU
> server, that might be a contention.
>>
>> * is it sensible to NOT have the thread pool handling other event
>> types? (e.g., session closed and exception)
> Again, it will depend on how expensive those operations are. Usually,
> I don't think it works using a thread from a pool for them.
>>
>> * Can someone explain why the DEFAULT_EVENT_SET in ExecutorFilter is
>> defined as it is? i.e.:
>> private static IoEventType[] DEFAULT_EVENT_SET = new IoEventType[] {
>> IoEventType.EXCEPTION_CAUGHT,
>> IoEventType.MESSAGE_RECEIVED,
>> IoEventType.MESSAGE_SENT,
>> IoEventType.SESSION_CLOSED,
>> IoEventType.SESSION_IDLE,
>> IoEventType.SESSION_OPENED
>> };
>> Why are all these events included?
>>
>> Perhaps someone more knowledgeable about Mina can offer an explanation.
> My understanding is that the Executor filter just adds an executor in
> the chain which will be used to spread the load on many threads. This
> is an optimization, rather than something you need to use. If you
> don't use it, your program will work fine.
>
> Now, considering that all your filters are thread safe (and the codec
> is probably the part which has to be double checked, as it's the
> filter user's will write), you will be safe whatever you do.
>
> If you have a lot of work to do when handling a SESSION_CLOSED or a
> SESSION_OPENED event (for instance, you need to open/close a database
> connection), then using an executor for those events might be better.
>
> The default is to handle all the events, I have no idea if it's a good
> politic or not. One might object that handling only MESSAGE_RECEIVED
> and MESSAGE_SENT is strange, as there are other events available.
> Anyway, this is configurable, so ...
>
> My 2 cts...
>
--
*Patrizio Munzi*
Product Specialist
Viale Bruno Buozzi, 19 - 00197 Roma (Italy)
tel: +39 06 4543 3540
fax: +39 06 4543 3587
mobile: +39 393 7195 164
mail: patrizio.munzi@eris4.com <ma...@eris4.com>
web: http://www.eris4.com <http://www.eris4.com/>
skype: eris4_munzi <skype:eris4_munzi?add>
Re: OrderedThreadPoolExecutor shutdown problem in MINA 2.0.0-M4
Posted by Emmanuel Lecharny <el...@apache.org>.
David Rosenstrauch wrote:
> Hmmm .... this seemed to solve the problem for me:
>
> readerThreadPool = new OrderedThreadPoolExecutor();
> filterChain.addLast("readExecutor", new
> ExecutorFilter(readerThreadPool, IoEventType.MESSAGE_RECEIVED));
> filterChain.addLast("codec", new ProtocolCodecFilter(codecFactory));
> writerThreadPool = new OrderedThreadPoolExecutor();
> filterChain.addLast("writeExecutor", new
> ExecutorFilter(writerThreadPool, IoEventType.WRITE));
>
>
> ... and then on session close:
>
> readerThreadPool.shutdown();
> writerThreadPool.shutdown();
>
>
> I'm guessing that what going on previously is that when I was using
> the single thread pool I wasn't explicitly specifying event types
> that the thread pool should be used for, and so it was also being used
> to handle events like session closed or (in your case) exception
> caught. And since the thread pool was already shut down, it wasn't
> able to handle those events.
>
> That said, I'm a bit unclear about how this thread pool stuff works in
> Mina 2.0. Is it the code I wrote above considered good practice? i.e.:
>
> * is it sensible to have a thread pool handling MESSAGE_RECEIVED, and
> WRITE?
It depends on how expensive those operations are. The best would be to
do some stress test, and see if yur program is bound by the CPU, by the
bandwith or by the tasks handled in the filters or handler for received
or write messages. Typically, if encoding or decoding a message is a
costly operation, without an executor on a N-way CPU server, that might
be a contention.
>
> * is it sensible to NOT have the thread pool handling other event
> types? (e.g., session closed and exception)
Again, it will depend on how expensive those operations are. Usually, I
don't think it works using a thread from a pool for them.
>
> * Can someone explain why the DEFAULT_EVENT_SET in ExecutorFilter is
> defined as it is? i.e.:
> private static IoEventType[] DEFAULT_EVENT_SET = new IoEventType[] {
> IoEventType.EXCEPTION_CAUGHT,
> IoEventType.MESSAGE_RECEIVED,
> IoEventType.MESSAGE_SENT,
> IoEventType.SESSION_CLOSED,
> IoEventType.SESSION_IDLE,
> IoEventType.SESSION_OPENED
> };
> Why are all these events included?
>
> Perhaps someone more knowledgeable about Mina can offer an explanation.
My understanding is that the Executor filter just adds an executor in
the chain which will be used to spread the load on many threads. This is
an optimization, rather than something you need to use. If you don't use
it, your program will work fine.
Now, considering that all your filters are thread safe (and the codec is
probably the part which has to be double checked, as it's the filter
user's will write), you will be safe whatever you do.
If you have a lot of work to do when handling a SESSION_CLOSED or a
SESSION_OPENED event (for instance, you need to open/close a database
connection), then using an executor for those events might be better.
The default is to handle all the events, I have no idea if it's a good
politic or not. One might object that handling only MESSAGE_RECEIVED and
MESSAGE_SENT is strange, as there are other events available. Anyway,
this is configurable, so ...
My 2 cts...
--
--
cordialement, regards,
Emmanuel Lécharny
www.iktek.com
directory.apache.org
Re: OrderedThreadPoolExecutor shutdown problem in MINA 2.0.0-M4
Posted by David Rosenstrauch <da...@darose.net>.
Hmmm .... this seemed to solve the problem for me:
readerThreadPool = new OrderedThreadPoolExecutor();
filterChain.addLast("readExecutor", new ExecutorFilter(readerThreadPool,
IoEventType.MESSAGE_RECEIVED));
filterChain.addLast("codec", new ProtocolCodecFilter(codecFactory));
writerThreadPool = new OrderedThreadPoolExecutor();
filterChain.addLast("writeExecutor", new
ExecutorFilter(writerThreadPool, IoEventType.WRITE));
... and then on session close:
readerThreadPool.shutdown();
writerThreadPool.shutdown();
I'm guessing that what going on previously is that when I was using the
single thread pool I wasn't explicitly specifying event types that the
thread pool should be used for, and so it was also being used to handle
events like session closed or (in your case) exception caught. And
since the thread pool was already shut down, it wasn't able to handle
those events.
That said, I'm a bit unclear about how this thread pool stuff works in
Mina 2.0. Is it the code I wrote above considered good practice? i.e.:
* is it sensible to have a thread pool handling MESSAGE_RECEIVED, and WRITE?
* is it sensible to NOT have the thread pool handling other event types?
(e.g., session closed and exception)
* Can someone explain why the DEFAULT_EVENT_SET in ExecutorFilter is
defined as it is? i.e.:
private static IoEventType[] DEFAULT_EVENT_SET = new IoEventType[] {
IoEventType.EXCEPTION_CAUGHT,
IoEventType.MESSAGE_RECEIVED,
IoEventType.MESSAGE_SENT,
IoEventType.SESSION_CLOSED,
IoEventType.SESSION_IDLE,
IoEventType.SESSION_OPENED
};
Why are all these events included?
Perhaps someone more knowledgeable about Mina can offer an explanation.
Thanks,
DR
Patrizio Munzi wrote:
> Hi everybody,
>
> I'm having the following exception after the shutdown of the
> OrderedThreadPoolExecutor.
> It looks not an exception I'm responsible for.
>
> I add an OrderedThreadPoolExecutor to my FilterChain as following:
>
> ----------------------------
> ExecutorFilter executorFilter = new ExecutorFilter();
> connector.getFilterChain().addLast("executor", executorFilter);
> ----------------------------
>
> then I shutdown it in the following way:
>
> ----------------------------
> CloseFuture closeFuture=session.close(false);
> closeFuture.awaitUninterruptibly();
> executorFilter.destroy();
> ----------------------------
>
> with the following exception thrown:
>
> ----------------------------
> 750 [NioProcessor-1] WARN
> org.apache.mina.core.filterchain.DefaultIoFilterChain - Unexpected
> exception from exceptionCaught handler.
> java.util.concurrent.RejectedExecutionException
> at
> java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1477)
>
> at
> org.apache.mina.filter.executor.OrderedThreadPoolExecutor.rejectTask(OrderedThreadPoolExecutor.java:309)
>
> at
> org.apache.mina.filter.executor.OrderedThreadPoolExecutor.execute(OrderedThreadPoolExecutor.java:272)
>
> at
> org.apache.mina.filter.executor.ExecutorFilter.fireEvent(ExecutorFilter.java:535)
>
> at
> org.apache.mina.filter.executor.ExecutorFilter.exceptionCaught(ExecutorFilter.java:593)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:480)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1100(DefaultIoFilterChain.java:48)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.exceptionCaught(DefaultIoFilterChain.java:797)
>
> at
> org.apache.mina.core.filterchain.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:112)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:480)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$1100(DefaultIoFilterChain.java:48)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.exceptionCaught(DefaultIoFilterChain.java:797)
>
> at
> org.apache.mina.core.filterchain.IoFilterAdapter.exceptionCaught(IoFilterAdapter.java:112)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextExceptionCaught(DefaultIoFilterChain.java:480)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain.fireExceptionCaught(DefaultIoFilterChain.java:468)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextSessionClosed(DefaultIoFilterChain.java:397)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$900(DefaultIoFilterChain.java:48)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.sessionClosed(DefaultIoFilterChain.java:787)
>
> at
> org.apache.mina.filter.codec.ProtocolCodecFilter.sessionClosed(ProtocolCodecFilter.java:363)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextSessionClosed(DefaultIoFilterChain.java:395)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain.access$900(DefaultIoFilterChain.java:48)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain$EntryImpl$1.sessionClosed(DefaultIoFilterChain.java:787)
>
> at
> org.apache.mina.core.filterchain.IoFilterAdapter.sessionClosed(IoFilterAdapter.java:96)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain.callNextSessionClosed(DefaultIoFilterChain.java:395)
>
> at
> org.apache.mina.core.filterchain.DefaultIoFilterChain.fireSessionClosed(DefaultIoFilterChain.java:388)
>
> at
> org.apache.mina.core.service.IoServiceListenerSupport.fireSessionDestroyed(IoServiceListenerSupport.java:211)
>
> at
> org.apache.mina.core.polling.AbstractPollingIoProcessor.removeNow(AbstractPollingIoProcessor.java:489)
>
> at
> org.apache.mina.core.polling.AbstractPollingIoProcessor.remove(AbstractPollingIoProcessor.java:458)
>
> at
> org.apache.mina.core.polling.AbstractPollingIoProcessor.access$600(AbstractPollingIoProcessor.java:57)
>
> at
> org.apache.mina.core.polling.AbstractPollingIoProcessor$Processor.run(AbstractPollingIoProcessor.java:872)
>
> at
> org.apache.mina.util.NamePreservingRunnable.run(NamePreservingRunnable.java:65)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
>
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
>
> at java.lang.Thread.run(Thread.java:595)
> ----------------------------
>
> Thanks for reading.