You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mina.apache.org by "Trustin Lee (JIRA)" <ji...@apache.org> on 2007/08/18 06:56:30 UTC
[jira] Issue Comment Edited: (DIRMINA-301) New Multi threaded
SocketIOProcessor to improve fairness of socket reads/writes
[ https://issues.apache.org/jira/browse/DIRMINA-301?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12520820 ]
trustin edited comment on DIRMINA-301 at 8/17/07 9:55 PM:
--------------------------------------------------------------
I am sorry for getting back to you very late. You know, I don't have such a blazingly fast device. ;)
I've just resolved DIRMINA-305 (SocketIoProcessor is biased to write operations). I'm very curious how my changes compare to the MultiThreadSocketIoProcessor. Currently, the maximum read/write bytes are 2 * socket buffer size. Please feel free to modify the '<< 1' operations in SocketIoProcessor if you want to compare each other more precisely.
was (Author: trustin):
I am sorry for getting back to you. You know, I don't have such a blazingly fast device. ;)
I've just resolved DIRMINA-305 (SocketIoProcessor is biased to write operations). I'm very curious how my changes compare to the MultiThreadSocketIoProcessor. Currently, the maximum read/write bytes are 2 * socket buffer size. Please feel free to modify the '<< 1' operations in SocketIoProcessor if you want to compare each other more precisely.
> New Multi threaded SocketIOProcessor to improve fairness of socket reads/writes
> -------------------------------------------------------------------------------
>
> Key: DIRMINA-301
> URL: https://issues.apache.org/jira/browse/DIRMINA-301
> Project: MINA
> Issue Type: Improvement
> Components: Transport
> Affects Versions: 1.0.0
> Environment: Problem on all platforms, example test ran on a 8 Way Opteron
> Reporter: Martin Ritchie
> Attachments: JIRA-301-Release-4.zip, Mina Multi Thread SocketIOProcessor(MinaHead).zip, Mina Multi Thread SocketIOProcessor.zip, Mina Multi Thread SocketIOProcessor.zip, MultiThreadSocketIOProcessor-java-1.0-proposal-R4.1-src.zip
>
>
> The current SocketIOProcessor uses a single thread to do both reads and socket flushes. During some testing of Apache Qpid (A messaging broker) which uses mina as a transport we ran in to two problems.
> The first was the client that produces lots of data failed with an OutOfMemoryException, as mentioned in DIRMINA-206.
> The second is that under constant load the broker cannot read and write from the same socket at the same time. I have created a MultiThreaded SocketIOProcessor that allows reads and writes to occur simultaneously. On low core boxes the performance is similar to the existing SocketIOProcessor. However, on higher core boxes the Multi threaded box can be more than twice the speed.
> The attached zip file is an attempt to resolve the second.
> To run the test:
> run "ant " to build everything.
> The memory requirements for each process are shown in parenthesis, these were just the largest numbers shown in top as the processes ran.
> Then
> "ant acceptor_mina"(1.2G) or "ant acceptor_multi"(250M) to run the listener process.
> This process simply sends the received message back down the same socket.
> Then run the corresponding writer
> "ant writer_mina"(560M) or "ant writer_multi"(540M)
> The results from each of the writers I ran on an 8way Opteron are shown below.
> The out of memory issue on the writer can be demonstrated buy running:
> "ant writer_mina_mem" or "ant writer_multi_mem"
> The increased throughput on the Multi Threaded SocketIOProcessor should allow it survive the low memory setting.
> ~/dev/TempProjects/mina-2006-11-02-1056/Mina Multi Thread SocketIOProcessor$ ant writer_mina
> Buildfile: build.xml
> writer_mina:
> [java] main 2006-11-02 10:57:36,932 INFO [apache.mina.SocketIOTest.WriterTest] Starting 2k test
> [java] main 2006-11-02 10:57:36,933 WARN [apache.mina.SocketIOTest.WriterTest] Using MINA NIO
> [java] main 2006-11-02 10:57:36,999 INFO [apache.mina.SocketIOTest.WriterTest] Attempting connection to localhost/127.0.0.1:9999
> [java] main 2006-11-02 10:57:37,035 INFO [apache.mina.SocketIOTest.WriterTest] Connection completed
> [java] Thread-2 2006-11-02 10:57:37,038 INFO [apache.mina.SocketIOTest.WriterTest] Starting to send 200000 buffers of 2048B
> [java] Thread-2 2006-11-02 10:57:43,409 INFO [apache.mina.SocketIOTest.WriterTest] All buffers sent; waiting for receipt from server
> [java] Thread-2 2006-11-02 10:57:58,652 INFO [apache.mina.SocketIOTest.WriterTest] Completed
> [java] Thread-2 2006-11-02 10:57:58,652 INFO [apache.mina.SocketIOTest.WriterTest] Total time waiting for server after last write: 15243
> [java] Thread-2 2006-11-02 10:57:58,652 INFO [apache.mina.SocketIOTest.WriterTest] Total time: 21613
> [java] Thread-2 2006-11-02 10:57:58,652 INFO [apache.mina.SocketIOTest.WriterTest] MB per second: 18951
> [java] Thread-2 2006-11-02 10:57:58,654 INFO [apache.mina.SocketIOTest.WriterTest] Average chunk time: 1.0000000000011369ms
> [java] Thread-2 2006-11-02 10:57:58,654 INFO [apache.mina.SocketIOTest.WriterTest] Maximum WriteRequestQueue size: 171836
> [java] Thread-2 2006-11-02 10:57:58,654 INFO [apache.mina.SocketIOTest.WriterTest] Closing session
> ~/dev/TempProjects/mina-2006-11-02-1056/Mina Multi Thread SocketIOProcessor$ ant writer_multi
> Buildfile: build.xml
> writer_multi:
> [java] main 2006-11-02 10:58:10,544 INFO [apache.mina.SocketIOTest.WriterTest] Starting 2k test
> [java] main 2006-11-02 10:58:10,546 WARN [apache.mina.SocketIOTest.WriterTest] Using MultiThread NIO
> [java] main 2006-11-02 10:58:10,620 INFO [apache.mina.SocketIOTest.WriterTest] Attempting connection to localhost/127.0.0.1:9999
> [java] main 2006-11-02 10:58:10,675 INFO [apache.mina.SocketIOTest.WriterTest] Connection completed
> [java] Thread-3 2006-11-02 10:58:10,678 INFO [apache.mina.SocketIOTest.WriterTest] Starting to send 200000 buffers of 2048B
> [java] Thread-3 2006-11-02 10:58:15,464 INFO [apache.mina.SocketIOTest.WriterTest] All buffers sent; waiting for receipt from server
> [java] Thread-3 2006-11-02 10:58:20,214 INFO [apache.mina.SocketIOTest.WriterTest] Completed
> [java] Thread-3 2006-11-02 10:58:20,215 INFO [apache.mina.SocketIOTest.WriterTest] Total time waiting for server after last write: 4750
> [java] Thread-3 2006-11-02 10:58:20,215 INFO [apache.mina.SocketIOTest.WriterTest] Total time: 9537
> [java] Thread-3 2006-11-02 10:58:20,215 INFO [apache.mina.SocketIOTest.WriterTest] MB per second: 42948
> [java] Thread-3 2006-11-02 10:58:20,216 INFO [apache.mina.SocketIOTest.WriterTest] Average chunk time: 1.0ms
> [java] Thread-3 2006-11-02 10:58:20,216 INFO [apache.mina.SocketIOTest.WriterTest] Maximum WriteRequestQueue size: 159054
> [java] Thread-3 2006-11-02 10:58:20,216 INFO [apache.mina.SocketIOTest.WriterTest] Closing session
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.