You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@thrift.apache.org by "Steve Jiang (JIRA)" <ji...@apache.org> on 2011/05/09 19:38:03 UTC
[jira] [Created] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Java nonblocking server with more than one thread for select and handling IO
----------------------------------------------------------------------------
Key: THRIFT-1167
URL: https://issues.apache.org/jira/browse/THRIFT-1167
Project: Thrift
Issue Type: New Feature
Components: Java - Library
Reporter: Steve Jiang
Attachments: threadedselectorthrift2.diff
I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047397#comment-13047397 ]
Vijay commented on THRIFT-1167:
-------------------------------
Hi Steve,
I think the selector doesn't wakeup all the selectors at once... hence the duplicate keys will be less.
PS: I think if we like this approach i can include throttling too... let me know.
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: TSHSHAServer-Patch-MultiThreaded_v3.patch, threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Steve Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13052717#comment-13052717 ]
Steve Jiang commented on THRIFT-1167:
-------------------------------------
Hi Bryan, are there any blockers that I can help with for getting this patch in?
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: TSHSHAServer-Patch-MultiThreaded_v3.patch, threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Steve Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047432#comment-13047432 ]
Steve Jiang commented on THRIFT-1167:
-------------------------------------
For long-lived connections the approaches are equivalent, as the assignment of a connection to a thread happens only at accept time in both cases. If you want to rebalance per-request, it'd obviously be a tradeoff.
As to performance, I'm speaking from experience in non-JVM environments. Do you time to do a benchmark? I'm a bit busy right now so it'd take a few days for me to get to it.
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: TSHSHAServer-Patch-MultiThreaded_v3.patch, threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Steve Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Jiang updated THRIFT-1167:
--------------------------------
Attachment: (was: threadedselectorthrift2.diff)
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Steve Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044614#comment-13044614 ]
Steve Jiang commented on THRIFT-1167:
-------------------------------------
Hi Bryan, do you have time to take a look at this? Any chance it can make it into thrift 0.7?
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: threadedselectorthrift2.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050635#comment-13050635 ]
Vijay commented on THRIFT-1167:
-------------------------------
Hi Steve,
It was really really hard to get the difference in a real application test.... But i did notice a skew in of number of connections handled by the threads than others (In my approach)... Hence i think your approach is better than the patch which was attached by me...
Hopping to see this patch committed soon :)
Thanks!
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: TSHSHAServer-Patch-MultiThreaded_v3.patch, threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Steve Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Jiang updated THRIFT-1167:
--------------------------------
Attachment: threadedselectorthrift3.diff
Sure, this is an architecture similar to that used by other high-throughput servers. Here's a brief summary of the changes from HsHa:
THsHaServer has an async selector thread that is responsible for accepting new connections and selecting for IO, reading until full request frames are buffered. It then hands off processing to a worker pool (executor) to service the requests.
The threaded selector server splits the role of the selector thread into multiple threads:
An accept thread responsible for accepting connections on the listen socket and handing them off to Selector threads
A collection of (rather than one) Selector threads each responsible for a subset of connected client sockets.
The worker pool (Executor) remains, handling invocations once requests have been read.
This solves two potential issues with HsHa:
1. In an RPC-heavy app server where the thrift processor does relatively little work (e.g. a service for generating UUIDs), a significant portion of CPU time is spent handling selection on sockets and doing reading/writing from the network. The HsHa server handles selection on all sockets and IO on a single thread, which on a multi-core platform limits the request throughput to a single core. This leaves the worker pool that can run on the remaining cores underutilized. Having socket selection and IO split across multiple threads increases maximum throughput in these situations.
2. The HsHa server accepts connections on the selector thread as fast as possible, even when the executor service is saturated and starts throwing RejectedExecutionException. In this case connections are closed whenever exceptions are thrown (after a request has already been read on the connection) potentially causing a lot of flapping. The FAIR_ACCEPT accept policy in the new server submits the accept socket handoff to the same executor service that handles invocations, which results in a close of a new connection immediately if the executor is saturated.
A potential downside to this design is there are additional context switches because accept happens on a separate thread.
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047446#comment-13047446 ]
Vijay commented on THRIFT-1167:
-------------------------------
Sure.... let me give it a try...
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: TSHSHAServer-Patch-MultiThreaded_v3.patch, threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury reassigned THRIFT-1167:
-------------------------------------
Assignee: Bryan Duxbury
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: threadedselectorthrift2.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vijay updated THRIFT-1167:
--------------------------
Attachment: TSHSHAServer-Patch-MultiThreaded_v3.patch
Hi Steve and Bryan,
I think OS does a good job in load balancing and do we really need to implement our own LB?
I implemented something similar for cassandra... Plz see the below result...
I can modify the existing codebase and provide a more integrated patch if you think this approach makes sense.
// total writes into cassandra (*2 will provide you the approx number of read and write into the socket).
lgmacd-vparth:Cass vparthasarathy$ grep "insert writing" /var/log/cassandra/system.log |wc -l
1103
// total
lgmacd-vparth:Cass vparthasarathy$ grep "eciving packets" /var/log/cassandra/system.log|wc -l
2221
// split per thread.
lgmacd-vparth:Cass vparthasarathy$ grep "eciving packets" /var/log/cassandra/system.log|awk '{print $2}'|sort |uniq -c
137 [Selector-Thread-0]
139 [Selector-Thread-10]
138 [Selector-Thread-11]
139 [Selector-Thread-12]
139 [Selector-Thread-13]
139 [Selector-Thread-14]
139 [Selector-Thread-15]
139 [Selector-Thread-1]
139 [Selector-Thread-2]
139 [Selector-Thread-3]
139 [Selector-Thread-4]
139 [Selector-Thread-5]
139 [Selector-Thread-6]
139 [Selector-Thread-7]
139 [Selector-Thread-8]
139 [Selector-Thread-9]
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: TSHSHAServer-Patch-MultiThreaded_v3.patch, threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Steve Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046285#comment-13046285 ]
Steve Jiang commented on THRIFT-1167:
-------------------------------------
The sockets are currently owned by a single thread (assigned round-robin at accept time) for the lifetime of the connection. The server allows overriding "createSelectorThreadLoadBalancer" to provide a smarter assignment strategy. However, this is only balancing at accept time so a particular thread may still be assigned a disproportionate number of long-lived, high-throughput connections but no worse than the current single select thread model.
I haven't run into this problem, but this could be changed so that there's an opportunity to re-assign a connection whenever a request is completed. However, that strategy might introduce more context switches in the non-problematic cases. Did you have other ideas in mind?
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Steve Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steve Jiang updated THRIFT-1167:
--------------------------------
Attachment: threadedselectorthrift2.diff
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Attachments: threadedselectorthrift2.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Bryan Duxbury resolved THRIFT-1167.
-----------------------------------
Resolution: Fixed
Fix Version/s: 0.8
Assignee: Steve Jiang (was: Bryan Duxbury)
I finally fully reviewed this, and it's great stuff. I cleaned up a few minor things (spacing mostly, along with converting the FrameBuffer states into an Enum), and committed this to TRUNK. Really looking forward to seeing the performance improvements! Thanks Steve!
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Steve Jiang
> Fix For: 0.8
>
> Attachments: TSHSHAServer-Patch-MultiThreaded_v3.patch, threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047413#comment-13047413 ]
Vijay commented on THRIFT-1167:
-------------------------------
Hi Steve,
Cool, can we infer that (as isValid is the overhead for frequently re-connecting clients which we are talking about, didn't see any in my tests though) will cancel out with the load balancing over head which we might have? And for long standing connection (connection pooling) in-fact it will have better performant?
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: TSHSHAServer-Patch-MultiThreaded_v3.patch, threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13044616#comment-13044616 ]
Bryan Duxbury commented on THRIFT-1167:
---------------------------------------
@Steve -
First off, sorry for taking so long to look at this patch.
Second, it looks like the patch doesn't apply cleanly to TRUNK anymore. There were a few rejections. Not sure why.
Third, I'd really like to understand the architecture before I dive really deep into the code. Can you comment with a short writeup of the approach?
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: threadedselectorthrift2.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086746#comment-13086746 ]
Hudson commented on THRIFT-1167:
--------------------------------
Integrated in Thrift #223 (See [https://builds.apache.org/job/Thrift/223/])
THRIFT-1167. java: Java nonblocking server with more than one thread for select and handling IO
This patch refactors the nonblocking server hierarchy and adds in a new server that has a threaded selector pool as well as a threaded invoker pool.
Patch: Steve Jiang
bryanduxbury : http://svn.apache.org/viewvc/?view=rev&rev=1158977
Files :
* /thrift/trunk/lib/java/src/org/apache/thrift/server/Invocation.java
* /thrift/trunk/lib/java/src/org/apache/thrift/server/TThreadedSelectorServer.java
* /thrift/trunk/lib/java/test/org/apache/thrift/server/TestThreadedSelectorServer.java
* /thrift/trunk/lib/java/src/org/apache/thrift/server/THsHaServer.java
* /thrift/trunk/lib/java/src/org/apache/thrift/server/AbstractNonblockingServer.java
* /thrift/trunk/lib/java/src/org/apache/thrift/server/TNonblockingServer.java
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Steve Jiang
> Fix For: 0.8
>
> Attachments: TSHSHAServer-Patch-MultiThreaded_v3.patch, threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046279#comment-13046279 ]
Bryan Duxbury commented on THRIFT-1167:
---------------------------------------
OK, that helps a lot.
Here's one question I've been trying to solve. In this architecture, does a given socket get bound to a particular select thread for the lifetime of the connection, or do the sockets move around between selectors opportunistically?
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Steve Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030884#comment-13030884 ]
Steve Jiang commented on THRIFT-1167:
-------------------------------------
Oops, I meant to say: "this server allows better throughput because IO is not limited by a single thread."
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: threadedselectorthrift2.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Bryan Duxbury (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13046287#comment-13046287 ]
Bryan Duxbury commented on THRIFT-1167:
---------------------------------------
OK, that makes sense. I generally don't expect that most people will end up with really skewed selectors, at least for reasonably large numbers of clients.
I had an alternative that is definitely not better, so let's not even talk about it :)
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Steve Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047373#comment-13047373 ]
Steve Jiang commented on THRIFT-1167:
-------------------------------------
Vijay, it looks like we took a similar approach. A couple of comments:
In your server all the IO threads are selecting on the server socket, and you let whichever one accepts first "win" while the losers are doing a small bit of unnecessary work. In the case of frequent accepts I've seen cases where this is enough of a penalty to matter, it depends on the situation. As far as the loadbalancing, I think the round-robin default will give a similar result to what kernel thread scheduling gives in most cases, with an option to do something smarter if needed.
There's nothing throttling accepts if the invoker pool is saturated, which can make bad situations worse. That is a must-have for my app.
That being said, it's a smaller change and doesn't touch the existing servers so it's less risky to commit. I'd probably subclass if this is the version thrift goes with.
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: TSHSHAServer-Patch-MultiThreaded_v3.patch, threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (THRIFT-1167) Java nonblocking server with more
than one thread for select and handling IO
Posted by "Steve Jiang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/THRIFT-1167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13047405#comment-13047405 ]
Steve Jiang commented on THRIFT-1167:
-------------------------------------
Right, I didn't mean all would be woken up but as long as > 1 thread is awoken, there is extra work being done.
> Java nonblocking server with more than one thread for select and handling IO
> ----------------------------------------------------------------------------
>
> Key: THRIFT-1167
> URL: https://issues.apache.org/jira/browse/THRIFT-1167
> Project: Thrift
> Issue Type: New Feature
> Components: Java - Library
> Reporter: Steve Jiang
> Assignee: Bryan Duxbury
> Attachments: TSHSHAServer-Patch-MultiThreaded_v3.patch, threadedselectorthrift3.diff
>
>
> I've used the HsHa server model to write a server that uses a thread for accept and a separate, configurable number of Selector threads to handle IO. I'd like to contribute this back to Thrift.
> For apps that are RPC-heavy and require little computation from the executor pool running on multi-core architectures, this server allows gets throughput as IO is not limited by one CPU.
> Please take a look at the attached patch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira