You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Erik Onnen (JIRA)" <ji...@apache.org> on 2010/08/24 07:20:15 UTC
[jira] Created: (CASSANDRA-1424) Allow fine-tuning of Thrift TCP
behavior for Thrift-managed sockets
Allow fine-tuning of Thrift TCP behavior for Thrift-managed sockets
-------------------------------------------------------------------
Key: CASSANDRA-1424
URL: https://issues.apache.org/jira/browse/CASSANDRA-1424
Project: Cassandra
Issue Type: Improvement
Components: Core
Affects Versions: 0.7 beta 1, 0.7 beta 2, 0.7.0
Reporter: Erik Onnen
Priority: Minor
Fix For: 0.7.0
Would be nice to be able to control low-level TCP socket behaviors for the Thrift-managed sockets used by o.a.c.thrift.CassandraDaemon.
Blatantly stole the idea from here: http://mail-archives.apache.org/mod_mbox/hadoop-hive-commits/201006.mbox/%3C20100616044535.E781723889CB@eris.apache.org%3E
Patch attached, implemented to retain default behavior with current system but allows definition of the following in cassandra.yaml:
* rpc_keepalive (default:false)
* rpc_reuse_addrs (default:false)
* rpc_send_buff_size_in_bytes (default:undefined)
* rpc_recv_buff_size_in_bytes (default:undefined)
Wasn't sure how/if to write a unit test so could use some advice there.
Verified that configs are picked up correctly and the socket accepts them on Linux 2.6 kernel.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1424) Allow fine-tuning of Thrift TCP
behavior for Thrift-managed sockets
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12908600#action_12908600 ]
Jonathan Ellis commented on CASSANDRA-1424:
-------------------------------------------
Is there a valid reason to turn off keepalive or reuseaddr? I'd be happy to just gobally enable those.
> Allow fine-tuning of Thrift TCP behavior for Thrift-managed sockets
> -------------------------------------------------------------------
>
> Key: CASSANDRA-1424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1424
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.7 beta 1, 0.7 beta 2, 0.7.0
> Reporter: Erik Onnen
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: CAS-1424.patch
>
>
> Would be nice to be able to control low-level TCP socket behaviors for the Thrift-managed sockets used by o.a.c.thrift.CassandraDaemon.
> Blatantly stole the idea from here: http://mail-archives.apache.org/mod_mbox/hadoop-hive-commits/201006.mbox/%3C20100616044535.E781723889CB@eris.apache.org%3E
> Patch attached, implemented to retain default behavior with current system but allows definition of the following in cassandra.yaml:
> * rpc_keepalive (default:false)
> * rpc_reuse_addrs (default:false)
> * rpc_send_buff_size_in_bytes (default:undefined)
> * rpc_recv_buff_size_in_bytes (default:undefined)
> Wasn't sure how/if to write a unit test so could use some advice there.
> Verified that configs are picked up correctly and the socket accepts them on Linux 2.6 kernel.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Issue Comment Edited: (CASSANDRA-1424) Allow fine-tuning of
Thrift TCP behavior for Thrift-managed sockets
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907881#action_12907881 ]
Chris Goffinet edited comment on CASSANDRA-1424 at 9/10/10 12:53 AM:
---------------------------------------------------------------------
I support this as well, we are doing similar things at Digg. I like the keepAlive support. I think we should also support letting our users pick which ThriftServer. There are pro's and con's to all of this and forcing Thread per connection IMHO is not the right way to go.
was (Author: lenn0x):
I support this as well, we are doing similar things at Digg. I'll try to see if I can work up a patch if folks feel its useful as well? I like the keepAlive support. I think we should also support letting our users pick which ThriftServer. There are pro's and con's to all of this and forcing Thread per connection IMHO is not the right way to go.
> Allow fine-tuning of Thrift TCP behavior for Thrift-managed sockets
> -------------------------------------------------------------------
>
> Key: CASSANDRA-1424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1424
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.7 beta 1, 0.7 beta 2, 0.7.0
> Reporter: Erik Onnen
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: CAS-1424.patch
>
>
> Would be nice to be able to control low-level TCP socket behaviors for the Thrift-managed sockets used by o.a.c.thrift.CassandraDaemon.
> Blatantly stole the idea from here: http://mail-archives.apache.org/mod_mbox/hadoop-hive-commits/201006.mbox/%3C20100616044535.E781723889CB@eris.apache.org%3E
> Patch attached, implemented to retain default behavior with current system but allows definition of the following in cassandra.yaml:
> * rpc_keepalive (default:false)
> * rpc_reuse_addrs (default:false)
> * rpc_send_buff_size_in_bytes (default:undefined)
> * rpc_recv_buff_size_in_bytes (default:undefined)
> Wasn't sure how/if to write a unit test so could use some advice there.
> Verified that configs are picked up correctly and the socket accepts them on Linux 2.6 kernel.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1424) Allow fine-tuning of Thrift TCP
behavior for Thrift-managed sockets
Posted by "Erik Onnen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Erik Onnen updated CASSANDRA-1424:
----------------------------------
Attachment: CAS-1424.patch
> Allow fine-tuning of Thrift TCP behavior for Thrift-managed sockets
> -------------------------------------------------------------------
>
> Key: CASSANDRA-1424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1424
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.7 beta 1, 0.7 beta 2, 0.7.0
> Reporter: Erik Onnen
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: CAS-1424.patch
>
>
> Would be nice to be able to control low-level TCP socket behaviors for the Thrift-managed sockets used by o.a.c.thrift.CassandraDaemon.
> Blatantly stole the idea from here: http://mail-archives.apache.org/mod_mbox/hadoop-hive-commits/201006.mbox/%3C20100616044535.E781723889CB@eris.apache.org%3E
> Patch attached, implemented to retain default behavior with current system but allows definition of the following in cassandra.yaml:
> * rpc_keepalive (default:false)
> * rpc_reuse_addrs (default:false)
> * rpc_send_buff_size_in_bytes (default:undefined)
> * rpc_recv_buff_size_in_bytes (default:undefined)
> Wasn't sure how/if to write a unit test so could use some advice there.
> Verified that configs are picked up correctly and the socket accepts them on Linux 2.6 kernel.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1424) Allow fine-tuning of Thrift TCP
behavior for Thrift-managed sockets
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12911008#action_12911008 ]
Hudson commented on CASSANDRA-1424:
-----------------------------------
Integrated in Cassandra #539 (See [https://hudson.apache.org/hudson/job/Cassandra/539/])
add options to configure Thrift socket keepalive and buffer sizes
patch by Erik Onnen and jbellis for CASSANDRA-1424
> Allow fine-tuning of Thrift TCP behavior for Thrift-managed sockets
> -------------------------------------------------------------------
>
> Key: CASSANDRA-1424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1424
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Erik Onnen
> Assignee: Erik Onnen
> Priority: Minor
> Fix For: 0.7 beta 2
>
> Attachments: 1424-v2.txt, CAS-1424.patch
>
>
> Would be nice to be able to control low-level TCP socket behaviors for the Thrift-managed sockets used by o.a.c.thrift.CassandraDaemon.
> Blatantly stole the idea from here: http://mail-archives.apache.org/mod_mbox/hadoop-hive-commits/201006.mbox/%3C20100616044535.E781723889CB@eris.apache.org%3E
> Patch attached, implemented to retain default behavior with current system but allows definition of the following in cassandra.yaml:
> * rpc_keepalive (default:false)
> * rpc_reuse_addrs (default:false)
> * rpc_send_buff_size_in_bytes (default:undefined)
> * rpc_recv_buff_size_in_bytes (default:undefined)
> Wasn't sure how/if to write a unit test so could use some advice there.
> Verified that configs are picked up correctly and the socket accepts them on Linux 2.6 kernel.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1424) Allow fine-tuning of Thrift TCP
behavior for Thrift-managed sockets
Posted by "Chris Goffinet (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907881#action_12907881 ]
Chris Goffinet commented on CASSANDRA-1424:
-------------------------------------------
I support this as well, we are doing similar things at Digg. I'll try to see if I can work up a patch if folks feel its useful as well? I like the keepAlive support. I think we should also support letting our users pick which ThriftServer. There are pro's and con's to all of this and forcing Thread per connection IMHO is not the right way to go.
> Allow fine-tuning of Thrift TCP behavior for Thrift-managed sockets
> -------------------------------------------------------------------
>
> Key: CASSANDRA-1424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1424
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.7 beta 1, 0.7 beta 2, 0.7.0
> Reporter: Erik Onnen
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: CAS-1424.patch
>
>
> Would be nice to be able to control low-level TCP socket behaviors for the Thrift-managed sockets used by o.a.c.thrift.CassandraDaemon.
> Blatantly stole the idea from here: http://mail-archives.apache.org/mod_mbox/hadoop-hive-commits/201006.mbox/%3C20100616044535.E781723889CB@eris.apache.org%3E
> Patch attached, implemented to retain default behavior with current system but allows definition of the following in cassandra.yaml:
> * rpc_keepalive (default:false)
> * rpc_reuse_addrs (default:false)
> * rpc_send_buff_size_in_bytes (default:undefined)
> * rpc_recv_buff_size_in_bytes (default:undefined)
> Wasn't sure how/if to write a unit test so could use some advice there.
> Verified that configs are picked up correctly and the socket accepts them on Linux 2.6 kernel.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1424) Allow fine-tuning of Thrift TCP
behavior for Thrift-managed sockets
Posted by "Erik Onnen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907880#action_12907880 ]
Erik Onnen commented on CASSANDRA-1424:
---------------------------------------
Bump?
> Allow fine-tuning of Thrift TCP behavior for Thrift-managed sockets
> -------------------------------------------------------------------
>
> Key: CASSANDRA-1424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1424
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.7 beta 1, 0.7 beta 2, 0.7.0
> Reporter: Erik Onnen
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: CAS-1424.patch
>
>
> Would be nice to be able to control low-level TCP socket behaviors for the Thrift-managed sockets used by o.a.c.thrift.CassandraDaemon.
> Blatantly stole the idea from here: http://mail-archives.apache.org/mod_mbox/hadoop-hive-commits/201006.mbox/%3C20100616044535.E781723889CB@eris.apache.org%3E
> Patch attached, implemented to retain default behavior with current system but allows definition of the following in cassandra.yaml:
> * rpc_keepalive (default:false)
> * rpc_reuse_addrs (default:false)
> * rpc_send_buff_size_in_bytes (default:undefined)
> * rpc_recv_buff_size_in_bytes (default:undefined)
> Wasn't sure how/if to write a unit test so could use some advice there.
> Verified that configs are picked up correctly and the socket accepts them on Linux 2.6 kernel.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1424) Allow fine-tuning of Thrift TCP
behavior for Thrift-managed sockets
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1424:
--------------------------------------
Affects Version/s: (was: 0.7 beta 1)
(was: 0.7.0)
(was: 0.7 beta 2)
> Allow fine-tuning of Thrift TCP behavior for Thrift-managed sockets
> -------------------------------------------------------------------
>
> Key: CASSANDRA-1424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1424
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Erik Onnen
> Assignee: Erik Onnen
> Priority: Minor
> Fix For: 0.7 beta 2
>
> Attachments: 1424-v2.txt, CAS-1424.patch
>
>
> Would be nice to be able to control low-level TCP socket behaviors for the Thrift-managed sockets used by o.a.c.thrift.CassandraDaemon.
> Blatantly stole the idea from here: http://mail-archives.apache.org/mod_mbox/hadoop-hive-commits/201006.mbox/%3C20100616044535.E781723889CB@eris.apache.org%3E
> Patch attached, implemented to retain default behavior with current system but allows definition of the following in cassandra.yaml:
> * rpc_keepalive (default:false)
> * rpc_reuse_addrs (default:false)
> * rpc_send_buff_size_in_bytes (default:undefined)
> * rpc_recv_buff_size_in_bytes (default:undefined)
> Wasn't sure how/if to write a unit test so could use some advice there.
> Verified that configs are picked up correctly and the socket accepts them on Linux 2.6 kernel.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-1424) Allow fine-tuning of Thrift TCP
behavior for Thrift-managed sockets
Posted by "Erik Onnen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12908842#action_12908842 ]
Erik Onnen commented on CASSANDRA-1424:
---------------------------------------
I've seen one or two hyper-secure environments where reuseaddr was frowned upon, that's about the only thing I can think of. If keepalives are dropped by policy (mobile networks for example) then it's wasteful but not harmful to have it on.
As long as there are ways to disable the behavior, I'm all for making them default on.
> Allow fine-tuning of Thrift TCP behavior for Thrift-managed sockets
> -------------------------------------------------------------------
>
> Key: CASSANDRA-1424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1424
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.7 beta 1, 0.7 beta 2, 0.7.0
> Reporter: Erik Onnen
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: CAS-1424.patch
>
>
> Would be nice to be able to control low-level TCP socket behaviors for the Thrift-managed sockets used by o.a.c.thrift.CassandraDaemon.
> Blatantly stole the idea from here: http://mail-archives.apache.org/mod_mbox/hadoop-hive-commits/201006.mbox/%3C20100616044535.E781723889CB@eris.apache.org%3E
> Patch attached, implemented to retain default behavior with current system but allows definition of the following in cassandra.yaml:
> * rpc_keepalive (default:false)
> * rpc_reuse_addrs (default:false)
> * rpc_send_buff_size_in_bytes (default:undefined)
> * rpc_recv_buff_size_in_bytes (default:undefined)
> Wasn't sure how/if to write a unit test so could use some advice there.
> Verified that configs are picked up correctly and the socket accepts them on Linux 2.6 kernel.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-1424) Allow fine-tuning of Thrift TCP
behavior for Thrift-managed sockets
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-1424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-1424:
--------------------------------------
Attachment: 1424-v2.txt
did some cleanup (v2 attached).
reuseaddr is a no-op tho b/c it's being set during accept, not bind. (TSocket out of the box always sets reuseaddr.) if you care deeply about making it optionally false, you'll have to override more of TSocket, otherwise, i'm happy to drop that until someone actually complains.
> Allow fine-tuning of Thrift TCP behavior for Thrift-managed sockets
> -------------------------------------------------------------------
>
> Key: CASSANDRA-1424
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1424
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.7 beta 1, 0.7 beta 2, 0.7.0
> Reporter: Erik Onnen
> Priority: Minor
> Fix For: 0.7.0
>
> Attachments: 1424-v2.txt, CAS-1424.patch
>
>
> Would be nice to be able to control low-level TCP socket behaviors for the Thrift-managed sockets used by o.a.c.thrift.CassandraDaemon.
> Blatantly stole the idea from here: http://mail-archives.apache.org/mod_mbox/hadoop-hive-commits/201006.mbox/%3C20100616044535.E781723889CB@eris.apache.org%3E
> Patch attached, implemented to retain default behavior with current system but allows definition of the following in cassandra.yaml:
> * rpc_keepalive (default:false)
> * rpc_reuse_addrs (default:false)
> * rpc_send_buff_size_in_bytes (default:undefined)
> * rpc_recv_buff_size_in_bytes (default:undefined)
> Wasn't sure how/if to write a unit test so could use some advice there.
> Verified that configs are picked up correctly and the socket accepts them on Linux 2.6 kernel.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.