You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Brandon Williams (Created) (JIRA)" <ji...@apache.org> on 2011/10/18 19:57:11 UTC
[jira] [Created] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Allow configuration of storage protocol socket buffer
-----------------------------------------------------
Key: CASSANDRA-3378
URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
Project: Cassandra
Issue Type: New Feature
Components: Core
Reporter: Brandon Williams
Priority: Minor
Fix For: 1.0.1
Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Posted by "Michał Michalski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500952#comment-13500952 ]
Michał Michalski commented on CASSANDRA-3378:
---------------------------------------------
Thanks :-)
I'll try to find some time soon and submit a patch (hopefully) by the end of the week.
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Priority: Minor
> Labels: lhf
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Posted by "Michał Michalski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500116#comment-13500116 ]
Michał Michalski commented on CASSANDRA-3378:
---------------------------------------------
I started to investigate this task (lfh label sounds like a good point to get deeper into Cassandra architecture ;-) ), but as I'm new to Cassandra internal architecture, I'm not really sure if I understand the subject of this task. To be more precise - I have a problem with understanding what does "storage protocol" really refers to in this case?
At first I thought it refers to Thrift (and ThriftServer started by CassandraDaemon), but when I was investigating this issue (by checking how "looking at rpc_[send,recv]_buff_size_in_bytes" setting is set and where) I discovered that ThriftServer already uses the setting mentioned in this task.
My second idea was "native server" (org.apache.cassandra.transport.Server) which I found in CassandraDaemon too. It uses ServerBootstrap created using NioServerSocketChannelFactory and it seems possible to add such setting there, but I wonder if this is really the service this issue tells about? Or maybe I'm completely wrong? Could you give me some clue?
Sorry if it's a silly question, but I'm confused ;-)
BTW: Is there maybe a doc that tells a bit more on how Cassandra works in terms of "internal" architecture/dependencies/protocols etc.?
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Priority: Minor
> Labels: lhf
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506571#comment-13506571 ]
Brandon Williams commented on CASSANDRA-3378:
---------------------------------------------
You actually need to set the receive buffer size in IncomingTcpConnection. It sounds like the lack of a buffer for compression is either an oversight or is buffered somewhere deeper in the stack. I'll run some tests when we're ready.
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Assignee: Michał Michalski
> Priority: Minor
> Labels: lhf
> Attachments: cassandra-3378-v1.patch
>
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Posted by "Michał Michalski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michał Michalski updated CASSANDRA-3378:
----------------------------------------
Attachment: cassandra-3378-v1.patch
I've attached an initial patch. As I mentioned I had a little concern on what to do with the default buffer size that is set for BufferedOutputStream and why is it set only for uncompressed OutputStream. I've read CASSANDRA-1896 and I understand why it's set to 4096, but I can only guess why it's not set for compressed stream too. Anyway - I'm pretty sure that user settings should overwrite this default 4096, but I wonder if default of 4096 should stay in code (so default buffer size values for compressed and uncompressed streams are handled separately) or maybe it should be set by default in cassandra.yaml. However, second solution (which I picked for now) causes compressed output stream to have default buffer size of 4096 bytes too, so it leads back to my first concern: is this a behaviour we expect?
Waiting for a comment and I'll improve this patch according to your suggestion.
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Priority: Minor
> Labels: lhf
> Attachments: cassandra-3378-v1.patch
>
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Posted by "Michał Michalski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502305#comment-13502305 ]
Michał Michalski commented on CASSANDRA-3378:
---------------------------------------------
Just pondering... In the existing code there's a BufferedOutputStream with buffer size of 4096 bytes at first, which can be then "overwritten" by a stream with default buffer size (no specified size is passed to constructor) if compression is enabled.
My concern is: if there's a reason for setting different buffer size for compressed and non-compressed connection by default, is this valid to apply the same value (provided by user in config) for both connection types?
Shouldn't they be distinguished on a config level too? For example by "splitting" storage_send_buff_size_bytes (that's how I temporarily named this new param) into two options: storage_uncompressed_send_buff_size_bytes and storage_compressed_send_buff_size_bytes?
Or maybe we assume that for this special environment (high latency connection) the performance gain achieved by using larger buffer is big enough to just ignore the case I mentioned and use the same value in these two places in code?
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Priority: Minor
> Labels: lhf
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3378) Allow configuration
of storage protocol socket buffer
Posted by "Chris Goffinet (Issue Comment Edited) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159157#comment-13159157 ]
Chris Goffinet edited comment on CASSANDRA-3378 at 11/29/11 8:45 AM:
---------------------------------------------------------------------
Assigning to Harish, starting him out with something small as his first patch.
was (Author: lenn0x):
Assigning to Harish, starting him out with some small as his first patch.
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Assignee: Harish Doddi
> Priority: Minor
> Fix For: 1.0.5
>
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-3378:
----------------------------------------
Reviewer: brandon.williams
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Assignee: Michał Michalski
> Priority: Minor
> Labels: lhf
> Attachments: cassandra-3378-v1.patch
>
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Posted by "Chris Goffinet (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13159157#comment-13159157 ]
Chris Goffinet commented on CASSANDRA-3378:
-------------------------------------------
Assigning to Harish, starting him out with some small as his first patch.
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Assignee: Harish Doddi
> Priority: Minor
> Fix For: 1.0.5
>
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13500512#comment-13500512 ]
Brandon Williams commented on CASSANDRA-3378:
---------------------------------------------
"storage protocol" is the internode communication protocol Cassandra uses, which is completely custom (ie, not thrift-based or anything like that.) OutboundTcpConnection is the class you need to modify to set the socket buffer.
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Priority: Minor
> Labels: lhf
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Posted by "Vladimir Sverzhinsky (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Vladimir Sverzhinsky reassigned CASSANDRA-3378:
-----------------------------------------------
Assignee: Vladimir Sverzhinsky (was: Harish Doddi)
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Assignee: Vladimir Sverzhinsky
> Priority: Minor
> Labels: lhf
> Fix For: 1.2
>
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-3378:
--------------------------------------
Fix Version/s: (was: 1.2.0)
Assignee: (was: Vladimir Sverzhinsky)
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Priority: Minor
> Labels: lhf
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Posted by "Chris Goffinet (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Chris Goffinet reassigned CASSANDRA-3378:
-----------------------------------------
Assignee: Harish Doddi
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Assignee: Harish Doddi
> Priority: Minor
> Fix For: 1.0.5
>
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3378) Allow configuration of storage
protocol socket buffer
Posted by "Michał Michalski (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Michał Michalski updated CASSANDRA-3378:
----------------------------------------
Attachment: cassandra-3378-v2.patch
Thanks for reviewing, Brandon.
Changes in v2:
1) Remove receive buffer setting from OutboundTcpConnection - as it's being used for Messaging purposes only and not really needed there
2) Set receive buffer (only) for IncommingTcpConnection. My concern is the way of changing this parameter in a kind of "wide" way - not only for streaming (for which it makes sense for me), but much earlier, so it "affects" messaging parts of code too. I'm unsure of the impact it may have on the messaging service in general - I don't have that much experience with Cassandra & network programming to judge it "just like that".
> Allow configuration of storage protocol socket buffer
> -----------------------------------------------------
>
> Key: CASSANDRA-3378
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3378
> Project: Cassandra
> Issue Type: New Feature
> Components: Core
> Reporter: Brandon Williams
> Assignee: Michał Michalski
> Priority: Minor
> Labels: lhf
> Attachments: cassandra-3378-v1.patch, cassandra-3378-v2.patch
>
>
> Similar to rpc_[send,recv]_buff_size_in_bytes, we should expose this for high latency connections.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira