You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "paul cannon (JIRA)" <ji...@apache.org> on 2012/08/14 11:07:38 UTC
[jira] [Created] (CASSANDRA-4539) potential backwards
incompatibility in native protocol
paul cannon created CASSANDRA-4539:
--------------------------------------
Summary: potential backwards incompatibility in native protocol
Key: CASSANDRA-4539
URL: https://issues.apache.org/jira/browse/CASSANDRA-4539
Project: Cassandra
Issue Type: Improvement
Components: API
Affects Versions: 1.2
Reporter: paul cannon
Assignee: Sylvain Lebresne
Priority: Minor
Fix For: 1.2
In the text of the native_protocol.spec document, it explains the format for a notation called {{[option]}}, which should represent "{{a pair of <id><value>}}".
In doing a first-draft implementation of the protocol for the python driver, though, I found that I had a misunderstanding. I read that section as saying that {{<value>}} was a {{[value]}}, and that it could have a length of 0 (i.e., the {{[int]}} on the front of the {{[value]}} could be 0). However, it turns out that {{<value>}} might not be there at all, or might be *two* {{[value]}}'s, depending on the option id and message context.
I'm not a fan of this, since
* A protocol parsing library can't simply implement a single function to read in {{[option]}}'s, since the length of the value part is dependent on the message context
* If we add a new native data type (a new option id which could be used inside a {{<col_spec_i>}} in a RESULT message), then older clients will not know how to read past the value part. Of course they won't know how to interpret the data or deserialize later rows of that unknown type - that's not the problem - the problem is that the older client should be able just to mark that column as unparseable and still handle the rest of the columns.
Can we make {{<value>}} be a {{[value]}}, the contents of which can be re-interpreted as a {{[string]}}, an {{[option]}}, two {{[option]}}'s, or whatever?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4539) potential backwards
incompatibility in native protocol
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-4539:
----------------------------------------
Attachment: 4539.txt
Attaching patch to do the change described above, that is use a simple string map for the STARTUP options.
> potential backwards incompatibility in native protocol
> ------------------------------------------------------
>
> Key: CASSANDRA-4539
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4539
> Project: Cassandra
> Issue Type: Improvement
> Components: API
> Affects Versions: 1.2.0 beta 1
> Reporter: paul cannon
> Assignee: Sylvain Lebresne
> Priority: Minor
> Labels: cql, native_protocol
> Fix For: 1.2.0 beta 1
>
> Attachments: 4539.txt
>
>
> In the text of the native_protocol.spec document, it explains the format for a notation called {{[option]}}, which should represent "{{a pair of <id><value>}}".
> In doing a first-draft implementation of the protocol for the python driver, though, I found that I had a misunderstanding. I read that section as saying that {{<value>}} was a {{[value]}}, and that it could have a length of 0 (i.e., the {{[int]}} on the front of the {{[value]}} could be 0). However, it turns out that {{<value>}} might not be there at all, or might be *two* {{[value]}}'s, depending on the option id and message context.
> I'm not a fan of this, since
> * A protocol parsing library can't simply implement a single function to read in {{[option]}}'s, since the length of the value part is dependent on the message context
> * If we add a new native data type (a new option id which could be used inside a {{<col_spec_i>}} in a RESULT message), then older clients will not know how to read past the value part. Of course they won't know how to interpret the data or deserialize later rows of that unknown type - that's not the problem - the problem is that the older client should be able just to mark that column as unparseable and still handle the rest of the columns.
> Can we make {{<value>}} be a {{[value]}}, the contents of which can be re-interpreted as a {{[string]}}, an {{[option]}}, two {{[option]}}'s, or whatever?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4539) potential backwards
incompatibility in native protocol
Posted by "paul cannon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13450286#comment-13450286 ]
paul cannon commented on CASSANDRA-4539:
----------------------------------------
+1.
> potential backwards incompatibility in native protocol
> ------------------------------------------------------
>
> Key: CASSANDRA-4539
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4539
> Project: Cassandra
> Issue Type: Improvement
> Components: API
> Affects Versions: 1.2.0 beta 1
> Reporter: paul cannon
> Assignee: Sylvain Lebresne
> Priority: Minor
> Labels: cql, native_protocol
> Fix For: 1.2.0 beta 1
>
> Attachments: 4539.txt
>
>
> In the text of the native_protocol.spec document, it explains the format for a notation called {{[option]}}, which should represent "{{a pair of <id><value>}}".
> In doing a first-draft implementation of the protocol for the python driver, though, I found that I had a misunderstanding. I read that section as saying that {{<value>}} was a {{[value]}}, and that it could have a length of 0 (i.e., the {{[int]}} on the front of the {{[value]}} could be 0). However, it turns out that {{<value>}} might not be there at all, or might be *two* {{[value]}}'s, depending on the option id and message context.
> I'm not a fan of this, since
> * A protocol parsing library can't simply implement a single function to read in {{[option]}}'s, since the length of the value part is dependent on the message context
> * If we add a new native data type (a new option id which could be used inside a {{<col_spec_i>}} in a RESULT message), then older clients will not know how to read past the value part. Of course they won't know how to interpret the data or deserialize later rows of that unknown type - that's not the problem - the problem is that the older client should be able just to mark that column as unparseable and still handle the rest of the columns.
> Can we make {{<value>}} be a {{[value]}}, the contents of which can be re-interpreted as a {{[string]}}, an {{[option]}}, two {{[option]}}'s, or whatever?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4539) potential backwards
incompatibility in native protocol
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447834#comment-13447834 ]
Sylvain Lebresne commented on CASSANDRA-4539:
---------------------------------------------
bq. I read that section as saying that <value> was a [value]
Hum, I see how one could be confused :). Though the idea is really that the two are not related at all. I've renamed [value] to [bytes] in commit 06269b9 to hopefully make if more clear that those are not related.
Now the main _raison d'être_ of [option] is to have a compact way to describe the type in the query response metadata. For that, and given that it's something sent very frequently, I really think we should keep it the way it is. I'm all for making client library implementers job easy as much as possible but adding even one byte per type is just not worth it in that case. And I don't think it's much of a problem in the case we add new data type as long as we fix the option ids for a given protocol version. And if we add a new type, the server can very well use the 'custom' option id until we decide to bump the protocol version to add a specific option id for that new type.
Now for STARTUP, I'll admit that reusing [option] might have been a bit overkill. It would probably be simpler/more flexible to change STARTUP to take some map<string, string> (and to have OPTIONS messages to return a map<string, [string list]>. I'll make that change.
> potential backwards incompatibility in native protocol
> ------------------------------------------------------
>
> Key: CASSANDRA-4539
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4539
> Project: Cassandra
> Issue Type: Improvement
> Components: API
> Affects Versions: 1.2.0 beta 1
> Reporter: paul cannon
> Assignee: Sylvain Lebresne
> Priority: Minor
> Labels: cql, native_protocol
> Fix For: 1.2.0 beta 1
>
>
> In the text of the native_protocol.spec document, it explains the format for a notation called {{[option]}}, which should represent "{{a pair of <id><value>}}".
> In doing a first-draft implementation of the protocol for the python driver, though, I found that I had a misunderstanding. I read that section as saying that {{<value>}} was a {{[value]}}, and that it could have a length of 0 (i.e., the {{[int]}} on the front of the {{[value]}} could be 0). However, it turns out that {{<value>}} might not be there at all, or might be *two* {{[value]}}'s, depending on the option id and message context.
> I'm not a fan of this, since
> * A protocol parsing library can't simply implement a single function to read in {{[option]}}'s, since the length of the value part is dependent on the message context
> * If we add a new native data type (a new option id which could be used inside a {{<col_spec_i>}} in a RESULT message), then older clients will not know how to read past the value part. Of course they won't know how to interpret the data or deserialize later rows of that unknown type - that's not the problem - the problem is that the older client should be able just to mark that column as unparseable and still handle the rest of the columns.
> Can we make {{<value>}} be a {{[value]}}, the contents of which can be re-interpreted as a {{[string]}}, an {{[option]}}, two {{[option]}}'s, or whatever?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4539) potential backwards
incompatibility in native protocol
Posted by "paul cannon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13434599#comment-13434599 ]
paul cannon commented on CASSANDRA-4539:
----------------------------------------
Another potential problem is if a client program wants to use a STARTUP option not yet understood by the driver. If an option's value part is always a {{[value]}}, then the driver doesn't have to guess how to send that argument to the server.
(It might also be worthwhile to make the values in STARTUP optionlists always be {{[string]}} s, so that apps don't have to bother with potentially encoding binary values themselves when the driver doesn't recognize those options.)
> potential backwards incompatibility in native protocol
> ------------------------------------------------------
>
> Key: CASSANDRA-4539
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4539
> Project: Cassandra
> Issue Type: Improvement
> Components: API
> Affects Versions: 1.2
> Reporter: paul cannon
> Assignee: Sylvain Lebresne
> Priority: Minor
> Labels: cql, native_protocol
> Fix For: 1.2
>
>
> In the text of the native_protocol.spec document, it explains the format for a notation called {{[option]}}, which should represent "{{a pair of <id><value>}}".
> In doing a first-draft implementation of the protocol for the python driver, though, I found that I had a misunderstanding. I read that section as saying that {{<value>}} was a {{[value]}}, and that it could have a length of 0 (i.e., the {{[int]}} on the front of the {{[value]}} could be 0). However, it turns out that {{<value>}} might not be there at all, or might be *two* {{[value]}}'s, depending on the option id and message context.
> I'm not a fan of this, since
> * A protocol parsing library can't simply implement a single function to read in {{[option]}}'s, since the length of the value part is dependent on the message context
> * If we add a new native data type (a new option id which could be used inside a {{<col_spec_i>}} in a RESULT message), then older clients will not know how to read past the value part. Of course they won't know how to interpret the data or deserialize later rows of that unknown type - that's not the problem - the problem is that the older client should be able just to mark that column as unparseable and still handle the rest of the columns.
> Can we make {{<value>}} be a {{[value]}}, the contents of which can be re-interpreted as a {{[string]}}, an {{[option]}}, two {{[option]}}'s, or whatever?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4539) potential backwards
incompatibility in native protocol
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4539:
--------------------------------------
Reviewer: thepaul
> potential backwards incompatibility in native protocol
> ------------------------------------------------------
>
> Key: CASSANDRA-4539
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4539
> Project: Cassandra
> Issue Type: Improvement
> Components: API
> Affects Versions: 1.2.0 beta 1
> Reporter: paul cannon
> Assignee: Sylvain Lebresne
> Priority: Minor
> Labels: cql, native_protocol
> Fix For: 1.2.0 beta 1
>
> Attachments: 4539.txt
>
>
> In the text of the native_protocol.spec document, it explains the format for a notation called {{[option]}}, which should represent "{{a pair of <id><value>}}".
> In doing a first-draft implementation of the protocol for the python driver, though, I found that I had a misunderstanding. I read that section as saying that {{<value>}} was a {{[value]}}, and that it could have a length of 0 (i.e., the {{[int]}} on the front of the {{[value]}} could be 0). However, it turns out that {{<value>}} might not be there at all, or might be *two* {{[value]}}'s, depending on the option id and message context.
> I'm not a fan of this, since
> * A protocol parsing library can't simply implement a single function to read in {{[option]}}'s, since the length of the value part is dependent on the message context
> * If we add a new native data type (a new option id which could be used inside a {{<col_spec_i>}} in a RESULT message), then older clients will not know how to read past the value part. Of course they won't know how to interpret the data or deserialize later rows of that unknown type - that's not the problem - the problem is that the older client should be able just to mark that column as unparseable and still handle the rest of the columns.
> Can we make {{<value>}} be a {{[value]}}, the contents of which can be re-interpreted as a {{[string]}}, an {{[option]}}, two {{[option]}}'s, or whatever?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira