You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Brandon Williams (JIRA)" <ji...@apache.org> on 2012/06/27 20:18:44 UTC
[jira] [Created] (CASSANDRA-4383) Binary encoding of vnode tokens
Brandon Williams created CASSANDRA-4383:
-------------------------------------------
Summary: Binary encoding of vnode tokens
Key: CASSANDRA-4383
URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
Project: Cassandra
Issue Type: Improvement
Reporter: Brandon Williams
Assignee: Brandon Williams
Fix For: 1.2
Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444935#comment-13444935 ]
Brandon Williams commented on CASSANDRA-4383:
---------------------------------------------
bq. The approach taken here continues to make me a little hesitant simply because, I think, it introduces for the first time a need for proper ordering of STATE transmission/reception. I don't have a clear-enough understanding of how the underlying messaging works to know if we can firmly rely on that or not
In this case, it's ok, because SS only reacts to STATUS, so we can change any other gossip state before changing that one and, regardless of how many gossip events fire in the meantime, be guaranteed that other hosts react to the STATUS update after all other gossip state changes are received.
bq. What I had in mind was (at least) something like a sendState{Normal,Bootstrap,...}(Collection<token> tokens) to encapsulate those operations sensitive to ordering. Gossiper.addLocalApplicationStates(...) still makes it too easy to do the wrong thing.
That's basically syntactic sugar over {{Gossiper.addLocalApplicationStates(...)}} which we'd still need as a building block. I've given up on my previous attempt which actually did not provide atomicity, and it looks like doing that would require rewriting EndpointState to use the Reference and SnapTree approach similar to AtomicSortedColumns, and that is way, way out of scope for this ticket (especially since we don't actually need it)
bq. And in addition to getHostId, usesHostId also seems better suited to the Gossiper.
You're right, updated patch moves those methods to Gossiper. I kept the name {{getHostId}} there though since, by asking the gossiper, you should know you're getting the hostId as it is *in gossip*, which may be a necessary thing instead of tMD for non-ring members (or old removed members, or whatever.)
bq. But, mostly I meant that it doesn't read as well as the old code that clearly did one thing when the version was < X, and another when it was >= Y.
That makes sense, I brought the old NET_VERSION checks back, but relocated to the gossiper.
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13439816#comment-13439816 ]
Eric Evans commented on CASSANDRA-4383:
---------------------------------------
In so much that I fully understand the implications, it looks OK to me. A few things:
You're using the presence of a hostID as an implicit indication of the version, and a determination of whether or not to try decoding more than just the legacy token. Going back to a test of the MS version might read better, and wouldn't mask a bug where MS >= 1.2 but a host ID wasn't set.
In {{SS.bootstrap()}}, I think the TOKENS state should be set before the STATUS normal. Also, since that's not the only place where the order of those 2 things matters, it might make sense to encapsulate that in a method, (or at the very least, clearly comment them).
And, of course as you mention, there are the tests... :)
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 4383-v1.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: fix-notifications.txt
We are guaranteed that both states will be received together, however, these states are held in a map, then iterated and applied as well as notifying in the same loop. Since map isn't sorted, sometimes this puts STATUS before TOKENS and immediately notifies for it, causing the NPE since TOKENS isn't set yet. This can also cause problems on token changes, since the status event will fire with the old tokens and never update tMD with the new ones.
Patch to make two loops instead, one to set states, and the other to handle notifications.
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt, fix-notifications.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: (was: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt)
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Issue Type: Sub-task (was: Improvement)
Parent: CASSANDRA-4119
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: (was: 0002-Fix-tests.txt)
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: 0003-Add-tokens-and-status-atomically.txt
0002-Fix-tests.txt
0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt
Updated patchset with working tests. Patch 0003 is something we can take or leave, it just seemed to make sense to have a way to atomically set two gossip states at once in case the gossiper fires in between adding the first and second state, however it's ok for us to gossip TOKENS without STATUS, since STATUS is what fires events. I'm also not 100% certain it actually adds them atomically, since EndpointState is backed by NBHM.
The last wrinkle is the case of bootstrapping a node without using vnodes, and without specifying a token. This technically still works, but produces a harmless NPE on the existing nodes when they first see the state and look for TOKENS. I'm hesitant to allow this to pass through though since it could paper over a truly serious bug, and bootstrapping in this fashion is already deprecated by vnodes. Thoughts?
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt, 0003-Add-tokens-and-status-atomically.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams reopened CASSANDRA-4383:
-----------------------------------------
Ok, something is up with the second trace. Debugging this is currently very painful with CASSANDRA-4615 in the way but I'll get it sorted.
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eric Evans reopened CASSANDRA-4383:
-----------------------------------
I'm seeing these:
{noformat}
java.lang.NullPointerException
at org.apache.cassandra.utils.UUIDGen.decompose(UUIDGen.java:120)
at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpointInternal(HintedHandOffManager.java:304)
at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:250)
at org.apache.cassandra.db.HintedHandOffManager.access$400(HintedHandOffManager.java:87)
at org.apache.cassandra.db.HintedHandOffManager$4.runMayThrow(HintedHandOffManager.java:433)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
... and some of these:
{noformat}
java.lang.NullPointerException
at org.apache.cassandra.service.StorageService.getApplicationStateValue(StorageService.java:1117)
at org.apache.cassandra.service.StorageService.getTokensFor(StorageService.java:1125)
at org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:1196)
at org.apache.cassandra.service.StorageService.onChange(StorageService.java:1103)
at org.apache.cassandra.gms.Gossiper.doNotifications(Gossiper.java:915)
at org.apache.cassandra.gms.Gossiper.applyNewStates(Gossiper.java:906)
at org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:868)
at org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:43)
at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:55)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{noformat}
In both cases it looks like an event is firing before HOST_ID or TOKENS respectively are being set.
To reproduce I use ccm to run a 3 node cluster, replication factor 2. I load about 70k rows at CL = ONE, read them back at CL = ALL, and then restart all of the nodes.
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: (was: 0002-Fix-tests.txt)
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Reviewer: urandom
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2
>
> Attachments: 4383-v1.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: 0002-Fix-tests.txt
0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: (was: 4383-v1.txt)
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: 0002-Fix-tests.txt
0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444176#comment-13444176 ]
Eric Evans commented on CASSANDRA-4383:
---------------------------------------
The approach taken here continues to make me a little hesitant simply because, I think, it introduces for the first time a need for proper ordering of STATE transmission/reception. I don't have a clear-enough understanding of how the underlying messaging works to know if we can firmly rely on that or not (I take you at your word that we can), but it is significant enough to warrant a mention, if for the interface alone.
And on that note...
bq. Patch 0003 is something we can take or leave, it just seemed to make sense to have a way to atomically set two gossip states at once in case the gossiper fires in between adding the first and second state, however it's ok for us to gossip TOKENS without STATUS, since STATUS is what fires events. I'm also not 100% certain it actually adds them atomically, since EndpointState is backed by NBHM.
What I had in mind was (at least) something like a {{sendState\{Normal,Bootstrap,...\}(Collection<token> tokens)}} to encapsulate those operations sensitive to ordering. {{Gossiper.addLocalApplicationStates(...)}} still makes it too easy to do the wrong thing.
A couple of other things:
{{StorageService.getHostId(ep)}} creates a second means of obtaining a host ID, and it's not at all obvious that it should only be used in assigning the value returned by the method of the _same name_ in {{TokenMetadata}}. At a minimum, I think this should be given a new name, one that makes obvious its purpose. However, wouldn't encapsulation be better anyway if this were a {{Gossiper}} method?
And in addition to {{getHostId}}, {{usesHostId}} also seems better suited to the {{Gossiper}}.
Finally, is there any reason that {{TokenSerializer.(de)serialize(...)}} shouldn't be static? What is the instance buying us?
----
As for the whole "...using the presence of a hostID as an implicit indication of the version...", I was unaware of CASSANDRA-4317 (e6530cc3) and (somehow )mistakenly took that as part of this change. Sorry about that.
bq. Well, it's six one way and a half dozen the other. We can look at NET_VERSION instead, but it was also introduced in 1.2, so it's effectively the same thing... and you could have the bug in the opposite direction
If we look for a NET_VERSION that's not there (and should be), then the error is a missing NET_VERSION. If NET_VERSION is >= 1.2, or exists at all if you prefer the implicit (I don't), and there is no HOST_ID, then we have a bug in transmitting/reception of HOST_ID.
But, mostly I meant that it doesn't read as well as the old code that clearly did one thing when the version was < X, and another when it was >= Y.
Anyway, this ship has sailed as far as this ticket goes, so if you'd prefer discuss it elsewhere then that's fine. There is no need to hold any of this against this particular issue/change.
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt, 0003-Add-tokens-and-status-atomically.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426213#comment-13426213 ]
Brandon Williams commented on CASSANDRA-4383:
---------------------------------------------
bq. If I follow you, we'll still need to ensure that all of the legacy bits continue to decode properly as UTF8, yes? Isn't that just relocating the problem?
If we promote HOST_ID and TOKENS we can do whatever we like there. I guess we'll just start accepting bytes in VV for those methods since there's no need to mess with the others.
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2
>
> Attachments: 4383-v1.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13443456#comment-13443456 ]
Brandon Williams commented on CASSANDRA-4383:
---------------------------------------------
bq. You're using the presence of a hostID as an implicit indication of the version, and a determination of whether or not to try decoding more than just the legacy token. Going back to a test of the MS version might read better, and wouldn't mask a bug where MS >= 1.2 but a host ID wasn't set.
Well, it's six one way and a half dozen the other. We can look at NET_VERSION instead, but it was also introduced in 1.2, so it's effectively the same thing... and you could have the bug in the opposite direction :(
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt, 0003-Add-tokens-and-status-atomically.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13447385#comment-13447385 ]
Eric Evans commented on CASSANDRA-4383:
---------------------------------------
bq. That's basically syntactic sugar over Gossiper.addLocalApplicationStates(...) which we'd still need as a building block. I've given up on my previous attempt which actually did not provide atomicity, and it looks like doing that would require rewriting EndpointState to use the Reference and SnapTree approach similar to AtomicSortedColumns, and that is way, way out of scope for this ticket (especially since we don't actually need it)
I was more or less looking for someway of safeguarding against future reordering of those two ops. A comment warning future patch-writers would probably work too.
And on that note, in {{StorageService.bootstrap(...)}}, shouldn't {{TOKENS}} come first?
One other minor nit, {{VersionedValue.tokens(...)}} should use {{TokenSerializer.serialize(...)}} statically.
bq. I fixed [the harmless bootstrap NPE] in this patch too, by simple confirming that TOKENS is actually present for the host in handleStateBootstrap, and if not, just handle things the old way since it's a legacy-style bootstrap.
Probably out of scope since this patch never changed anything in this regard, but I wonder if {{StorageService.handleState\{Bootstrap,Normal\}(...)}} wouldn't benefit from some logging when they encounter a non-host ID enabled endpoint. It's a rare but noteworthy event. If you don't want to slip that change in pre-commit, let me know and I might open another ticket for that.
Aside from this, LGTM +1
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13448850#comment-13448850 ]
Eric Evans commented on CASSANDRA-4383:
---------------------------------------
+1
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt, fix-notifications.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams resolved CASSANDRA-4383.
-----------------------------------------
Resolution: Fixed
Committed.
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt, fix-notifications.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt
Patch to promote hostId and tokens to their own app states. I decided to suppress tokens in the gossipinfo output, and likewise left hostIds as strings, since the savings from encoding those in binary is not much and maintaining a list of things we do need to convert back to a human readable format will get tedious. Having a huge list of tokens in gossipinfo is actually a bit counterproductive for troubleshooting gossip issues anyway, and the information is available elsewhere.
What's left now is about a zillion tests to rewrite (again) to support the new states.
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 4383-v1.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: 4383-v1.txt
Attached is a first draft. Some notes:
* we have to keep the first token as text for backward-compatibility, but after that we can encode the rest in binary.
* simply using binary is good enough, since CASSANDRA-4139 will give us varint encoding for free, and beyond that CASSANDRA-3127 gives us compression.
I say first draft, because while everything works with this patch, during the implementation I found some things that, while perhaps beyond the original scope of this ticket, are things we should probably address:
* the logic for serialization and deserialization is split between VV and SS, which is kind of ugly, and we probably need to deser for gossipinfo to both make it useful and avoid annoying flashing the terminal. These are things I could have done, though, I just haven't yet.
* The game of appending things to STATUS and carefully splitting to avoid accidentally tripping over the VV delimiter is both something I'd like to stop doing, and slightly dangerous.
* Since VV uses strings, we have to use the latin-1 codepage to pass the binary tokens to avoid having any bytes eaten by string encoding. This is a bit hackish.
To solve the STATUS and pieces[] problem, I suggest we stop appending things to it right now. Currently LEAVING is the one-off where HOST_ID is NOT included, and there's nothing we can do about that while maintaining compatibility. So what I suggest is we make that the norm, and promote HOST_ID to a new ApplicationState, which will simplify the "do I need to look for a hostId?" checks since the state will be guaranteed to be there for new-style nodes. Similarly, I think we should promote the serialized tokens to a TOKENS ApplicationState, so we can stop deftly avoiding tripping over our string delimiter there. Old-style nodes will still do the the split on STATUS and we'll keep putting the first token there for that, but new-style nodes can process TOKENS directly and safely.
Finally, to avoid the latin-1 hack, we should probably think about converting VV to accepting and writing bytes directly.
Thoughts?
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2
>
> Attachments: 4383-v1.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: (was: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt)
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: (was: 0003-Add-tokens-and-status-atomically.txt)
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: (was: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt)
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams resolved CASSANDRA-4383.
-----------------------------------------
Resolution: Fixed
This is actually CASSANDRA-4384. It does not repro without hints.
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Eric Evans (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13426208#comment-13426208 ]
Eric Evans commented on CASSANDRA-4383:
---------------------------------------
bq. Attached is a first draft ...
Looks good to me.
bq. the logic for serialization and deserialization is split between VV and SS, which is kind of ugly, and we probably need to deser for gossipinfo to both make it useful and avoid annoying flashing the terminal. These are things I could have done, though, I just haven't yet.
Agreed.
bq. The game of appending things to STATUS and carefully splitting to avoid accidentally tripping over the VV delimiter is both something I'd like to stop doing, and slightly dangerous.
Makes sense. If we are committing to no longer appending to status, you could also remove the encoding of the number of tokens in the patch above.
bq. Finally, to avoid the latin-1 hack, we should probably think about converting VV to accepting and writing bytes directly.
{quote}
To solve the STATUS and pieces[] problem, I suggest we stop appending things to it right now. Currently LEAVING is the one-off where HOST_ID is NOT included, and there's nothing we can do about that while maintaining compatibility. So what I suggest is we make that the norm, and promote HOST_ID to a new ApplicationState, which will simplify the "do I need to look for a hostId?" checks since the state will be guaranteed to be there for new-style nodes. Similarly, I think we should promote the serialized tokens to a TOKENS ApplicationState, so we can stop deftly avoiding tripping over our string delimiter there. Old-style nodes will still do the the split on STATUS and we'll keep putting the first token there for that, but new-style nodes can process TOKENS directly and safely.
{quote}
Makes sense to me.
bq. Finally, to avoid the latin-1 hack, we should probably think about converting VV to accepting and writing bytes directly.
If I follow you, we'll still need to ensure that all of the legacy bits continue to decode properly as UTF8, yes? Isn't that just relocating the problem?
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2
>
> Attachments: 4383-v1.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4383:
----------------------------------------
Attachment: (was: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt)
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 4383-v1.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425126#comment-13425126 ]
Brandon Williams commented on CASSANDRA-4383:
---------------------------------------------
As a small bonus, if we promote HOST_ID then representing that in binary is trivial too.
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2
>
> Attachments: 4383-v1.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4383) Binary encoding of vnode tokens
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444944#comment-13444944 ]
Brandon Williams commented on CASSANDRA-4383:
---------------------------------------------
Quoting myself:
bq. The last wrinkle is the case of bootstrapping a node without using vnodes, and without specifying a token. This technically still works, but produces a harmless NPE on the existing nodes when they first see the state and look for TOKENS. I'm hesitant to allow this to pass through though since it could paper over a truly serious bug, and bootstrapping in this fashion is already deprecated by vnodes.
I fixed that in this patch too, by simple confirming that TOKENS is actually present for the host in handleStateBootstrap, and if not, just handle things the old way since it's a legacy-style bootstrap.
> Binary encoding of vnode tokens
> -------------------------------
>
> Key: CASSANDRA-4383
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4383
> Project: Cassandra
> Issue Type: Sub-task
> Reporter: Brandon Williams
> Assignee: Brandon Williams
> Fix For: 1.2.0 beta 1
>
> Attachments: 0001-Add-HOST_ID-and-TOKENS-app-states-binary-serialization.txt, 0002-Fix-tests.txt
>
>
> Since after CASSANDRA-4317 we can know which version a remote node is using (that is, whether it is vnode-aware or not) this a good opportunity to change the token encoding to binary, since with a default of 256 tokens per node even a fixed-length 16 byte encoding per token provides a great deal of savings in gossip traffic over a text representation.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira