You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Sylvain Lebresne (JIRA)" <ji...@apache.org> on 2012/06/18 10:11:42 UTC
[jira] [Created] (CASSANDRA-4351) Consider storing more
informations on peers in system tables
Sylvain Lebresne created CASSANDRA-4351:
-------------------------------------------
Summary: Consider storing more informations on peers in system tables
Key: CASSANDRA-4351
URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Sylvain Lebresne
Priority: Minor
Fix For: 1.2
Currently, the only thing we keep in system tables about other peers is their token and IP addresses. We should probably also record the new ring_id, but since CASSANDRA-4018 makes system table easily queriable, may it could be worth adding some more information (basically most of what we gossip could be a candidate (schema UUID, status, C* version, ...)) as a simple way to expose the ring state to users (even if it's just a "view" of the ring state from one specific node I believe it's still nice).
Of course that means storing information that may not be absolutely needed by the server, but I'm not sure there is much harm to that.
Note that doing this cleanly may require changing the schema of current system tables but as long as we do that in the 1.2 timeframe it's ok (since the concerned system table 'local' and 'peers' are news anyway).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4351) Consider storing more
informations on peers in system tables
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13432725#comment-13432725 ]
Sylvain Lebresne commented on CASSANDRA-4351:
---------------------------------------------
The idea for this ticket is that in an ideal world, I could imagine to have a schema like:
{noformat}
CREATE TABLE System.peers (
ring_id uuid PRIMARY KEY,
tokens set<blob>,
peer inet,
schema_version uuid, // so that client can check if the cluster is in agreement
release_version text, // quick check to see where we're at of a rolling upgrade
rpc_address,
dc text,
rack text // a smart client that want to discover node could use that to maintain it's metadata
)
{noformat}
Now the main thing I'm not sure is that currently we have a bunch of cases where we call SystemTable.removeTokens() that I'm not sure how to adapt to such schema. That is, I'm not fully sure I understand why removeTokens is called in some of those cases.
> Consider storing more informations on peers in system tables
> -------------------------------------------------------------
>
> Key: CASSANDRA-4351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2
>
>
> Currently, the only thing we keep in system tables about other peers is their token and IP addresses. We should probably also record the new ring_id, but since CASSANDRA-4018 makes system table easily queriable, may it could be worth adding some more information (basically most of what we gossip could be a candidate (schema UUID, status, C* version, ...)) as a simple way to expose the ring state to users (even if it's just a "view" of the ring state from one specific node I believe it's still nice).
> Of course that means storing information that may not be absolutely needed by the server, but I'm not sure there is much harm to that.
> Note that doing this cleanly may require changing the schema of current system tables but as long as we do that in the 1.2 timeframe it's ok (since the concerned system table 'local' and 'peers' are news anyway).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4351) Consider storing more
informations on peers in system tables
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-4351:
----------------------------------------
Attachment: 0002-Save-tokens-as-strings.txt
0001-4351.txt
Attaching a patch for this. It pretty much change the peers table to look like what's described above. It also change the local table to use an actual set instead of some hand-made serialization of the tokens.
I'm also attaching a second patch to write tokens as string instead of bytes. The pros being that if we do that, the output of reading the local and peers table is more readable (and I don't think the slightly higher space on the disk matter). That being said, if people really prefer keeping them as blob, I won't insist too much.
> Consider storing more informations on peers in system tables
> -------------------------------------------------------------
>
> Key: CASSANDRA-4351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-4351.txt, 0002-Save-tokens-as-strings.txt
>
>
> Currently, the only thing we keep in system tables about other peers is their token and IP addresses. We should probably also record the new ring_id, but since CASSANDRA-4018 makes system table easily queriable, may it could be worth adding some more information (basically most of what we gossip could be a candidate (schema UUID, status, C* version, ...)) as a simple way to expose the ring state to users (even if it's just a "view" of the ring state from one specific node I believe it's still nice).
> Of course that means storing information that may not be absolutely needed by the server, but I'm not sure there is much harm to that.
> Note that doing this cleanly may require changing the schema of current system tables but as long as we do that in the 1.2 timeframe it's ok (since the concerned system table 'local' and 'peers' are news anyway).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4351) Consider storing more
informations on peers in system tables
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470087#comment-13470087 ]
Sylvain Lebresne commented on CASSANDRA-4351:
---------------------------------------------
Committed 0001 but holding this open a little long to see if we decide something on the 2nd part.
bq. what if instead we change LongToken to/fromString to hex-encode with a constant width, the way CASSANDRA-4550 wanted? Of course then we'd need to switch it to unsigned comparison
That's not a bad idea (since we have no backward compatibility problem) so why not (and switching to unsigned comparison is probably not a big deal (though we do have to be careful about the fact that the minimum token shouldn't be a valid token, so tokens value will have to be in [1, 2^64-1])). That being said, I'm not sure about the "instead" in the sentence above. Was that to be understood as "in addition" to 0002?
> Consider storing more informations on peers in system tables
> -------------------------------------------------------------
>
> Key: CASSANDRA-4351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-4351.txt, 0002-Save-tokens-as-strings.txt
>
>
> Currently, the only thing we keep in system tables about other peers is their token and IP addresses. We should probably also record the new ring_id, but since CASSANDRA-4018 makes system table easily queriable, may it could be worth adding some more information (basically most of what we gossip could be a candidate (schema UUID, status, C* version, ...)) as a simple way to expose the ring state to users (even if it's just a "view" of the ring state from one specific node I believe it's still nice).
> Of course that means storing information that may not be absolutely needed by the server, but I'm not sure there is much harm to that.
> Note that doing this cleanly may require changing the schema of current system tables but as long as we do that in the 1.2 timeframe it's ok (since the concerned system table 'local' and 'peers' are news anyway).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4351) Consider storing more
informations on peers in system tables
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13432725#comment-13432725 ]
Sylvain Lebresne commented on CASSANDRA-4351:
---------------------------------------------
The idea for this ticket is that in an ideal world, I could imagine to have a schema like:
{noformat}
CREATE TABLE System.peers (
ring_id uuid PRIMARY KEY,
tokens set<blob>,
peer inet,
schema_version uuid, // so that client can check if the cluster is in agreement
release_version text, // quick check to see where we're at of a rolling upgrade
rpc_address,
dc text,
rack text // a smart client that want to discover node could use that to maintain it's metadata
)
{noformat}
Now the main thing I'm not sure is that currently we have a bunch of cases where we call SystemTable.removeTokens() that I'm not sure how to adapt to such schema. That is, I'm not fully sure I understand why removeTokens is called in some of those cases.
> Consider storing more informations on peers in system tables
> -------------------------------------------------------------
>
> Key: CASSANDRA-4351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2
>
>
> Currently, the only thing we keep in system tables about other peers is their token and IP addresses. We should probably also record the new ring_id, but since CASSANDRA-4018 makes system table easily queriable, may it could be worth adding some more information (basically most of what we gossip could be a candidate (schema UUID, status, C* version, ...)) as a simple way to expose the ring state to users (even if it's just a "view" of the ring state from one specific node I believe it's still nice).
> Of course that means storing information that may not be absolutely needed by the server, but I'm not sure there is much harm to that.
> Note that doing this cleanly may require changing the schema of current system tables but as long as we do that in the 1.2 timeframe it's ok (since the concerned system table 'local' and 'peers' are news anyway).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4351) Consider storing more
informations on peers in system tables
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470456#comment-13470456 ]
Jonathan Ellis commented on CASSANDRA-4351:
-------------------------------------------
You're right, 0002 is useful even if we change m3p, +1
> Consider storing more informations on peers in system tables
> -------------------------------------------------------------
>
> Key: CASSANDRA-4351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-4351.txt, 0002-Save-tokens-as-strings.txt
>
>
> Currently, the only thing we keep in system tables about other peers is their token and IP addresses. We should probably also record the new ring_id, but since CASSANDRA-4018 makes system table easily queriable, may it could be worth adding some more information (basically most of what we gossip could be a candidate (schema UUID, status, C* version, ...)) as a simple way to expose the ring state to users (even if it's just a "view" of the ring state from one specific node I believe it's still nice).
> Of course that means storing information that may not be absolutely needed by the server, but I'm not sure there is much harm to that.
> Note that doing this cleanly may require changing the schema of current system tables but as long as we do that in the 1.2 timeframe it's ok (since the concerned system table 'local' and 'peers' are news anyway).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4351) Consider storing more
informations on peers in system tables
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne updated CASSANDRA-4351:
----------------------------------------
Fix Version/s: (was: 1.2.0)
1.2.0 beta 2
> Consider storing more informations on peers in system tables
> -------------------------------------------------------------
>
> Key: CASSANDRA-4351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-4351.txt, 0002-Save-tokens-as-strings.txt
>
>
> Currently, the only thing we keep in system tables about other peers is their token and IP addresses. We should probably also record the new ring_id, but since CASSANDRA-4018 makes system table easily queriable, may it could be worth adding some more information (basically most of what we gossip could be a candidate (schema UUID, status, C* version, ...)) as a simple way to expose the ring state to users (even if it's just a "view" of the ring state from one specific node I believe it's still nice).
> Of course that means storing information that may not be absolutely needed by the server, but I'm not sure there is much harm to that.
> Note that doing this cleanly may require changing the schema of current system tables but as long as we do that in the 1.2 timeframe it's ok (since the concerned system table 'local' and 'peers' are news anyway).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4351) Consider storing more
informations on peers in system tables
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13396021#comment-13396021 ]
Jonathan Ellis commented on CASSANDRA-4351:
-------------------------------------------
Sounds reasonable.
> Consider storing more informations on peers in system tables
> -------------------------------------------------------------
>
> Key: CASSANDRA-4351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2
>
>
> Currently, the only thing we keep in system tables about other peers is their token and IP addresses. We should probably also record the new ring_id, but since CASSANDRA-4018 makes system table easily queriable, may it could be worth adding some more information (basically most of what we gossip could be a candidate (schema UUID, status, C* version, ...)) as a simple way to expose the ring state to users (even if it's just a "view" of the ring state from one specific node I believe it's still nice).
> Of course that means storing information that may not be absolutely needed by the server, but I'm not sure there is much harm to that.
> Note that doing this cleanly may require changing the schema of current system tables but as long as we do that in the 1.2 timeframe it's ok (since the concerned system table 'local' and 'peers' are news anyway).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4351) Consider storing more
informations on peers in system tables
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470442#comment-13470442 ]
Jonathan Ellis commented on CASSANDRA-4351:
-------------------------------------------
No, I did mean instead, since then the human-readable token as seen in nodetool is the same as what you'd get from cqlsh.
> Consider storing more informations on peers in system tables
> -------------------------------------------------------------
>
> Key: CASSANDRA-4351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-4351.txt, 0002-Save-tokens-as-strings.txt
>
>
> Currently, the only thing we keep in system tables about other peers is their token and IP addresses. We should probably also record the new ring_id, but since CASSANDRA-4018 makes system table easily queriable, may it could be worth adding some more information (basically most of what we gossip could be a candidate (schema UUID, status, C* version, ...)) as a simple way to expose the ring state to users (even if it's just a "view" of the ring state from one specific node I believe it's still nice).
> Of course that means storing information that may not be absolutely needed by the server, but I'm not sure there is much harm to that.
> Note that doing this cleanly may require changing the schema of current system tables but as long as we do that in the 1.2 timeframe it's ok (since the concerned system table 'local' and 'peers' are news anyway).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4351) Consider storing more
informations on peers in system tables
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13469728#comment-13469728 ]
Jonathan Ellis commented on CASSANDRA-4351:
-------------------------------------------
+1 on 0001.
For 0002 ... what if instead we change LongToken (the default on 1.2 and what people "should" be using) to/fromString to hex-encode with a constant width, the way CASSANDRA-4550 wanted? Of course then we'd need to switch it to unsigned comparison to be consistent with what the hex implies. Dunno, maybe it's not worth the trouble.
> Consider storing more informations on peers in system tables
> -------------------------------------------------------------
>
> Key: CASSANDRA-4351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-4351.txt, 0002-Save-tokens-as-strings.txt
>
>
> Currently, the only thing we keep in system tables about other peers is their token and IP addresses. We should probably also record the new ring_id, but since CASSANDRA-4018 makes system table easily queriable, may it could be worth adding some more information (basically most of what we gossip could be a candidate (schema UUID, status, C* version, ...)) as a simple way to expose the ring state to users (even if it's just a "view" of the ring state from one specific node I believe it's still nice).
> Of course that means storing information that may not be absolutely needed by the server, but I'm not sure there is much harm to that.
> Note that doing this cleanly may require changing the schema of current system tables but as long as we do that in the 1.2 timeframe it's ok (since the concerned system table 'local' and 'peers' are news anyway).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4351) Consider storing more
informations on peers in system tables
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sylvain Lebresne resolved CASSANDRA-4351.
-----------------------------------------
Resolution: Fixed
Reviewer: jbellis
Assignee: Sylvain Lebresne
Alright, 0002 committed. I don't know if we want to bother with the change to M3P then, but we can always do that in another ticket in any case, so closing that one now.
> Consider storing more informations on peers in system tables
> -------------------------------------------------------------
>
> Key: CASSANDRA-4351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Assignee: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-4351.txt, 0002-Save-tokens-as-strings.txt
>
>
> Currently, the only thing we keep in system tables about other peers is their token and IP addresses. We should probably also record the new ring_id, but since CASSANDRA-4018 makes system table easily queriable, may it could be worth adding some more information (basically most of what we gossip could be a candidate (schema UUID, status, C* version, ...)) as a simple way to expose the ring state to users (even if it's just a "view" of the ring state from one specific node I believe it's still nice).
> Of course that means storing information that may not be absolutely needed by the server, but I'm not sure there is much harm to that.
> Note that doing this cleanly may require changing the schema of current system tables but as long as we do that in the 1.2 timeframe it's ok (since the concerned system table 'local' and 'peers' are news anyway).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4351) Consider storing more
informations on peers in system tables
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13470450#comment-13470450 ]
Sylvain Lebresne commented on CASSANDRA-4351:
---------------------------------------------
Oh I see. Was mislead by the fact that cqlsh don't print blob in hex currently. But if we fix that then yes, that'd be the same, at least for Murmur3Partitioner.
Though I do note that Murmur3Partitioner is only an option for brand new clusters (i.e. almost no-one will have Murmur3Partitioner at first). But again, I don't care too much. I suppose that in the long run, with vnodes, the pretty printing of tokens won't be very useful anymore.
> Consider storing more informations on peers in system tables
> -------------------------------------------------------------
>
> Key: CASSANDRA-4351
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4351
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Sylvain Lebresne
> Priority: Minor
> Fix For: 1.2.0 beta 2
>
> Attachments: 0001-4351.txt, 0002-Save-tokens-as-strings.txt
>
>
> Currently, the only thing we keep in system tables about other peers is their token and IP addresses. We should probably also record the new ring_id, but since CASSANDRA-4018 makes system table easily queriable, may it could be worth adding some more information (basically most of what we gossip could be a candidate (schema UUID, status, C* version, ...)) as a simple way to expose the ring state to users (even if it's just a "view" of the ring state from one specific node I believe it's still nice).
> Of course that means storing information that may not be absolutely needed by the server, but I'm not sure there is much harm to that.
> Note that doing this cleanly may require changing the schema of current system tables but as long as we do that in the 1.2 timeframe it's ok (since the concerned system table 'local' and 'peers' are news anyway).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira