You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (Created) (JIRA)" <ji...@apache.org> on 2012/03/08 08:05:09 UTC
[jira] [Created] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Add column metadata to system columnfamilies
--------------------------------------------
Key: CASSANDRA-4018
URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
Project: Cassandra
Issue Type: Improvement
Reporter: Jonathan Ellis
Priority: Minor
Fix For: 1.1.1
CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285570#comment-13285570 ]
Sylvain Lebresne commented on CASSANDRA-4018:
---------------------------------------------
I pushed at https://github.com/pcmanus/cassandra/commits/4018-10 an additional commit on top of 4018-9. It does create a ClientState for processInternal; a clientState object is not very expensive and it's easier than trying to hack the code to work without clientState imo. Another problem is that currently you can't do range_slice query on the system tables before the tokens are set, but the request was to actually read the saved token, so that thrown a assertionError in the TokenMetadata.ringIterator. The commit fix this by specializing getNaturalEndpoints for localStrategy, which is enough. But I would be fine if we decide that during the SystemTable loading we decide to stick with lower level CFStore methods to query.
Another problem is CASSANDRA-4296, so the branch should also be rebased on top of that (which is not the case of my github branch btw).
Even with that, I still get one last unit test failure. The problem is that the new 'peers' table uses row keys that are not necessarily UTF8, but the tests are using CollatingOrderPreservingPartitioner so this throws an exception (in BootStrapperTest). Not sure what's the best fix for that.
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290182#comment-13290182 ]
Jonathan Ellis commented on CASSANDRA-4018:
-------------------------------------------
SGTM.
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457681#comment-13457681 ]
Hudson commented on CASSANDRA-4018:
-----------------------------------
Integrated in Cassandra #2117 (See [https://builds.apache.org/job/Cassandra/2117/])
cqlsh: adapt to CASSANDRA-4018 system cf changes (Revision 9ec77eb66ec4106fde1b8ecaf17c2aca521fe9f0)
Result = ABORTED
sylvain :
Files :
* bin/cqlsh
* pylib/cqlshlib/cql3handling.py
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2.0 beta 1
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Jonathan Ellis (Assigned) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-4018:
-----------------------------------------
Assignee: Jonathan Ellis
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.1.1
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Jonathan Ellis (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-4018:
--------------------------------------
Fix Version/s: (was: 1.1.1)
1.2
(Moving to 1.2 since it's going to require rewriting HH to use composites.)
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290150#comment-13290150 ]
Sylvain Lebresne commented on CASSANDRA-4018:
---------------------------------------------
bq. if we're going to have it in peers I'd rather take it out of the local info table
The one thing to be careful if we do that is that we may want a way to identify the local token other than the IP address, in case the change it's IP. Maybe we could just add a boolean 'is_local' for that. But why not.
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13282526#comment-13282526 ]
Sylvain Lebresne commented on CASSANDRA-4018:
---------------------------------------------
Some remarks on the patches:
* The patch removes the gc_grace (of 3 months) for schema tables, but schema tables really need their gc_grace.
* The second patch allows uppercase for properties in create table, but CASSANDRA-4278 has a more generic solution for that. It'd probably be better to rebase this on top of CASSANDRA-4278 (once it's committed).
* In HHOM, the table definition in the comments don't correspond to the actual schema. And since the rest seems to expect the schema in the comment, I'm not sure this work as expected (as a side note, it'd be safer to reuse the actual CF comparator rather that redefining the compositeType).
* Given that the peers table will later store potentially multiple token per host, it seems 'peers' is not the best name. Mostly this will be an inverted index for tokens, maybe 'tokens' or 'tokens_maps' or something like that would be more suited? But on a more general level, if we had CASSANDRA-3647, we could imagine to have a peers system table, where the key would be the unique host id and we would store as info the inet address and the list of token. Just saying.
* In hints, following CASSANDRA-4120, the row key is the host id, not the token, so we should probably rename it in the schema (and maybe use the UUID type).
* In SystemTable, when inserting into Peers, it use the colum name "token", but the table definition use the name "peer" (which is better since it's the inet address that this column store).
* In SystemTable, when fetching keys in Peers, instead of doing the query manually and doing a resultify, why not use CQL3 for the query itself?
* In SystemTable.getLocalHostId, the PEERS_CF is queried instead of LOCAL_CF
* In NodeIdCf, the id is actually a UUID (and the code does rely on the sorting being the one of (time) UUID).
Otherwise, after this patch we will just have IndexCf and NodeIdCf that will have case sensitive names. Do we want to do something about that? I guess that's not the most useful table to be queried by users, so probably it's not a big deal, but wanted to mention it to get other opinions.
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285013#comment-13285013 ]
Jonathan Ellis commented on CASSANDRA-4018:
-------------------------------------------
Addressed above and pushed to https://github.com/jbellis/cassandra/branches/4018-9. (Changes were rebased except for the last commit, which is new.)
Note that this is against current trunk now including CASSANDRA-4278, but it does not work; getting "COMMENT is not a valid keyword argument" from compile(). Bug in 4278?
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290069#comment-13290069 ]
Sylvain Lebresne commented on CASSANDRA-4018:
---------------------------------------------
I've rebased the previous branch to https://github.com/pcmanus/cassandra/commits/4018-11. There is also two additional commits:
* the first one fixes a few typos/mismatch in the columns names
* the second one adds the local token in the peers table. My rational is that for tools, it'll probably be easier to have the peers table be representative of the full ring, without having to query the local table too. Yet SystemTable.loadTokens() expects we don't return the local token in the result, so the code makes sure to exclude it.
All unit tests are passing on that branch and I've (manually) checked all the system table with cqlsh. So I'm personally +1 on that branch but obviously if we're good with my last changes.
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290123#comment-13290123 ]
Jonathan Ellis commented on CASSANDRA-4018:
-------------------------------------------
bq. the second one adds the local token in the peers table
I'd rather have a single source of truth to avoid confusion and potential error, so if we're going to have it in peers I'd rather take it out of the local info table.
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281949#comment-13281949 ]
Jonathan Ellis commented on CASSANDRA-4018:
-------------------------------------------
rebased to https://github.com/jbellis/cassandra/branches/4018-8
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13288969#comment-13288969 ]
Jonathan Ellis commented on CASSANDRA-4018:
-------------------------------------------
Ripped out COPP in CASSANDRA-2479, are we good now? :)
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290180#comment-13290180 ]
Sylvain Lebresne commented on CASSANDRA-4018:
---------------------------------------------
Yes, actually there is a bunch of info a node have on it's peer that we don't write in the system table (the ring_id, but things like other nodes version, shema version etc...). We don't need to persist them per-se internally, but doing it would allow to expose it to the user (for example, for the schema version, that would give a very simple way for client to check the cluster is fully up to date on the schema).
I could typically imagine changing peers and local a bit to have peers store all the things we know about our peers (including the local node) that we deem useful to expose, and then maybe have a 'configuration' table (or 'local' but I like 'configuration :)) for whatever make sense only for the local node.
That being said, maybe it's worth committing what we have (maybe without my last patch for now), and open a separate ticket to see if we want to go a step further.
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290176#comment-13290176 ]
Jonathan Ellis commented on CASSANDRA-4018:
-------------------------------------------
What about that fancy new ring_id?
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4018) Add column metadata to system
columnfamilies
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285328#comment-13285328 ]
Jonathan Ellis commented on CASSANDRA-4018:
-------------------------------------------
The problem was that 4278 wasn't in trunk yet, just 1.1.
re-pushed (--force) to 4018-9 with some fixes.
The last commit to add a native CQL query to peers turns out to not work; it NPEs in hasColumnFamily without a ClientState. Not sure of a good fix here. Should we split this to another ticket?
Forgot to comment on a couple other things above:
- I like "peers" over "tokens" since it's basically saying, "here's the information I have about the other nodes in the cluster." "tokens" is a lot more vague since we use tokens all over the place; it's less immediately clear to someone examining the system keyspace what it's doing
- I left NodeIdCf and IndexCf alone since we can't just throw them away and start over case-insensitive (like we have the luxury of doing with hints). And I didn't want to write a bunch of sstable-renaming code... Can push this to another ticket if you like.
> Add column metadata to system columnfamilies
> --------------------------------------------
>
> Key: CASSANDRA-4018
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4018
> Project: Cassandra
> Issue Type: Improvement
> Affects Versions: 1.1.0
> Reporter: Jonathan Ellis
> Assignee: Jonathan Ellis
> Priority: Minor
> Fix For: 1.2
>
>
> CASSANDRA-3792 adds this to the schema CFs; we should modernize the other system CFs as well
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira