You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jonathan Ellis (JIRA)" <ji...@apache.org> on 2011/03/23 18:00:05 UTC
[jira] [Created] (CASSANDRA-2369) support replication decisions
per-key
support replication decisions per-key
-------------------------------------
Key: CASSANDRA-2369
URL: https://issues.apache.org/jira/browse/CASSANDRA-2369
Project: Cassandra
Issue Type: New Feature
Reporter: Jonathan Ellis
Assignee: Jonathan Ellis
Priority: Minor
Fix For: 0.8
Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2369) support replication decisions
per-key
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13050572#comment-13050572 ]
Jonathan Ellis commented on CASSANDRA-2369:
-------------------------------------------
bq. There is a similar problem on bootstrap and node movement. Instead of asking a single replica to stream data from the range a new node is assuming, it will have to ask all replicas that may have rows for that range to make sure it doesn't miss any.
Additionally, each source replica would have to deserialize each row in the range to make sure it's actually supposed to be streamed, which would slow streaming down significantly.
Starting to think this is not worth trying to support.
> support replication decisions per-key
> -------------------------------------
>
> Key: CASSANDRA-2369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2369
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.0
>
>
> Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2369) support replication decisions
per-key
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048862#comment-13048862 ]
Vijay commented on CASSANDRA-2369:
----------------------------------
Can we say, you cannot set the replication factor more than the default replication factor (which we can rename it sometime later saying max replication factor)... if set... replica repair can construct a tree based on the partitioner algo (getReplicationToken) for all the replicas till we have max?
> support replication decisions per-key
> -------------------------------------
>
> Key: CASSANDRA-2369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2369
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.0
>
>
> Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2369) support replication decisions
per-key
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048234#comment-13048234 ]
Vijay commented on CASSANDRA-2369:
----------------------------------
Hi Jonathan, Will do... I should have a proto-type ready by next week or so... Thanks!
> support replication decisions per-key
> -------------------------------------
>
> Key: CASSANDRA-2369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2369
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.0
>
>
> Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-2369) support replication decisions
per-key
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-2369.
---------------------------------------
Resolution: Won't Fix
Fix Version/s: (was: 1.0)
Assignee: (was: Vijay)
Haven't been able to think of a way around the above problems. Closing as wontfix.
> support replication decisions per-key
> -------------------------------------
>
> Key: CASSANDRA-2369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2369
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Priority: Minor
>
> Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2369) support replication decisions
per-key
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-2369:
-----------------------------------------
Assignee: paul cannon (was: Jonathan Ellis)
I think the goal here is to end up with AbstractReplicationStrategy.getNaturalEndpoints take a DecoratedKey parameter instead of a Token.
Ultimately, even assuming that we always start by decorating the key with a token might be assuming too much (and want to decide based on the raw ByteBuffer key) but let's not borrow trouble; it should be a lot easier to make the DecoratedKey change since existing code that wants a token can easily be satisfied that way.
> support replication decisions per-key
> -------------------------------------
>
> Key: CASSANDRA-2369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2369
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: paul cannon
> Priority: Minor
> Fix For: 1.0
>
>
> Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2369) support replication decisions
per-key
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049855#comment-13049855 ]
Jonathan Ellis commented on CASSANDRA-2369:
-------------------------------------------
No, it's worse than that.
Let me give an example of a simple multi-node, multi-DC cluster: nodes A and M in DC1, nodes B and N in DC2. So node A, M, B, and N have keys in ranges (M-A], (A, M], (N, B], (B, N], respectively.
If I write a row K with NTS {DC1: 1, DC2:2}, then I know it will be on nodes M and N. So far so good.
What if I now repair node M? It knows it has to compare its data for range (A, B] with that data on node B, and range (B, M] with that data on node N. So it builds a merkle tree for each range, and requests that B and N do so as well, then they exchange trees to see if things are in sync.
How does this change if we introduce this partitioner? M can no longer assume that keys it has for range (A, B] should also be replicated to node M, and vice versa. You would have to build a separate tree for each replica, i.e. instead of just a tree for (A, B], each replica would need to build a tree for (A, B]-that-belongs-on-M, and another tree for (A, B)-that-belongs-on-B, and so forth for as many possible replicas as exist.
There is a similar problem on bootstrap and node movement. Instead of asking a single replica to stream data from the range a new node is assuming, it will have to ask _all_ replicas that may have rows for that range to make sure it doesn't miss any.
> support replication decisions per-key
> -------------------------------------
>
> Key: CASSANDRA-2369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2369
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.0
>
>
> Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2369) support replication decisions
per-key
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049899#comment-13049899 ]
Sylvain Lebresne commented on CASSANDRA-2369:
---------------------------------------------
Let me also add that if you allow that, load balancing will be a bitch. One may argue it should be the problem of whomever wants to use this, but I'm not sure that providing tools that make foot-shooting too easy is such a good idea.
> support replication decisions per-key
> -------------------------------------
>
> Key: CASSANDRA-2369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2369
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.0
>
>
> Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CASSANDRA-2369) support replication decisions
per-key
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reassigned CASSANDRA-2369:
-----------------------------------------
Assignee: Vijay (was: paul cannon)
Want to take a stab at this, Vijay?
> support replication decisions per-key
> -------------------------------------
>
> Key: CASSANDRA-2369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2369
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.0
>
>
> Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2369) support replication decisions
per-key
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13048855#comment-13048855 ]
Jonathan Ellis commented on CASSANDRA-2369:
-------------------------------------------
Repair is going to be tough -- we currently assume that for each replication range the set of peer replicas that have those rows is the same. I don't see a good fix for that.
> support replication decisions per-key
> -------------------------------------
>
> Key: CASSANDRA-2369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2369
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.0
>
>
> Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2369) support replication decisions
per-key
Posted by "Vijay (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13049871#comment-13049871 ]
Vijay commented on CASSANDRA-2369:
----------------------------------
Hi Jonathan, Yeah, i started coding the repair case (1 to 1 repair, something similar to what you mentioned above) quickly realized the complexity it buys... than any good... Max replication can help with the boot strap and repair (but wont be efficient at per sure)...
> support replication decisions per-key
> -------------------------------------
>
> Key: CASSANDRA-2369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2369
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: Vijay
> Priority: Minor
> Fix For: 1.0
>
>
> Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2369) support replication decisions
per-key
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13025976#comment-13025976 ]
Jonathan Ellis commented on CASSANDRA-2369:
-------------------------------------------
The more I think about this the more I suspect that what I'm asking for here is a return to the assumption that keys:tokens are 1:1, i.e., incompatible with CASSANDRA-1034.
It's actually even worse than that, because nodes own token _ranges_: if I have tokens X Y and Z that sort in that order, and a node has range (T, Z] then they all have to be on that node, no matter what the original keys were.
So I think the right approach is not to mess with ReplicationStrategy api, but rather to build some kind of row-aware partitioner that generates tokens with extra metadata, e.g., a (hash, {dc1: 2, dc2: 0, dc3: 1}) tuple. Then the strategy would only have to run logic similar to NTS, except instead of per-keyspace strategy options it would get them from the Token object.
> support replication decisions per-key
> -------------------------------------
>
> Key: CASSANDRA-2369
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2369
> Project: Cassandra
> Issue Type: New Feature
> Reporter: Jonathan Ellis
> Assignee: paul cannon
> Priority: Minor
> Fix For: 1.0
>
>
> Currently the replicationstrategy gets a token and a keyspace with which to decide how to place replicas. for per-row replication this is insufficient because tokenization is lossy (CASSANDRA-1034).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira