You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Peter Schuller (Created) (JIRA)" <ji...@apache.org> on 2012/02/13 01:48:59 UTC
[jira] [Created] (CASSANDRA-3901) write endpoints are not treated
correctly, breaking consistency guarantees
write endpoints are not treated correctly, breaking consistency guarantees
--------------------------------------------------------------------------
Key: CASSANDRA-3901
URL: https://issues.apache.org/jira/browse/CASSANDRA-3901
Project: Cassandra
Issue Type: Sub-task
Components: Core
Reporter: Peter Schuller
Assignee: Peter Schuller
Priority: Critical
I had a nagging feeling this was the case ever since I started wanting CASSANDRA-3833 and thinking about hot to handle the association between nodes in the read set and nodes in the write set.
I may be wrong (please point me in the direct direction if so), but I see no code anywhere that tries to (1) apply consistency level to currently normal endpoints only, and (2) "connect" a given read endpoint with a future write endpoint such that they are tied together for consistency purposes (parts of these concerns probably is covered by CASSANDRA-2434 but that ticket is more general).
To be more clear about the problem: Suppose we have a ring of nodes, with a single node bootstrapping. Now, for a given row key suppose reads are served by A, B and C while writes are to go to A, B, C and D. In other words, D is the node bootstrapping. Suppose RF is 3 and A,B,C,D is ring order. There are a few things required for correct behavior:
* Writes acked by D must never be treated as sufficient to satisfy consistency level since until it is part of the read set it does not count towards CL on reads.
* Writes acked by B must *not* be treated as sufficient to satisfy consistency level *unless* the same write is *also* acked by D, because once D enters the ring, B will no longer be counting towards CL on reads. The only alternative is to make the read succeed and disallow D from entering the ring.
We don't seem to be handling this at all (and it becomes more complicated with arbitrary transitions).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3901) write endpoints are not treated
correctly, breaking consistency guarantees
Posted by "Peter Schuller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464356#comment-13464356 ]
Peter Schuller commented on CASSANDRA-3901:
-------------------------------------------
As an aside, streaming from multiple sources and discarding un-needed data in real-time based on streaming data you have and comparing, overlaps in mechanism with what you might want to do for repair in order to avoid the size explosion problem completely.
> write endpoints are not treated correctly, breaking consistency guarantees
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-3901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3901
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
>
> I had a nagging feeling this was the case ever since I started wanting CASSANDRA-3833 and thinking about hot to handle the association between nodes in the read set and nodes in the write set.
> I may be wrong (please point me in the direct direction if so), but I see no code anywhere that tries to (1) apply consistency level to currently normal endpoints only, and (2) "connect" a given read endpoint with a future write endpoint such that they are tied together for consistency purposes (parts of these concerns probably is covered by CASSANDRA-2434 but that ticket is more general).
> To be more clear about the problem: Suppose we have a ring of nodes, with a single node bootstrapping. Now, for a given row key suppose reads are served by A, B and C while writes are to go to A, B, C and D. In other words, D is the node bootstrapping. Suppose RF is 3 and A,B,C,D is ring order. There are a few things required for correct behavior:
> * Writes acked by D must never be treated as sufficient to satisfy consistency level since until it is part of the read set it does not count towards CL on reads.
> * Writes acked by B must *not* be treated as sufficient to satisfy consistency level *unless* the same write is *also* acked by D, because once D enters the ring, B will no longer be counting towards CL on reads. The only alternative is to make the read succeed and disallow D from entering the ring.
> We don't seem to be handling this at all (and it becomes more complicated with arbitrary transitions).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3901) write endpoints are not treated
correctly, breaking consistency guarantees
Posted by "Peter Schuller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464355#comment-13464355 ]
Peter Schuller commented on CASSANDRA-3901:
-------------------------------------------
If the patch in 833 causes writes to go to all possible pending ranges, then I agree it's the same.
That said, *bootstrap* is still broken overall due to the way data is sourced for streaming. I don't know whether we should re-phrase this ticket to be more general or file a new one.
The main issue is that when streaming, you have to be streaming from the node that you're replacing for a particular range. Otherwise you're violating consistency.
If the node you're streaming from has actually died permanently and there is data loss, you'd have to stream from *all* replicas to satisfy consistency. That is currently not very tenable due to how compaction and streaming works (size explosion would be the result).
> write endpoints are not treated correctly, breaking consistency guarantees
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-3901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3901
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
>
> I had a nagging feeling this was the case ever since I started wanting CASSANDRA-3833 and thinking about hot to handle the association between nodes in the read set and nodes in the write set.
> I may be wrong (please point me in the direct direction if so), but I see no code anywhere that tries to (1) apply consistency level to currently normal endpoints only, and (2) "connect" a given read endpoint with a future write endpoint such that they are tied together for consistency purposes (parts of these concerns probably is covered by CASSANDRA-2434 but that ticket is more general).
> To be more clear about the problem: Suppose we have a ring of nodes, with a single node bootstrapping. Now, for a given row key suppose reads are served by A, B and C while writes are to go to A, B, C and D. In other words, D is the node bootstrapping. Suppose RF is 3 and A,B,C,D is ring order. There are a few things required for correct behavior:
> * Writes acked by D must never be treated as sufficient to satisfy consistency level since until it is part of the read set it does not count towards CL on reads.
> * Writes acked by B must *not* be treated as sufficient to satisfy consistency level *unless* the same write is *also* acked by D, because once D enters the ring, B will no longer be counting towards CL on reads. The only alternative is to make the read succeed and disallow D from entering the ring.
> We don't seem to be handling this at all (and it becomes more complicated with arbitrary transitions).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-3901) write endpoints are not treated
correctly, breaking consistency guarantees
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-3901.
---------------------------------------
Resolution: Duplicate
> write endpoints are not treated correctly, breaking consistency guarantees
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-3901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3901
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
>
> I had a nagging feeling this was the case ever since I started wanting CASSANDRA-3833 and thinking about hot to handle the association between nodes in the read set and nodes in the write set.
> I may be wrong (please point me in the direct direction if so), but I see no code anywhere that tries to (1) apply consistency level to currently normal endpoints only, and (2) "connect" a given read endpoint with a future write endpoint such that they are tied together for consistency purposes (parts of these concerns probably is covered by CASSANDRA-2434 but that ticket is more general).
> To be more clear about the problem: Suppose we have a ring of nodes, with a single node bootstrapping. Now, for a given row key suppose reads are served by A, B and C while writes are to go to A, B, C and D. In other words, D is the node bootstrapping. Suppose RF is 3 and A,B,C,D is ring order. There are a few things required for correct behavior:
> * Writes acked by D must never be treated as sufficient to satisfy consistency level since until it is part of the read set it does not count towards CL on reads.
> * Writes acked by B must *not* be treated as sufficient to satisfy consistency level *unless* the same write is *also* acked by D, because once D enters the ring, B will no longer be counting towards CL on reads. The only alternative is to make the read succeed and disallow D from entering the ring.
> We don't seem to be handling this at all (and it becomes more complicated with arbitrary transitions).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3901) write endpoints are not treated
correctly, breaking consistency guarantees
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464435#comment-13464435 ]
Jonathan Ellis commented on CASSANDRA-3901:
-------------------------------------------
bq. The main issue is that when streaming, you have to be streaming from the node that you're replacing for a particular range
Right, that part IS CASSANDRA-2434. Here's my very first comment over there:
bq. ISTM the easiest fix is to always stream from the node that will be removed from the replicas for each range, unless given permission from the operator to choose a replica that is closer / less dead.
> write endpoints are not treated correctly, breaking consistency guarantees
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-3901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3901
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
>
> I had a nagging feeling this was the case ever since I started wanting CASSANDRA-3833 and thinking about hot to handle the association between nodes in the read set and nodes in the write set.
> I may be wrong (please point me in the direct direction if so), but I see no code anywhere that tries to (1) apply consistency level to currently normal endpoints only, and (2) "connect" a given read endpoint with a future write endpoint such that they are tied together for consistency purposes (parts of these concerns probably is covered by CASSANDRA-2434 but that ticket is more general).
> To be more clear about the problem: Suppose we have a ring of nodes, with a single node bootstrapping. Now, for a given row key suppose reads are served by A, B and C while writes are to go to A, B, C and D. In other words, D is the node bootstrapping. Suppose RF is 3 and A,B,C,D is ring order. There are a few things required for correct behavior:
> * Writes acked by D must never be treated as sufficient to satisfy consistency level since until it is part of the read set it does not count towards CL on reads.
> * Writes acked by B must *not* be treated as sufficient to satisfy consistency level *unless* the same write is *also* acked by D, because once D enters the ring, B will no longer be counting towards CL on reads. The only alternative is to make the read succeed and disallow D from entering the ring.
> We don't seem to be handling this at all (and it becomes more complicated with arbitrary transitions).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3901) write endpoints are not treated
correctly, breaking consistency guarantees
Posted by "paul cannon (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207156#comment-13207156 ]
paul cannon commented on CASSANDRA-3901:
----------------------------------------
I don't believe that the proposed fix for CASSANDRA-2434 covers these concerns at all. I guess you could say that the *scope* of 2434 covers this, but I think it's separate enough to deserve its own ticket, as you've done.
> write endpoints are not treated correctly, breaking consistency guarantees
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-3901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3901
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Priority: Critical
>
> I had a nagging feeling this was the case ever since I started wanting CASSANDRA-3833 and thinking about hot to handle the association between nodes in the read set and nodes in the write set.
> I may be wrong (please point me in the direct direction if so), but I see no code anywhere that tries to (1) apply consistency level to currently normal endpoints only, and (2) "connect" a given read endpoint with a future write endpoint such that they are tied together for consistency purposes (parts of these concerns probably is covered by CASSANDRA-2434 but that ticket is more general).
> To be more clear about the problem: Suppose we have a ring of nodes, with a single node bootstrapping. Now, for a given row key suppose reads are served by A, B and C while writes are to go to A, B, C and D. In other words, D is the node bootstrapping. Suppose RF is 3 and A,B,C,D is ring order. There are a few things required for correct behavior:
> * Writes acked by D must never be treated as sufficient to satisfy consistency level since until it is part of the read set it does not count towards CL on reads.
> * Writes acked by B must *not* be treated as sufficient to satisfy consistency level *unless* the same write is *also* acked by D, because once D enters the ring, B will no longer be counting towards CL on reads. The only alternative is to make the read succeed and disallow D from entering the ring.
> We don't seem to be handling this at all (and it becomes more complicated with arbitrary transitions).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3901) write endpoints are not treated
correctly, breaking consistency guarantees
Posted by "Peter Schuller (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13464382#comment-13464382 ]
Peter Schuller commented on CASSANDRA-3901:
-------------------------------------------
Also, I don't remember anymore: But if moves and decomms don't end up being subsumed by the pending ranges case, those would have to be addressed.
> write endpoints are not treated correctly, breaking consistency guarantees
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-3901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3901
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
>
> I had a nagging feeling this was the case ever since I started wanting CASSANDRA-3833 and thinking about hot to handle the association between nodes in the read set and nodes in the write set.
> I may be wrong (please point me in the direct direction if so), but I see no code anywhere that tries to (1) apply consistency level to currently normal endpoints only, and (2) "connect" a given read endpoint with a future write endpoint such that they are tied together for consistency purposes (parts of these concerns probably is covered by CASSANDRA-2434 but that ticket is more general).
> To be more clear about the problem: Suppose we have a ring of nodes, with a single node bootstrapping. Now, for a given row key suppose reads are served by A, B and C while writes are to go to A, B, C and D. In other words, D is the node bootstrapping. Suppose RF is 3 and A,B,C,D is ring order. There are a few things required for correct behavior:
> * Writes acked by D must never be treated as sufficient to satisfy consistency level since until it is part of the read set it does not count towards CL on reads.
> * Writes acked by B must *not* be treated as sufficient to satisfy consistency level *unless* the same write is *also* acked by D, because once D enters the ring, B will no longer be counting towards CL on reads. The only alternative is to make the read succeed and disallow D from entering the ring.
> We don't seem to be handling this at all (and it becomes more complicated with arbitrary transitions).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3901) write endpoints are not treated
correctly, breaking consistency guarantees
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459781#comment-13459781 ]
Jonathan Ellis commented on CASSANDRA-3901:
-------------------------------------------
I couldn't find the fix in trunk so I dug a bit deeper and found that I reverted the fix the day after committing it. I don't remember why (this was over a year ago). Reopened 833.
> write endpoints are not treated correctly, breaking consistency guarantees
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-3901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3901
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
>
> I had a nagging feeling this was the case ever since I started wanting CASSANDRA-3833 and thinking about hot to handle the association between nodes in the read set and nodes in the write set.
> I may be wrong (please point me in the direct direction if so), but I see no code anywhere that tries to (1) apply consistency level to currently normal endpoints only, and (2) "connect" a given read endpoint with a future write endpoint such that they are tied together for consistency purposes (parts of these concerns probably is covered by CASSANDRA-2434 but that ticket is more general).
> To be more clear about the problem: Suppose we have a ring of nodes, with a single node bootstrapping. Now, for a given row key suppose reads are served by A, B and C while writes are to go to A, B, C and D. In other words, D is the node bootstrapping. Suppose RF is 3 and A,B,C,D is ring order. There are a few things required for correct behavior:
> * Writes acked by D must never be treated as sufficient to satisfy consistency level since until it is part of the read set it does not count towards CL on reads.
> * Writes acked by B must *not* be treated as sufficient to satisfy consistency level *unless* the same write is *also* acked by D, because once D enters the ring, B will no longer be counting towards CL on reads. The only alternative is to make the read succeed and disallow D from entering the ring.
> We don't seem to be handling this at all (and it becomes more complicated with arbitrary transitions).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3901) write endpoints are not treated
correctly, breaking consistency guarantees
Posted by "Peter Schuller (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207175#comment-13207175 ]
Peter Schuller commented on CASSANDRA-3901:
-------------------------------------------
You're probably right. I didn't re-read 2434 again (it's long and it takes careful reading to follow the discussion), and mostly wanted to give a nod towards it in case it did.
> write endpoints are not treated correctly, breaking consistency guarantees
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-3901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3901
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Priority: Critical
>
> I had a nagging feeling this was the case ever since I started wanting CASSANDRA-3833 and thinking about hot to handle the association between nodes in the read set and nodes in the write set.
> I may be wrong (please point me in the direct direction if so), but I see no code anywhere that tries to (1) apply consistency level to currently normal endpoints only, and (2) "connect" a given read endpoint with a future write endpoint such that they are tied together for consistency purposes (parts of these concerns probably is covered by CASSANDRA-2434 but that ticket is more general).
> To be more clear about the problem: Suppose we have a ring of nodes, with a single node bootstrapping. Now, for a given row key suppose reads are served by A, B and C while writes are to go to A, B, C and D. In other words, D is the node bootstrapping. Suppose RF is 3 and A,B,C,D is ring order. There are a few things required for correct behavior:
> * Writes acked by D must never be treated as sufficient to satisfy consistency level since until it is part of the read set it does not count towards CL on reads.
> * Writes acked by B must *not* be treated as sufficient to satisfy consistency level *unless* the same write is *also* acked by D, because once D enters the ring, B will no longer be counting towards CL on reads. The only alternative is to make the read succeed and disallow D from entering the ring.
> We don't seem to be handling this at all (and it becomes more complicated with arbitrary transitions).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3901) write endpoints are not treated
correctly, breaking consistency guarantees
Posted by "Peter Schuller (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13216616#comment-13216616 ]
Peter Schuller commented on CASSANDRA-3901:
-------------------------------------------
When fixing this, we must consider saved endpoints. My reading now is that saved endpoints only constitute fully joined nodes, and when starting up we'd be potentially serving thrift traffic before becoming aware of bootstrapping nodes that imply pending ranges. (This relates to the general problem of waiting to serve thrift traffic before reaching a point of believing we understand the ring state, which we currently don't, and can result in e.g. massive amounts of timeouts when going up on thrift prior to gossip kicking hosts up.) Even if we fix the write path for this ticket, the fix won't be complete without actually ensuring the write path has a consistent ring state too.
> write endpoints are not treated correctly, breaking consistency guarantees
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-3901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3901
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
> Priority: Critical
>
> I had a nagging feeling this was the case ever since I started wanting CASSANDRA-3833 and thinking about hot to handle the association between nodes in the read set and nodes in the write set.
> I may be wrong (please point me in the direct direction if so), but I see no code anywhere that tries to (1) apply consistency level to currently normal endpoints only, and (2) "connect" a given read endpoint with a future write endpoint such that they are tied together for consistency purposes (parts of these concerns probably is covered by CASSANDRA-2434 but that ticket is more general).
> To be more clear about the problem: Suppose we have a ring of nodes, with a single node bootstrapping. Now, for a given row key suppose reads are served by A, B and C while writes are to go to A, B, C and D. In other words, D is the node bootstrapping. Suppose RF is 3 and A,B,C,D is ring order. There are a few things required for correct behavior:
> * Writes acked by D must never be treated as sufficient to satisfy consistency level since until it is part of the read set it does not count towards CL on reads.
> * Writes acked by B must *not* be treated as sufficient to satisfy consistency level *unless* the same write is *also* acked by D, because once D enters the ring, B will no longer be counting towards CL on reads. The only alternative is to make the read succeed and disallow D from entering the ring.
> We don't seem to be handling this at all (and it becomes more complicated with arbitrary transitions).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3901) write endpoints are not treated
correctly, breaking consistency guarantees
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459764#comment-13459764 ]
Jonathan Ellis commented on CASSANDRA-3901:
-------------------------------------------
Am I misreading this, or is this the same as CASSANDRA-833? See especially https://issues.apache.org/jira/browse/CASSANDRA-833?focusedCommentId=13028232&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13028232
> write endpoints are not treated correctly, breaking consistency guarantees
> --------------------------------------------------------------------------
>
> Key: CASSANDRA-3901
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3901
> Project: Cassandra
> Issue Type: Sub-task
> Components: Core
> Reporter: Peter Schuller
> Assignee: Peter Schuller
>
> I had a nagging feeling this was the case ever since I started wanting CASSANDRA-3833 and thinking about hot to handle the association between nodes in the read set and nodes in the write set.
> I may be wrong (please point me in the direct direction if so), but I see no code anywhere that tries to (1) apply consistency level to currently normal endpoints only, and (2) "connect" a given read endpoint with a future write endpoint such that they are tied together for consistency purposes (parts of these concerns probably is covered by CASSANDRA-2434 but that ticket is more general).
> To be more clear about the problem: Suppose we have a ring of nodes, with a single node bootstrapping. Now, for a given row key suppose reads are served by A, B and C while writes are to go to A, B, C and D. In other words, D is the node bootstrapping. Suppose RF is 3 and A,B,C,D is ring order. There are a few things required for correct behavior:
> * Writes acked by D must never be treated as sufficient to satisfy consistency level since until it is part of the read set it does not count towards CL on reads.
> * Writes acked by B must *not* be treated as sufficient to satisfy consistency level *unless* the same write is *also* acked by D, because once D enters the ring, B will no longer be counting towards CL on reads. The only alternative is to make the read succeed and disallow D from entering the ring.
> We don't seem to be handling this at all (and it becomes more complicated with arbitrary transitions).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira