[jira] [Commented] (CASSANDRA-11804) Read request at proper CL returns ReadTimeout occasionally

["YCozy (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-11804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066764#comment-17066764 ] 

YCozy commented on CASSANDRA-11804:
-----------------------------------

We saw a similar issue in 3.11.5 when querying data using cqlsh. We had a three node cluster, and got the following error when two nodes are partitioned away:
{code:java}
<stdin>:1:ReadTimeout: Error from server: code=1200 [Coordinator node timed out waiting for replica nodes' responses] message="Operation timed out - received only 0 responses." info={'received_responses': 0, 'required_responses': 1, 'consistency': 'ONE'}{code}

> Read request at proper CL returns ReadTimeout occasionally
> ----------------------------------------------------------
>
>                 Key: CASSANDRA-11804
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11804
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: C* 3.4 | Ruby-driver 3.0-rc2
>            Reporter: Kishan Karunaratne
>            Priority: Normal
>             Fix For: 3.11.x
>
>
> I have a 3-node cluster with a keyspace with RF=3, with some data inserted. I'm using a DowngradingConsistency retry policy on the client. Performing a query at CL=ALL with one node blocked (ccm pause/SIGSTOP), the query fails and returns a ReadTimeout to the client as expected. The following is in the debug log:
> {noformat}
> ReadCallback.java:126 - Timed out; received 2 of 3 responses (including data)
> {noformat}
> Now, the driver automatically retries once, and in this case at QUORUM.  The client receives a ReadTimeout once more: "Cassandra::Errors::ReadTimeoutError: Operation timed out - received only 1 responses." This ReadTimeout should not have happened because we still have 2 good nodes/hosts to retrieve data from. This is in the debug log:
> {noformat}
> ReadCallback.java:126 - Timed out; received 1 of 2 responses (including data)
> {noformat}
> The weird part is that the query does occasionally succeed (at QUORUM), I'd say < 50% of the time. I did the same experiment with two nodes blocked and I get "Cassandra::Errors::ReadTimeoutError: Operation timed out - received only 0 responses.":
> {noformat}
> ReadCallback.java:126 - Timed out; received 1 of 3 responses (including data)
> ReadCallback.java:126 - Timed out; received 0 of 1 responses
> {noformat}
> I expect this query to have succeeded at ONE, because there is one good node left (and it's the one used as the coordinator node). In both cases, I feel like the coordinator node doesn't count itself as a replica for the CL.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org