[jira] [Created] (CASSANDRA-6600) Huge read latency with LOCAL_ONE when < RF nodes are up

["Duncan Sands (JIRA)" <ji...@apache.org>]

Duncan Sands created CASSANDRA-6600:
---------------------------------------

             Summary: Huge read latency with LOCAL_ONE when < RF nodes are up
                 Key: CASSANDRA-6600
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6600
             Project: Cassandra
          Issue Type: Bug
          Components: Core
            Reporter: Duncan Sands


I recently upgraded a multi data centre cluster from 1.2.12 to 2.0.4.

In one data centre there are 3 nodes with an RF of 3.  Clients are reading from these nodes using CQL3 and LOCAL_ONE.  At one point during the upgrade 1 node was down, so less than RF nodes were up.  Read latency went from < 1ms to > 1 second.  Once all nodes were up, read latency went back down to < 1ms.  If I stop a node then read latency shoots back up again.

This is not due to my client as I was able to reproduce this as follows.  With all RF nodes up:
  connect to a node using cqlsh
  set the consistency level to LOCAL_ONE
  use cqlsh to read a few values from a random table - it completes instantly
  bring down one of the other nodes in the same data centre
  do the same query again in cqlsh.  It times out with "Request did not complete within rpc_timeout".



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)