[jira] [Created] (CASSANDRA-8316) "Did not get positive replies from all endpoints" error on incremental repair

["Loic Lambiel (JIRA)" <ji...@apache.org>]

Loic Lambiel created CASSANDRA-8316:
---------------------------------------

             Summary:  "Did not get positive replies from all endpoints" error on incremental repair
                 Key: CASSANDRA-8316
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8316
             Project: Cassandra
          Issue Type: Bug
          Components: Core
         Environment: cassandra 2.1.2
            Reporter: Loic Lambiel


Hi,

I've got an issue with incremental repairs on our production 15 nodes 2.1.2 (new cluster, not yet loaded, RF=3)

After having successfully performed an incremental repair (-par -inc) on 3 nodes, I started receiving "Repair failed with error Did not get positive replies from all endpoints." from nodetool on all remaining nodes :

[2014-11-14 09:12:36,488] Starting repair command #3, repairing 108 ranges for keyspace xxxx (seq=false, full=false)
[2014-11-14 09:12:47,919] Repair failed with error Did not get positive replies from all endpoints.

All the nodes are up and running and the local system log shows that the repair commands got started and that's it.

I've also noticed that soon after the repair, several nodes started having more cpu load indefinitely without any particular reason (no tasks / queries, nothing in the logs). I then restarted C* on these nodes and retried the repair on several nodes, which were successful until facing the issue again.

I tried to repro on our 3 nodes preproduction cluster without success

It looks like I'm not the only one having this issue: http://www.mail-archive.com/user%40cassandra.apache.org/msg39145.html

Any idea?

Thanks

Loic




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)