[jira] [Created] (CASSANDRA-10285) Compaction running indefinitely on system.hints

["Alan Boudreault (JIRA)" <ji...@apache.org>]

Alan Boudreault created CASSANDRA-10285:
-------------------------------------------

             Summary: Compaction running indefinitely on system.hints 
                 Key: CASSANDRA-10285
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10285
             Project: Cassandra
          Issue Type: Bug
            Reporter: Alan Boudreault


During my hints storage benchmarks, I've experienced an issue using C* 2.2. The hints was never replayed. After a while (more than 24H ...), I noticed that there were still compactions running on system.hints and that some new ones was triggered every 10-20 minutes.

To reproduce, we create a cluster of 2 nodes. RF=2 and we generate hints by shutting down the node2.
{code}
ccm create --install-dir=`pwd` -n 2 local && ccm start
ccm node1 stress -- write n=50M -rate threads=300 -port jmx=7198 -errors ignore -schema replication\(factor=2\)

# wait 5-6 seconds to get the schema creation propagated on all nodes, then in another window, stop node 2
ccm node2 stop

# wait the initial 50M writes are finished, bring back node2 up and write another 50M keys.
ccm node2 start
ccm node1 stress -- write n=50M -rate threads=300 -port jmx=7198 -errors ignore -schema replication\(factor=2\)

# You should get the initial compaction finished after 15-20 minutes. You can set the mb throughput to 0 to get that done faster. 

# Monitor the node1. the hints will never be replayed and you should see compactions happening indefinitely. 
{code}

//cc [~krummas] [~yukim]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)