[jira] [Created] (CASSANDRA-6998) HintedHandoff - expired hints may block future hints deliveries

["Scooletz (JIRA)" <ji...@apache.org>]

Scooletz created CASSANDRA-6998:
-----------------------------------

             Summary: HintedHandoff - expired hints may block future hints deliveries
                 Key: CASSANDRA-6998
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6998
             Project: Cassandra
          Issue Type: Bug
          Components: Core
         Environment: - cluster of two DCs: DC1, DC2
- keyspace using NetworkTopologyStrategy (replication factors for both DCs)
- heavy load (write:read, 100:1) with LOCAL_QUORUM using Java driver setup with DC awareness, writing to DC1

            Reporter: Scooletz
             Fix For: 2.0.3


For tests purposes, DC2 was shut down for 1 day. The _hints_ table was filled with millions of rows. Now, when _HintedHandOffManager_ tries to _doDeliverHintsToEndpoint_  it queries the store with QueryFilter.getSliceFilter which counts deleted (TTLed) cells and throws org.apache.cassandra.db.filter.TombstoneOverwhelmingException. 
Throwing this exception stops the manager from running compaction as it is run only after successful handoff. This leaves the HH practically disabled till administrator runs truncateAllHints. 
Wouldn't it be nicer if on org.apache.cassandra.db.filter.TombstoneOverwhelmingException run compaction? That would remove TTLed hints leaving whole HH mechanism in a healthy state.



--
This message was sent by Atlassian JIRA
(v6.2#6252)