Re: Crash with TombstoneOverwhelmingException

[David Tinker <da...@gmail.com>]

We are seeing the exact same exception in our logs. Is there any workaround?

We never delete rows but we do a lot of updates. Is that where the
tombstones are coming from?

On Wed, Dec 25, 2013 at 5:24 PM, Sanjeeth Kumar <sa...@exotel.in> wrote:
> Hi all,
>   One of my cassandra nodes crashes with the following exception
> periodically -
> ERROR [HintedHandoff:33] 2013-12-25 20:29:22,276 SliceQueryFilter.java (line
> 200) Scanned over 100000 tombstones; query aborted (see tombstone_fail_thr
> eshold)
> ERROR [HintedHandoff:33] 2013-12-25 20:29:22,278 CassandraDaemon.java (line
> 187) Exception in thread Thread[HintedHandoff:33,1,main]
> org.apache.cassandra.db.filter.TombstoneOverwhelmingException
>         at
> org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:201)
>         at
> org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:122)
>         at
> org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:80)
>         at
> org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:72)
>         at
> org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:297)
>         at
> org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:53)
>         at
> org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1487)
>         at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1306)
>         at
> org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:351)
>         at
> org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:309)
>         at
> org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:92)
>         at
> org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:530)
>         at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>         at java.lang.Thread.run(Thread.java:744)
>
> Why does this happen? Does this relate to any incorrect config value?
>
> The Cassandra Version I'm running is
> ReleaseVersion: 2.0.3
>
> - Sanjeeth
>



-- 
http://qdb.io/ Persistent Message Queues With Replay and #RabbitMQ Integration

Re: Crash with TombstoneOverwhelmingException

[Cyril Scetbon <cy...@free.fr>]

With cassandra an update is equivalent to an insert

Cyril Scetbon

> Le 14 janv. 2014 à 08:38, David Tinker <da...@gmail.com> a écrit :
> 
> We never delete rows but we do a lot of updates. Is that where the
> tombstones are coming from?