[jira] [Created] (CASSANDRA-6403) Division by zero Exception in HintedHandoff and CompactionExecutor

["Gabriel Wicke (JIRA)" <ji...@apache.org>]

Gabriel Wicke created CASSANDRA-6403:
----------------------------------------

             Summary: Division by zero Exception in HintedHandoff and CompactionExecutor
                 Key: CASSANDRA-6403
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6403
             Project: Cassandra
          Issue Type: Bug
          Components: Core
         Environment: Cassandra 2.0.3 RC, Linux (Ubuntu Precise), OpenJDK 7
            Reporter: Gabriel Wicke


In write load testing I'm getting division by zero exceptions after running for a while:

ERROR [HintedHandoff:2] 2013-11-23 20:44:41,411 CassandraDaemon.java (line 187) Exception in thread Thread[HintedHandoff:2,1,main]
java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.ArithmeticException: / by zero
        at org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:464)
        at org.apache.cassandra.db.HintedHandOffManager.deliverHintsToEndpoint(HintedHandOffManager.java:309)
        at org.apache.cassandra.db.HintedHandOffManager.access$300(HintedHandOffManager.java:92)
        at org.apache.cassandra.db.HintedHandOffManager$4.run(HintedHandOffManager.java:530)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
Caused by: java.util.concurrent.ExecutionException: java.lang.ArithmeticException: / by zero
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
        at java.util.concurrent.FutureTask.get(FutureTask.java:111)
        at org.apache.cassandra.db.HintedHandOffManager.doDeliverHintsToEndpoint(HintedHandOffManager.java:460)
        ... 6 more

ERROR [CompactionExecutor:8] 2013-11-23 21:34:01,493 CassandraDaemon.java (line 187) Exception in thread Thread[CompactionExecutor:8,1,RMI Runtime]
java.lang.ArithmeticException: / by zero
        at org.apache.cassandra.db.compaction.ParallelCompactionIterable.<init>(ParallelCompactionIterable.java:59)
        at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:126)
        at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
        at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
        at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
        at org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:296)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)


The nodes that encounter this error seem to hold onto a lot of memory which is not freed even after the write load is stopped. With the write load continuing they eventually run out of heap. nodetool compact dies with the same exception.



--
This message was sent by Atlassian JIRA
(v6.1#6144)