You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Mikhail Stepura (JIRA)" <ji...@apache.org> on 2014/09/05 22:43:28 UTC
[jira] [Commented] (CASSANDRA-7892) Hashmap IllegalStateException
in anticompaction
[ https://issues.apache.org/jira/browse/CASSANDRA-7892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14123540#comment-14123540 ]
Mikhail Stepura commented on CASSANDRA-7892:
--------------------------------------------
It looks like we call {{sstableIterator.remove()}} for already removed {{sstable}}.
/cc [~yukim] [~krummas] [~lyubent] should we add a {{break}} to the 2nd if-branch ({{!sstableRange.intersects(r)}}) as well?
{code:title=CompactionManager.performAnticompaction()}
while (sstableIterator.hasNext())
{
SSTableReader sstable = sstableIterator.next();
for (Range<Token> r : Range.normalize(ranges))
{
...
if (r.contains(sstableRange))
{
logger.info("SSTable {} fully contained in range {}, mutating repairedAt instead of anticompacting", sstable, r);
sstable.descriptor.getMetadataSerializer().mutateRepairedAt(sstable.descriptor, repairedAt);
sstable.reloadSSTableMetadata();
mutatedRepairStatuses.add(sstable);
sstableIterator.remove();
break;
}
else if (!sstableRange.intersects(r))
{
logger.info("SSTable {} ({}) does not intersect repaired range {}, not touching repairedAt.", sstable, sstableRange, r);
nonAnticompacting.add(sstable);
sstableIterator.remove();
// to break here? <--------------
}
else
{
logger.info("SSTable {} ({}) will be anticompacted on range {}", sstable, sstableRange, r);
}
}
}
{code}
> Hashmap IllegalStateException in anticompaction
> -----------------------------------------------
>
> Key: CASSANDRA-7892
> URL: https://issues.apache.org/jira/browse/CASSANDRA-7892
> Project: Cassandra
> Issue Type: Bug
> Environment: Cassandra 2.1 rc 5
> Reporter: Johan Bjork
>
> {noformat}
> ERROR [AntiEntropyStage:4] 2014-09-05 19:55:06,439 CassandraDaemon.java:166 - Exception in thread Thread[AntiEntropyStage:4,5,main]
> java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalStateException
> at org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:117) ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62) ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_60]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ~[na:1.7.0_60]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_60]
> Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalStateException
> at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:411) ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:400) ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at org.apache.cassandra.repair.RepairMessageVerbHandler.doVerb(RepairMessageVerbHandler.java:113) ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> ... 4 common frames omitted
> Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException
> at java.util.concurrent.FutureTask.report(FutureTask.java:122) ~[na:1.7.0_60]
> at java.util.concurrent.FutureTask.get(FutureTask.java:188) ~[na:1.7.0_60]
> at org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:407) ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> ... 6 common frames omitted
> Caused by: java.lang.IllegalStateException: null
> at java.util.HashMap$HashIterator.remove(HashMap.java:938) ~[na:1.7.0_60]
> at org.apache.cassandra.db.compaction.CompactionManager.performAnticompaction(CompactionManager.java:425) ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:381) ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_60]
> ... 3 common frames omitted
> {noformat}
> And on a different host:
> {noformat}
> ERROR [CompactionExecutor:43] 2014-09-05 19:55:06,376 CassandraDaemon.java:166 - Exception in thread Thread[CompactionExecutor:4
> 3,1,main]
> java.lang.IllegalStateException: null
> at java.util.HashMap$HashIterator.remove(HashMap.java:938) ~[na:1.7.0_60]
> at org.apache.cassandra.db.compaction.CompactionManager.performAnticompaction(CompactionManager.java:425) ~[apache-cassa
> ndra-2.1.0-rc5.jar:2.1.0-rc5]
> at org.apache.cassandra.db.compaction.CompactionManager$6.runMayThrow(CompactionManager.java:381) ~[apache-cassandra-2.1
> .0-rc5.jar:2.1.0-rc5]
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[apache-cassandra-2.1.0-rc5.jar:2.1.0-rc5]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_60]
> at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_60]
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) ~[na:1.7.0_60]
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_60]
> at java.lang.Thread.run(Thread.java:745) [na:1.7.0_60]
> {noformat}
> The exceptions happened in our test environment after I added a bunch of new-nodes at the same time that was in a different (previously non-existing) datacenter. The datacenter is determined using GossipingPropertyFileSnitch and started repairs on a bunch of nodes at the same time using
> {noformat}
> nodetool -h <node> repair -inc -par
> {noformat}
> The nodes in the old datacenter had previously been repaired using the -inc option.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)