You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Yuki Morishita (JIRA)" <ji...@apache.org> on 2012/10/24 00:44:12 UTC
[jira] [Reopened] (CASSANDRA-3306) Error in
LeveledCompactionStrategy
[ https://issues.apache.org/jira/browse/CASSANDRA-3306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yuki Morishita reopened CASSANDRA-3306:
---------------------------------------
Assignee: Yuki Morishita
This error actually happens on 1.1. And I can easily reproduce with unit test(Test code attached).
{code}
[junit] ERROR 17:34:46,696 Fatal exception in thread Thread[CompactionExecutor:3,1,main]
[junit] java.lang.AssertionError: Expecting new size of 2, got 1 while replacing [SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-1-Data.db'), SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-5-Data.db'), SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-4-Data.db'), SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-2-Data.db')] by [SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-6-Data.db')] in View(pending_count=0, sstables=[SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-1-Data.db'), SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-2-Data.db'), SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-4-Data.db'), SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-4-Data.db'), SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-5-Data.db')], compacting=[SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-1-Data.db'), SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-5-Data.db'), SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-4-Data.db'), SSTableReader(path='build/test/cassandra/data/Keyspace1/Standard1/Keyspace1-Standard1-hf-2-Data.db')])
[junit] at org.apache.cassandra.db.DataTracker$View.newSSTables(DataTracker.java:651)
[junit] at org.apache.cassandra.db.DataTracker$View.replace(DataTracker.java:616)
[junit] at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:320)
[junit] at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:253)
[junit] at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:994)
[junit] at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:200)
[junit] at org.apache.cassandra.db.compaction.CompactionManager$1.runMayThrow(CompactionManager.java:154)
[junit] at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
[junit] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
[junit] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
[junit] at java.util.concurrent.FutureTask.run(FutureTask.java:138)
[junit] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
[junit] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
[junit] at java.lang.Thread.run(Thread.java:680)
{code}
The cause is actually in streaming. StreamInSession can add duplicate reference to SSTable to DataTracker when it is left even after stream session finishes. This typically happens when source node is marked as dead by FailureDetector during streaming session(GC storm is the one I saw) and keep sending file in same session after the node comes back.
> Error in LeveledCompactionStrategy
> ----------------------------------
>
> Key: CASSANDRA-3306
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3306
> Project: Cassandra
> Issue Type: Bug
> Affects Versions: 1.0.0
> Reporter: Radim Kolar
> Assignee: Yuki Morishita
>
> during stress testing, i always get this error making leveledcompaction strategy unusable. Should be easy to reproduce - just write fast.
> ERROR [CompactionExecutor:6] 2011-10-04 15:48:52,179 AbstractCassandraDaemon.java (line 133) Fatal exception in thread Thread[CompactionExecutor:6,5,main]
> java.lang.AssertionError
> at org.apache.cassandra.db.DataTracker$View.newSSTables(DataTracker.java:580)
> at org.apache.cassandra.db.DataTracker$View.replace(DataTracker.java:546)
> at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:268)
> at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:232)
> at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:960)
> at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:199)
> at org.apache.cassandra.db.compaction.LeveledCompactionTask.execute(LeveledCompactionTask.java:47)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:131)
> at org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:114)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:662)
> and this is in json data for table:
> {
> "generations" : [ {
> "generation" : 0,
> "members" : [ 459, 460, 461, 462, 463, 464, 465, 466, 467, 468, 469, 470, 471, 472, 473, 474, 475, 476, 477, 478, 479, 480, 481, 482, 483, 484 ]
> }, {
> "generation" : 1,
> "members" : [ ]
> }, {
> "generation" : 2,
> "members" : [ ]
> }, {
> "generation" : 3,
> "members" : [ ]
> }, {
> "generation" : 4,
> "members" : [ ]
> }, {
> "generation" : 5,
> "members" : [ ]
> }, {
> "generation" : 6,
> "members" : [ ]
> }, {
> "generation" : 7,
> "members" : [ ]
> } ]
> }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira