You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Brandon Williams (JIRA)" <ji...@apache.org> on 2012/10/12 16:47:05 UTC
[jira] [Created] (CASSANDRA-4799) assertion failure in leveled
compaction test
Brandon Williams created CASSANDRA-4799:
-------------------------------------------
Summary: assertion failure in leveled compaction test
Key: CASSANDRA-4799
URL: https://issues.apache.org/jira/browse/CASSANDRA-4799
Project: Cassandra
Issue Type: Bug
Components: Core
Reporter: Brandon Williams
Assignee: Yuki Morishita
Fix For: 1.2.0
It's somewhat rare, but I'm regularly seeing this failure on trunk:
{{noformat}}
[junit] Testcase: testValidationMultipleSSTablePerLevel(org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest): FAILED
[junit] null
[junit] junit.framework.AssertionFailedError
[junit] at org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest.testValidationMultipleSSTablePerLevel(LeveledCompactionStrategyTest.java:78)
[junit]
[junit]
[junit] Test org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED
{{noformat}}
I suspect there's a deeper problem, since this is a pretty fundamental assertion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4799) assertion failure in leveled
compaction test
Posted by "Yuki Morishita (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476329#comment-13476329 ]
Yuki Morishita commented on CASSANDRA-4799:
-------------------------------------------
I found two problems here.
1) Test does not completely setup necessary SSTables and causes above AssertionError.
LCS test uses CFS#forceFlush to generate enough SSTables at the beginning of the test, but since the method is asynchronous call, there is a chance that test proceeds without sufficient SSTables to fill up L2 and causes AE at strat.getLevelSize(2) > 0.
Fix for this is to change forceFlush to forceBlockingFlush.
(diff: https://github.com/yukim/cassandra/commit/0d16efc6d7592e61f15598d5a4e3cc81d2760007)
2) Repair sometimes causes AssertionError with LCS
During the test, I encountered below error several times.
{code}
ERROR [ValidationExecutor:1] 2012-10-12 14:39:18,660 SchemaLoader.java (line 73) Fatal exception in thread Thread[ValidationExecutor:1,1,main]
java.lang.AssertionError
at org.apache.cassandra.db.compaction.LeveledCompactionStrategy.getScanners(LeveledCompactionStrategy.java:183)
at org.apache.cassandra.db.compaction.CompactionManager$ValidationCompactionIterable.<init>(CompactionManager.java:879)
at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:743)
at org.apache.cassandra.db.compaction.CompactionManager.access$700(CompactionManager.java:71)
at org.apache.cassandra.db.compaction.CompactionManager$7.call(CompactionManager.java:481)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
{code}
AssertionError comes from following assert code in LCS#getScanners:
{code}
for (SSTableReader sstable : sstables)
{
int level = manifest.levelOf(sstable);
assert level >= 0;
byLevel.get(level).add(sstable);
}
{code}
LeveledManifest#levelOf method returns level of SSTable or -1 if SSTable is not yet added to LeveledManifest. Here, each SSTable comes from CF's DataTracker.
Every time SSTable is written by flush/compaction, it gets added to DataTracker and after that, added to LeveledManifest if you are using LCS.
If above repair code is executed between those two, you will get AssertionError.
My proposed fix is to remove assertion and treat SSTables that do not belong LeveledManifest yet as L0 SSTables.
(diff: https://github.com/yukim/cassandra/commit/c8a0fb9a9128e47ec3d07926eb26f6fd93664f52)
Problem (2) also exists in 1.1 branch, but without assertion. We may want to fix this in 1.1 too.
> assertion failure in leveled compaction test
> --------------------------------------------
>
> Key: CASSANDRA-4799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4799
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Brandon Williams
> Assignee: Yuki Morishita
> Fix For: 1.2.0
>
>
> It's somewhat rare, but I'm regularly seeing this failure on trunk:
> {noformat}
> [junit] Testcase: testValidationMultipleSSTablePerLevel(org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest): FAILED
> [junit] null
> [junit] junit.framework.AssertionFailedError
> [junit] at org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest.testValidationMultipleSSTablePerLevel(LeveledCompactionStrategyTest.java:78)
> [junit]
> [junit]
> [junit] Test org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED
> {noformat}
> I suspect there's a deeper problem, since this is a pretty fundamental assertion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CASSANDRA-4799) assertion failure in leveled
compaction test
Posted by "Yuki Morishita (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yuki Morishita resolved CASSANDRA-4799.
---------------------------------------
Resolution: Fixed
Fix Version/s: 1.1.7
Reviewer: jbellis
Committed to trunk as well as 1.1 branch.
> assertion failure in leveled compaction test
> --------------------------------------------
>
> Key: CASSANDRA-4799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4799
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Brandon Williams
> Assignee: Yuki Morishita
> Fix For: 1.1.7, 1.2.0
>
>
> It's somewhat rare, but I'm regularly seeing this failure on trunk:
> {noformat}
> [junit] Testcase: testValidationMultipleSSTablePerLevel(org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest): FAILED
> [junit] null
> [junit] junit.framework.AssertionFailedError
> [junit] at org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest.testValidationMultipleSSTablePerLevel(LeveledCompactionStrategyTest.java:78)
> [junit]
> [junit]
> [junit] Test org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED
> {noformat}
> I suspect there's a deeper problem, since this is a pretty fundamental assertion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-4799) assertion failure in leveled
compaction test
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13476663#comment-13476663 ]
Jonathan Ellis commented on CASSANDRA-4799:
-------------------------------------------
+1
(I wish we could keep that assert but I don't have a better solution.)
> assertion failure in leveled compaction test
> --------------------------------------------
>
> Key: CASSANDRA-4799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4799
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Brandon Williams
> Assignee: Yuki Morishita
> Fix For: 1.2.0
>
>
> It's somewhat rare, but I'm regularly seeing this failure on trunk:
> {noformat}
> [junit] Testcase: testValidationMultipleSSTablePerLevel(org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest): FAILED
> [junit] null
> [junit] junit.framework.AssertionFailedError
> [junit] at org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest.testValidationMultipleSSTablePerLevel(LeveledCompactionStrategyTest.java:78)
> [junit]
> [junit]
> [junit] Test org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED
> {noformat}
> I suspect there's a deeper problem, since this is a pretty fundamental assertion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-4799) assertion failure in leveled
compaction test
Posted by "Brandon Williams (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-4799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brandon Williams updated CASSANDRA-4799:
----------------------------------------
Description:
It's somewhat rare, but I'm regularly seeing this failure on trunk:
{noformat}
[junit] Testcase: testValidationMultipleSSTablePerLevel(org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest): FAILED
[junit] null
[junit] junit.framework.AssertionFailedError
[junit] at org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest.testValidationMultipleSSTablePerLevel(LeveledCompactionStrategyTest.java:78)
[junit]
[junit]
[junit] Test org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED
{noformat}
I suspect there's a deeper problem, since this is a pretty fundamental assertion.
was:
It's somewhat rare, but I'm regularly seeing this failure on trunk:
{{noformat}}
[junit] Testcase: testValidationMultipleSSTablePerLevel(org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest): FAILED
[junit] null
[junit] junit.framework.AssertionFailedError
[junit] at org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest.testValidationMultipleSSTablePerLevel(LeveledCompactionStrategyTest.java:78)
[junit]
[junit]
[junit] Test org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED
{{noformat}}
I suspect there's a deeper problem, since this is a pretty fundamental assertion.
> assertion failure in leveled compaction test
> --------------------------------------------
>
> Key: CASSANDRA-4799
> URL: https://issues.apache.org/jira/browse/CASSANDRA-4799
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Reporter: Brandon Williams
> Assignee: Yuki Morishita
> Fix For: 1.2.0
>
>
> It's somewhat rare, but I'm regularly seeing this failure on trunk:
> {noformat}
> [junit] Testcase: testValidationMultipleSSTablePerLevel(org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest): FAILED
> [junit] null
> [junit] junit.framework.AssertionFailedError
> [junit] at org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest.testValidationMultipleSSTablePerLevel(LeveledCompactionStrategyTest.java:78)
> [junit]
> [junit]
> [junit] Test org.apache.cassandra.db.compaction.LeveledCompactionStrategyTest FAILED
> {noformat}
> I suspect there's a deeper problem, since this is a pretty fundamental assertion.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira