You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Kjetil Valstadsve (JIRA)" <ji...@apache.org> on 2011/01/07 15:23:45 UTC
[jira] Commented: (CASSANDRA-1947) Cassandra dies on (presumed) bad
row during compaction
[ https://issues.apache.org/jira/browse/CASSANDRA-1947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978792#action_12978792 ]
Kjetil Valstadsve commented on CASSANDRA-1947:
----------------------------------------------
According to tjake (chatting on #cassandra), bad rows are readily detectable, but there hasn't been a good, general way to handle and report them.
For this, I would suggest a new property on the keyspace and/or column family, to indicate whether loss of data is acceptable. If set to true, it would be acceptable for cassandra to detect the bad row, log it, remove it, and continue.
Our use case is data that are generated on demand and occasionally refreshed. The occasional bad row wouldn't hurt us.
Statistics on the history of bad row detection would be nice, though, of course.
> Cassandra dies on (presumed) bad row during compaction
> ------------------------------------------------------
>
> Key: CASSANDRA-1947
> URL: https://issues.apache.org/jira/browse/CASSANDRA-1947
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Affects Versions: 0.7.0 rc 3
> Reporter: Kjetil Valstadsve
>
> My Cassandra has, unfortunately, ended up with a bad row somewhere. This consistently results in the following stacktrace and an abrupt death, shortly after (re)start:
> {code}
> ERROR [CompactionExecutor:1] 2011-01-06 12:47:56,057 AbstractCassandraDaemon.java Fatal exception in thread Thread[CompactionExecutor:1,1,main]
> java.lang.OutOfMemoryError: Java heap space
> at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
> at org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:277)
> at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:94)
> at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:35)
> at org.apache.cassandra.io.sstable.SSTableIdentityIterator.next(SSTableIdentityIterator.java:101)
> at org.apache.cassandra.io.sstable.SSTableIdentityIterator.next(SSTableIdentityIterator.java:34)
> at org.apache.commons.collections.iterators.CollatingIterator.set(CollatingIterator.java:284)
> at org.apache.commons.collections.iterators.CollatingIterator.least(CollatingIterator.java:326)
> at org.apache.commons.collections.iterators.CollatingIterator.next(CollatingIterator.java:230)
> at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:68)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
> at com.google.common.collect.Iterators$7.computeNext(Iterators.java:604)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
> at org.apache.cassandra.db.ColumnIndexer.serializeInternal(ColumnIndexer.java:76)
> at org.apache.cassandra.db.ColumnIndexer.serialize(ColumnIndexer.java:50)
> at org.apache.cassandra.io.LazilyCompactedRow.<init>(LazilyCompactedRow.java:88)
> at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:136)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:107)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:42)
> at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
> at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
> at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
> at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:323)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
> at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> {code}
> Looks to me like there should be a guard against bogus bytebuffer sizes, somewhere along this path.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.