You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@cassandra.apache.org by Aaron Morton <aa...@thelastpickle.com> on 2011/02/08 22:40:48 UTC

error casting Column to SuperColumn during compaction. ? CASSANDRA-1992 ?

I got the error below on an newish 0.7.0 cluster with the following...

- no schema changes. 
- original RF at 1, changed to 3 via cassandra-cli and repair run
- stable node membership, i.e. no nodes added 

Was thinking it may have to do with  CASSANDRA-1992 (see http://www.mail-archive.com/user@cassandra.apache.org/msg09276.html) but I've not seen this error associated with that issue before. I can apply the patch or run the head on the 0.7 branch if that will help. 

May be able to dig into it further later today. (not ruling out me doing stupid things yet)

Aaron

ERROR [CompactionExecutor:1] 2011-02-08 16:59:35,380 AbstractCassandraDaemon.java (line org.apache.cassandra.service.AbstractCassandraDaemon) Fatal exception in thread Thread[CompactionExecutor:1,1,main]
java.lang.ClassCastException: org.apache.cassandra.db.Column cannot be cast to org.apache.cassandra.db.SuperColumn
        at org.apache.cassandra.db.SuperColumnSerializer.serialize(SuperColumn.java:329)
        at org.apache.cassandra.db.SuperColumnSerializer.serialize(SuperColumn.java:313)
        at org.apache.cassandra.db.ColumnFamilySerializer.serializeForSSTable(ColumnFamilySerializer.java:87)
        at org.apache.cassandra.db.ColumnFamilySerializer.serializeWithIndexes(ColumnFamilySerializer.java:106)
        at org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:97)
        at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:138)
        at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:107)
        at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:42)
        at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
        at org.apache.commons.collections.iterators.FilterIterator.setNextObjecand such starting t(FilterIterator.java:183)
        at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
        at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:323)
        at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
        at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTaskjava:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)


Then after restart started getting...


ERROR [CompactionExecutor:1] 2011-02-09 10:39:15,496 PrecompactedRow.java (line org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:82)) Skipping row DecoratedKey(71445620198865609512646197760056087250, 2f77686174696674686973776572656d617374657266696c652f746e6e5f73686f74735f3035305f3030352f6d664964) in /local1/cassandra/data/junkbox/ObjectIndex-e-216-Data.db
java.io.IOException: Corrupt (negative) value length encountered
        at org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:274)
        at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:94)
        at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:35)
        at org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:129)
        at org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:137)
        at org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:78)
        at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:138)
        at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:107)
        at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:42)
        at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
        at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
        at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
        at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:323)
        at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
        at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)
ERROR [CompactionExecutor:1] 2011-02-09 10:39:15,497 PrecompactedRow.java (line org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:82)) Skipping row DecoratedKey(71446871696297746749786898510505891557, 2f77686174696674686973776572656d617374657266696c652f746e6e5f73686f74735f6b63685f3032342f6d657461) in /local1/cassandra/data/junkbox/ObjectIndex-e-216-Data.db
org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid column name length 0
        at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:68)
        at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:35)
        at org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:129)
        at org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:137)
        at org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:78)
        at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:138)
        at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:107)
        at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:42)
        at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
        at com.google.commoncollect.AbstractIterator.hasNext(AbstractIterator.java:131)
        at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
        at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
        at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:323)
        at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
        at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:619)

Re: error casting Column to SuperColumn during compaction. ? CASSANDRA-1992 ?

Posted by Jonathan Ellis <jb...@gmail.com>.

Looks like https://issues.apache.org/jira/browse/CASSANDRA-1992.

On Tue, Feb 8, 2011 at 3:40 PM, Aaron Morton <aa...@thelastpickle.com> wrote:
> I got the error below on an newish 0.7.0 cluster with the following...
> - no schema changes.
> - original RF at 1, changed to 3 via cassandra-cli and repair run
> - stable node membership, i.e. no nodes added
> Was thinking it may have to do with  CASSANDRA-1992
> (see http://www.mail-archive.com/user@cassandra.apache.org/msg09276.html)
> but I've not seen this error associated with that issue before. I can apply
> the patch or run the head on the 0.7 branch if that will help.
> May be able to dig into it further later today. (not ruling out me doing
> stupid things yet)
> Aaron
> ERROR [CompactionExecutor:1] 2011-02-08 16:59:35,380
> AbstractCassandraDaemon.java (line
> org.apache.cassandra.service.AbstractCassandraDaemon) Fatal exception in
> thread Thread[CompactionExecutor:1,1,main]
> java.lang.ClassCastException: org.apache.cassandra.db.Column cannot be cast
> to org.apache.cassandra.db.SuperColumn
>         at
> org.apache.cassandra.db.SuperColumnSerializer.serialize(SuperColumn.java:329)
>         at
> org.apache.cassandra.db.SuperColumnSerializer.serialize(SuperColumn.java:313)
>         at
> org.apache.cassandra.dbColumnFamilySerializer.serializeForSSTable(ColumnFamilySerializer.java:87)
>         at
> org.apache.cassandra.db.ColumnFamilySerializer.serializeWithIndexes(ColumnFamilySerializer.java:106)
>         at
> org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:97)
>         at
> org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:138)
>         at
> org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:107)
>         at
> org.apachecassandra.io.CompactionIterator.getReduced(CompactionIterator.java:42)
>         at
> org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
>         at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
>         at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
>         at
> org.apache.commons.collections.iterators.FilterIterator.setNextObjecand such
> starting t(FilterIterator.java:183)
>         at
> org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
>         at
> org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:323)
>         at
> org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
>         at
> org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
>         at java.utilconcurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
>
> Then after restart started getting...
>
> ERROR [CompactionExecutor:1] 2011-02-09 10:39:15,496 PrecompactedRow.java
> (line
> org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:82))
> Skipping row DecoratedKey(71445620198865609512646197760056087250,
> 2f77686174696674686973776572656d617374657266696c652f746e6e5f73686f74735f3035305f3030352f6d664964)
> in /local1/cassandra/data/junkbox/ObjectIndex-e-216-Data.db
> java.io.IOException: Corrupt (negative) value length encountered
>         at
> org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:274)
>         at
> orgapache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:94)
>         at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:35)
>         at
> org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:129)
>         at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:137)
>         at
> org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:78)
>         at
> org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:138)
>         at
> org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:107)
>         at
> org.apache.cassandraio.CompactionIterator.getReduced(CompactionIterator.java:42)
>         at
> org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
>         at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
>         at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
>         at
> org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
>         at
> org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
>         at
> org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:323)
>         at
> org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
>         at
> org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
>         at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
> ERROR [CompactionExecutor:1] 2011-02-09 10:39:15,497 PrecompactedRow.java
> (line
> org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:82))
> Skipping row DecoratedKey(71446871696297746749786898510505891557,
> 2f77686174696674686973776572656d617374657266696c652f746e6e5f73686f74735f6b63685f3032342f6d657461)
> in /local1/cassandra/data/junkbox/ObjectIndex-e-216-Data.db
> org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid
> column name length 0
>         at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:68)
>         at
> org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:35)
>         at
> org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:129)
>         at
> org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:137)
>         at
> org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:78)
>         at
> org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:138)
>         at
> org.apache.cassandra.ioCompactionIterator.getReduced(CompactionIterator.java:107)
>         at
> org.apache.cassandra.io.CompactionIteratorgetReduced(CompactionIterator.java:42)
>         at
> org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
>         at
> com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
>         at
> com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
>         at
> org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
>         at
> org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
>         at
> org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:323)
>         at
> org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122)
>         at
> org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92)
>         at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com