You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@cassandra.apache.org by Edmond Lau <ed...@ooyala.com> on 2009/10/15 19:51:46 UTC
cassandra fatal error - "The name should match the name of the
current column or super column"
I'm using the cassandra 0.4 release. I was loading a bunch of data
into cassandra when the thrift api started throwing
UnavailableExceptions. Checking the logs, I found errors that looked
like the following:
ERROR [ROW-MUTATION-STAGE:2935] 2009-10-15 17:32:52,518
DebuggableThreadPoolExecutor.java (line 85) Error in
ThreadPoolExecutor
java.lang.IllegalArgumentException: The name should match the name of
the current column or super column
at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
at org.apache.cassandra.db.Memtable.resolve(Memtable.java:156)
at org.apache.cassandra.db.Memtable.put(Memtable.java:139)
at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:450)
at org.apache.cassandra.db.Table.apply(Table.java:608)
at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:205)
at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:79)
at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:39)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
ERROR [ROW-MUTATION-STAGE:2935] 2009-10-15 17:32:52,519
CassandraDaemon.java (line 71) Fatal exception in thread
Thread[ROW-MUTATION-STAGE:2935,5,main]
java.lang.IllegalArgumentException: The name should match the name of
the current column or super column
at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
at org.apache.cassandra.db.Memtable.resolve(Memtable.java:156)
at org.apache.cassandra.db.Memtable.put(Memtable.java:139)
at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:450)
at org.apache.cassandra.db.Table.apply(Table.java:608)
at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:205)
at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:79)
at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:39)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Stopping and starting the cluster gives me something similar:
ERROR - Error in executor futuretask
java.util.concurrent.ExecutionException:
java.lang.IllegalArgumentException: The name should match the name of
the current column or super column
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.logFutureExceptions(DebuggableThreadPoolExecutor.java:95)
at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor.afterExecute(DebuggableScheduledThreadPoolExecutor.java:50)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.lang.IllegalArgumentException: The name should match
the name of the current column or super column
at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
at org.apache.cassandra.db.ColumnFamily.resolve(ColumnFamily.java:408)
at org.apache.cassandra.db.ColumnFamilyStore.merge(ColumnFamilyStore.java:477)
at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:1078)
at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:689)
at org.apache.cassandra.db.MinorCompactionManager$1.call(MinorCompactionManager.java:165)
at org.apache.cassandra.db.MinorCompactionManager$1.call(MinorCompactionManager.java:162)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
... 2 more
I haven't yet tried to repro, but was wondering if anyone had any
insight on it. It looks like an update got routed to a super column
that wasn't responsible for it?
Edmond
Re: cassandra fatal error - "The name should match the name of the
current column or super column"
Posted by Edmond Lau <ed...@ooyala.com>.
I re-read my response, and in case it was unclear, I meant:
I applied the patch but wasn't able to reuse the old tables. The
patch seems to be working fine after I nuked the data though.
Edmond
On Mon, Oct 19, 2009 at 5:16 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> Thanks for following up!
>
> On Mon, Oct 19, 2009 at 7:11 PM, Edmond Lau <ed...@ooyala.com> wrote:
>> I wasn't able to apply the patch and reuse the old tables, but after
>> nuking the data, I'm no longer running into the issue anymore.
>>
>> On Fri, Oct 16, 2009 at 3:16 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>> Those are both consistent with a different bug that was fixed for
>>> 0.4.1 (now released officially!)
>>>
>>> -Jonathan
>>>
>>> On Fri, Oct 16, 2009 at 4:59 PM, Edmond Lau <ed...@ooyala.com> wrote:
>>>> Jonathan -
>>>>
>>>> I patched in your latest change that dropped the assertions and tried
>>>> to restart my cluster on my old data. 2 of 5 nodes still failed to
>>>> start, with different errors.
>>>>
>>>> One dies with a generic EOFException during recovery:
>>>>
>>>> INFO - Compacting
>>>> [/var/lib/cassandra/data/Analytics/movie-53-Data.db,/var/lib/cassandra/data/Analytics/movie-54-Data.db,/var/lib/cassandra/data/Analytics/movie-55-Data.db,/var/lib/cassandra/data/Analytics/movie-56-Data.db]
>>>> INFO - Completed flushing Memtable(country)@1074231936
>>>> ERROR - Exception encountered during startup.
>>>> java.io.EOFException
>>>> at java.io.DataInputStream.readFully(DataInputStream.java:180)
>>>> at java.io.DataInputStream.readUTF(DataInputStream.java:592)
>>>> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
>>>> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:222)
>>>> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:320)
>>>> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:58)
>>>> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:91)
>>>> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:167)
>>>> Exception encountered during startup.
>>>>
>>>> The other dies with a more interesting:
>>>>
>>>> INFO [main] 2009-10-16 21:42:17,082 Memtable.java (line 186) Flushing
>>>> Memtable(provider)@322343927
>>>> INFO [main] 2009-10-16 21:42:17,179 Memtable.java (line 220)
>>>> Completed flushing Memtable(provider)@322343927
>>>> ERROR [main] 2009-10-16 21:42:17,338 CassandraDaemon.java (line 185)
>>>> Exception encountered during startup.
>>>> java.lang.RuntimeException: Unable to load comparator class ''.
>>>> probably this means you have obsolete sstables lying around
>>>> at org.apache.cassandra.db.ColumnFamilySerializer.readComparator(ColumnFamilySerializer.java:108)
>>>> at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:83)
>>>> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:229)
>>>> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:320)
>>>> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:58)
>>>> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:91)
>>>> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:167)
>>>> Caused by: java.lang.ClassNotFoundException:
>>>> at java.lang.Class.forName0(Native Method)
>>>> at java.lang.Class.forName(Class.java:169)
>>>> at org.apache.cassandra.db.ColumnFamilySerializer.readComparator(ColumnFamilySerializer.java:104)
>>>> ... 6 more
>>>>
>>>> There may be some unprintable garbage characters in the class name.
>>>> They show up in emacs but aren't visible in the system log file.
>>>>
>>>> Would you expect the previous fatal error to have corrupted my data?
>>>>
>>>> Edmond
>>>>
>>>> On Fri, Oct 16, 2009 at 1:23 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>>>> Don't know if you tried this yet, but Eric noticed a bug in my patch.
>>>>> There's a new one now, fixed.
>>>>>
>>>>> -Jonathan
>>>>>
>>>>
>>>
>>
>
Re: cassandra fatal error - "The name should match the name of the
current column or super column"
Posted by Jonathan Ellis <jb...@gmail.com>.
Thanks for following up!
On Mon, Oct 19, 2009 at 7:11 PM, Edmond Lau <ed...@ooyala.com> wrote:
> I wasn't able to apply the patch and reuse the old tables, but after
> nuking the data, I'm no longer running into the issue anymore.
>
> On Fri, Oct 16, 2009 at 3:16 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>> Those are both consistent with a different bug that was fixed for
>> 0.4.1 (now released officially!)
>>
>> -Jonathan
>>
>> On Fri, Oct 16, 2009 at 4:59 PM, Edmond Lau <ed...@ooyala.com> wrote:
>>> Jonathan -
>>>
>>> I patched in your latest change that dropped the assertions and tried
>>> to restart my cluster on my old data. 2 of 5 nodes still failed to
>>> start, with different errors.
>>>
>>> One dies with a generic EOFException during recovery:
>>>
>>> INFO - Compacting
>>> [/var/lib/cassandra/data/Analytics/movie-53-Data.db,/var/lib/cassandra/data/Analytics/movie-54-Data.db,/var/lib/cassandra/data/Analytics/movie-55-Data.db,/var/lib/cassandra/data/Analytics/movie-56-Data.db]
>>> INFO - Completed flushing Memtable(country)@1074231936
>>> ERROR - Exception encountered during startup.
>>> java.io.EOFException
>>> at java.io.DataInputStream.readFully(DataInputStream.java:180)
>>> at java.io.DataInputStream.readUTF(DataInputStream.java:592)
>>> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
>>> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:222)
>>> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:320)
>>> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:58)
>>> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:91)
>>> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:167)
>>> Exception encountered during startup.
>>>
>>> The other dies with a more interesting:
>>>
>>> INFO [main] 2009-10-16 21:42:17,082 Memtable.java (line 186) Flushing
>>> Memtable(provider)@322343927
>>> INFO [main] 2009-10-16 21:42:17,179 Memtable.java (line 220)
>>> Completed flushing Memtable(provider)@322343927
>>> ERROR [main] 2009-10-16 21:42:17,338 CassandraDaemon.java (line 185)
>>> Exception encountered during startup.
>>> java.lang.RuntimeException: Unable to load comparator class ''.
>>> probably this means you have obsolete sstables lying around
>>> at org.apache.cassandra.db.ColumnFamilySerializer.readComparator(ColumnFamilySerializer.java:108)
>>> at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:83)
>>> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:229)
>>> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:320)
>>> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:58)
>>> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:91)
>>> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:167)
>>> Caused by: java.lang.ClassNotFoundException:
>>> at java.lang.Class.forName0(Native Method)
>>> at java.lang.Class.forName(Class.java:169)
>>> at org.apache.cassandra.db.ColumnFamilySerializer.readComparator(ColumnFamilySerializer.java:104)
>>> ... 6 more
>>>
>>> There may be some unprintable garbage characters in the class name.
>>> They show up in emacs but aren't visible in the system log file.
>>>
>>> Would you expect the previous fatal error to have corrupted my data?
>>>
>>> Edmond
>>>
>>> On Fri, Oct 16, 2009 at 1:23 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>>> Don't know if you tried this yet, but Eric noticed a bug in my patch.
>>>> There's a new one now, fixed.
>>>>
>>>> -Jonathan
>>>>
>>>
>>
>
Re: cassandra fatal error - "The name should match the name of the
current column or super column"
Posted by Edmond Lau <ed...@ooyala.com>.
I wasn't able to apply the patch and reuse the old tables, but after
nuking the data, I'm no longer running into the issue anymore.
On Fri, Oct 16, 2009 at 3:16 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> Those are both consistent with a different bug that was fixed for
> 0.4.1 (now released officially!)
>
> -Jonathan
>
> On Fri, Oct 16, 2009 at 4:59 PM, Edmond Lau <ed...@ooyala.com> wrote:
>> Jonathan -
>>
>> I patched in your latest change that dropped the assertions and tried
>> to restart my cluster on my old data. 2 of 5 nodes still failed to
>> start, with different errors.
>>
>> One dies with a generic EOFException during recovery:
>>
>> INFO - Compacting
>> [/var/lib/cassandra/data/Analytics/movie-53-Data.db,/var/lib/cassandra/data/Analytics/movie-54-Data.db,/var/lib/cassandra/data/Analytics/movie-55-Data.db,/var/lib/cassandra/data/Analytics/movie-56-Data.db]
>> INFO - Completed flushing Memtable(country)@1074231936
>> ERROR - Exception encountered during startup.
>> java.io.EOFException
>> at java.io.DataInputStream.readFully(DataInputStream.java:180)
>> at java.io.DataInputStream.readUTF(DataInputStream.java:592)
>> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
>> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:222)
>> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:320)
>> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:58)
>> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:91)
>> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:167)
>> Exception encountered during startup.
>>
>> The other dies with a more interesting:
>>
>> INFO [main] 2009-10-16 21:42:17,082 Memtable.java (line 186) Flushing
>> Memtable(provider)@322343927
>> INFO [main] 2009-10-16 21:42:17,179 Memtable.java (line 220)
>> Completed flushing Memtable(provider)@322343927
>> ERROR [main] 2009-10-16 21:42:17,338 CassandraDaemon.java (line 185)
>> Exception encountered during startup.
>> java.lang.RuntimeException: Unable to load comparator class ''.
>> probably this means you have obsolete sstables lying around
>> at org.apache.cassandra.db.ColumnFamilySerializer.readComparator(ColumnFamilySerializer.java:108)
>> at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:83)
>> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:229)
>> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:320)
>> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:58)
>> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:91)
>> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:167)
>> Caused by: java.lang.ClassNotFoundException:
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:169)
>> at org.apache.cassandra.db.ColumnFamilySerializer.readComparator(ColumnFamilySerializer.java:104)
>> ... 6 more
>>
>> There may be some unprintable garbage characters in the class name.
>> They show up in emacs but aren't visible in the system log file.
>>
>> Would you expect the previous fatal error to have corrupted my data?
>>
>> Edmond
>>
>> On Fri, Oct 16, 2009 at 1:23 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>> Don't know if you tried this yet, but Eric noticed a bug in my patch.
>>> There's a new one now, fixed.
>>>
>>> -Jonathan
>>>
>>
>
Re: cassandra fatal error - "The name should match the name of the
current column or super column"
Posted by Jonathan Ellis <jb...@gmail.com>.
Those are both consistent with a different bug that was fixed for
0.4.1 (now released officially!)
-Jonathan
On Fri, Oct 16, 2009 at 4:59 PM, Edmond Lau <ed...@ooyala.com> wrote:
> Jonathan -
>
> I patched in your latest change that dropped the assertions and tried
> to restart my cluster on my old data. 2 of 5 nodes still failed to
> start, with different errors.
>
> One dies with a generic EOFException during recovery:
>
> INFO - Compacting
> [/var/lib/cassandra/data/Analytics/movie-53-Data.db,/var/lib/cassandra/data/Analytics/movie-54-Data.db,/var/lib/cassandra/data/Analytics/movie-55-Data.db,/var/lib/cassandra/data/Analytics/movie-56-Data.db]
> INFO - Completed flushing Memtable(country)@1074231936
> ERROR - Exception encountered during startup.
> java.io.EOFException
> at java.io.DataInputStream.readFully(DataInputStream.java:180)
> at java.io.DataInputStream.readUTF(DataInputStream.java:592)
> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:222)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:320)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:58)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:91)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:167)
> Exception encountered during startup.
>
> The other dies with a more interesting:
>
> INFO [main] 2009-10-16 21:42:17,082 Memtable.java (line 186) Flushing
> Memtable(provider)@322343927
> INFO [main] 2009-10-16 21:42:17,179 Memtable.java (line 220)
> Completed flushing Memtable(provider)@322343927
> ERROR [main] 2009-10-16 21:42:17,338 CassandraDaemon.java (line 185)
> Exception encountered during startup.
> java.lang.RuntimeException: Unable to load comparator class ''.
> probably this means you have obsolete sstables lying around
> at org.apache.cassandra.db.ColumnFamilySerializer.readComparator(ColumnFamilySerializer.java:108)
> at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:83)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:229)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:320)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:58)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:91)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:167)
> Caused by: java.lang.ClassNotFoundException:
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.cassandra.db.ColumnFamilySerializer.readComparator(ColumnFamilySerializer.java:104)
> ... 6 more
>
> There may be some unprintable garbage characters in the class name.
> They show up in emacs but aren't visible in the system log file.
>
> Would you expect the previous fatal error to have corrupted my data?
>
> Edmond
>
> On Fri, Oct 16, 2009 at 1:23 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>> Don't know if you tried this yet, but Eric noticed a bug in my patch.
>> There's a new one now, fixed.
>>
>> -Jonathan
>>
>
Re: cassandra fatal error - "The name should match the name of the
current column or super column"
Posted by Edmond Lau <ed...@ooyala.com>.
Jonathan -
I patched in your latest change that dropped the assertions and tried
to restart my cluster on my old data. 2 of 5 nodes still failed to
start, with different errors.
One dies with a generic EOFException during recovery:
INFO - Compacting
[/var/lib/cassandra/data/Analytics/movie-53-Data.db,/var/lib/cassandra/data/Analytics/movie-54-Data.db,/var/lib/cassandra/data/Analytics/movie-55-Data.db,/var/lib/cassandra/data/Analytics/movie-56-Data.db]
INFO - Completed flushing Memtable(country)@1074231936
ERROR - Exception encountered during startup.
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:180)
at java.io.DataInputStream.readUTF(DataInputStream.java:592)
at java.io.DataInputStream.readUTF(DataInputStream.java:547)
at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:222)
at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:320)
at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:58)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:91)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:167)
Exception encountered during startup.
The other dies with a more interesting:
INFO [main] 2009-10-16 21:42:17,082 Memtable.java (line 186) Flushing
Memtable(provider)@322343927
INFO [main] 2009-10-16 21:42:17,179 Memtable.java (line 220)
Completed flushing Memtable(provider)@322343927
ERROR [main] 2009-10-16 21:42:17,338 CassandraDaemon.java (line 185)
Exception encountered during startup.
java.lang.RuntimeException: Unable to load comparator class ''.
probably this means you have obsolete sstables lying around
at org.apache.cassandra.db.ColumnFamilySerializer.readComparator(ColumnFamilySerializer.java:108)
at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:83)
at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:229)
at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:320)
at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:58)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:91)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:167)
Caused by: java.lang.ClassNotFoundException:
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at org.apache.cassandra.db.ColumnFamilySerializer.readComparator(ColumnFamilySerializer.java:104)
... 6 more
There may be some unprintable garbage characters in the class name.
They show up in emacs but aren't visible in the system log file.
Would you expect the previous fatal error to have corrupted my data?
Edmond
On Fri, Oct 16, 2009 at 1:23 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> Don't know if you tried this yet, but Eric noticed a bug in my patch.
> There's a new one now, fixed.
>
> -Jonathan
>
Re: cassandra fatal error - "The name should match the name of the
current column or super column"
Posted by Jonathan Ellis <jb...@gmail.com>.
Don't know if you tried this yet, but Eric noticed a bug in my patch.
There's a new one now, fixed.
-Jonathan
Re: cassandra fatal error - "The name should match the name of the
current column or super column"
Posted by Edmond Lau <ed...@ooyala.com>.
Thanks Jonathan for the fast response time. I'll patch your fix and
give it a whirl.
On Thu, Oct 15, 2009 at 11:47 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> Aha! :)
>
> Could you test the patch attached to
> https://issues.apache.org/jira/browse/CASSANDRA-493 ?
>
> (If you're using the binary release, you can get the source from
> https://svn.apache.org/repos/asf/incubator/cassandra/tags/cassandra-0.4.0-final/
> and build with "ant")
>
> thanks,
>
> -Jonathan
>
> On Thu, Oct 15, 2009 at 1:35 PM, Edmond Lau <ed...@ooyala.com> wrote:
>> Yes, I'm using UTF8Type as the comparator.
>>
>> On Thu, Oct 15, 2009 at 11:33 AM, Jonathan Ellis <jb...@gmail.com> wrote:
>>> Quicker question: Is this ColumnFamily using UTF8Type as its comparator?
>>>
>>> On Thu, Oct 15, 2009 at 1:17 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>>> The exception on restart is occurring during a compaction of
>>>> already-written sstables. It logs what the files are beforehand
>>>> ("Compacting [...]"). Could you gzip up those sstables and send those
>>>> to me somehow (the Data, Index, and Filter files) along with the
>>>> ColumnFamily definition from your config so I can try some tests out?
>>>>
>>>> -Jonathan
>>>>
>>>> On Thu, Oct 15, 2009 at 12:51 PM, Edmond Lau <ed...@ooyala.com> wrote:
>>>>> I'm using the cassandra 0.4 release. I was loading a bunch of data
>>>>> into cassandra when the thrift api started throwing
>>>>> UnavailableExceptions. Checking the logs, I found errors that looked
>>>>> like the following:
>>>>>
>>>>> ERROR [ROW-MUTATION-STAGE:2935] 2009-10-15 17:32:52,518
>>>>> DebuggableThreadPoolExecutor.java (line 85) Error in
>>>>> ThreadPoolExecutor
>>>>> java.lang.IllegalArgumentException: The name should match the name of
>>>>> the current column or super column
>>>>> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
>>>>> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
>>>>> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
>>>>> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:156)
>>>>> at org.apache.cassandra.db.Memtable.put(Memtable.java:139)
>>>>> at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:450)
>>>>> at org.apache.cassandra.db.Table.apply(Table.java:608)
>>>>> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:205)
>>>>> at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:79)
>>>>> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:39)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>> at java.lang.Thread.run(Thread.java:619)
>>>>> ERROR [ROW-MUTATION-STAGE:2935] 2009-10-15 17:32:52,519
>>>>> CassandraDaemon.java (line 71) Fatal exception in thread
>>>>> Thread[ROW-MUTATION-STAGE:2935,5,main]
>>>>> java.lang.IllegalArgumentException: The name should match the name of
>>>>> the current column or super column
>>>>> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
>>>>> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
>>>>> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
>>>>> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:156)
>>>>> at org.apache.cassandra.db.Memtable.put(Memtable.java:139)
>>>>> at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:450)
>>>>> at org.apache.cassandra.db.Table.apply(Table.java:608)
>>>>> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:205)
>>>>> at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:79)
>>>>> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:39)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>> at java.lang.Thread.run(Thread.java:619)
>>>>>
>>>>> Stopping and starting the cluster gives me something similar:
>>>>>
>>>>> ERROR - Error in executor futuretask
>>>>> java.util.concurrent.ExecutionException:
>>>>> java.lang.IllegalArgumentException: The name should match the name of
>>>>> the current column or super column
>>>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>>> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.logFutureExceptions(DebuggableThreadPoolExecutor.java:95)
>>>>> at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor.afterExecute(DebuggableScheduledThreadPoolExecutor.java:50)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>> at java.lang.Thread.run(Thread.java:619)
>>>>> Caused by: java.lang.IllegalArgumentException: The name should match
>>>>> the name of the current column or super column
>>>>> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
>>>>> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
>>>>> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
>>>>> at org.apache.cassandra.db.ColumnFamily.resolve(ColumnFamily.java:408)
>>>>> at org.apache.cassandra.db.ColumnFamilyStore.merge(ColumnFamilyStore.java:477)
>>>>> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:1078)
>>>>> at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:689)
>>>>> at org.apache.cassandra.db.MinorCompactionManager$1.call(MinorCompactionManager.java:165)
>>>>> at org.apache.cassandra.db.MinorCompactionManager$1.call(MinorCompactionManager.java:162)
>>>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>> ... 2 more
>>>>>
>>>>> I haven't yet tried to repro, but was wondering if anyone had any
>>>>> insight on it. It looks like an update got routed to a super column
>>>>> that wasn't responsible for it?
>>>>>
>>>>> Edmond
>>>>>
>>>>
>>>
>>
>
Re: cassandra fatal error - "The name should match the name of the
current column or super column"
Posted by Jonathan Ellis <jb...@gmail.com>.
Aha! :)
Could you test the patch attached to
https://issues.apache.org/jira/browse/CASSANDRA-493 ?
(If you're using the binary release, you can get the source from
https://svn.apache.org/repos/asf/incubator/cassandra/tags/cassandra-0.4.0-final/
and build with "ant")
thanks,
-Jonathan
On Thu, Oct 15, 2009 at 1:35 PM, Edmond Lau <ed...@ooyala.com> wrote:
> Yes, I'm using UTF8Type as the comparator.
>
> On Thu, Oct 15, 2009 at 11:33 AM, Jonathan Ellis <jb...@gmail.com> wrote:
>> Quicker question: Is this ColumnFamily using UTF8Type as its comparator?
>>
>> On Thu, Oct 15, 2009 at 1:17 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>> The exception on restart is occurring during a compaction of
>>> already-written sstables. It logs what the files are beforehand
>>> ("Compacting [...]"). Could you gzip up those sstables and send those
>>> to me somehow (the Data, Index, and Filter files) along with the
>>> ColumnFamily definition from your config so I can try some tests out?
>>>
>>> -Jonathan
>>>
>>> On Thu, Oct 15, 2009 at 12:51 PM, Edmond Lau <ed...@ooyala.com> wrote:
>>>> I'm using the cassandra 0.4 release. I was loading a bunch of data
>>>> into cassandra when the thrift api started throwing
>>>> UnavailableExceptions. Checking the logs, I found errors that looked
>>>> like the following:
>>>>
>>>> ERROR [ROW-MUTATION-STAGE:2935] 2009-10-15 17:32:52,518
>>>> DebuggableThreadPoolExecutor.java (line 85) Error in
>>>> ThreadPoolExecutor
>>>> java.lang.IllegalArgumentException: The name should match the name of
>>>> the current column or super column
>>>> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
>>>> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
>>>> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
>>>> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:156)
>>>> at org.apache.cassandra.db.Memtable.put(Memtable.java:139)
>>>> at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:450)
>>>> at org.apache.cassandra.db.Table.apply(Table.java:608)
>>>> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:205)
>>>> at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:79)
>>>> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:39)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:619)
>>>> ERROR [ROW-MUTATION-STAGE:2935] 2009-10-15 17:32:52,519
>>>> CassandraDaemon.java (line 71) Fatal exception in thread
>>>> Thread[ROW-MUTATION-STAGE:2935,5,main]
>>>> java.lang.IllegalArgumentException: The name should match the name of
>>>> the current column or super column
>>>> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
>>>> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
>>>> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
>>>> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:156)
>>>> at org.apache.cassandra.db.Memtable.put(Memtable.java:139)
>>>> at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:450)
>>>> at org.apache.cassandra.db.Table.apply(Table.java:608)
>>>> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:205)
>>>> at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:79)
>>>> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:39)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:619)
>>>>
>>>> Stopping and starting the cluster gives me something similar:
>>>>
>>>> ERROR - Error in executor futuretask
>>>> java.util.concurrent.ExecutionException:
>>>> java.lang.IllegalArgumentException: The name should match the name of
>>>> the current column or super column
>>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.logFutureExceptions(DebuggableThreadPoolExecutor.java:95)
>>>> at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor.afterExecute(DebuggableScheduledThreadPoolExecutor.java:50)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:619)
>>>> Caused by: java.lang.IllegalArgumentException: The name should match
>>>> the name of the current column or super column
>>>> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
>>>> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
>>>> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
>>>> at org.apache.cassandra.db.ColumnFamily.resolve(ColumnFamily.java:408)
>>>> at org.apache.cassandra.db.ColumnFamilyStore.merge(ColumnFamilyStore.java:477)
>>>> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:1078)
>>>> at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:689)
>>>> at org.apache.cassandra.db.MinorCompactionManager$1.call(MinorCompactionManager.java:165)
>>>> at org.apache.cassandra.db.MinorCompactionManager$1.call(MinorCompactionManager.java:162)
>>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> ... 2 more
>>>>
>>>> I haven't yet tried to repro, but was wondering if anyone had any
>>>> insight on it. It looks like an update got routed to a super column
>>>> that wasn't responsible for it?
>>>>
>>>> Edmond
>>>>
>>>
>>
>
Re: cassandra fatal error - "The name should match the name of the
current column or super column"
Posted by Edmond Lau <ed...@ooyala.com>.
Yes, I'm using UTF8Type as the comparator.
On Thu, Oct 15, 2009 at 11:33 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> Quicker question: Is this ColumnFamily using UTF8Type as its comparator?
>
> On Thu, Oct 15, 2009 at 1:17 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>> The exception on restart is occurring during a compaction of
>> already-written sstables. It logs what the files are beforehand
>> ("Compacting [...]"). Could you gzip up those sstables and send those
>> to me somehow (the Data, Index, and Filter files) along with the
>> ColumnFamily definition from your config so I can try some tests out?
>>
>> -Jonathan
>>
>> On Thu, Oct 15, 2009 at 12:51 PM, Edmond Lau <ed...@ooyala.com> wrote:
>>> I'm using the cassandra 0.4 release. I was loading a bunch of data
>>> into cassandra when the thrift api started throwing
>>> UnavailableExceptions. Checking the logs, I found errors that looked
>>> like the following:
>>>
>>> ERROR [ROW-MUTATION-STAGE:2935] 2009-10-15 17:32:52,518
>>> DebuggableThreadPoolExecutor.java (line 85) Error in
>>> ThreadPoolExecutor
>>> java.lang.IllegalArgumentException: The name should match the name of
>>> the current column or super column
>>> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
>>> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
>>> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
>>> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:156)
>>> at org.apache.cassandra.db.Memtable.put(Memtable.java:139)
>>> at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:450)
>>> at org.apache.cassandra.db.Table.apply(Table.java:608)
>>> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:205)
>>> at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:79)
>>> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:39)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:619)
>>> ERROR [ROW-MUTATION-STAGE:2935] 2009-10-15 17:32:52,519
>>> CassandraDaemon.java (line 71) Fatal exception in thread
>>> Thread[ROW-MUTATION-STAGE:2935,5,main]
>>> java.lang.IllegalArgumentException: The name should match the name of
>>> the current column or super column
>>> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
>>> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
>>> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
>>> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:156)
>>> at org.apache.cassandra.db.Memtable.put(Memtable.java:139)
>>> at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:450)
>>> at org.apache.cassandra.db.Table.apply(Table.java:608)
>>> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:205)
>>> at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:79)
>>> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:39)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:619)
>>>
>>> Stopping and starting the cluster gives me something similar:
>>>
>>> ERROR - Error in executor futuretask
>>> java.util.concurrent.ExecutionException:
>>> java.lang.IllegalArgumentException: The name should match the name of
>>> the current column or super column
>>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.logFutureExceptions(DebuggableThreadPoolExecutor.java:95)
>>> at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor.afterExecute(DebuggableScheduledThreadPoolExecutor.java:50)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>> at java.lang.Thread.run(Thread.java:619)
>>> Caused by: java.lang.IllegalArgumentException: The name should match
>>> the name of the current column or super column
>>> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
>>> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
>>> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
>>> at org.apache.cassandra.db.ColumnFamily.resolve(ColumnFamily.java:408)
>>> at org.apache.cassandra.db.ColumnFamilyStore.merge(ColumnFamilyStore.java:477)
>>> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:1078)
>>> at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:689)
>>> at org.apache.cassandra.db.MinorCompactionManager$1.call(MinorCompactionManager.java:165)
>>> at org.apache.cassandra.db.MinorCompactionManager$1.call(MinorCompactionManager.java:162)
>>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>> ... 2 more
>>>
>>> I haven't yet tried to repro, but was wondering if anyone had any
>>> insight on it. It looks like an update got routed to a super column
>>> that wasn't responsible for it?
>>>
>>> Edmond
>>>
>>
>
Re: cassandra fatal error - "The name should match the name of the
current column or super column"
Posted by Jonathan Ellis <jb...@gmail.com>.
Quicker question: Is this ColumnFamily using UTF8Type as its comparator?
On Thu, Oct 15, 2009 at 1:17 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> The exception on restart is occurring during a compaction of
> already-written sstables. It logs what the files are beforehand
> ("Compacting [...]"). Could you gzip up those sstables and send those
> to me somehow (the Data, Index, and Filter files) along with the
> ColumnFamily definition from your config so I can try some tests out?
>
> -Jonathan
>
> On Thu, Oct 15, 2009 at 12:51 PM, Edmond Lau <ed...@ooyala.com> wrote:
>> I'm using the cassandra 0.4 release. I was loading a bunch of data
>> into cassandra when the thrift api started throwing
>> UnavailableExceptions. Checking the logs, I found errors that looked
>> like the following:
>>
>> ERROR [ROW-MUTATION-STAGE:2935] 2009-10-15 17:32:52,518
>> DebuggableThreadPoolExecutor.java (line 85) Error in
>> ThreadPoolExecutor
>> java.lang.IllegalArgumentException: The name should match the name of
>> the current column or super column
>> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
>> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
>> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
>> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:156)
>> at org.apache.cassandra.db.Memtable.put(Memtable.java:139)
>> at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:450)
>> at org.apache.cassandra.db.Table.apply(Table.java:608)
>> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:205)
>> at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:79)
>> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:39)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:619)
>> ERROR [ROW-MUTATION-STAGE:2935] 2009-10-15 17:32:52,519
>> CassandraDaemon.java (line 71) Fatal exception in thread
>> Thread[ROW-MUTATION-STAGE:2935,5,main]
>> java.lang.IllegalArgumentException: The name should match the name of
>> the current column or super column
>> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
>> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
>> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
>> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:156)
>> at org.apache.cassandra.db.Memtable.put(Memtable.java:139)
>> at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:450)
>> at org.apache.cassandra.db.Table.apply(Table.java:608)
>> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:205)
>> at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:79)
>> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:39)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:619)
>>
>> Stopping and starting the cluster gives me something similar:
>>
>> ERROR - Error in executor futuretask
>> java.util.concurrent.ExecutionException:
>> java.lang.IllegalArgumentException: The name should match the name of
>> the current column or super column
>> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.logFutureExceptions(DebuggableThreadPoolExecutor.java:95)
>> at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor.afterExecute(DebuggableScheduledThreadPoolExecutor.java:50)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:619)
>> Caused by: java.lang.IllegalArgumentException: The name should match
>> the name of the current column or super column
>> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
>> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
>> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
>> at org.apache.cassandra.db.ColumnFamily.resolve(ColumnFamily.java:408)
>> at org.apache.cassandra.db.ColumnFamilyStore.merge(ColumnFamilyStore.java:477)
>> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:1078)
>> at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:689)
>> at org.apache.cassandra.db.MinorCompactionManager$1.call(MinorCompactionManager.java:165)
>> at org.apache.cassandra.db.MinorCompactionManager$1.call(MinorCompactionManager.java:162)
>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> ... 2 more
>>
>> I haven't yet tried to repro, but was wondering if anyone had any
>> insight on it. It looks like an update got routed to a super column
>> that wasn't responsible for it?
>>
>> Edmond
>>
>
Re: cassandra fatal error - "The name should match the name of the
current column or super column"
Posted by Jonathan Ellis <jb...@gmail.com>.
The exception on restart is occurring during a compaction of
already-written sstables. It logs what the files are beforehand
("Compacting [...]"). Could you gzip up those sstables and send those
to me somehow (the Data, Index, and Filter files) along with the
ColumnFamily definition from your config so I can try some tests out?
-Jonathan
On Thu, Oct 15, 2009 at 12:51 PM, Edmond Lau <ed...@ooyala.com> wrote:
> I'm using the cassandra 0.4 release. I was loading a bunch of data
> into cassandra when the thrift api started throwing
> UnavailableExceptions. Checking the logs, I found errors that looked
> like the following:
>
> ERROR [ROW-MUTATION-STAGE:2935] 2009-10-15 17:32:52,518
> DebuggableThreadPoolExecutor.java (line 85) Error in
> ThreadPoolExecutor
> java.lang.IllegalArgumentException: The name should match the name of
> the current column or super column
> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:156)
> at org.apache.cassandra.db.Memtable.put(Memtable.java:139)
> at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:450)
> at org.apache.cassandra.db.Table.apply(Table.java:608)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:205)
> at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:79)
> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:39)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> ERROR [ROW-MUTATION-STAGE:2935] 2009-10-15 17:32:52,519
> CassandraDaemon.java (line 71) Fatal exception in thread
> Thread[ROW-MUTATION-STAGE:2935,5,main]
> java.lang.IllegalArgumentException: The name should match the name of
> the current column or super column
> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
> at org.apache.cassandra.db.Memtable.resolve(Memtable.java:156)
> at org.apache.cassandra.db.Memtable.put(Memtable.java:139)
> at org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:450)
> at org.apache.cassandra.db.Table.apply(Table.java:608)
> at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:205)
> at org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:79)
> at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:39)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
>
> Stopping and starting the cluster gives me something similar:
>
> ERROR - Error in executor futuretask
> java.util.concurrent.ExecutionException:
> java.lang.IllegalArgumentException: The name should match the name of
> the current column or super column
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.logFutureExceptions(DebuggableThreadPoolExecutor.java:95)
> at org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor.afterExecute(DebuggableScheduledThreadPoolExecutor.java:50)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:888)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.lang.IllegalArgumentException: The name should match
> the name of the current column or super column
> at org.apache.cassandra.db.SuperColumn.putColumn(SuperColumn.java:208)
> at org.apache.cassandra.db.ColumnFamily.addColumn(ColumnFamily.java:200)
> at org.apache.cassandra.db.ColumnFamily.addColumns(ColumnFamily.java:127)
> at org.apache.cassandra.db.ColumnFamily.resolve(ColumnFamily.java:408)
> at org.apache.cassandra.db.ColumnFamilyStore.merge(ColumnFamilyStore.java:477)
> at org.apache.cassandra.db.ColumnFamilyStore.doFileCompaction(ColumnFamilyStore.java:1078)
> at org.apache.cassandra.db.ColumnFamilyStore.doCompaction(ColumnFamilyStore.java:689)
> at org.apache.cassandra.db.MinorCompactionManager$1.call(MinorCompactionManager.java:165)
> at org.apache.cassandra.db.MinorCompactionManager$1.call(MinorCompactionManager.java:162)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:207)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> ... 2 more
>
> I haven't yet tried to repro, but was wondering if anyone had any
> insight on it. It looks like an update got routed to a super column
> that wasn't responsible for it?
>
> Edmond
>