You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@cassandra.apache.org by Sam Hodgson <ho...@hotmail.com> on 2013/04/09 15:26:34 UTC

Added column does not sort as the last column at

Hi All,
I just upgraded from Cassandra 1.1.7 to 1.2.3 and im now seeing a lot of the following error in my output.log, cant find much on the web about it:
ERROR 11:56:01,317 Exception in thread Thread[ReadStage:7236,5,main] java.lang.AssertionError: Added column does not sort as the last column at org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:131) at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:109) at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:104) at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:171) at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:136) at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:84) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:294) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1363) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1220) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1132) at org.apache.cassandra.db.Table.getRow(Table.java:348) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:70) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1052) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1578) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)
I dont know Java and im struggling to identify exactly which query is causing this, is there a way to trace this back to a specific query?
Any help appreciated.
Sam

RE: Added column does not sort as the last column at

Posted by Sam Hodgson <ho...@hotmail.com>.

Thanks for this Aaron - I eventually found some data corruption in the keys of the affected CF's which i think was causing the issue.

> Subject: Re: Added column does not sort as the last column at
> From: aaron@thelastpickle.com
> Date: Thu, 11 Apr 2013 22:08:17 +1200
> To: user@cassandra.apache.org
> 
> To reduce possibilities, have you changed a super CF to a standard CF recently ? 
> 
> Can you isolate this to specific CF ? 
> 
> Have you changed the comparators / schema recently ? 
> 
> Cheers
> 
> -----------------
> Aaron Morton
> Freelance Cassandra Consultant
> New Zealand
> 
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 10/04/2013, at 1:26 AM, Sam Hodgson <ho...@hotmail.com> wrote:
> 
> > Hi All,
> > 
> > 
> > 
> > I just upgraded from Cassandra 1.1.7 to 1.2.3 and im now seeing a lot of the following error in my output.log, cant find much on the web about it:
> > 
> > 
> > 
> > ERROR 11:56:01,317 Exception in thread Thread[ReadStage:7236,5,main] java.lang.AssertionError: Added column does not sort as the last column at org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:131) at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:109) at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:104) at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:171) at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:136) at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:84) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:294) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1363) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1220) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1132) at org.apache.cassandra.db.Table.getRow(Table.java:348) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:70) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1052) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1578) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)
> > 
> > 
> > 
> > I dont know Java and im struggling to identify exactly which query is causing this, is there a way to trace this back to a specific query?
> > 
> > 
> > 
> > Any help appreciated.
> > 
> > 
> > 
> > Sam
> > 
>

Re: Added column does not sort as the last column at

Posted by aaron morton <aa...@thelastpickle.com>.

To reduce possibilities, have you changed a super CF to a standard CF recently ? 

Can you isolate this to specific CF ? 

Have you changed the comparators / schema recently ? 

Cheers

-----------------
Aaron Morton
Freelance Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 10/04/2013, at 1:26 AM, Sam Hodgson <ho...@hotmail.com> wrote:

> Hi All,
> 
> 
> 
> I just upgraded from Cassandra 1.1.7 to 1.2.3 and im now seeing a lot of the following error in my output.log, cant find much on the web about it:
> 
> 
> 
> ERROR 11:56:01,317 Exception in thread Thread[ReadStage:7236,5,main] java.lang.AssertionError: Added column does not sort as the last column at org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:131) at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:109) at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:104) at org.apache.cassandra.db.filter.SliceQueryFilter.collectReducedColumns(SliceQueryFilter.java:171) at org.apache.cassandra.db.filter.QueryFilter.collateColumns(QueryFilter.java:136) at org.apache.cassandra.db.filter.QueryFilter.collateOnDiskAtom(QueryFilter.java:84) at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:294) at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65) at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1363) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1220) at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1132) at org.apache.cassandra.db.Table.getRow(Table.java:348) at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:70) at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1052) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1578) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)
> 
> 
> 
> I dont know Java and im struggling to identify exactly which query is causing this, is there a way to trace this back to a specific query?
> 
> 
> 
> Any help appreciated.
> 
> 
> 
> Sam
>