[jira] [Commented] (CASSANDRA-6411) Issue with reading from sstable

["Jeremy Jongsma (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-6411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843431#comment-13843431 ] 

Jeremy Jongsma commented on CASSANDRA-6411:
-------------------------------------------

We have been seeing this a lot too on Cassandra 1.2.10.1. On a development cluster with a very minimal data set and usage profile, we see this roughly once a week - almost always on Monday morning for some reason. The only maintenance tasks we run nightly are:

Backups: 
  nodetool clearsnapshot
  nodetool snapshot -t $DATE

General maintenance:
  nodetool flush
  nodetool compact
  nodetool repair --partitioner-range

None of these commands return any errors when they are run, but on Monday morning some queries trigger the error in the original report. The only fix is nodetool scrub / nodetool repair on the affected node.


> Issue with reading from sstable
> -------------------------------
>
>                 Key: CASSANDRA-6411
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6411
>             Project: Cassandra
>          Issue Type: Bug
>          Components: API
>            Reporter: Mike Konobeevskiy
>
> With Cassandra 1.2.5 this happens almost every week. 
> java.lang.RuntimeException: org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.EOFException: EOF after 5105 bytes out of 19815
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1582)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:724)
> Caused by: org.apache.cassandra.io.sstable.CorruptSSTableException: java.io.EOFException: EOF after 5105 bytes out of 19815
> 	at org.apache.cassandra.db.columniterator.SimpleSliceReader.<init>(SimpleSliceReader.java:91)
> 	at org.apache.cassandra.db.columniterator.SSTableSliceIterator.createReader(SSTableSliceIterator.java:68)
> 	at org.apache.cassandra.db.columniterator.SSTableSliceIterator.<init>(SSTableSliceIterator.java:44)
> 	at org.apache.cassandra.db.filter.SliceQueryFilter.getSSTableColumnIterator(SliceQueryFilter.java:101)
> 	at org.apache.cassandra.db.filter.QueryFilter.getSSTableColumnIterator(QueryFilter.java:68)
> 	at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:274)
> 	at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:65)
> 	at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1357)
> 	at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1214)
> 	at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1126)
> 	at org.apache.cassandra.db.Table.getRow(Table.java:347)
> 	at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:70)
> 	at org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1052)
> 	at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1578)
> 	... 3 more
> Caused by: java.io.EOFException: EOF after 5105 bytes out of 19815
> 	at org.apache.cassandra.io.util.FileUtils.skipBytesFully(FileUtils.java:350)
> 	at org.apache.cassandra.utils.ByteBufferUtil.skipShortLength(ByteBufferUtil.java:382)
> 	at org.apache.cassandra.db.columniterator.SimpleSliceReader.<init>(SimpleSliceReader.java:72)
> 	... 16 more
> This is occurring roughly weekly with quite minimal usage.
> Recreation of CF does not help.



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)