[jira] [Comment Edited] (CASSANDRA-8192) Better error logging on corrupt compressed SSTables: currently AssertionError in Memory.java

["Marcus Eriksson (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-8192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14256721#comment-14256721 ] 

Marcus Eriksson edited comment on CASSANDRA-8192 at 12/23/14 7:57 AM:
----------------------------------------------------------------------

I need to create a file like this:
{code}
        DataOutputStream dos = new DataOutputStream(new FileOutputStream("/tmp/comp_info"));
        dos.writeUTF("LZ4Compressor"); // compressor name
        dos.writeInt(0);// option count
        dos.writeInt(0);//chunk length
        dos.writeLong(0); // data length
        dos.writeInt(0); // chunk count
        dos.close();
{code}
to reproduce (dunno how common it is for file corruption to create files like that (ie, being zeroed out after a number of bytes)?)

Anyway, could we add an assert where we serialize the CompressionInfo to disk to make sure we don't have a bug there? And, if it is corrupt data, we should probably check if chunkCount <= 0?


was (Author: krummas):
I need to create a file like this:
{code}
        DataOutputStream dos = new DataOutputStream(new FileOutputStream("/tmp/comp_info"));
        dos.writeUTF("LZ4Compressor"); // compressor name
        dos.writeInt(0);// option count
        dos.writeInt(0);//chunk length
        dos.writeLong(0); // data length
        dos.writeInt(0); // chunk count
        dos.close();
{code}
to reproduce (dunno how common it is for file corruption to create files like that?)

Anyway, could we add an assert where we serialize the CompressionInfo to disk to make sure we don't have a bug there? And, if it is corrupt data, we should probably check if chunkCount <= 0?

> Better error logging on corrupt compressed SSTables: currently AssertionError in Memory.java
> --------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-8192
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8192
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>         Environment: Windows-7-32 bit, 3GB RAM, Java 1.7.0_67
>            Reporter: Andreas Schnitzerling
>            Assignee: Joshua McKenzie
>            Priority: Minor
>             Fix For: 2.1.3
>
>         Attachments: 8192_v1.txt, cassandra.bat, cassandra.yaml, logdata-onlinedata-ka-196504-CompressionInfo.zip, printChunkOffsetErrors.txt, system-compactions_in_progress-ka-47594-CompressionInfo.zip, system-sstable_activity-jb-25-Filter.zip, system.log, system_AssertionTest.log
>
>
> Since update of 1 of 12 nodes from 2.1.0-rel to 2.1.1-rel Exception during start up.
> {panel:title=system.log}
> ERROR [SSTableBatchOpen:1] 2014-10-27 09:44:00,079 CassandraDaemon.java:153 - Exception in thread Thread[SSTableBatchOpen:1,5,main]
> java.lang.AssertionError: null
> 	at org.apache.cassandra.io.util.Memory.size(Memory.java:307) ~[apache-cassandra-2.1.1.jar:2.1.1]
> 	at org.apache.cassandra.io.compress.CompressionMetadata.<init>(CompressionMetadata.java:135) ~[apache-cassandra-2.1.1.jar:2.1.1]
> 	at org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:83) ~[apache-cassandra-2.1.1.jar:2.1.1]
> 	at org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:50) ~[apache-cassandra-2.1.1.jar:2.1.1]
> 	at org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:48) ~[apache-cassandra-2.1.1.jar:2.1.1]
> 	at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:766) ~[apache-cassandra-2.1.1.jar:2.1.1]
> 	at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:725) ~[apache-cassandra-2.1.1.jar:2.1.1]
> 	at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:402) ~[apache-cassandra-2.1.1.jar:2.1.1]
> 	at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:302) ~[apache-cassandra-2.1.1.jar:2.1.1]
> 	at org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:438) ~[apache-cassandra-2.1.1.jar:2.1.1]
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.7.0_55]
> 	at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_55]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_55]
> 	at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
> {panel}
> In the attached log you can still see as well CASSANDRA-8069 and CASSANDRA-6283.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)