You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "James Golick (JIRA)" <ji...@apache.org> on 2009/08/18 18:43:14 UTC
[jira] Created: (CASSANDRA-370) Commit log replay issues
Commit log replay issues
------------------------
Key: CASSANDRA-370
URL: https://issues.apache.org/jira/browse/CASSANDRA-370
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.4
Environment: OS X 10.5.7
Reporter: James Golick
Priority: Critical
I've been having a bunch of trouble replaying commit logs. I've seen various exceptions, including:
java.lang.NegativeArraySizeException
at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
Exception encountered during startup.
java.lang.NegativeArraySizeException
at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
I also got this:
java.lang.RuntimeException: Unable to load comparator class 'org.apache.cassandra.db.marshal.UTF8Typ'. probably this means you have obsolete sstables lying around
at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:525)
at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserializeEmpty(ColumnFamily.java:535)
at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserialize(ColumnFamily.java:500)
at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:225)
at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
Caused by: java.lang.ClassNotFoundException: org/apache/cassandra/db/marshal/UTF8Typ
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:521)
... 7 more
and this:
java.io.EOFException
at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
at java.io.DataInputStream.readUTF(DataInputStream.java:572)
at java.io.DataInputStream.readUTF(DataInputStream.java:547)
at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218)
at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
Exception encountered during startup.
Not sure if any of them are related.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-370) Commit log replay issues
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747045#action_12747045 ]
Sammy Yu commented on CASSANDRA-370:
------------------------------------
+1 on both 0001 and 0002 patches
> Commit log replay issues
> ------------------------
>
> Key: CASSANDRA-370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-370
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Environment: OS X 10.5.7
> Reporter: James Golick
> Assignee: Jonathan Ellis
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 0001-CASSANDRA-370-avoid-opening-multiple-writers-for-the-c.txt, 0002-read-bytes-doesn-t-automatically-throw-EOFException-i.txt
>
>
> I've been having a bunch of trouble replaying commit logs. I've seen various exceptions, including:
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> I also got this:
> java.lang.RuntimeException: Unable to load comparator class 'org.apache.cassandra.db.marshal.UTF8Typ'. probably this means you have obsolete sstables lying around
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:525)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserializeEmpty(ColumnFamily.java:535)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserialize(ColumnFamily.java:500)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:225)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Caused by: java.lang.ClassNotFoundException: org/apache/cassandra/db/marshal/UTF8Typ
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:521)
> ... 7 more
> and this:
> java.io.EOFException
> at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
> at java.io.DataInputStream.readUTF(DataInputStream.java:572)
> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> Not sure if any of them are related.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Reopened: (CASSANDRA-370) Commit log replay issues
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis reopened CASSANDRA-370:
--------------------------------------
patch 01 causes issues. reverted. (02 is still applied.)
> Commit log replay issues
> ------------------------
>
> Key: CASSANDRA-370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-370
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Environment: OS X 10.5.7
> Reporter: James Golick
> Assignee: Jonathan Ellis
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 0001-CASSANDRA-370-avoid-opening-multiple-writers-for-the-c.txt, 0002-read-bytes-doesn-t-automatically-throw-EOFException-i.txt
>
>
> I've been having a bunch of trouble replaying commit logs. I've seen various exceptions, including:
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> I also got this:
> java.lang.RuntimeException: Unable to load comparator class 'org.apache.cassandra.db.marshal.UTF8Typ'. probably this means you have obsolete sstables lying around
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:525)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserializeEmpty(ColumnFamily.java:535)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserialize(ColumnFamily.java:500)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:225)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Caused by: java.lang.ClassNotFoundException: org/apache/cassandra/db/marshal/UTF8Typ
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:521)
> ... 7 more
> and this:
> java.io.EOFException
> at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
> at java.io.DataInputStream.readUTF(DataInputStream.java:572)
> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> Not sure if any of them are related.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-370) Commit log replay issues
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-370.
--------------------------------------
Resolution: Fixed
committed
> Commit log replay issues
> ------------------------
>
> Key: CASSANDRA-370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-370
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Environment: OS X 10.5.7
> Reporter: James Golick
> Assignee: Jonathan Ellis
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 0001-CASSANDRA-370-avoid-opening-multiple-writers-for-the-c.txt, 0002-read-bytes-doesn-t-automatically-throw-EOFException-i.txt, 01-fixed.diff
>
>
> I've been having a bunch of trouble replaying commit logs. I've seen various exceptions, including:
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> I also got this:
> java.lang.RuntimeException: Unable to load comparator class 'org.apache.cassandra.db.marshal.UTF8Typ'. probably this means you have obsolete sstables lying around
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:525)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserializeEmpty(ColumnFamily.java:535)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserialize(ColumnFamily.java:500)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:225)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Caused by: java.lang.ClassNotFoundException: org/apache/cassandra/db/marshal/UTF8Typ
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:521)
> ... 7 more
> and this:
> java.io.EOFException
> at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
> at java.io.DataInputStream.readUTF(DataInputStream.java:572)
> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> Not sure if any of them are related.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-370) Commit log replay issues
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-370:
-------------------------------------
Attachment: 01-fixed.diff
fixed -- wasn't seeking back to current-end-of-CL in the header update code, so it would start clobbering the hell out of itself as soon as that code path ran.
> Commit log replay issues
> ------------------------
>
> Key: CASSANDRA-370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-370
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Environment: OS X 10.5.7
> Reporter: James Golick
> Assignee: Jonathan Ellis
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 0001-CASSANDRA-370-avoid-opening-multiple-writers-for-the-c.txt, 0002-read-bytes-doesn-t-automatically-throw-EOFException-i.txt, 01-fixed.diff
>
>
> I've been having a bunch of trouble replaying commit logs. I've seen various exceptions, including:
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> I also got this:
> java.lang.RuntimeException: Unable to load comparator class 'org.apache.cassandra.db.marshal.UTF8Typ'. probably this means you have obsolete sstables lying around
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:525)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserializeEmpty(ColumnFamily.java:535)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserialize(ColumnFamily.java:500)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:225)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Caused by: java.lang.ClassNotFoundException: org/apache/cassandra/db/marshal/UTF8Typ
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:521)
> ... 7 more
> and this:
> java.io.EOFException
> at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
> at java.io.DataInputStream.readUTF(DataInputStream.java:572)
> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> Not sure if any of them are related.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-370) Commit log replay issues
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747434#action_12747434 ]
Hudson commented on CASSANDRA-370:
----------------------------------
Integrated in Cassandra #177 (See [http://hudson.zones.apache.org/hudson/job/Cassandra/177/])
[fixed version] avoid opening multiple writers for the current file; the buffered nature of the global logWriter_ could cause problems
patch by jbellis; reviewed for by Sammy Yu
Revert " avoid opening multiple writers for the current file; the buffered nature of the global logWriter_ could cause problems"
read(bytes) doesn't automatically throw EOFException if it reads less than asked for, so we need to check for that.
patch by jbellis; reviewed by Sammy Yu for
avoid opening multiple writers for the current file; the buffered nature of the global logWriter_ could cause problems
patch by jbellis; reviewed by Sammy Yu for
> Commit log replay issues
> ------------------------
>
> Key: CASSANDRA-370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-370
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Environment: OS X 10.5.7
> Reporter: James Golick
> Assignee: Jonathan Ellis
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 0001-CASSANDRA-370-avoid-opening-multiple-writers-for-the-c.txt, 0002-read-bytes-doesn-t-automatically-throw-EOFException-i.txt, 01-fixed.diff
>
>
> I've been having a bunch of trouble replaying commit logs. I've seen various exceptions, including:
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> I also got this:
> java.lang.RuntimeException: Unable to load comparator class 'org.apache.cassandra.db.marshal.UTF8Typ'. probably this means you have obsolete sstables lying around
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:525)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserializeEmpty(ColumnFamily.java:535)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserialize(ColumnFamily.java:500)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:225)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Caused by: java.lang.ClassNotFoundException: org/apache/cassandra/db/marshal/UTF8Typ
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:521)
> ... 7 more
> and this:
> java.io.EOFException
> at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
> at java.io.DataInputStream.readUTF(DataInputStream.java:572)
> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> Not sure if any of them are related.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-370) Commit log replay issues
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-370:
-------------------------------------
Attachment: 0002-read-bytes-doesn-t-automatically-throw-EOFException-i.txt
0001-CASSANDRA-370-avoid-opening-multiple-writers-for-the-c.txt
> Commit log replay issues
> ------------------------
>
> Key: CASSANDRA-370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-370
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Environment: OS X 10.5.7
> Reporter: James Golick
> Priority: Critical
> Attachments: 0001-CASSANDRA-370-avoid-opening-multiple-writers-for-the-c.txt, 0002-read-bytes-doesn-t-automatically-throw-EOFException-i.txt
>
>
> I've been having a bunch of trouble replaying commit logs. I've seen various exceptions, including:
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> I also got this:
> java.lang.RuntimeException: Unable to load comparator class 'org.apache.cassandra.db.marshal.UTF8Typ'. probably this means you have obsolete sstables lying around
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:525)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserializeEmpty(ColumnFamily.java:535)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserialize(ColumnFamily.java:500)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:225)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Caused by: java.lang.ClassNotFoundException: org/apache/cassandra/db/marshal/UTF8Typ
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:521)
> ... 7 more
> and this:
> java.io.EOFException
> at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
> at java.io.DataInputStream.readUTF(DataInputStream.java:572)
> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> Not sure if any of them are related.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-370) Commit log replay issues
Posted by "Sammy Yu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747099#action_12747099 ]
Sammy Yu commented on CASSANDRA-370:
------------------------------------
01-fixed.diff +1 Tested it for real this time. Unit test passes and our production system no longer exhibits this bad behavior.
> Commit log replay issues
> ------------------------
>
> Key: CASSANDRA-370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-370
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Environment: OS X 10.5.7
> Reporter: James Golick
> Assignee: Jonathan Ellis
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 0001-CASSANDRA-370-avoid-opening-multiple-writers-for-the-c.txt, 0002-read-bytes-doesn-t-automatically-throw-EOFException-i.txt, 01-fixed.diff
>
>
> I've been having a bunch of trouble replaying commit logs. I've seen various exceptions, including:
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> I also got this:
> java.lang.RuntimeException: Unable to load comparator class 'org.apache.cassandra.db.marshal.UTF8Typ'. probably this means you have obsolete sstables lying around
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:525)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserializeEmpty(ColumnFamily.java:535)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserialize(ColumnFamily.java:500)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:225)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Caused by: java.lang.ClassNotFoundException: org/apache/cassandra/db/marshal/UTF8Typ
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:521)
> ... 7 more
> and this:
> java.io.EOFException
> at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
> at java.io.DataInputStream.readUTF(DataInputStream.java:572)
> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> Not sure if any of them are related.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (CASSANDRA-370) Commit log replay issues
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-370:
-------------------------------------
Fix Version/s: 0.4
Assignee: Jonathan Ellis
> Commit log replay issues
> ------------------------
>
> Key: CASSANDRA-370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-370
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Environment: OS X 10.5.7
> Reporter: James Golick
> Assignee: Jonathan Ellis
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 0001-CASSANDRA-370-avoid-opening-multiple-writers-for-the-c.txt, 0002-read-bytes-doesn-t-automatically-throw-EOFException-i.txt
>
>
> I've been having a bunch of trouble replaying commit logs. I've seen various exceptions, including:
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> I also got this:
> java.lang.RuntimeException: Unable to load comparator class 'org.apache.cassandra.db.marshal.UTF8Typ'. probably this means you have obsolete sstables lying around
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:525)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserializeEmpty(ColumnFamily.java:535)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserialize(ColumnFamily.java:500)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:225)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Caused by: java.lang.ClassNotFoundException: org/apache/cassandra/db/marshal/UTF8Typ
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:521)
> ... 7 more
> and this:
> java.io.EOFException
> at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
> at java.io.DataInputStream.readUTF(DataInputStream.java:572)
> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> Not sure if any of them are related.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Commented: (CASSANDRA-370) Commit log replay issues
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12744615#action_12744615 ]
Jonathan Ellis commented on CASSANDRA-370:
------------------------------------------
02
read(bytes) doesn't automatically throw EOFException if it reads less than asked for, so we need to check for that maniually
01
avoid opening multiple writers for the current file; the buffered nature of the global
logWriter_ could cause problems
james, in all likelihood the problem fixed by 02 is what you were running into, and that patch should fix replaying your old commitlog as-is (magic! :)
> Commit log replay issues
> ------------------------
>
> Key: CASSANDRA-370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-370
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Environment: OS X 10.5.7
> Reporter: James Golick
> Priority: Critical
> Attachments: 0001-CASSANDRA-370-avoid-opening-multiple-writers-for-the-c.txt, 0002-read-bytes-doesn-t-automatically-throw-EOFException-i.txt
>
>
> I've been having a bunch of trouble replaying commit logs. I've seen various exceptions, including:
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> I also got this:
> java.lang.RuntimeException: Unable to load comparator class 'org.apache.cassandra.db.marshal.UTF8Typ'. probably this means you have obsolete sstables lying around
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:525)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserializeEmpty(ColumnFamily.java:535)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserialize(ColumnFamily.java:500)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:225)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Caused by: java.lang.ClassNotFoundException: org/apache/cassandra/db/marshal/UTF8Typ
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:521)
> ... 7 more
> and this:
> java.io.EOFException
> at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
> at java.io.DataInputStream.readUTF(DataInputStream.java:572)
> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> Not sure if any of them are related.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Resolved: (CASSANDRA-370) Commit log replay issues
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis resolved CASSANDRA-370.
--------------------------------------
Resolution: Fixed
committed
> Commit log replay issues
> ------------------------
>
> Key: CASSANDRA-370
> URL: https://issues.apache.org/jira/browse/CASSANDRA-370
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.4
> Environment: OS X 10.5.7
> Reporter: James Golick
> Assignee: Jonathan Ellis
> Priority: Critical
> Fix For: 0.4
>
> Attachments: 0001-CASSANDRA-370-avoid-opening-multiple-writers-for-the-c.txt, 0002-read-bytes-doesn-t-automatically-throw-EOFException-i.txt
>
>
> I've been having a bunch of trouble replaying commit logs. I've seen various exceptions, including:
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> java.lang.NegativeArraySizeException
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:273)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> I also got this:
> java.lang.RuntimeException: Unable to load comparator class 'org.apache.cassandra.db.marshal.UTF8Typ'. probably this means you have obsolete sstables lying around
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:525)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserializeEmpty(ColumnFamily.java:535)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.deserialize(ColumnFamily.java:500)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:225)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Caused by: java.lang.ClassNotFoundException: org/apache/cassandra/db/marshal/UTF8Typ
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:169)
> at org.apache.cassandra.db.ColumnFamily$ColumnFamilySerializer.readComparator(ColumnFamily.java:521)
> ... 7 more
> and this:
> java.io.EOFException
> at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
> at java.io.DataInputStream.readUTF(DataInputStream.java:572)
> at java.io.DataInputStream.readUTF(DataInputStream.java:547)
> at org.apache.cassandra.db.RowSerializer.deserialize(Row.java:218)
> at org.apache.cassandra.db.CommitLog.recover(CommitLog.java:284)
> at org.apache.cassandra.db.RecoveryManager.doRecovery(RecoveryManager.java:63)
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:95)
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:156)
> Exception encountered during startup.
> Not sure if any of them are related.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.