[jira] [Commented] (CASSANDRA-11995) Commitlog replaced with all NULs

["John Marsten (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-11995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15644485#comment-15644485 ] 

John Marsten commented on CASSANDRA-11995:
------------------------------------------

We observed this phenomenon running Apache Cassandra 3.8 on Windows Server 2012 VM. The precise history and state of the VM in question was unclear, but apparently the administrators took a live snapshot, and after reverting to the snapshot, Cassandra would no longer start due to the error above. In this case, the commit log file was 48MB of NUL bytes. The Cassandra system log showed an ungraceful termination and a handful of warnings related to slow commit log syncs, but no errors prior to the CommitLogReplayException on subsequent startups.

Cassandra was configured for batch mode commit log syncing with a window size of 50ms. It's not clear whether the system was writing to Cassandra at the time of the snapshot.

> Commitlog replaced with all NULs
> --------------------------------
>
>                 Key: CASSANDRA-11995
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11995
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: Windows 10 Enterprise 1511
> DataStax Cassandra Community Server 2.2.3
>            Reporter: James Howe
>
> I noticed this morning that Cassandra was failing to start, after being shut down on Friday.
> {code}
> ERROR 09:13:37 Exiting due to error while processing commit log during initialization.
> org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException: Could not read commit log descriptor in file C:\Program Files\DataStax Community\data\commitlog\CommitLog-5-1465571056722.log
> 	at org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:622) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:302) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:147) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:189) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:169) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:273) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:513) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:622) [apache-cassandra-2.2.3.jar:2.2.3]
> {code}
> Checking the referenced file reveals it comprises 33,554,432 NUL bytes.
> No logs (stdout, stderr, prunsrv) from the shutdown show any other issues and appear exactly as normal.
> Is installed as a service via DataStax's distribution.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)