[jira] [Commented] (CASSANDRA-11995) Commitlog replaced with all NULs

["Robert Dennington (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-11995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15861177#comment-15861177 ] 

Robert Dennington commented on CASSANDRA-11995:
-----------------------------------------------

I've been seeing this issue quite frequently on my dev machine.

For various reasons, that machine is currently a bit unstable and tends to lock up quite often, meaning I'm forced to hit the reset button on the front, which results in ungraceful termination of all the running programs. Since I use it for other things, in some boots of the system the Cassandra server starts but I never actually connect to it or use it in any way. In those cases, if I perform a forced reset of the machine while Cassandra is in a "running but hasn't been accessed yet" state, I almost always encounter this issue.

I was able to work around the problem by just deleting the commit log that was filled with NULs. I checked the tables and data and nothing seems to be missing in the database, so this definitely seems like a minor glitch. It looks like the Cassandra server is creating an empty commit log file, but the header for it isn't being synced to disk, possibly because there haven't been any changes to data happening which would warrant bothering with one. Then if there is a forced reset while the server is in this state, it isn't able to start up again on reboot because there is a commit log filled with NUL bytes that lacks a header.

> Commitlog replaced with all NULs
> --------------------------------
>
>                 Key: CASSANDRA-11995
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11995
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: Windows 10 Enterprise 1511
> DataStax Cassandra Community Server 2.2.3
>            Reporter: James Howe
>
> I noticed this morning that Cassandra was failing to start, after being shut down on Friday.
> {code}
> ERROR 09:13:37 Exiting due to error while processing commit log during initialization.
> org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException: Could not read commit log descriptor in file C:\Program Files\DataStax Community\data\commitlog\CommitLog-5-1465571056722.log
> 	at org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:622) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:302) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:147) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:189) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:169) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:273) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:513) [apache-cassandra-2.2.3.jar:2.2.3]
> 	at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:622) [apache-cassandra-2.2.3.jar:2.2.3]
> {code}
> Checking the referenced file reveals it comprises 33,554,432 (32 * 1024 * 1024) NUL bytes.
> No logs (stdout, stderr, prunsrv) from the shutdown show any other issues and appear exactly as normal.
> Is installed as a service via DataStax's distribution.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)