[jira] [Issue Comment Edited] (CASSANDRA-3232) Startup after abrupt shutdown may result in failure to delete a temporary SSTable

["Ophir Radnitz (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108608#comment-13108608 ] 

Ophir Radnitz edited comment on CASSANDRA-3232 at 9/20/11 11:43 AM:
--------------------------------------------------------------------

Very possible, although upon further checking it turns out the file that failed to be deleted does not exist. Could this mean the storage is somehow out of sync with the file system?

Moreover, Cassandra fails to load after such failure. I think there's a point to be made about wrapping the call to ColumnFamilyStore#scrubDataDirectories in AbstractCassandraDaemon#setup, logging errors and continuing to load anyway.

      was (Author: liqweed):
    Very possible, although upon further checking in turns out the file that failed to be deleted does not exist. Could this mean the storage is somehow out of sync with the file system?

Moreover, Cassandra fails to load after such failure. I think there's a point to be made about wrapping the call to ColumnFamilyStore#scrubDataDirectories in AbstractCassandraDaemon#setup, logging errors and continuing to load anyway.
  
> Startup after abrupt shutdown may result in failure to delete a temporary SSTable
> ---------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-3232
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-3232
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.8.5
>         Environment: Windows 7 64bit
>            Reporter: Ophir Radnitz
>            Priority: Minor
>
> After killing the server and starting it up again, once in 3-4 times we're encountering the described problem.
> A relevant stack trace:
> {noformat}
> 11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable)
> 11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[cassandra_server,5,main]
> java.io.IOError: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
>         at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
>         at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
>         at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
>         at org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
>         at org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
>         at com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
>         at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
>         at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
>         at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
>         at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
>         ... 6 more
> 11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.io.IOError: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
>         at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
>         at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
>         at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
>         at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
>         at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
>         at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>         at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
>         at org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
>         at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
>         ... 9 more
> 11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
> 11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\session_hits-g-456
> ...
> {noformat}

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira