You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Ophir Radnitz (JIRA)" <ji...@apache.org> on 2011/09/20 11:41:08 UTC
[jira] [Created] (CASSANDRA-3232) Startup after abrupt shutdown may
result in failure to delete a temporary SSTable
Startup after abrupt shutdown may result in failure to delete a temporary SSTable
---------------------------------------------------------------------------------
Key: CASSANDRA-3232
URL: https://issues.apache.org/jira/browse/CASSANDRA-3232
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.8.5
Environment: Windows 7 64bit
Reporter: Ophir Radnitz
Priority: Minor
After killing the server and starting it up again, once in 3-4 times we're encountering the described problem.
A relevant stack trace:
{{
11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable)
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[cassandra_server,5,main]
java.io.IOError: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
at org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
at org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
at com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
... 6 more
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[FlushWriter:1,5,main]
java.io.IOError: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
at org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
... 9 more
11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\session_hits-g-456
...
}}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3232) Startup after abrupt shutdown
may result in failure to delete a temporary SSTable
Posted by "Ophir Radnitz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108608#comment-13108608 ]
Ophir Radnitz commented on CASSANDRA-3232:
------------------------------------------
Very possible, although upon further checking in turns out the file that failed to be deleted does not exist. Could this mean the storage is somehow out of sync with the file system?
Moreover, Cassandra fails to load after such failure. I think there's a point to be made about wrapping the call to ColumnFamilyStore#scrubDataDirectories in AbstractCassandraDaemon#setup, logging errors and continuing to load anyway.
> Startup after abrupt shutdown may result in failure to delete a temporary SSTable
> ---------------------------------------------------------------------------------
>
> Key: CASSANDRA-3232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3232
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.5
> Environment: Windows 7 64bit
> Reporter: Ophir Radnitz
> Priority: Minor
>
> After killing the server and starting it up again, once in 3-4 times we're encountering the described problem.
> A relevant stack trace:
> {noformat}
> 11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable)
> 11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[cassandra_server,5,main]
> java.io.IOError: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
> at org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
> at org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
> at com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
> ... 6 more
> 11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.io.IOError: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
> at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
> at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
> at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
> at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
> at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
> at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
> at org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
> at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
> ... 9 more
> 11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
> 11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\session_hits-g-456
> ...
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3232) Startup after abrupt shutdown
may result in failure to delete a temporary SSTable
Posted by "Ophir Radnitz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108615#comment-13108615 ]
Ophir Radnitz commented on CASSANDRA-3232:
------------------------------------------
I neglected to mention a probably important detail - I'm running an embedded Cassandra instance using EmbeddedCassandraService and do my writes via the StorageProxy API to avoid thrift serialization.
> Startup after abrupt shutdown may result in failure to delete a temporary SSTable
> ---------------------------------------------------------------------------------
>
> Key: CASSANDRA-3232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3232
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.5
> Environment: Windows 7 64bit
> Reporter: Ophir Radnitz
> Priority: Minor
>
> After killing the server and starting it up again, once in 3-4 times we're encountering the described problem.
> A relevant stack trace:
> {noformat}
> 11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable)
> 11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[cassandra_server,5,main]
> java.io.IOError: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
> at org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
> at org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
> at com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
> ... 6 more
> 11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.io.IOError: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
> at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
> at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
> at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
> at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
> at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
> at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
> at org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
> at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
> ... 9 more
> 11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
> 11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\session_hits-g-456
> ...
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Issue Comment Edited] (CASSANDRA-3232) Startup after abrupt
shutdown may result in failure to delete a temporary SSTable
Posted by "Ophir Radnitz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108608#comment-13108608 ]
Ophir Radnitz edited comment on CASSANDRA-3232 at 9/20/11 11:43 AM:
--------------------------------------------------------------------
Very possible, although upon further checking it turns out the file that failed to be deleted does not exist. Could this mean the storage is somehow out of sync with the file system?
Moreover, Cassandra fails to load after such failure. I think there's a point to be made about wrapping the call to ColumnFamilyStore#scrubDataDirectories in AbstractCassandraDaemon#setup, logging errors and continuing to load anyway.
was (Author: liqweed):
Very possible, although upon further checking in turns out the file that failed to be deleted does not exist. Could this mean the storage is somehow out of sync with the file system?
Moreover, Cassandra fails to load after such failure. I think there's a point to be made about wrapping the call to ColumnFamilyStore#scrubDataDirectories in AbstractCassandraDaemon#setup, logging errors and continuing to load anyway.
> Startup after abrupt shutdown may result in failure to delete a temporary SSTable
> ---------------------------------------------------------------------------------
>
> Key: CASSANDRA-3232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3232
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.5
> Environment: Windows 7 64bit
> Reporter: Ophir Radnitz
> Priority: Minor
>
> After killing the server and starting it up again, once in 3-4 times we're encountering the described problem.
> A relevant stack trace:
> {noformat}
> 11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable)
> 11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[cassandra_server,5,main]
> java.io.IOError: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
> at org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
> at org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
> at com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
> ... 6 more
> 11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.io.IOError: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
> at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
> at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
> at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
> at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
> at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
> at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
> at org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
> at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
> ... 9 more
> 11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
> 11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\session_hits-g-456
> ...
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-3232) Startup after abrupt shutdown
may result in failure to delete a temporary SSTable
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108596#comment-13108596 ]
Jonathan Ellis commented on CASSANDRA-3232:
-------------------------------------------
Sounds like Windows hasn't realized the old owner is dead yet and thus refuses the delete?
> Startup after abrupt shutdown may result in failure to delete a temporary SSTable
> ---------------------------------------------------------------------------------
>
> Key: CASSANDRA-3232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3232
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.5
> Environment: Windows 7 64bit
> Reporter: Ophir Radnitz
> Priority: Minor
>
> After killing the server and starting it up again, once in 3-4 times we're encountering the described problem.
> A relevant stack trace:
> {noformat}
> 11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable)
> 11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[cassandra_server,5,main]
> java.io.IOError: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
> at org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
> at org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
> at com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
> ... 6 more
> 11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.io.IOError: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
> at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
> at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
> at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
> at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
> at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
> at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
> at org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
> at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
> ... 9 more
> 11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
> 11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\session_hits-g-456
> ...
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CASSANDRA-3232) Startup after abrupt shutdown may
result in failure to delete a temporary SSTable
Posted by "Ophir Radnitz (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-3232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ophir Radnitz updated CASSANDRA-3232:
-------------------------------------
Description:
After killing the server and starting it up again, once in 3-4 times we're encountering the described problem.
A relevant stack trace:
{noformat}
11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable)
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[cassandra_server,5,main]
java.io.IOError: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
at org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
at org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
at com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
... 6 more
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[FlushWriter:1,5,main]
java.io.IOError: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
at org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
... 9 more
11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\session_hits-g-456
...
{noformat}
was:
After killing the server and starting it up again, once in 3-4 times we're encountering the described problem.
A relevant stack trace:
{{
11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable)
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[cassandra_server,5,main]
java.io.IOError: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
at org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
at org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
at com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
... 6 more
11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[FlushWriter:1,5,main]
java.io.IOError: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)
Caused by: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
at org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
... 9 more
11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\session_hits-g-456
...
}}
> Startup after abrupt shutdown may result in failure to delete a temporary SSTable
> ---------------------------------------------------------------------------------
>
> Key: CASSANDRA-3232
> URL: https://issues.apache.org/jira/browse/CASSANDRA-3232
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.8.5
> Environment: Windows 7 64bit
> Reporter: Ophir Radnitz
> Priority: Minor
>
> After killing the server and starting it up again, once in 3-4 times we're encountering the described problem.
> A relevant stack trace:
> {noformat}
> 11:10:44 ColumnFamilyStore [INFO] Removing compacted SSTable files (see http://wiki.apache.org/cassandra/MemtableSSTable)
> 11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[cassandra_server,5,main]
> java.io.IOError: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:158)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:484)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:173)
> at org.apache.cassandra.service.AbstractCassandraDaemon.init(AbstractCassandraDaemon.java:237)
> at org.apache.cassandra.service.EmbeddedCassandraService.start(EmbeddedCassandraService.java:57)
> at com.clarisite.clingine.dataaccesslayer.cassandra.EmbeddedCassandra$1.run(EmbeddedCassandra.java:42)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: Failed to delete c:\cassandra\data\system\LocationInfo-tmp-g-24-Statistics.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:52)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:42)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:151)
> ... 6 more
> 11:10:44 AbstractCassandraDaemon [ERROR] Fatal exception in thread Thread[FlushWriter:1,5,main]
> java.io.IOError: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
> at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:225)
> at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:190)
> at org.apache.cassandra.io.sstable.SSTableWriter.closeAndOpenReader(SSTableWriter.java:173)
> at org.apache.cassandra.db.Memtable.writeSortedContents(Memtable.java:253)
> at org.apache.cassandra.db.Memtable.access$400(Memtable.java:49)
> at org.apache.cassandra.db.Memtable$3.runMayThrow(Memtable.java:270)
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.io.IOException: rename failed of c:\cassandra\data\system\LocationInfo-g-24-Index.db
> at org.apache.cassandra.utils.FBUtilities.renameWithConfirm(FBUtilities.java:383)
> at org.apache.cassandra.io.sstable.SSTableWriter.rename(SSTableWriter.java:220)
> ... 9 more
> 11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\resources-g-1
> 11:10:44 SSTableReader [INFO] Opening c:\cassandra\data\20110914\session_hits-g-456
> ...
> {noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira