You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Thibaut (JIRA)" <ji...@apache.org> on 2011/02/21 12:57:38 UTC
[jira] Created: (CASSANDRA-2206) Startup fails due to cassandra
trying to delete inexisting file
Startup fails due to cassandra trying to delete inexisting file
---------------------------------------------------------------
Key: CASSANDRA-2206
URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
Project: Cassandra
Issue Type: Bug
Components: Core
Affects Versions: 0.7.2
Environment: linux
Reporter: Thibaut
Fix For: 0.7.3
Hi,
On one of our nodes, startup fails due to cassandra trying to delete a nonexistant "Data" file (see below).
Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured.
Directory listing:
total 109M
-rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
-rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
-rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
-rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
-rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
-rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
-rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
-rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
-rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
-rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
-rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
-rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
-rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
-rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
-rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
-rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
-rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
-rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
-rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
-rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
-rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
-rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
-rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
-rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
-rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
-rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
-rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
-rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
Cassandra log:
/software/cassandra/bin/cassandra
rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
INFO 12:47:29,030 Heap size: 2614493184/2614493184
INFO 12:47:29,031 JNA not found. Native methods will be disabled.
INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml
INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
WARN 12:47:29,623 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored.
ERROR 12:47:29,638 Exception encountered during startup.
java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
... 4 more
Exception encountered during startup.
java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2206) Startup fails due to cassandra
trying to delete nonexisting file
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2206:
--------------------------------------
Attachment: 2206.txt
> Startup fails due to cassandra trying to delete nonexisting file
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.2
> Environment: linux
> Reporter: Thibaut
> Assignee: Jonathan Ellis
> Fix For: 0.7.3
>
> Attachments: 2206.txt
>
>
> Hi,
> On one of our nodes, startup fails due to cassandra trying to delete a nonexistant "Data" file (see below).
> Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured.
> Directory listing:
> total 109M
> -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
> -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
> -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
> -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
> -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
> -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
> -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
> -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
> -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
> -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
> -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
> -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
> -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
> -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
> -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
> -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
> -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
> -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
> -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
> -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
> -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
> -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
> -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
> -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
> -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
> -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
> -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
> Cassandra log:
> /software/cassandra/bin/cassandra
> rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
> root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
> INFO 12:47:29,030 Heap size: 2614493184/2614493184
> INFO 12:47:29,031 JNA not found. Native methods will be disabled.
> INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml
> INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
> INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
> INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
> INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
> INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
> INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
> INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
> INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
> WARN 12:47:29,623 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored.
> ERROR 12:47:29,638 Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
> Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Issue Comment Edited: (CASSANDRA-2206) Startup fails due to
cassandra trying to delete nonexisting file
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997424#comment-12997424 ]
Jonathan Ellis edited comment on CASSANDRA-2206 at 2/21/11 2:50 PM:
--------------------------------------------------------------------
You can manually fix this by removing the "tmp" from all the tmp-f-1707 files.
was (Author: jbellis):
You can manually fix this by removing the "tmp" from *tmp-f-1707*.
> Startup fails due to cassandra trying to delete nonexisting file
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.2
> Environment: linux
> Reporter: Thibaut
> Fix For: 0.7.3
>
> Attachments: 2206.txt
>
>
> Hi,
> On one of our nodes, startup fails due to cassandra trying to delete a nonexistant "Data" file (see below).
> Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured.
> Directory listing:
> total 109M
> -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
> -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
> -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
> -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
> -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
> -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
> -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
> -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
> -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
> -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
> -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
> -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
> -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
> -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
> -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
> -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
> -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
> -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
> -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
> -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
> -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
> -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
> -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
> -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
> -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
> -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
> -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
> Cassandra log:
> /software/cassandra/bin/cassandra
> rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
> root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
> INFO 12:47:29,030 Heap size: 2614493184/2614493184
> INFO 12:47:29,031 JNA not found. Native methods will be disabled.
> INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml
> INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
> INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
> INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
> INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
> INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
> INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
> INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
> INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
> WARN 12:47:29,623 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored.
> ERROR 12:47:29,638 Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
> Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2206) Startup fails due to cassandra
trying to delete nonexisting file
Posted by "Gary Dusbabek (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12998885#comment-12998885 ]
Gary Dusbabek commented on CASSANDRA-2206:
------------------------------------------
+1
> Startup fails due to cassandra trying to delete nonexisting file
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.2
> Environment: linux
> Reporter: Thibaut
> Assignee: Jonathan Ellis
> Fix For: 0.7.3
>
> Attachments: 2206-v2.txt, 2206.txt
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Hi,
> On one of our nodes, startup fails due to cassandra trying to delete a nonexistant "Data" file (see below).
> Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured.
> Directory listing:
> total 109M
> -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
> -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
> -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
> -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
> -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
> -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
> -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
> -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
> -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
> -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
> -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
> -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
> -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
> -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
> -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
> -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
> -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
> -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
> -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
> -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
> -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
> -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
> -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
> -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
> -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
> -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
> -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
> Cassandra log:
> /software/cassandra/bin/cassandra
> rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
> root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
> INFO 12:47:29,030 Heap size: 2614493184/2614493184
> INFO 12:47:29,031 JNA not found. Native methods will be disabled.
> INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml
> INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
> INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
> INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
> INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
> INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
> INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
> INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
> INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
> WARN 12:47:29,623 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored.
> ERROR 12:47:29,638 Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
> Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2206) Startup fails due to cassandra
trying to delete nonexisting file
Posted by "Thibaut (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Thibaut updated CASSANDRA-2206:
-------------------------------
Summary: Startup fails due to cassandra trying to delete nonexisting file (was: Startup fails due to cassandra trying to delete inexisting file)
> Startup fails due to cassandra trying to delete nonexisting file
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.2
> Environment: linux
> Reporter: Thibaut
> Fix For: 0.7.3
>
>
> Hi,
> On one of our nodes, startup fails due to cassandra trying to delete a nonexistant "Data" file (see below).
> Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured.
> Directory listing:
> total 109M
> -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
> -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
> -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
> -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
> -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
> -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
> -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
> -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
> -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
> -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
> -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
> -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
> -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
> -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
> -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
> -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
> -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
> -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
> -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
> -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
> -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
> -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
> -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
> -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
> -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
> -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
> -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
> Cassandra log:
> /software/cassandra/bin/cassandra
> rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
> root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
> INFO 12:47:29,030 Heap size: 2614493184/2614493184
> INFO 12:47:29,031 JNA not found. Native methods will be disabled.
> INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml
> INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
> INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
> INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
> INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
> INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
> INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
> INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
> INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
> WARN 12:47:29,623 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored.
> ERROR 12:47:29,638 Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
> Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2206) Startup fails due to cassandra
trying to delete nonexisting file
Posted by "Matthew F. Dennis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12998218#comment-12998218 ]
Matthew F. Dennis commented on CASSANDRA-2206:
----------------------------------------------
renaming -data files last doesn't actually fix the problem.
You can test this by running
{noformat}
cd /var/lib/cassandra/data/system
touch Schema-tmp-f-763-Data.db Schema-f-763-Filter.db
{noformat}
before starting cassandra.
This would simulate a crash between renaming all the the components besides -data and renaming -data (though really a crash any time between when the first component is renamed and before -data is renamed will exhibit the issue).
> Startup fails due to cassandra trying to delete nonexisting file
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.2
> Environment: linux
> Reporter: Thibaut
> Assignee: Jonathan Ellis
> Fix For: 0.7.3
>
> Attachments: 2206.txt
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Hi,
> On one of our nodes, startup fails due to cassandra trying to delete a nonexistant "Data" file (see below).
> Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured.
> Directory listing:
> total 109M
> -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
> -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
> -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
> -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
> -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
> -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
> -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
> -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
> -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
> -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
> -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
> -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
> -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
> -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
> -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
> -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
> -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
> -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
> -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
> -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
> -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
> -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
> -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
> -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
> -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
> -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
> -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
> Cassandra log:
> /software/cassandra/bin/cassandra
> rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
> root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
> INFO 12:47:29,030 Heap size: 2614493184/2614493184
> INFO 12:47:29,031 JNA not found. Native methods will be disabled.
> INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml
> INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
> INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
> INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
> INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
> INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
> INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
> INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
> INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
> WARN 12:47:29,623 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored.
> ERROR 12:47:29,638 Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
> Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2206) Startup fails due to cassandra
trying to delete nonexisting file
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2206:
--------------------------------------
Attachment: (was: 2206.txt)
> Startup fails due to cassandra trying to delete nonexisting file
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.2
> Environment: linux
> Reporter: Thibaut
> Assignee: Jonathan Ellis
> Fix For: 0.7.3
>
> Attachments: 2206.txt
>
>
> Hi,
> On one of our nodes, startup fails due to cassandra trying to delete a nonexistant "Data" file (see below).
> Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured.
> Directory listing:
> total 109M
> -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
> -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
> -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
> -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
> -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
> -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
> -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
> -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
> -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
> -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
> -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
> -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
> -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
> -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
> -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
> -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
> -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
> -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
> -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
> -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
> -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
> -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
> -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
> -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
> -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
> -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
> -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
> Cassandra log:
> /software/cassandra/bin/cassandra
> rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
> root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
> INFO 12:47:29,030 Heap size: 2614493184/2614493184
> INFO 12:47:29,031 JNA not found. Native methods will be disabled.
> INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml
> INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
> INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
> INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
> INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
> INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
> INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
> INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
> INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
> WARN 12:47:29,623 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored.
> ERROR 12:47:29,638 Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
> Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2206) Startup fails due to cassandra
trying to delete nonexisting file
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12997424#comment-12997424 ]
Jonathan Ellis commented on CASSANDRA-2206:
-------------------------------------------
You can manually fix this by removing the "tmp" from *tmp-f-1707*.
> Startup fails due to cassandra trying to delete nonexisting file
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.2
> Environment: linux
> Reporter: Thibaut
> Fix For: 0.7.3
>
>
> Hi,
> On one of our nodes, startup fails due to cassandra trying to delete a nonexistant "Data" file (see below).
> Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured.
> Directory listing:
> total 109M
> -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
> -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
> -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
> -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
> -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
> -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
> -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
> -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
> -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
> -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
> -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
> -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
> -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
> -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
> -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
> -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
> -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
> -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
> -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
> -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
> -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
> -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
> -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
> -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
> -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
> -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
> -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
> Cassandra log:
> /software/cassandra/bin/cassandra
> rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
> root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
> INFO 12:47:29,030 Heap size: 2614493184/2614493184
> INFO 12:47:29,031 JNA not found. Native methods will be disabled.
> INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml
> INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
> INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
> INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
> INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
> INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
> INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
> INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
> INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
> WARN 12:47:29,623 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored.
> ERROR 12:47:29,638 Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
> Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2206) Startup fails due to cassandra
trying to delete nonexisting file
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2206:
--------------------------------------
Remaining Estimate: 1h
Original Estimate: 1h
> Startup fails due to cassandra trying to delete nonexisting file
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.2
> Environment: linux
> Reporter: Thibaut
> Assignee: Jonathan Ellis
> Fix For: 0.7.3
>
> Attachments: 2206.txt
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Hi,
> On one of our nodes, startup fails due to cassandra trying to delete a nonexistant "Data" file (see below).
> Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured.
> Directory listing:
> total 109M
> -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
> -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
> -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
> -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
> -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
> -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
> -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
> -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
> -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
> -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
> -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
> -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
> -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
> -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
> -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
> -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
> -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
> -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
> -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
> -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
> -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
> -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
> -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
> -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
> -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
> -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
> -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
> Cassandra log:
> /software/cassandra/bin/cassandra
> rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
> root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
> INFO 12:47:29,030 Heap size: 2614493184/2614493184
> INFO 12:47:29,031 JNA not found. Native methods will be disabled.
> INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml
> INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
> INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
> INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
> INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
> INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
> INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
> INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
> INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
> WARN 12:47:29,623 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored.
> ERROR 12:47:29,638 Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
> Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2206) Startup fails due to cassandra
trying to delete nonexisting file
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2206:
--------------------------------------
Attachment: 2206-v2.txt
v2 also fixes a bug where tmp and non-tmp Descriptors were considered equal, which is what confused the file scanning for scrub in your example
> Startup fails due to cassandra trying to delete nonexisting file
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.2
> Environment: linux
> Reporter: Thibaut
> Assignee: Jonathan Ellis
> Fix For: 0.7.3
>
> Attachments: 2206-v2.txt, 2206.txt
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Hi,
> On one of our nodes, startup fails due to cassandra trying to delete a nonexistant "Data" file (see below).
> Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured.
> Directory listing:
> total 109M
> -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
> -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
> -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
> -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
> -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
> -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
> -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
> -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
> -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
> -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
> -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
> -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
> -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
> -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
> -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
> -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
> -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
> -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
> -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
> -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
> -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
> -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
> -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
> -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
> -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
> -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
> -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
> Cassandra log:
> /software/cassandra/bin/cassandra
> rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
> root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
> INFO 12:47:29,030 Heap size: 2614493184/2614493184
> INFO 12:47:29,031 JNA not found. Native methods will be disabled.
> INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml
> INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
> INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
> INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
> INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
> INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
> INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
> INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
> INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
> WARN 12:47:29,623 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored.
> ERROR 12:47:29,638 Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
> Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (CASSANDRA-2206) Startup fails due to cassandra
trying to delete nonexisting file
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12998901#comment-12998901 ]
Hudson commented on CASSANDRA-2206:
-----------------------------------
Integrated in Cassandra-0.7 #316 (See [https://hudson.apache.org/hudson/job/Cassandra-0.7/316/])
improve detection and cleanup ofpartially-written sstables
patch by jbellis; reviewed by gdusbabek for CASSANDRA-2206
> Startup fails due to cassandra trying to delete nonexisting file
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.2
> Environment: linux
> Reporter: Thibaut
> Assignee: Jonathan Ellis
> Fix For: 0.7.3
>
> Attachments: 2206-v2.txt, 2206.txt
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> Hi,
> On one of our nodes, startup fails due to cassandra trying to delete a nonexistant "Data" file (see below).
> Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured.
> Directory listing:
> total 109M
> -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
> -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
> -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
> -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
> -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
> -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
> -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
> -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
> -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
> -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
> -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
> -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
> -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
> -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
> -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
> -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
> -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
> -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
> -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
> -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
> -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
> -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
> -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
> -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
> -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
> -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
> -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
> Cassandra log:
> /software/cassandra/bin/cassandra
> rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
> root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
> INFO 12:47:29,030 Heap size: 2614493184/2614493184
> INFO 12:47:29,031 JNA not found. Native methods will be disabled.
> INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml
> INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
> INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
> INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
> INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
> INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
> INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
> INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
> INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
> WARN 12:47:29,623 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored.
> ERROR 12:47:29,638 Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
> Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (CASSANDRA-2206) Startup fails due to cassandra
trying to delete nonexisting file
Posted by "Jonathan Ellis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Ellis updated CASSANDRA-2206:
--------------------------------------
Attachment: 2206.txt
Patch to fix this for new files by making sure we rename -Data last.
> Startup fails due to cassandra trying to delete nonexisting file
> ----------------------------------------------------------------
>
> Key: CASSANDRA-2206
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2206
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Affects Versions: 0.7.2
> Environment: linux
> Reporter: Thibaut
> Fix For: 0.7.3
>
> Attachments: 2206.txt
>
>
> Hi,
> On one of our nodes, startup fails due to cassandra trying to delete a nonexistant "Data" file (see below).
> Why that file is missing is another mistery... The log file entries don't show any ERROR messages before cassandra restarted (for reasons I don't know) and this error occured.
> Directory listing:
> total 109M
> -rw-r--r-- 1 root root 51M 2011-02-21 05:25 table_task-f-1666-Data.db
> -rw-r--r-- 1 root root 243K 2011-02-21 05:25 table_task-f-1666-Filter.db
> -rw-r--r-- 1 root root 6.1M 2011-02-21 05:25 table_task-f-1666-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 05:25 table_task-f-1666-Statistics.db
> -rw-r--r-- 1 root root 9.8M 2011-02-21 11:36 table_task-f-1703-Data.db
> -rw-r--r-- 1 root root 57K 2011-02-21 11:36 table_task-f-1703-Filter.db
> -rw-r--r-- 1 root root 1.3M 2011-02-21 11:36 table_task-f-1703-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:36 table_task-f-1703-Statistics.db
> -rw-r--r-- 1 root root 292K 2011-02-21 11:42 table_task-f-1704-Data.db
> -rw-r--r-- 1 root root 1.7K 2011-02-21 11:42 table_task-f-1704-Filter.db
> -rw-r--r-- 1 root root 42K 2011-02-21 11:42 table_task-f-1704-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:42 table_task-f-1704-Statistics.db
> -rw-r--r-- 1 root root 364K 2011-02-21 11:52 table_task-f-1705-Data.db
> -rw-r--r-- 1 root root 2.0K 2011-02-21 11:52 table_task-f-1705-Filter.db
> -rw-r--r-- 1 root root 50K 2011-02-21 11:52 table_task-f-1705-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:52 table_task-f-1705-Statistics.db
> -rw-r--r-- 1 root root 535K 2011-02-21 12:10 table_task-f-1706-Data.db
> -rw-r--r-- 1 root root 2.8K 2011-02-21 12:10 table_task-f-1706-Filter.db
> -rw-r--r-- 1 root root 70K 2011-02-21 12:10 table_task-f-1706-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:10 table_task-f-1706-Statistics.db
> -rw-r--r-- 1 root root 11M 2011-02-21 12:11 table_task-f-1707-Data.db
> -rw-r--r-- 1 root root 18M 2011-02-21 09:47 table_task_meta-f-417-Data.db
> -rw-r--r-- 1 root root 271K 2011-02-21 09:47 table_task_meta-f-417-Filter.db
> -rw-r--r-- 1 root root 6.7M 2011-02-21 09:47 table_task_meta-f-417-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 09:47 table_task_meta-f-417-Statistics.db
> -rw-r--r-- 1 root root 1.2M 2011-02-21 10:47 table_task_meta-f-418-Data.db
> -rw-r--r-- 1 root root 18K 2011-02-21 10:47 table_task_meta-f-418-Filter.db
> -rw-r--r-- 1 root root 460K 2011-02-21 10:47 table_task_meta-f-418-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 10:47 table_task_meta-f-418-Statistics.db
> -rw-r--r-- 1 root root 791K 2011-02-21 11:47 table_task_meta-f-419-Data.db
> -rw-r--r-- 1 root root 13K 2011-02-21 11:47 table_task_meta-f-419-Filter.db
> -rw-r--r-- 1 root root 311K 2011-02-21 11:47 table_task_meta-f-419-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 11:47 table_task_meta-f-419-Statistics.db
> -rw-r--r-- 1 root root 57K 2011-02-21 12:11 table_task-tmp-f-1707-Filter.db
> -rw-r--r-- 1 root root 1.4M 2011-02-21 12:11 table_task-tmp-f-1707-Index.db
> -rw-r--r-- 1 root root 4.2K 2011-02-21 12:11 table_task-tmp-f-1707-Statistics.db
> Cassandra log:
> /software/cassandra/bin/cassandra
> rm: cannot remove `/software/cassandra/lib/jna.jar': No such file or directory
> root@intr1n3:/cassandra/data/table_task# INFO 12:47:29,020 Logging initialized
> INFO 12:47:29,030 Heap size: 2614493184/2614493184
> INFO 12:47:29,031 JNA not found. Native methods will be disabled.
> INFO 12:47:29,038 Loading settings from file:/software/cassandra/conf/cassandra.yaml
> INFO 12:47:29,320 DiskAccessMode is standard, indexAccessMode is mmap
> INFO 12:47:29,332 Creating new commitlog segment /hd1/cassandra_md5/commitlog/CommitLog-1298288849332.log
> INFO 12:47:29,422 Opening /cassandra/data/system/Schema-f-244
> INFO 12:47:29,434 Opening /cassandra/data/system/Migrations-f-244
> INFO 12:47:29,437 Opening /cassandra/data/system/LocationInfo-f-137
> INFO 12:47:29,440 Opening /cassandra/data/system/HintsColumnFamily-f-352
> INFO 12:47:29,441 Opening /cassandra/data/system/HintsColumnFamily-f-353
> INFO 12:47:29,465 Loading schema version 54bc134e-2229-11e0-9159-fdf0b6b4b562
> WARN 12:47:29,623 Schema definitions were defined both locally and in cassandra.yaml. Definitions in cassandra.yaml were ignored.
> ERROR 12:47:29,638 Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
> Exception encountered during startup.
> java.io.IOError: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:145)
> at org.apache.cassandra.db.ColumnFamilyStore.scrubDataDirectories(ColumnFamilyStore.java:468)
> at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:153)
> at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:316)
> at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:79)
> Caused by: java.io.IOException: Failed to delete /cassandra/data/table_task/table_task-tmp-f-1707-Data.db
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> at org.apache.cassandra.io.sstable.SSTable.delete(SSTable.java:133)
> ... 4 more
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira