You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Jay Zhuang (JIRA)" <ji...@apache.org> on 2018/03/12 04:15:00 UTC
[jira] [Created] (CASSANDRA-14308) Remove invalid SSTables from
interrupted compaction
Jay Zhuang created CASSANDRA-14308:
--------------------------------------
Summary: Remove invalid SSTables from interrupted compaction
Key: CASSANDRA-14308
URL: https://issues.apache.org/jira/browse/CASSANDRA-14308
Project: Cassandra
Issue Type: Bug
Components: Compaction
Reporter: Jay Zhuang
Assignee: Jay Zhuang
If the JVM crash while compaction is in progress, the incompleted SSTable won't be cleaned up, which causes {{Stats component is missing for sstable}} error in the start up log:
{noformat}
ERROR [SSTableBatchOpen:3] 2018-03-11 00:17:35,597 CassandraDaemon.java:207 - Exception in thread Thread[SSTableBatchOpen:3,5,main]
java.lang.AssertionError: Stats component is missing for sstable /cassandra/data/keyspace/table-id/mc-12345-big
at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:458) ~[apache-cassandra-3.0.14.jar:3.0.14]
at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:374) ~[apache-cassandra-3.0.14.jar:3.0.14]
at org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:533) ~[apache-cassandra-3.0.14.jar:3.0.14]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_121]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_121]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_121]
at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:79) [apache-cassandra-3.0.14.jar:3.0.14]
at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_121]
{noformat}
The accumulated incompleted SSTables could take lots of space, especially for STCS, which could have very large SSTables.
Here is the script we use to delete the SSTables after node is restarted:
{noformat}
grep 'Stats component is missing for sstable' $SYSTEM_LOG | awk '{print $8}' > ~/invalid_sstables ; for ss in `cat ~/invalid_sstables`; do echo == $ss; ll $ss*; sudo rm $ss* ; done
{noformat}
I would suggest to remove these incompleted SSTables while startup.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org