You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Andreas Schnitzerling (JIRA)" <ji...@apache.org> on 2013/12/02 13:03:37 UTC
[jira] [Commented] (CASSANDRA-6283) Windows 7 data files keept open
/ can't be deleted after compaction.
[ https://issues.apache.org/jira/browse/CASSANDRA-6283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13836458#comment-13836458 ]
Andreas Schnitzerling commented on CASSANDRA-6283:
--------------------------------------------------
Over weekend I got the following:
{panel:title=system.log}
ERROR [STREAM-IN-/10.6.8.78] 2013-11-29 17:57:20,266 StreamSession.java (line 410) [Stream #ea9bd2c0-5916-11e3-a5b5-b13a5fe180aa] Streaming error occurred
java.lang.RuntimeException: Outgoing stream handler has been closed
at org.apache.cassandra.streaming.ConnectionHandler.sendMessage(ConnectionHandler.java:175)
at org.apache.cassandra.streaming.StreamSession.prepare(StreamSession.java:436)
at org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:358)
at org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:293)
at java.lang.Thread.run(Unknown Source)
WARN [STREAM-IN-/10.6.8.78] 2013-11-29 17:57:20,266 StreamResultFuture.java (line 210) [Stream #ea9bd2c0-5916-11e3-a5b5-b13a5fe180aa] Stream failed
ERROR [NonPeriodicTasks:1] 2013-11-29 17:58:54,266 SSTableDeletingTask.java (line 81) Unable to delete D:\Programme\cassandra\data\system\schema_columns\system-schema_columns-jb-468-Data.db (it will be removed on server restart; we'll also retry after GC)
ERROR [NonPeriodicTasks:1] 2013-11-29 17:58:54,266 SSTableDeletingTask.java (line 81) Unable to delete D:\Programme\cassandra\data\system\schema_columns\system-schema_columns-jb-469-Data.db (it will be removed on server restart; we'll also retry after GC)
ERROR [NonPeriodicTasks:1] 2013-11-29 17:58:58,446 SSTableDeletingTask.java (line 81) Unable to delete D:\Programme\cassandra\data\system\schema_columnfamilies\system-schema_columnfamilies-jb-525-Data.db (it will be removed on server restart; we'll also retry after GC)
ERROR [NonPeriodicTasks:1] 2013-11-29 17:58:58,446 SSTableDeletingTask.java (line 81) Unable to delete D:\Programme\cassandra\data\system\schema_columnfamilies\system-schema_columnfamilies-jb-527-Data.db (it will be removed on server restart; we'll also retry after GC)
{panel}
The deleting problem occurs only once. So my GC patch seems to work. But there's no log of the leakdetect-patch.
{panel:title=system.log 10.6.8.78}
ERROR [CompactionExecutor:40] 2013-11-29 17:56:16,368 CassandraDaemon.java (line 187) Exception in thread Thread[CompactionExecutor:40,1,main]
java.util.concurrent.RejectedExecutionException: ThreadPoolExecutor has shut down
at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor$1.rejectedExecution(DebuggableThreadPoolExecutor.java:61)
at java.util.concurrent.ThreadPoolExecutor.reject(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.execute(Unknown Source)
at org.apache.cassandra.concurrent.DebuggableThreadPoolExecutor.execute(DebuggableThreadPoolExecutor.java:145)
at java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
at org.apache.cassandra.db.ColumnFamilyStore.switchMemtable(ColumnFamilyStore.java:752)
at org.apache.cassandra.db.ColumnFamilyStore.forceFlush(ColumnFamilyStore.java:817)
at org.apache.cassandra.db.SystemKeyspace.forceBlockingFlush(SystemKeyspace.java:420)
at org.apache.cassandra.db.SystemKeyspace.finishCompaction(SystemKeyspace.java:197)
at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:225)
at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:197)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
{panel}
> Windows 7 data files keept open / can't be deleted after compaction.
> --------------------------------------------------------------------
>
> Key: CASSANDRA-6283
> URL: https://issues.apache.org/jira/browse/CASSANDRA-6283
> Project: Cassandra
> Issue Type: Bug
> Components: Core
> Environment: Windows 7 (32) / Java 1.7.0.45
> Reporter: Andreas Schnitzerling
> Priority: Critical
> Labels: newbie, patch, test
> Fix For: 2.0.4
>
> Attachments: leakdetect.patch, screenshot-1.jpg, system.log
>
>
> Files cannot be deleted, patch CASSANDRA-5383 (Win7 deleting problem) doesn't help on Win-7 on Cassandra 2.0.2. Even 2.1 Snapshot is not running. The cause is: Opened file handles seem to be lost and not closed properly. Win 7 blames, that another process is still using the file (but its obviously cassandra). Only restart of the server makes the files deleted. But after heavy using (changes) of tables, there are about 24K files in the data folder (instead of 35 after every restart) and Cassandra crashes. I experiminted and I found out, that a finalizer fixes the problem. So after GC the files will be deleted (not optimal, but working fine). It runs now 2 days continously without problem. Possible fix/test:
> I wrote the following finalizer at the end of class org.apache.cassandra.io.util.RandomAccessReader:
> {code:title=RandomAccessReader.java|borderStyle=solid}
> @Override
> protected void finalize() throws Throwable {
> deallocate();
> super.finalize();
> }
> {code}
> Can somebody test / develop / patch it? Thx.
--
This message was sent by Atlassian JIRA
(v6.1#6144)