[jira] Resolved: (CASSANDRA-1858) File descriptors to sstables not closed (even for sstables that have been deleted due to compaction)

["Jonathan Ellis (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CASSANDRA-1858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis resolved CASSANDRA-1858.
---------------------------------------

    Resolution: Duplicate

if he saw it in rc1, it's probably CASSANDRA-1790 (fixed in rc2)

> File descriptors to sstables not closed (even for sstables that have been deleted due to compaction)
> ----------------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-1858
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-1858
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.7.0 rc 1
>         Environment: Ubuntu 8.04 with Java 1.6.0_22-b04
>            Reporter: Josep M. Blanquer
>
> The Cassandra process doesn't let go of filedescriptors to sstables.
> It was initially spotted due to the fact that the disk utilization was really high, while the data dirs of sstables held much less (nicely compacted) data.
> Performing an "lsof" lists the Cassandra process having basically all open FD's pointing to lots and lots of deleted files. In fact, I saw the "-1-Data.db" in the list...which tells me that not even the first sstable was let go.
> Restarting the cassandra process obviously gets rid of the references. I've also manually triggered a GC to try to make sure compaction is completed...to no avail. Holding FD's for a high-throughput write servers is a big problem since it accumulates a huge amount of disk space due to compactions..etc...so it's very easy to run out of space.
> There's nothing special in this setup, so I think it'd be easily reproducible. I get this in a:
> * 4-node cluster, 1 Keyspace, 1 CF, RF=2
> * the cluster has basically been stood up, and blasted with inserts (now it has 200GB, 50 each)...but it doesn't get reads (I'm performing just loading data tests).
> * While the loading is happening, one can observe that the process keeps compacting and creating new sstables...but never lets the FD's go.
> * even if I trigger compactions and cleanups...the disk util only continues to go up...i.e., all FD's are still open, plus the compated sstables are added.
> * at a particular node I see about 11 fd's for current sstables, and 1250 fd's pointing to deleted sstables.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.