[jira] [Created] (CASSANDRA-8586) support millions of sstables by lazily acquiring/caching/dropping filehandles

["Tupshin Harper (JIRA)" <ji...@apache.org>]

Tupshin Harper created CASSANDRA-8586:
-----------------------------------------

             Summary: support millions of sstables by lazily acquiring/caching/dropping filehandles
                 Key: CASSANDRA-8586
                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8586
             Project: Cassandra
          Issue Type: New Feature
            Reporter: Tupshin Harper
            Assignee: Aleksey Yeschenko


This might turn into a meta ticket if other obstacles are found in the goal of supporting a huge number of sstables.

Technically, the only gap that I know of to prevent us from supporting absurd numbers of sstables is the fact that we hold on to an open filehandle for every single sstable. 

For use cases that are willing to take a hit to read-performance in order to achieve high densities and low write amplification, a mechanism for only retaining file handles for recently read sstables could be very valuable.

This will allow for alternate compaction strategies and compaction strategy tuning that don't try to optimize for read performance as aggresively.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)