[jira] Commented: (CASSANDRA-299) make table directory creation lazy

["Arin Sarkissian (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12742013#action_12742013 ] 

Arin Sarkissian commented on CASSANDRA-299:
-------------------------------------------

http://github.com/phatduckk/Cassandra/commit/e2b5774db232f1d419e22089444682dbee1d3f51

What's going on in the patch is that  tables are no longer open()'ed and onStart()'ed in CassandraDaemon.

The whole process is lazy...
When a table is first opened we will make sure it has a dir in each <DataFileDirectories>
Once that's done we start the table which will start each of its CF

If I understand correctly what I've done so far is just 1/2 of  what Jonathan mentioned in his comment. It sounds like Jonathan's saying we should still open() and onStart() Tables in CassandraDaemon but only those that have subdirs in <DataFileDirectories>.

Combining that with my lazy opne()/onStart() refector will work but I want to make sure that's what you're getting at before I submit a patch...

> make table directory creation lazy
> ----------------------------------
>
>                 Key: CASSANDRA-299
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-299
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jonathan Ellis
>            Priority: Minor
>             Fix For: 0.5
>
>
> checking that each subdir for each table is present on startup -- _every_ startup -- could be a real pita.
> i think that to support 100k tables (not impossible, in a hosted-cassandra-as-a-service scenario) we're going to want to make table dir creation lazy.
> then we would want to make scanning for sstables faster by only doing one listdir call per datadir, to see which table subdirs are present, and then checking only those for sstable files.  this would involve some re-org of the onstart code.
> (note that we don't want to prune directories if there are no sstables left in them, since we'd end up re-creating them at some point anyway; we just want to allow the lack of a table subdir to imply the same thing as an empty one.)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.