[jira] [Assigned] (CASSANDRA-2427) Heuristic or hard cap to prevent fragmented commit logs from bringing down the server

["Jonathan Ellis (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/CASSANDRA-2427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jonathan Ellis reassigned CASSANDRA-2427:
-----------------------------------------

    Assignee: paul cannon

How about this:
- create a commitlog-space-cap setting; if space gets above that, flush every dirty CF in the oldest segment and remove it
- remove the per-CF memtable_flush_after_minutes setting in favor of the above (this fixes the flush-storm problem of all your CFs flushing at once when you configure this too low)
- while we're at it, remove the commitlog segment size config setting, nobody has ever changed it to my knowledge in 2+ years

(I don't think we want to add an additional check for "is there enough space for a new segment" at segment creation time; if there isn't enough space for that there probably isn't enough to flush either and we're screwed either way. So it would be complexity w/o a purpose.)

> Heuristic or hard cap to prevent fragmented commit logs from bringing down the server
> -------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-2427
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2427
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Benjamin Coverston
>            Assignee: paul cannon
>              Labels: commitlog, hardening
>             Fix For: 1.0
>
>
> Widely divergent write rates on column families can cause the commit log segments to fragment. In some cases we have seen the commit log partition overrun.
> One solution here would be to create a heuristic for segment fragmentation to trigger a flush (commit log segments/memtable) or simply track the free disk space and force a global flush when the disk gets to 80% capacity.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira