[jira] [Comment Edited] (CASSANDRA-8771) Remove commit log segment recycling

["Ariel Weisberg (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-8771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14316754#comment-14316754 ] 

Ariel Weisberg edited comment on CASSANDRA-8771 at 2/11/15 6:49 PM:
--------------------------------------------------------------------

There is also the option of calling fallocate via JNA. What that does is an open question for me as it is file system specific. Red Hat in their performance guide claim that it will encourage better allocation.

With compression, at least the current proposed implementation it is a little undesirable because it doesn't get very close to the target size of 32 megabytes after compression shrinks the data in the segment.


was (Author: aweisberg):
There is also the option of calling fallocate via JNA. What that does is an open question for me as it is file system specific. Red Hat in their performance guide claim that it will encourage better allocation.

> Remove commit log segment recycling
> -----------------------------------
>
>                 Key: CASSANDRA-8771
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8771
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Ariel Weisberg
>              Labels: commitlog
>
> For discussion
> Commit log segment recycling introduces a lot of complexity in the existing code.
> CASSANDRA-8729 is a side effect of commit log segment recycling and addressing it will require memory management code and thread coordination for memory that the filesystem will no longer handle for us.
> There is some discussion about what storage configurations actually benefit from preallocated files. Fast random access devices like SSDs, or non-volatile write caches etc. make the distinction not that great. 
> I haven't measured any difference in throughput for bulk appending vs overwriting although it was pointed out that I didn't test with concurrent IO streams.
> What would it take to make removing commit log segment recycling acceptable? Maybe a benchmark on a spinning disk that measures the performance impact of preallocation when there are other IO streams?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)