[jira] [Commented] (CASSANDRA-6809) Compressed Commit Log

["Alan Boudreault (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CASSANDRA-6809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14556583#comment-14556583 ] 

Alan Boudreault commented on CASSANDRA-6809:
--------------------------------------------

I did multiple tests locally and on cstar_perf. Everything looks good so far and the performance are great. I also tested the disk usage through the different compression algorithms and the benefit were good as expected: For the record, here are the links of the main cstar_perf tests:

http://cstar.datastax.com/tests/id/a5f48f90-007c-11e5-adde-42010af0688f
http://cstar.datastax.com/tests/id/8f0c4dfe-007c-11e5-aa79-42010af0688f
http://cstar.datastax.com/tests/id/de7512ea-007c-11e5-adde-42010af0688f

The DeflateCompressor is a bit slower, but IIRC, it is expected and not very used. 

> Compressed Commit Log
> ---------------------
>
>                 Key: CASSANDRA-6809
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6809
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Benedict
>            Assignee: Branimir Lambov
>            Priority: Minor
>              Labels: docs-impacting, performance, qa-resolved
>             Fix For: 2.2.0 beta 1
>
>         Attachments: ComitLogStress.java, logtest.txt
>
>
> It seems an unnecessary oversight that we don't compress the commit log. Doing so should improve throughput, but some care will need to be taken to ensure we use as much of a segment as possible. I propose decoupling the writing of the records from the segments. Basically write into a (queue of) DirectByteBuffer, and have the sync thread compress, say, ~64K chunks every X MB written to the CL (where X is ordinarily CLS size), and then pack as many of the compressed chunks into a CLS as possible.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)