[jira] [Commented] (OAK-5860) Compressed segments

["Michael Dürig (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-5860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15924041#comment-15924041 ] 

Michael Dürig commented on OAK-5860:
------------------------------------

Compressing the segments might not make too much sense when using memory mapped access as in that case it would defeat the purpose of the direct buffer access (i.e. would require copying the uncompressed segments into the heap). 

So I suggest to look at this issue in combination with OAK-5859, where one suggested approach is to move away from memory mapped access. 

> Compressed segments
> -------------------
>
>                 Key: OAK-5860
>                 URL: https://issues.apache.org/jira/browse/OAK-5860
>             Project: Jackrabbit Oak
>          Issue Type: New Feature
>          Components: segment-tar
>            Reporter: Michael Dürig
>              Labels: scalability
>             Fix For: 1.7.2
>
>
> It would be interesting to see the effect of compressing the segments within the tar files with a sufficiently effective and performant compression algorithm:
> * Can we increase overall throughput by trading CPU for IO?
> * Can we scale to bigger repositories (in number of nodes) by squeezing in more segments per MB and thus pushing out onset of thrashing?
> * What would be a good compression algorithm/library?
> * Can/should we make this optional? 
> * Migration and compatibility issues?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)