[jira] [Comment Edited] (OAK-4279) Rework offline compaction

["Alex Parvulescu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-4279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15309873#comment-15309873 ] 

Alex Parvulescu edited comment on OAK-4279 at 6/1/16 8:09 AM:
--------------------------------------------------------------

attaching patch. there are still some rough edges, but it would be good to validate the approach. [~mduerig] feedback appreciated!

[edit] 2 issues currently working on
* offline compaction will need to reduce the {{retainedGenerations}} to 1, otherwise it's not as effective
 * Compactor deduplication map (ex. compaction map) is now a simple map, it is prone to OOMEs if I don't add some protection (either drop items beyond a certain size, or persist to disk like the persisted map did)


was (Author: alex.parvulescu):
attaching patch. there are still some rough edges, but it would be good to validate the approach. [~mduerig] feedback appreciated!

> Rework offline compaction
> -------------------------
>
>                 Key: OAK-4279
>                 URL: https://issues.apache.org/jira/browse/OAK-4279
>             Project: Jackrabbit Oak
>          Issue Type: Task
>          Components: segment-tar
>            Reporter: Michael Dürig
>            Assignee: Alex Parvulescu
>            Priority: Blocker
>              Labels: compaction, gc
>             Fix For: 1.6
>
>         Attachments: OAK-4279-v0.patch
>
>
> The fix for OAK-3348 broke some of the previous functionality of offline compaction:
> * No more progress logging
> * Compaction is not interruptible any more (in the sense of OAK-3290)
> * Offline compaction could remove the ids of the segment node states to squeeze out some extra space. Those are only needed for later generations generated via online compaction. 
> We should probably implement offline compaction again through a dedicated {{Compactor}} class as it was done in {{oak-segment}} instead of relying on the de-duplication cache (aka online compaction). 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)