[jira] [Updated] (OAK-4015) Expedite commits from the compactor

["Michael Dürig (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OAK-4015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Michael Dürig updated OAK-4015:
-------------------------------
    Attachment: OAK-4015-histo.png
                OAK-4015-wait-time.png

The patch seems in fact to have a positive impact on the time the compactor needs to wait for the commit lock. Running {{SegmentCompactionIT}} for several days with an hourly compaction cycle I got the following mean/median wait times:

base: 15.5s / 10.4s  
patched: 1.7s / 2.7s

This is also reflected in the histogram (base on top, patched on bottom): !OAK-4015-histo.png!

Finally without the patch the lock wait time seems to depend on the size of the repository as it grows over time. With the patch the wait time seems to stay constant: !OAK-4015-wait-time.png!

[~alex.parvulescu], [~frm] FYI



> Expedite commits from the compactor
> -----------------------------------
>
>                 Key: OAK-4015
>                 URL: https://issues.apache.org/jira/browse/OAK-4015
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: segmentmk
>            Reporter: Michael Dürig
>              Labels: compaction, gc, perfomance
>             Fix For: 1.6
>
>         Attachments: OAK-4015-histo.png, OAK-4015-wait-time.png
>
>
> Concurrent commits during compaction cause those to be re-compacted. Currently it seems that the compaction thread can end up waiting for some time to acquire the commit lock [1], which in turn causes more commits to pile up to be re-compacted. I think this could be improved by tweaking the lock such that the compactor could jump ahead of the queue. I.e. use a lock which can be acquired in expedited mode. 
> [1] SegmentNodeStore#commitSemaphore



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)