[jira] [Commented] (OAK-4122) Replace the commit semaphore in the segment node store with a scheduler

["Andrei Dulceanu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15944933#comment-15944933 ] 

Andrei Dulceanu commented on OAK-4122:
--------------------------------------

I finished extracting all the logic regarding merging/checkpointing from {{SegmentNodeStore}} to a {{LockBasedScheduler}}. The API sketched in the previous round is not (yet) fully used, but will come into play once a queue-based scheduler will be implemented. So far the only tests which needed to be ignored are [1] (two tests) and [2]. As previously individual commits can be tracked in [3].

[~frm], [~mduerig], could you take a look at the branch? I'm interested if {{LockBasedScheduler}} makes it into a good baseline. Is there room for other improvements at this stage? Thanks!

[1] https://github.com/dulceanu/jackrabbit-oak/blob/features/scheduler/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/CheckpointTest.java#L103
[2] https://github.com/dulceanu/jackrabbit-oak/blob/features/scheduler/oak-segment-tar/src/test/java/org/apache/jackrabbit/oak/segment/MergeTest.java#L91
[3] https://github.com/dulceanu/jackrabbit-oak/find/features/scheduler

> Replace the commit semaphore in the segment node store with a scheduler
> -----------------------------------------------------------------------
>
>                 Key: OAK-4122
>                 URL: https://issues.apache.org/jira/browse/OAK-4122
>             Project: Jackrabbit Oak
>          Issue Type: New Feature
>          Components: segment-tar
>            Reporter: Michael Dürig
>            Assignee: Andrei Dulceanu
>              Labels: operations, performance, scalability, throughput
>             Fix For: 1.8, 1.7.3
>
>
> {{SegmentNodeStore}} currently uses a semaphore to coordinate concurrent commits thus relying on the scheduling algorithm of that implementation and ultimately of the JVM for in what order commits are processed. 
> I think it would be beneficial to replace that semaphore with an explicit queue of pending commit. This would allow us to implement a proper scheduler optimising for e.g. minimal system load, maximal throughput or minimal latency etc. A scheduler could e.g. give precedence to big commits and order commits along the order of its base revisions, which would decrease the amount of work to be done in rebasing. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)