[jira] [Commented] (OAK-4122) Replace the commit semaphore in the segment node store with a scheduler

["Michael Dürig (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15326976#comment-15326976 ] 

Michael Dürig commented on OAK-4122:
------------------------------------

After all the refactoring that happened in that last couple of weeks here is a sketch of how I think such a scheduler should fit in:

* The {{SegmentStore}} (e.g. {{FileStore}}) uses a {{Revions}} instance to track its head. This is a low level concern and each implementation should make an effort to minimise locking as much as possible. E.g. for {{TarRevisons}} used by the {{FileStore}} setting the head is usually just a "compare and swap" operation. Locking is only involved for obtaining exclusive access to the store (e.g. for forced commits) (See OAK-4015 for possible optimisations). 

* All functionality related to tracking the head of the {{SegmentNodeStore}} should go into a dedicated commit scheduler. This conceptionally involves {{commitSemaphore}}, {{changeDispatcher}}, {{refreshHead}}, {{locked()}}, etc. 

* The commit scheduler should replace the lock based implementation of committing changes with a queue based one as described before. 

> Replace the commit semaphore in the segment node store with a scheduler
> -----------------------------------------------------------------------
>
>                 Key: OAK-4122
>                 URL: https://issues.apache.org/jira/browse/OAK-4122
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: segment-tar
>            Reporter: Michael Dürig
>            Assignee: Francesco Mari
>              Labels: performance, scalability, throughput
>             Fix For: 1.6
>
>
> {{SegmentNodeStore}} currently uses a semaphore to coordinate concurrent commits thus relying on the scheduling algorithm of that implementation and ultimately of the JVM for in what order commits are processed. 
> I think it would be beneficial to replace that semaphore with an explicit queue of pending commit. This would allow us to implement a proper scheduler optimising for e.g. minimal system load, maximal throughput or minimal latency etc. A scheduler could e.g. give precedence to big commits and order commits along the order of its base revisions, which would decrease the amount of work to be done in rebasing. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)