[jira] [Commented] (OAK-4122) Replace the commit semaphore in the segment node store with a scheduler

["Andrei Dulceanu (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15830111#comment-15830111 ] 

Andrei Dulceanu commented on OAK-4122:
--------------------------------------

I created a branch [1] in my GitHub to track development for this issue. I just collected initial API proposed by [~frm] and [~mduerig] (with subsequent changes). I'm trying to better visualise now if/how recent improvements in {{segment-tar}} (e.g. {{Revisions}} abstraction being mentioned in an earlier comment) affects current design. If this is the case, what suggestions/additions do you see here?

On a different topic, as discussed recently, the low-hanging fruit in terms of scheduler benefits would be monitoring. For our current implementation of the {{SegmentNodeStore}} this was addressed in OAK-4742. Are there other monitoring operations needed there?

/cc [~mduerig], [~frm]

[1] https://github.com/dulceanu/jackrabbit-oak/commits/features/scheduler

> Replace the commit semaphore in the segment node store with a scheduler
> -----------------------------------------------------------------------
>
>                 Key: OAK-4122
>                 URL: https://issues.apache.org/jira/browse/OAK-4122
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: segment-tar
>            Reporter: Michael Dürig
>            Assignee: Andrei Dulceanu
>              Labels: operations, performance, scalability, throughput
>             Fix For: 1.8
>
>
> {{SegmentNodeStore}} currently uses a semaphore to coordinate concurrent commits thus relying on the scheduling algorithm of that implementation and ultimately of the JVM for in what order commits are processed. 
> I think it would be beneficial to replace that semaphore with an explicit queue of pending commit. This would allow us to implement a proper scheduler optimising for e.g. minimal system load, maximal throughput or minimal latency etc. A scheduler could e.g. give precedence to big commits and order commits along the order of its base revisions, which would decrease the amount of work to be done in rebasing. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)