[jira] [Comment Edited] (OAK-4122) Replace the commit semaphore in the segment node store with a scheduler

["Michael Dürig (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/OAK-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15223924#comment-15223924 ] 

Michael Dürig edited comment on OAK-4122 at 6/13/16 7:49 AM:
-------------------------------------------------------------

A couple of benefits an explicit commit queue would give us:

* We could implement sophisticated monitoring, which would give us statistics (e.g. max, min, mean, median 90% percentile etc.) about how long commits take, how long they stay on the queue, what the commit rate is etc. 

* We could favour certain commits. E.g. --commits from the compactor (see OAK-4015) could jump the queue--. Also checkpoint policies (see OAK-3587) could be implemented more easily though this mechanism. (*Edit:* Not sure about the compactor any more. Those commits happen on the lower file store level. We should probably stick with the lock approach outlined in OAK-4015 here). 

* Instead of racing the compactor against concurrent commits and rebasing the latter later in a number of compaction cycles (see OAK-2743), we could suspend the compactor on concurrent commits and have it resume where it left off afterwards. This would give us finer control over how we prioritise the compactor vs. normal commits. (OAK-3348 is a prerequisite for this bullet point). 

* We could try to parallelise (parts of) certain commits. This probably goes along with some POCs [~egli] has been working on lately. 

* As such queues would in most cases introduce a thread boundary we would shield the file store from {{InterruptedException}} caused by applications threads (see OAK-2609). 

Overall I think replacing the commit semaphore with an explicit queue will open give us some additional design space allowing us to generalise things we now have individual implementations for. 


was (Author: mduerig):
A couple of benefits an explicit commit queue would give us:

* We could implement sophisticated monitoring, which would give us statistics (e.g. max, min, mean, median 90% percentile etc.) about how long commits take, how long they stay on the queue, what the commit rate is etc. 

* We could favour certain commits. E.g. commits from the compactor (see OAK-4015) could jump the queue. Also checkpoint policies (see OAK-3587) could be implemented more easily though this mechanism. 

* Instead of racing the compactor against concurrent commits and rebasing the latter later in a number of compaction cycles (see OAK-2743), we could suspend the compactor on concurrent commits and have it resume where it left off afterwards. This would give us finer control over how we prioritise the compactor vs. normal commits. (OAK-3348 is a prerequisite for this bullet point). 

* We could try to parallelise (parts of) certain commits. This probably goes along with some POCs [~egli] has been working on lately. 

* As such queues would in most cases introduce a thread boundary we would shield the file store from {{InterruptedException}} caused by applications threads (see OAK-2609). 

Overall I think replacing the commit semaphore with an explicit queue will open give us some additional design space allowing us to generalise things we now have individual implementations for. 

> Replace the commit semaphore in the segment node store with a scheduler
> -----------------------------------------------------------------------
>
>                 Key: OAK-4122
>                 URL: https://issues.apache.org/jira/browse/OAK-4122
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: segment-tar
>            Reporter: Michael Dürig
>            Assignee: Francesco Mari
>              Labels: performance, scalability, throughput
>             Fix For: 1.6
>
>
> {{SegmentNodeStore}} currently uses a semaphore to coordinate concurrent commits thus relying on the scheduling algorithm of that implementation and ultimately of the JVM for in what order commits are processed. 
> I think it would be beneficial to replace that semaphore with an explicit queue of pending commit. This would allow us to implement a proper scheduler optimising for e.g. minimal system load, maximal throughput or minimal latency etc. A scheduler could e.g. give precedence to big commits and order commits along the order of its base revisions, which would decrease the amount of work to be done in rebasing. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)