[jira] [Commented] (SLING-5387) Provide support for running singleton jobs on non leader cluster nodes also

["Carsten Ziegeler (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/SLING-5387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15984393#comment-15984393 ] 

Carsten Ziegeler commented on SLING-5387:
-----------------------------------------

I think there are four scenarios:
a) an OSGi service is scheduled and a name property is provided
b) a job is scheduled using the API and a name is provided
c) an OSGi service is scheduled and a name property is not provided
d) a job is scheduled using the API and no name is provided

For a) and b) we can use the value of the name property as a unique key.
For c) and d) we can use the class name of the provided job as a unique key.

For distribution in a cluster, we could:
a) create a hash (md5) of the key
b) create an integer of the first two characters of the hash
c) distribute the job based on the integer from b) modulo the number of instances reported by discovery, the instances are sorted by instance id for this

This should give us a fairly equal distribution I hope

[~egli], [~chetanm] WDYT?

> Provide support for running singleton jobs on non leader cluster nodes also
> ---------------------------------------------------------------------------
>
>                 Key: SLING-5387
>                 URL: https://issues.apache.org/jira/browse/SLING-5387
>             Project: Sling
>          Issue Type: Improvement
>          Components: Commons
>            Reporter: Chetan Mehrotra
>            Assignee: Carsten Ziegeler
>             Fix For: Commons Scheduler 2.5.4
>
>
> With SLING-2979 support for running singleton jobs on specific instance was provided. In most cases we want to run  a job as singleton and not want to "pin" it to specific nodes. For this {{scheduler.runOn}} needs to be set to {{SINGLE}}.
> However per [current implementation|https://github.com/apache/sling/blob/org.apache.sling.commons.scheduler-2.4.14/src/main/java/org/apache/sling/commons/scheduler/impl/QuartzJobExecutor.java#L64] {{SINGLE}} is treated as {{LEADER}}. This effectively causes *all singleton* jobs to get executed on leader only thus putting extra load.
> For better utilization of cluster resources it should be possible to distribute such singleton jobs on other cluster nodes and still ensure that singleton contract is honoured!
> We would like to make use of this feature to ensure Oak AsyncIndexTask to run on different cluster nodes (OAK-2749) 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)