You are viewing a plain text version of this content. The canonical link for it is here.
Posted to oak-dev@jackrabbit.apache.org by Chetan Mehrotra <ch...@gmail.com> on 2016/07/19 04:55:16 UTC
Specifying threadpool name for periodic scheduled jobs (OAK-4563)
Hi Team,
While running Oak in Sling we rely on Sling Scheduler [1] to execute
the periodic jobs. By default Sling Scheduler uses a pool of 5 threads
to run all such periodic jobs in the system. Recently we saw an issue
OAK-4563 where due to some reason the pool got exhausted for long time
and that prevented the async indexing job to run for long time and
hence affected the query result.
To address that Sling now provides a new option (SLING-5831) where one
can specify the pool name to be used to execute a specific job. So we
can specify custom pool which can be used for Oak related jobs.
Now currently in Oak we use following types of periodic jobs
1. Async indexing - (Cluster Singleton)
2. Document Store - Journal GC (Cluster Singleton)
3. Document Store - LastRevRecovery
4. Statistic Collection - For timeseries data update in ChangeProcessor,
SegmentNodeStore GCMonitor
Now should we use
A - one single pool for all of the above
B - use the pool only for 1-3. The default pool would be of 5. So even
if #2 #3 are running
it would not hamper #1
Assuming #4 is not that critical to run and may consist of lots of jobs.
My suggestion would be to go for #B
Chetan Mehrotra
[1] https://sling.apache.org/documentation/bundles/scheduler-service-commons-scheduler.html
Re: Specifying threadpool name for periodic scheduled jobs (OAK-4563)
Posted by Chetan Mehrotra <ch...@gmail.com>.
On Tue, Jul 19, 2016 at 1:44 PM, Stefan Egli <st...@apache.org> wrote:
> I'd go for #A to limit cross-effects between oak and other layers.
Note that for #4 there can be multiple task scheduled. So if a system
has 100 JCR Listeners than there would be 1 task/listener to manage
the time series stats. These should be quick and non blocking though.
All other task are much more critical for repository to function
properly. Hence thoughts to go for #B where we have a dedicated pool
for those 'n' tasks. Where n is much small i.e. number of async lanes
+ 2 from DocumentNodeStore so far. So its easy to size
Chetan Mehrotra
Re: Specifying threadpool name for periodic scheduled jobs (OAK-4563)
Posted by Stefan Egli <st...@apache.org>.
I'd go for #A to limit cross-effects between oak and other layers.
The reason one would want to use the default pool for #4 is probably the
idea that you'd want to avoid "wasting" a thread in the oak-thread-pool
and rather rely on a shared one. But arguably, that should be an
optimization of the thread pool provider itself: that provider could be
more intelligent and allocate threads from an under-used pool elsewhere -
if that were more performant.
But from a logical point of view, I'd argue it's better to have an
oak-dedicated thread-pool.
Cheers,
Stefan
On 19/07/16 10:06, "Chetan Mehrotra" <ch...@gmail.com> wrote:
>On Tue, Jul 19, 2016 at 1:21 PM, Michael Dürig <md...@apache.org> wrote:
>> Not sure as I'm confused by your description of that option. I don't
>> understand which of 1, 2, 3 and 4 would run in the "default pool" and
>>which
>> should run in its own dedicated pool.
>
>#1, #2 and #3 would run in dedicated pool and each using same pool.
>Pool name would be 'oak'. Also see OAK-4563 for the patch
>
>While for #4 default pool would be used as those are non blocking and
>short tasks
>
>Chetan Mehrotra
Re: Specifying threadpool name for periodic scheduled jobs (OAK-4563)
Posted by Chetan Mehrotra <ch...@gmail.com>.
On Tue, Jul 19, 2016 at 1:21 PM, Michael Dürig <md...@apache.org> wrote:
> Not sure as I'm confused by your description of that option. I don't
> understand which of 1, 2, 3 and 4 would run in the "default pool" and which
> should run in its own dedicated pool.
#1, #2 and #3 would run in dedicated pool and each using same pool.
Pool name would be 'oak'. Also see OAK-4563 for the patch
While for #4 default pool would be used as those are non blocking and
short tasks
Chetan Mehrotra
Re: Specifying threadpool name for periodic scheduled jobs (OAK-4563)
Posted by Michael Dürig <md...@apache.org>.
On 19.7.16 9:26 , Chetan Mehrotra wrote:
> On Tue, Jul 19, 2016 at 12:54 PM, Michael D�rig <md...@apache.org> wrote:
>> For blocking or time intensive tasks I would go for a dedicated thread pool.
>
> So wrt current issue that means option #B ....?
Not sure as I'm confused by your description of that option. I don't
understand which of 1, 2, 3 and 4 would run in the "default pool" and
which should run in its own dedicated pool.
Michael
>
> Chetan Mehrotra
>
Re: Specifying threadpool name for periodic scheduled jobs (OAK-4563)
Posted by Chetan Mehrotra <ch...@gmail.com>.
On Tue, Jul 19, 2016 at 12:54 PM, Michael Dürig <md...@apache.org> wrote:
> For blocking or time intensive tasks I would go for a dedicated thread pool.
So wrt current issue that means option #B ....?
Chetan Mehrotra
Re: Specifying threadpool name for periodic scheduled jobs (OAK-4563)
Posted by Michael Dürig <md...@apache.org>.
On 19.7.16 6:55 , Chetan Mehrotra wrote:
> Now currently in Oak we use following types of periodic jobs
>
> 1. Async indexing - (Cluster Singleton)
> 2. Document Store - Journal GC (Cluster Singleton)
> 3. Document Store - LastRevRecovery
> 4. Statistic Collection - For timeseries data update in ChangeProcessor,
> SegmentNodeStore GCMonitor
>
> Now should we use
>
> A - one single pool for all of the above
> B - use the pool only for 1-3. The default pool would be of 5. So even
> if #2 #3 are running
> it would not hamper #1
>
> Assuming #4 is not that critical to run and may consist of lots of jobs.
>
> My suggestion would be to go for #B
In general I would use a common single thread pool for short,
non-blocking tasks. For blocking or time intensive tasks I would go for
a dedicated thread pool. Even better I wouldn't waste threads for
blocking tasks but structure the overall computation in a non blocking
way. This is a different topic though...
Michael
Re: Specifying threadpool name for periodic scheduled jobs (OAK-4563)
Posted by Vikas Saurabh <vi...@gmail.com>.
Hi,
> Now should we use
>
> A - one single pool for all of the above
> B - use the pool only for 1-3. The default pool would be of 5. So even
> if #2 #3 are running
> it would not hamper #1
While I'd want option#B to a better option, but I'd like to add one
quick bit - we'd need to also take care of that async indexing itself
could "potentially" require multiple tasks (lanes?? in async indexing
terminology).
While 5 threads would still take care of 3 async indexing lanes +
task#2 + task#3. But, I wonder, if we should further split as
async-index-pool (tasks for #1), background-jobs-pool (#2, #3, maybe
more??), #4 probably goes into default-pool.
Thanks,
Vikas
PS: the assumption is that async indexing task can "potentially" take
time (e.g. if there's binary extraction in the picture) + number of
async indexing lanes is configurable at repository creation time