You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@river.apache.org by Patricia Shanahan <pa...@acm.org> on 2011/07/01 04:02:17 UTC

Re: TaskManager

I'm having trouble squaring this with the description of
ThreadPoolExecutor. If I'm reading it right, it *can* be initialized to
set a low limit on the maximum number of threads, but can equally well
be set to have up to Integer.MAX_VALUE threads, effectively unlimited.

There are two forms of TaskManager behavior that I do not see an easy
way to reproduce using a simple ThreadPoolExecutor, runAfter and loadLevel.

runAfter my be better handled by the class issuing the tasks.

loadLevel permits specification of a ratio between threads and tasks, so
that threads are only created up to e.g. one third of the number of
tasks. I'm not sure that is a really good idea. It means leaving tasks
to wait even if we could create another thread.

Patricia



On 6/29/2011 7:06 AM, Christopher Dolan wrote:
> Just remember, we discussed this a year ago
> (http://mail-archives.apache.org/mod_mbox/incubator-river-dev/201006.mbo
> x/%3C4C12147A.8030601@zeus.net.au%3E) and some of us concluded that
> ThreadPoolExecutor has rather non-intuitive runtime behavior because
> it's optimized to not cause OutOfMemory problems. Not everyone had a
> problem with that behavior, but I'm not a fan.
>
> Chris
>
> P.S. can someone add a link to the Incubator mail archives to the web
> page? They are quite hard to find...
>    http://mail-archives.apache.org/mod_mbox/incubator-river-dev/
>    http://mail-archives.apache.org/mod_mbox/incubator-river-user/
>    http://mail-archives.apache.org/mod_mbox/incubator-river-commits/
>
>
> -----Original Message-----
> From: Peter Firmstone [mailto:jini@zeus.net.au]
> Sent: Sunday, June 26, 2011 5:18 PM
> To: dev@river.apache.org
> Subject: Re: TaskManager
>
>
>> I think the way to look at this is to examine the non-trivial runAfter
>> implementations. If they have common elements that can be helped by
> some
>> utility classes, write those classes. If not, deal with each
>> individually.
>>
>> We are now committed to 1.5, so if we did not have runAfter we could
>> replace TaskManager with java.util.concurrent.ThreadPoolExecutor.
>
> That's probably where we should be headed, since it reduces our
> maintenance burden.
>
> Peter.
>


Re: TaskManager

Posted by Patricia Shanahan <pa...@acm.org>.
The different thread pool implementations allow different constructor 
parameters. Perhaps a factory pattern?

I think it would make run time selection of an alternative 
implementation easier, but make it harder for a class to specify 
parameters. On the other hand, I'm not sure how informed and meaningful 
the per-using-class parameters are.

Also, does it make sense to have separate pools, if we remove runAfter 
as a thread pool feature? Maybe we should have one pool for all River 
code running under a single JVM? That would improve thread sharing, and 
simplify the parameters.

(I would oppose combining pools with runAfter, because of O(n^2) behavior).

Patricia


On 7/5/2011 1:12 AM, Tom Hobbs wrote:
> Would it be possible t extract the whole thing into an interface?  Have the
> river default implementation something similar to what we have now (at least
> in behaviour if not implementation) but allow users to swap in their own
> task execution implementations that may or may not throttle in some way.
>
> Just a thought.
>
>
>
> Grammar and spelling have been sacrificed on the altar of messaging via
> mobile device.
>
> On 1 Jul 2011 21:29, "Patricia Shanahan"<pa...@acm.org>  wrote:
>> On 7/1/2011 1:14 PM, Gregg Wonderly wrote:
>> ...
>>> If the system throughput is not meeting my needs, I'll apply admission
>>> controls, or "add hardware" to manage the issue. Anything else done by
>>> river only serves as a "limit" that can be very frustrating to debug and
>>> deal with.
>>
>> I would like to make any River controls e.g. on the number of threads
>> for some purpose configurable. A site that does not want any River
>> controls would configure the upper limits on threads to be
>> Integer.MAX_VALUE.
>>
>> If we get rid of the runAfter issue, we could possibly reduce the number
>> of separate thread pools, making configuration simpler. runAfter, as
>> currently implemented, introduces some O(n*n) behavior, where n is the
>> number of incomplete tasks.
>>
>> Patricia
>


Re: TaskManager

Posted by Tom Hobbs <tv...@googlemail.com>.
Would it be possible t extract the whole thing into an interface?  Have the
river default implementation something similar to what we have now (at least
in behaviour if not implementation) but allow users to swap in their own
task execution implementations that may or may not throttle in some way.

Just a thought.



Grammar and spelling have been sacrificed on the altar of messaging via
mobile device.

On 1 Jul 2011 21:29, "Patricia Shanahan" <pa...@acm.org> wrote:
> On 7/1/2011 1:14 PM, Gregg Wonderly wrote:
> ...
>> If the system throughput is not meeting my needs, I'll apply admission
>> controls, or "add hardware" to manage the issue. Anything else done by
>> river only serves as a "limit" that can be very frustrating to debug and
>> deal with.
>
> I would like to make any River controls e.g. on the number of threads
> for some purpose configurable. A site that does not want any River
> controls would configure the upper limits on threads to be
> Integer.MAX_VALUE.
>
> If we get rid of the runAfter issue, we could possibly reduce the number
> of separate thread pools, making configuration simpler. runAfter, as
> currently implemented, introduces some O(n*n) behavior, where n is the
> number of incomplete tasks.
>
> Patricia

Re: TaskManager

Posted by Patricia Shanahan <pa...@acm.org>.
On 7/1/2011 1:14 PM, Gregg Wonderly wrote:
...
> If the system throughput is not meeting my needs, I'll apply admission
> controls, or "add hardware" to manage the issue. Anything else done by
> river only serves as a "limit" that can be very frustrating to debug and
> deal with.

I would like to make any River controls e.g. on the number of threads
for some purpose configurable. A site that does not want any River
controls would configure the upper limits on threads to be
Integer.MAX_VALUE.

If we get rid of the runAfter issue, we could possibly reduce the number
of separate thread pools, making configuration simpler. runAfter, as
currently implemented, introduces some O(n*n) behavior, where n is the
number of incomplete tasks.

Patricia

Re: TaskManager

Posted by Gregg Wonderly <gr...@wonderly.org>.
On 7/1/2011 2:44 AM, Patricia Shanahan wrote:
> On 6/30/2011 11:38 PM, Dan Creswell wrote:
>> On 1 July 2011 03:22, Gregg Wonderly<gr...@wonderly.org> wrote:
>>> On 6/30/2011 9:02 PM, Patricia Shanahan wrote:
> ...
>>>> loadLevel permits specification of a ratio between threads and tasks, so
>>>> that threads are only created up to e.g. one third of the number of
>>>> tasks. I'm not sure that is a really good idea. It means leaving tasks
>>>> to wait even if we could create another thread.
>>>
>>
>> Yes, but tasks are waiting because you're choosing to limit resources.
>> That always happens. The choices are to increase the limit and utilise
>> more processor (assuming you have it) or have fewer tasks (often a
>> function of user-load so needs admission control etc to make that
>> stick).
>>
>>> I don't like this thought at all in a distributed system. A vast majority
>>> of tasks really need to be invoked immediately to maintain a reasonable
>>> chance at throughput.
>>>
>>
>> Equally a vast majority of tasks can only have a reasonable chance of
>> throughput in the presence of sufficient resource. The point of
>> TaskManager is to control resource consumption (including processor
>> utilisation) which naturally asserts a limit on throughput.
> ...
>
> Some resource limitation is definitely necessary, and supported by both
> TaskManager and ThreadPoolExecutor. In either implementation one can say
> "Use no more than 30 threads for this purpose.".
>
> That makes sense to me. Running too many threads increases resource
> utilization without increasing throughput. Even before that limit is
> reached, the need to share the system resources with other pools may
> limit the number of threads that can be allowed for one pool.
>
> The additional type of limitation that TaskManager supports and
> ThreadPoolExecutor does not is much weirder than that. TaskManager
> allows specification of a "load level" which is a ratio between the
> number of runnable tasks and the number of threads. For example, if the
> ratio is 3, then with 30 tasks it is limited to 10 threads. With 300
> tasks it is limited to 100 threads.
>
> If the system could efficiently accommodate 100 threads devoted to that
> pool, why not create 30 of them when there are 30 runnable tasks that
> could use them, rather than limiting the 30 tasks to 10 threads, keeping
> some of them waiting?

In a distributed system where users are just throwing requests into a system of 
"tasks", you can, as Dan says, have some admission control, because you no where 
the "non-system-dependency" control point is at.

When you are looking at the implementation of a distributed system, having 
"thread" controls or "resource limits" results in very difficult to understand 
failure modes.  Especially where there are no "timeouts" imposed or other "wait 
limits" available.  This is what I'm complaining about.  Let the designer of the 
system manage the "load" at the point of work insertion.  River's internals do 
nobody any favors by restricting "how many" or "when" things can happen.

If the system throughput is not meeting my needs, I'll apply admission controls, 
or "add hardware" to manage the issue.  Anything else done by river only serves 
as a "limit" that can be very frustrating to debug and deal with.

Gregg Wonderly




Re: TaskManager

Posted by Dan Creswell <da...@gmail.com>.
Inline....

On 1 July 2011 08:44, Patricia Shanahan <pa...@acm.org> wrote:
> On 6/30/2011 11:38 PM, Dan Creswell wrote:
>>
>> On 1 July 2011 03:22, Gregg Wonderly<gr...@wonderly.org>  wrote:
>>>
>>> On 6/30/2011 9:02 PM, Patricia Shanahan wrote:
>
> ...
>>>>
>>>> loadLevel permits specification of a ratio between threads and tasks, so
>>>> that threads are only created up to e.g. one third of the number of
>>>> tasks. I'm not sure that is a really good idea. It means leaving tasks
>>>> to wait even if we could create another thread.
>>>
>>
>> Yes, but tasks are waiting because you're choosing to limit resources.
>> That always happens. The choices are to increase the limit and utilise
>> more processor (assuming you have it) or have fewer tasks (often a
>> function of user-load so needs admission control etc to make that
>> stick).
>>
>>> I don't like this thought at all in a distributed system.  A vast
>>> majority
>>> of tasks really need to be invoked immediately to maintain a reasonable
>>> chance at throughput.
>>>
>>
>> Equally a vast majority of tasks can only have a reasonable chance of
>> throughput in the presence of sufficient resource. The point of
>> TaskManager is to control resource consumption (including processor
>> utilisation) which naturally asserts a limit on throughput.
>
> ...
>
> Some resource limitation is definitely necessary, and supported by both
> TaskManager and ThreadPoolExecutor. In either implementation one can say
> "Use no more than 30 threads for this purpose.".
>
> That makes sense to me. Running too many threads increases resource
> utilization without increasing throughput. Even before that limit is
> reached, the need to share the system resources with other pools may
> limit the number of threads that can be allowed for one pool.
>
> The additional type of limitation that TaskManager supports and
> ThreadPoolExecutor does not is much weirder than that. TaskManager
> allows specification of a "load level" which is a ratio between the
> number of runnable tasks and the number of threads. For example, if the
> ratio is 3, then with 30 tasks it is limited to 10 threads. With 300
> tasks it is limited to 100 threads.
>
> If the system could efficiently accommodate 100 threads devoted to that
> pool, why not create 30 of them when there are 30 runnable tasks that
> could use them, rather than limiting the 30 tasks to 10 threads, keeping
> some of them waiting?

I can't recall specific discussion about this but I would imagine this
could be an attempt at handling context switching costs. Basically if
task execution time is small relative to context switch cost
"clamping" the number of threads down to some ratio of tasks might
make sense. It's overall faster for those tasks to wait just a little
in a queue whilst an already running thread comes back around to pick
them up than it is to context switch out a thread with a task, pull
another in (and start it up) and start dispatching that new task.

>
> Patricia
>

Re: TaskManager

Posted by Patricia Shanahan <pa...@acm.org>.
On 6/30/2011 11:38 PM, Dan Creswell wrote:
> On 1 July 2011 03:22, Gregg Wonderly<gr...@wonderly.org>  wrote:
>> On 6/30/2011 9:02 PM, Patricia Shanahan wrote:
...
>>> loadLevel permits specification of a ratio between threads and tasks, so
>>> that threads are only created up to e.g. one third of the number of
>>> tasks. I'm not sure that is a really good idea. It means leaving tasks
>>> to wait even if we could create another thread.
>>
>
> Yes, but tasks are waiting because you're choosing to limit resources.
> That always happens. The choices are to increase the limit and utilise
> more processor (assuming you have it) or have fewer tasks (often a
> function of user-load so needs admission control etc to make that
> stick).
>
>> I don't like this thought at all in a distributed system.  A vast majority
>> of tasks really need to be invoked immediately to maintain a reasonable
>> chance at throughput.
>>
>
> Equally a vast majority of tasks can only have a reasonable chance of
> throughput in the presence of sufficient resource. The point of
> TaskManager is to control resource consumption (including processor
> utilisation) which naturally asserts a limit on throughput.
...

Some resource limitation is definitely necessary, and supported by both
TaskManager and ThreadPoolExecutor. In either implementation one can say
"Use no more than 30 threads for this purpose.".

That makes sense to me. Running too many threads increases resource
utilization without increasing throughput. Even before that limit is
reached, the need to share the system resources with other pools may
limit the number of threads that can be allowed for one pool.

The additional type of limitation that TaskManager supports and
ThreadPoolExecutor does not is much weirder than that. TaskManager
allows specification of a "load level" which is a ratio between the
number of runnable tasks and the number of threads. For example, if the
ratio is 3, then with 30 tasks it is limited to 10 threads. With 300
tasks it is limited to 100 threads.

If the system could efficiently accommodate 100 threads devoted to that
pool, why not create 30 of them when there are 30 runnable tasks that
could use them, rather than limiting the 30 tasks to 10 threads, keeping
some of them waiting?

Patricia

Re: TaskManager

Posted by Dan Creswell <da...@gmail.com>.
On 1 July 2011 03:22, Gregg Wonderly <gr...@wonderly.org> wrote:
> On 6/30/2011 9:02 PM, Patricia Shanahan wrote:
>>
>> I'm having trouble squaring this with the description of
>> ThreadPoolExecutor. If I'm reading it right, it *can* be initialized to
>> set a low limit on the maximum number of threads, but can equally well
>> be set to have up to Integer.MAX_VALUE threads, effectively unlimited.
>
> Yes, this is the case.  There are some other subtle behaviors which we just
> need to study to make sure that the "documented behavior" is what we need.
>
>> There are two forms of TaskManager behavior that I do not see an easy
>> way to reproduce using a simple ThreadPoolExecutor, runAfter and
>> loadLevel.
>>
>> runAfter my be better handled by the class issuing the tasks.
>
> There is also the Fork-Join framework which does allow some "arrangement" of
> order.  I'm not sure this is a great fit right now, but it is something to
> consider I think, especially for "notifications", "commits" and some other
> things that have two level ordering "function->tasks".
>
>> loadLevel permits specification of a ratio between threads and tasks, so
>> that threads are only created up to e.g. one third of the number of
>> tasks. I'm not sure that is a really good idea. It means leaving tasks
>> to wait even if we could create another thread.
>

Yes, but tasks are waiting because you're choosing to limit resources.
That always happens. The choices are to increase the limit and utilise
more processor (assuming you have it) or have fewer tasks (often a
function of user-load so needs admission control etc to make that
stick).

> I don't like this thought at all in a distributed system.  A vast majority
> of tasks really need to be invoked immediately to maintain a reasonable
> chance at throughput.
>

Equally a vast majority of tasks can only have a reasonable chance of
throughput in the presence of sufficient resource. The point of
TaskManager is to control resource consumption (including processor
utilisation) which naturally asserts a limit on throughput.

> Gregg
>
>> Patricia
>>
>>
>>
>> On 6/29/2011 7:06 AM, Christopher Dolan wrote:
>>>
>>> Just remember, we discussed this a year ago
>>> (http://mail-archives.apache.org/mod_mbox/incubator-river-dev/201006.mbo
>>> x/%3C4C12147A.8030601@zeus.net.au%3E) and some of us concluded that
>>> ThreadPoolExecutor has rather non-intuitive runtime behavior because
>>> it's optimized to not cause OutOfMemory problems. Not everyone had a
>>> problem with that behavior, but I'm not a fan.
>>>
>>> Chris
>>>
>>> P.S. can someone add a link to the Incubator mail archives to the web
>>> page? They are quite hard to find...
>>> http://mail-archives.apache.org/mod_mbox/incubator-river-dev/
>>> http://mail-archives.apache.org/mod_mbox/incubator-river-user/
>>> http://mail-archives.apache.org/mod_mbox/incubator-river-commits/
>>>
>>>
>>> -----Original Message-----
>>> From: Peter Firmstone [mailto:jini@zeus.net.au]
>>> Sent: Sunday, June 26, 2011 5:18 PM
>>> To: dev@river.apache.org
>>> Subject: Re: TaskManager
>>>
>>>
>>>> I think the way to look at this is to examine the non-trivial runAfter
>>>> implementations. If they have common elements that can be helped by
>>>
>>> some
>>>>
>>>> utility classes, write those classes. If not, deal with each
>>>> individually.
>>>>
>>>> We are now committed to 1.5, so if we did not have runAfter we could
>>>> replace TaskManager with java.util.concurrent.ThreadPoolExecutor.
>>>
>>> That's probably where we should be headed, since it reduces our
>>> maintenance burden.
>>>
>>> Peter.
>>>
>>
>>
>
>

Re: TaskManager

Posted by Gregg Wonderly <gr...@wonderly.org>.
On 6/30/2011 9:02 PM, Patricia Shanahan wrote:
> I'm having trouble squaring this with the description of
> ThreadPoolExecutor. If I'm reading it right, it *can* be initialized to
> set a low limit on the maximum number of threads, but can equally well
> be set to have up to Integer.MAX_VALUE threads, effectively unlimited.

Yes, this is the case.  There are some other subtle behaviors which we just need 
to study to make sure that the "documented behavior" is what we need.

> There are two forms of TaskManager behavior that I do not see an easy
> way to reproduce using a simple ThreadPoolExecutor, runAfter and loadLevel.
>
> runAfter my be better handled by the class issuing the tasks.

There is also the Fork-Join framework which does allow some "arrangement" of 
order.  I'm not sure this is a great fit right now, but it is something to 
consider I think, especially for "notifications", "commits" and some other 
things that have two level ordering "function->tasks".

> loadLevel permits specification of a ratio between threads and tasks, so
> that threads are only created up to e.g. one third of the number of
> tasks. I'm not sure that is a really good idea. It means leaving tasks
> to wait even if we could create another thread.

I don't like this thought at all in a distributed system.  A vast majority of 
tasks really need to be invoked immediately to maintain a reasonable chance at 
throughput.

Gregg

> Patricia
>
>
>
> On 6/29/2011 7:06 AM, Christopher Dolan wrote:
>> Just remember, we discussed this a year ago
>> (http://mail-archives.apache.org/mod_mbox/incubator-river-dev/201006.mbo
>> x/%3C4C12147A.8030601@zeus.net.au%3E) and some of us concluded that
>> ThreadPoolExecutor has rather non-intuitive runtime behavior because
>> it's optimized to not cause OutOfMemory problems. Not everyone had a
>> problem with that behavior, but I'm not a fan.
>>
>> Chris
>>
>> P.S. can someone add a link to the Incubator mail archives to the web
>> page? They are quite hard to find...
>> http://mail-archives.apache.org/mod_mbox/incubator-river-dev/
>> http://mail-archives.apache.org/mod_mbox/incubator-river-user/
>> http://mail-archives.apache.org/mod_mbox/incubator-river-commits/
>>
>>
>> -----Original Message-----
>> From: Peter Firmstone [mailto:jini@zeus.net.au]
>> Sent: Sunday, June 26, 2011 5:18 PM
>> To: dev@river.apache.org
>> Subject: Re: TaskManager
>>
>>
>>> I think the way to look at this is to examine the non-trivial runAfter
>>> implementations. If they have common elements that can be helped by
>> some
>>> utility classes, write those classes. If not, deal with each
>>> individually.
>>>
>>> We are now committed to 1.5, so if we did not have runAfter we could
>>> replace TaskManager with java.util.concurrent.ThreadPoolExecutor.
>>
>> That's probably where we should be headed, since it reduces our
>> maintenance burden.
>>
>> Peter.
>>
>
>