You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@felix.apache.org by Bob Paulin <bo...@bobpaulin.com> on 2014/08/28 15:00:46 UTC
Event Admin Tuning the Async Thread Pool
Hi,
I'd like to propose adding an additional configuration variable to Event
Admin that controls the sync to async thread ratio. Currently the code
sets an intelligent default that sets the async threads at 50% of the
sync threads as long as there are more than 5 total sync threads.
final int asyncThreadPoolSize = m_threadPoolSize > 5 ? m_threadPoolSize
/ 2 : 2;
This assumes an even split between post and send operations. However
this is not the case with many applications leaning heavily one way or
the other. I'd like to replace the above code with:
final int asyncThreadPoolSize = m_threadPoolSize > 5 ?
Math.floor(m_threadPoolSize * m_asyncToSyncThreadRatio) : 2;
With m_asyncToSyncThreadRatio defaulting to 0.5. This leaves the
intelligent defaults but allows the admin to tune the application based
on the application usage. This tuning can show significant performance
improvements. From the EventAdmin StressTestIT:
Scenerio 100% Async events
Events sent 1050000
Producer Threads 15
@ 50% async to sync ratio
PAXEXAM-PROBE-8762bbe7-91d9-4ff1-8562-38096af676da[org.apache.felix.eventadmin.ittests.StressTestIT]
: Finished tests, received 1050000 events in 6551ms
@ 100% async to sync ratio
PAXEXAM-PROBE-02e70ff6-36ff-4bed-ab00-f2bcb9440109[org.apache.felix.eventadmin.ittests.StressTestIT]
: Finished tests, received 1050000 events in 4871ms
Thoughts? Happy to submit a patch if there are no concerns.
- Bob Paulin
Re: Event Admin Tuning the Async Thread Pool
Posted by Carsten Ziegeler <cz...@apache.org>.
Hi Bob,
yes, the reason for the two pools is to actually avoid the deadlock
situation you mention.
Carsten
2014-08-28 16:32 GMT+02:00 Bob Paulin <bo...@bobpaulin.com>:
> Carsten,
>
> I agree self-tuning single threadpool would be a very interesting
> implementation. We should be able to adjust the maximum threads for a pool
> over a given interval. I think making it a single thread pool would be
> the tricky part since I think we could deadlock if the entire active
> threadpool was filled with async threads. But perhaps a timeout could be
> set to ensure that some of them get kicked out after a given amount of
> time. I'll do some more research on that but in the mean time I'll submit
> a jira for the config option.
>
> - Bob
>
> On 8/28/2014 8:25 AM, Carsten Ziegeler wrote:
>
>> Hi Bob,
>>
>> this sounds good to me, however it would be even better if the event admin
>> could find out the ratio and adjust it accordingly. I'm not against making
>> this configurable, but it might be hard to know the good value upfront.
>>
>> As a general node, it would be great, if we could simply have a single
>> thread pool :)
>>
>> But +1 for a patch in any case.
>>
>> Carsten
>>
>>
>> 2014-08-28 15:00 GMT+02:00 Bob Paulin <bo...@bobpaulin.com>:
>>
>> Hi,
>>>
>>> I'd like to propose adding an additional configuration variable to Event
>>> Admin that controls the sync to async thread ratio. Currently the code
>>> sets an intelligent default that sets the async threads at 50% of the
>>> sync
>>> threads as long as there are more than 5 total sync threads.
>>>
>>> final int asyncThreadPoolSize = m_threadPoolSize > 5 ? m_threadPoolSize /
>>> 2 : 2;
>>>
>>> This assumes an even split between post and send operations. However this
>>> is not the case with many applications leaning heavily one way or the
>>> other. I'd like to replace the above code with:
>>>
>>> final int asyncThreadPoolSize = m_threadPoolSize > 5 ?
>>> Math.floor(m_threadPoolSize * m_asyncToSyncThreadRatio) : 2;
>>>
>>> With m_asyncToSyncThreadRatio defaulting to 0.5. This leaves the
>>> intelligent defaults but allows the admin to tune the application based
>>> on
>>> the application usage. This tuning can show significant performance
>>> improvements. From the EventAdmin StressTestIT:
>>>
>>> Scenerio 100% Async events
>>> Events sent 1050000
>>> Producer Threads 15
>>>
>>>
>>> @ 50% async to sync ratio
>>> PAXEXAM-PROBE-8762bbe7-91d9-4ff1-8562-38096af676da[org.
>>> apache.felix.eventadmin.ittests.StressTestIT] : Finished tests, received
>>> 1050000 events in 6551ms
>>>
>>> @ 100% async to sync ratio
>>> PAXEXAM-PROBE-02e70ff6-36ff-4bed-ab00-f2bcb9440109[org.
>>> apache.felix.eventadmin.ittests.StressTestIT] : Finished tests, received
>>> 1050000 events in 4871ms
>>>
>>> Thoughts? Happy to submit a patch if there are no concerns.
>>>
>>> - Bob Paulin
>>>
>>>
>>
>>
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org
Re: Event Admin Tuning the Async Thread Pool
Posted by Bob Paulin <bo...@bobpaulin.com>.
Carsten,
I agree self-tuning single threadpool would be a very interesting
implementation. We should be able to adjust the maximum threads for a
pool over a given interval. I think making it a single thread pool
would be the tricky part since I think we could deadlock if the entire
active threadpool was filled with async threads. But perhaps a timeout
could be set to ensure that some of them get kicked out after a given
amount of time. I'll do some more research on that but in the mean time
I'll submit a jira for the config option.
- Bob
On 8/28/2014 8:25 AM, Carsten Ziegeler wrote:
> Hi Bob,
>
> this sounds good to me, however it would be even better if the event admin
> could find out the ratio and adjust it accordingly. I'm not against making
> this configurable, but it might be hard to know the good value upfront.
>
> As a general node, it would be great, if we could simply have a single
> thread pool :)
>
> But +1 for a patch in any case.
>
> Carsten
>
>
> 2014-08-28 15:00 GMT+02:00 Bob Paulin <bo...@bobpaulin.com>:
>
>> Hi,
>>
>> I'd like to propose adding an additional configuration variable to Event
>> Admin that controls the sync to async thread ratio. Currently the code
>> sets an intelligent default that sets the async threads at 50% of the sync
>> threads as long as there are more than 5 total sync threads.
>>
>> final int asyncThreadPoolSize = m_threadPoolSize > 5 ? m_threadPoolSize /
>> 2 : 2;
>>
>> This assumes an even split between post and send operations. However this
>> is not the case with many applications leaning heavily one way or the
>> other. I'd like to replace the above code with:
>>
>> final int asyncThreadPoolSize = m_threadPoolSize > 5 ?
>> Math.floor(m_threadPoolSize * m_asyncToSyncThreadRatio) : 2;
>>
>> With m_asyncToSyncThreadRatio defaulting to 0.5. This leaves the
>> intelligent defaults but allows the admin to tune the application based on
>> the application usage. This tuning can show significant performance
>> improvements. From the EventAdmin StressTestIT:
>>
>> Scenerio 100% Async events
>> Events sent 1050000
>> Producer Threads 15
>>
>>
>> @ 50% async to sync ratio
>> PAXEXAM-PROBE-8762bbe7-91d9-4ff1-8562-38096af676da[org.
>> apache.felix.eventadmin.ittests.StressTestIT] : Finished tests, received
>> 1050000 events in 6551ms
>>
>> @ 100% async to sync ratio
>> PAXEXAM-PROBE-02e70ff6-36ff-4bed-ab00-f2bcb9440109[org.
>> apache.felix.eventadmin.ittests.StressTestIT] : Finished tests, received
>> 1050000 events in 4871ms
>>
>> Thoughts? Happy to submit a patch if there are no concerns.
>>
>> - Bob Paulin
>>
>
>
Re: Event Admin Tuning the Async Thread Pool
Posted by Carsten Ziegeler <cz...@apache.org>.
Hi Bob,
this sounds good to me, however it would be even better if the event admin
could find out the ratio and adjust it accordingly. I'm not against making
this configurable, but it might be hard to know the good value upfront.
As a general node, it would be great, if we could simply have a single
thread pool :)
But +1 for a patch in any case.
Carsten
2014-08-28 15:00 GMT+02:00 Bob Paulin <bo...@bobpaulin.com>:
> Hi,
>
> I'd like to propose adding an additional configuration variable to Event
> Admin that controls the sync to async thread ratio. Currently the code
> sets an intelligent default that sets the async threads at 50% of the sync
> threads as long as there are more than 5 total sync threads.
>
> final int asyncThreadPoolSize = m_threadPoolSize > 5 ? m_threadPoolSize /
> 2 : 2;
>
> This assumes an even split between post and send operations. However this
> is not the case with many applications leaning heavily one way or the
> other. I'd like to replace the above code with:
>
> final int asyncThreadPoolSize = m_threadPoolSize > 5 ?
> Math.floor(m_threadPoolSize * m_asyncToSyncThreadRatio) : 2;
>
> With m_asyncToSyncThreadRatio defaulting to 0.5. This leaves the
> intelligent defaults but allows the admin to tune the application based on
> the application usage. This tuning can show significant performance
> improvements. From the EventAdmin StressTestIT:
>
> Scenerio 100% Async events
> Events sent 1050000
> Producer Threads 15
>
>
> @ 50% async to sync ratio
> PAXEXAM-PROBE-8762bbe7-91d9-4ff1-8562-38096af676da[org.
> apache.felix.eventadmin.ittests.StressTestIT] : Finished tests, received
> 1050000 events in 6551ms
>
> @ 100% async to sync ratio
> PAXEXAM-PROBE-02e70ff6-36ff-4bed-ab00-f2bcb9440109[org.
> apache.felix.eventadmin.ittests.StressTestIT] : Finished tests, received
> 1050000 events in 4871ms
>
> Thoughts? Happy to submit a patch if there are no concerns.
>
> - Bob Paulin
>
--
Carsten Ziegeler
Adobe Research Switzerland
cziegeler@apache.org