You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@helix.apache.org by Bo Liu <ne...@gmail.com> on 2018/07/16 18:26:48 UTC
Does participant create new thread to handle state transition messages?
Hi Helix team,
We observe that the number of newly created threads increases when there
are state transition messages to be processed, which has perf impact to
online traffic. Wanted to rule out if this is from the Participant internal
thread management.
--
Best regards,
Bo
Re: Does participant create new thread to handle state transition messages?
Posted by Lei Xia <lx...@apache.org>.
Hi, Bo
batchMessageExecutorService is used only when batchMessage is enabled,
which by default is not enabled unless you have explicitly set
"BATCH_MESSAGE_MODE" to true in either cluster config or ideal state. So
it should not create/terminate threads if it is idle. Do you have
thread-dump to show what thread were kept created?
Lei
On Mon, Jul 16, 2018 at 2:09 PM, Bo Liu <ne...@gmail.com> wrote:
> Thanks Lei.
>
> all the thread pools in _executorMap are fixed thread pools, which are
> unlikely to create threads after initialization.
>
> _batchMessageExecutorService looks a little suspicious, as it is a cached
> thread pool, which could terminate and create threads on the fly. However,
> I am not sure if it is used for state transitioning for OnlineOffline model?
>
>
>
> On Mon, Jul 16, 2018 at 12:33 PM Lei Xia <lx...@apache.org> wrote:
>
>> Hi, Bo
>>
>> Helix participant creates a thread-pool to handle the state transition
>> by default, and the application can supply its own thread-pool for specific
>> state-transition too. The default thread-pool size is 40, which is
>> configurable.
>>
>>
>> Lei
>>
>> On Mon, Jul 16, 2018 at 11:26 AM, Bo Liu <ne...@gmail.com> wrote:
>>
>>> Hi Helix team,
>>>
>>> We observe that the number of newly created threads increases when there
>>> are state transition messages to be processed, which has perf impact to
>>> online traffic. Wanted to rule out if this is from the Participant internal
>>> thread management.
>>>
>>> --
>>> Best regards,
>>> Bo
>>>
>>>
>>
>
> --
> Best regards,
> Bo
>
>
Re: Does participant create new thread to handle state transition messages?
Posted by Bo Liu <ne...@gmail.com>.
Thanks Lei.
all the thread pools in _executorMap are fixed thread pools, which are
unlikely to create threads after initialization.
_batchMessageExecutorService looks a little suspicious, as it is a cached
thread pool, which could terminate and create threads on the fly. However,
I am not sure if it is used for state transitioning for OnlineOffline model?
On Mon, Jul 16, 2018 at 12:33 PM Lei Xia <lx...@apache.org> wrote:
> Hi, Bo
>
> Helix participant creates a thread-pool to handle the state transition
> by default, and the application can supply its own thread-pool for specific
> state-transition too. The default thread-pool size is 40, which is
> configurable.
>
>
> Lei
>
> On Mon, Jul 16, 2018 at 11:26 AM, Bo Liu <ne...@gmail.com> wrote:
>
>> Hi Helix team,
>>
>> We observe that the number of newly created threads increases when there
>> are state transition messages to be processed, which has perf impact to
>> online traffic. Wanted to rule out if this is from the Participant internal
>> thread management.
>>
>> --
>> Best regards,
>> Bo
>>
>>
>
--
Best regards,
Bo
Re: Does participant create new thread to handle state transition messages?
Posted by Lei Xia <lx...@apache.org>.
Hi, Bo
Helix participant creates a thread-pool to handle the state transition
by default, and the application can supply its own thread-pool for specific
state-transition too. The default thread-pool size is 40, which is
configurable.
Lei
On Mon, Jul 16, 2018 at 11:26 AM, Bo Liu <ne...@gmail.com> wrote:
> Hi Helix team,
>
> We observe that the number of newly created threads increases when there
> are state transition messages to be processed, which has perf impact to
> online traffic. Wanted to rule out if this is from the Participant internal
> thread management.
>
> --
> Best regards,
> Bo
>
>