You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@helix.apache.org by DImuthu Upeksha <di...@gmail.com> on 2019/11/25 18:13:31 UTC
Task is running periodically
Hi Folks,
We have noticed a task in Helix cluster was running periodically even
though it was completed at each run. When we look at the logs of the
controller, I can see some Concurrent Modification exceptions [1]. However
this is a very rare occurrence. We have been using current Helix version
for few months on our production deployments but this is the first time we
have seen this behavior.
Do you have any insight into this?
We are using helix 0.8.2
[1] https://gist.github.com/DImuthuUpe/33db7bbe5d53fcc38dfc66eb0d45df55
Thanks
Dimuthu
Re: Task is running periodically
Posted by DImuthu Upeksha <di...@gmail.com>.
Hi Lee,
I use default configurations for rebalancing. However is this patch
available on 0.9.1?
Thanks
Dimuthu
On Mon, Nov 25, 2019 at 2:19 PM Hunter Lee <na...@gmail.com> wrote:
> Hi Dimuthu,
>
> Are you using periodicalRebalance? That might trigger a rebalance
> periodically even if there's no activity in the cluster.
>
> As for the ConcurrentModification exception, I believe there was a patch
> for that going from 0.8.2 to the latest on the upstream.
>
> Hunter
>
> On Mon, Nov 25, 2019 at 10:13 AM DImuthu Upeksha <
> dimuthu.upeksha2@gmail.com> wrote:
>
>> Hi Folks,
>>
>> We have noticed a task in Helix cluster was running periodically even
>> though it was completed at each run. When we look at the logs of the
>> controller, I can see some Concurrent Modification exceptions [1]. However
>> this is a very rare occurrence. We have been using current Helix version
>> for few months on our production deployments but this is the first time we
>> have seen this behavior.
>>
>> Do you have any insight into this?
>>
>> We are using helix 0.8.2
>>
>> [1] https://gist.github.com/DImuthuUpe/33db7bbe5d53fcc38dfc66eb0d45df55
>>
>> Thanks
>> Dimuthu
>>
>
Re: Task is running periodically
Posted by DImuthu Upeksha <di...@gmail.com>.
Hi Lee,
I use default configurations for rebalancing. However is this patch
available on 0.9.1?
Thanks
Dimuthu
On Mon, Nov 25, 2019 at 2:19 PM Hunter Lee <na...@gmail.com> wrote:
> Hi Dimuthu,
>
> Are you using periodicalRebalance? That might trigger a rebalance
> periodically even if there's no activity in the cluster.
>
> As for the ConcurrentModification exception, I believe there was a patch
> for that going from 0.8.2 to the latest on the upstream.
>
> Hunter
>
> On Mon, Nov 25, 2019 at 10:13 AM DImuthu Upeksha <
> dimuthu.upeksha2@gmail.com> wrote:
>
>> Hi Folks,
>>
>> We have noticed a task in Helix cluster was running periodically even
>> though it was completed at each run. When we look at the logs of the
>> controller, I can see some Concurrent Modification exceptions [1]. However
>> this is a very rare occurrence. We have been using current Helix version
>> for few months on our production deployments but this is the first time we
>> have seen this behavior.
>>
>> Do you have any insight into this?
>>
>> We are using helix 0.8.2
>>
>> [1] https://gist.github.com/DImuthuUpe/33db7bbe5d53fcc38dfc66eb0d45df55
>>
>> Thanks
>> Dimuthu
>>
>
Re: Task is running periodically
Posted by Hunter Lee <na...@gmail.com>.
Hi Dimuthu,
Are you using periodicalRebalance? That might trigger a rebalance
periodically even if there's no activity in the cluster.
As for the ConcurrentModification exception, I believe there was a patch
for that going from 0.8.2 to the latest on the upstream.
Hunter
On Mon, Nov 25, 2019 at 10:13 AM DImuthu Upeksha <di...@gmail.com>
wrote:
> Hi Folks,
>
> We have noticed a task in Helix cluster was running periodically even
> though it was completed at each run. When we look at the logs of the
> controller, I can see some Concurrent Modification exceptions [1]. However
> this is a very rare occurrence. We have been using current Helix version
> for few months on our production deployments but this is the first time we
> have seen this behavior.
>
> Do you have any insight into this?
>
> We are using helix 0.8.2
>
> [1] https://gist.github.com/DImuthuUpe/33db7bbe5d53fcc38dfc66eb0d45df55
>
> Thanks
> Dimuthu
>
Re: Task is running periodically
Posted by Hunter Lee <na...@gmail.com>.
Hi Dimuthu,
Are you using periodicalRebalance? That might trigger a rebalance
periodically even if there's no activity in the cluster.
As for the ConcurrentModification exception, I believe there was a patch
for that going from 0.8.2 to the latest on the upstream.
Hunter
On Mon, Nov 25, 2019 at 10:13 AM DImuthu Upeksha <di...@gmail.com>
wrote:
> Hi Folks,
>
> We have noticed a task in Helix cluster was running periodically even
> though it was completed at each run. When we look at the logs of the
> controller, I can see some Concurrent Modification exceptions [1]. However
> this is a very rare occurrence. We have been using current Helix version
> for few months on our production deployments but this is the first time we
> have seen this behavior.
>
> Do you have any insight into this?
>
> We are using helix 0.8.2
>
> [1] https://gist.github.com/DImuthuUpe/33db7bbe5d53fcc38dfc66eb0d45df55
>
> Thanks
> Dimuthu
>