You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@helix.apache.org by Junkai Xue <ju...@gmail.com> on 2022/02/03 21:34:06 UTC

Re: Dynamically increasing partition count for a resource

Hi Rishabh,

You can read the IdealState by HelixAdmin getResourceIdealState() function
and change the number of partitions in the IdealState. Once you are done
with it, you can set it back by setResourceIdealState() function in
HelixAdmin.

BTW, the number of partition changes does not work for Custom mode and SEMI
auto.

Best,

Junkai

On Mon, Jan 31, 2022 at 5:23 AM Rishabh Goyal <rg...@gmail.com> wrote:

> Hi All,
>
> I am new to Apache helix. My use case requires me to dynamically change
> partition count of a resource (count can be increased/decreased depending
> upon the requirement).
>
> I am not able to figure out how to do this through the HelixAdmin
> interface except for deletion and subsequent addition of the same resource.
> However that would lead to removal of all active subscriptions which is not
> something I can afford to do.
>
> Is there any way to change the setting dynamically for a resource ?
>
> Regards,
> Rishabh
>


-- 
Junkai Xue

Re: Dynamically increasing partition count for a resource

Posted by Junkai Xue <ju...@gmail.com>.
I guess you are using SEMI AUTO mode? Then you need to call rebalance()
function in HelixAdmin again if that's the case.

Best,

Junkai

On Wed, Feb 9, 2022 at 7:47 AM Rishabh Goyal <rg...@gmail.com> wrote:

> Hi Junkai,
>
> Thanks for prompt reply. I tried using the above mentioned method. I can
> see the partition count has increased but the list of partitions have not
> been updated. I tried updating partition count for one of my resources to
> 21 from 20 but _20th partition did not get created.
>
> Then I tried using the rebalance API provided by helixAdmin. Once
> rebalance was successful, I can see the new partition being created. Is
> this expected ?
>
> Regards,
> Rishabh
>
> On Fri, 4 Feb 2022 at 03:04, Junkai Xue <ju...@gmail.com> wrote:
>
>> Hi Rishabh,
>>
>> You can read the IdealState by HelixAdmin getResourceIdealState()
>> function and change the number of partitions in the IdealState. Once you
>> are done with it, you can set it back by setResourceIdealState() function
>> in HelixAdmin.
>>
>> BTW, the number of partition changes does not work for Custom mode and
>> SEMI auto.
>>
>> Best,
>>
>> Junkai
>>
>> On Mon, Jan 31, 2022 at 5:23 AM Rishabh Goyal <rg...@gmail.com>
>> wrote:
>>
>>> Hi All,
>>>
>>> I am new to Apache helix. My use case requires me to dynamically change
>>> partition count of a resource (count can be increased/decreased depending
>>> upon the requirement).
>>>
>>> I am not able to figure out how to do this through the HelixAdmin
>>> interface except for deletion and subsequent addition of the same resource.
>>> However that would lead to removal of all active subscriptions which is not
>>> something I can afford to do.
>>>
>>> Is there any way to change the setting dynamically for a resource ?
>>>
>>> Regards,
>>> Rishabh
>>>
>>
>>
>> --
>> Junkai Xue
>>
>

-- 
Junkai Xue

Re: Dynamically increasing partition count for a resource

Posted by Rishabh Goyal <rg...@gmail.com>.
Hi Junkai,

Thanks for prompt reply. I tried using the above mentioned method. I can
see the partition count has increased but the list of partitions have not
been updated. I tried updating partition count for one of my resources to
21 from 20 but _20th partition did not get created.

Then I tried using the rebalance API provided by helixAdmin. Once rebalance
was successful, I can see the new partition being created. Is this expected
?

Regards,
Rishabh

On Fri, 4 Feb 2022 at 03:04, Junkai Xue <ju...@gmail.com> wrote:

> Hi Rishabh,
>
> You can read the IdealState by HelixAdmin getResourceIdealState() function
> and change the number of partitions in the IdealState. Once you are done
> with it, you can set it back by setResourceIdealState() function in
> HelixAdmin.
>
> BTW, the number of partition changes does not work for Custom mode and
> SEMI auto.
>
> Best,
>
> Junkai
>
> On Mon, Jan 31, 2022 at 5:23 AM Rishabh Goyal <rg...@gmail.com>
> wrote:
>
>> Hi All,
>>
>> I am new to Apache helix. My use case requires me to dynamically change
>> partition count of a resource (count can be increased/decreased depending
>> upon the requirement).
>>
>> I am not able to figure out how to do this through the HelixAdmin
>> interface except for deletion and subsequent addition of the same resource.
>> However that would lead to removal of all active subscriptions which is not
>> something I can afford to do.
>>
>> Is there any way to change the setting dynamically for a resource ?
>>
>> Regards,
>> Rishabh
>>
>
>
> --
> Junkai Xue
>