You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@helix.apache.org by Kanak Biscuitwala <ka...@hotmail.com> on 2014/03/09 04:23:31 UTC
API Refactoring: Current Status
Hi,
Just as an update, I've spent some of today (and will spend some of tomorrow) improving the process for creating a resource. This involves significant changes to the RebalancerConfig and its builders. The goal is twofold: make the RebalancerConfig hierarchy much simpler and less confusing for both users and developers.
If this part of the code is done right, it should simplify the required work related to CRUD for resources.
Kanak
Re: API Refactoring: Current Status
Posted by Sandeep Nayak <os...@gmail.com>.
Thanks Kanak,
I pulled the latest and will start looking at the changes. I will send
a few questions I have while looking at the model classes which I will
send shortly.
Sandeep
On Sun, Mar 9, 2014 at 4:49 PM, Kanak Biscuitwala <ka...@hotmail.com> wrote:
> I just pushed the new RebalancerConfig code. It's still in the controller.rebalancer.config package. For the most part, the usage is not dramatically different from what it was before. Here's how it's different:
>
> 1. The spaghetti code dealing with going between IdealState and RebalancerConfig is gone now, making things much more extensible.
> 2. No dependence on Jackson anymore. Our default config types will always be written as IdealState, and the user-defined rebalancers can define their own serializers.
> 3. Read-only with builders. The builders can take a previous RebalancerConfig as input to support updates.
> 4. Removed a lot of code duplication and if statements. When we get the HelixAdministrator wired up, creating a resource should look relatively clean.
> 5. Allow for a default base builder, as suggested by Sandeep.
>
> Work left to do:
>
> 1. Get rid of AbstractRebalancerConfig. All of the methods there should be part of the API.
> 2. Validation on the builders. For instance, the resource ID should always be passed.
> 3. Integration with the rest of the API changes
>
> Creating a full auto rebalancer config looks like this:
>
> FullAutoRebalancerConfig rebalancerConfig =
> new FullAutoRebalancerConfig.Builder().withResourceId(resourceId).withReplicaCount(1)
> .withPartitionCount(8).withStateModelDefId(stateModelDef.getStateModelDefId()).build();
>
> OR
>
> RebalancerConfig rebalancerConfig =
> new BasicRebalancerConfig.Builder().withResourceId(resourceId).withReplicaCount(1)
> .withPartitionCount(8).withStateModelDefId(stateModelDef.getStateModelDefId())
> .withMode(RebalanceMode.FULL_AUTO).build();
>
> and both usages call the same methods underneath! When we decide which resource commands to support, we'll be able to wrap this code.
>
> Kanak
>
> ----------------------------------------
>> From: kanak.b@hotmail.com
>> To: dev@helix.apache.org
>> Subject: RE: API Refactoring: Current Status
>> Date: Sun, 9 Mar 2014 11:21:31 -0700
>>
>>
>> Hi Kishore,
>>
>> I will post it to the forked repo once I resolve all the compile errors.
>>
>> Kanak
>>
>> ----------------------------------------
>>> Date: Sun, 9 Mar 2014 00:09:40 -0800
>>> Subject: Re: API Refactoring: Current Status
>>> From: g.kishore@gmail.com
>>> To: dev@helix.apache.org
>>>
>>> Hi Kanak,
>>>
>>> Where can I look at the code.
>>>
>>> thanks,
>>> Kishore G
>>>
>>>
>>> On Sat, Mar 8, 2014 at 7:23 PM, Kanak Biscuitwala <ka...@hotmail.com>wrote:
>>>
>>>> Hi,
>>>>
>>>> Just as an update, I've spent some of today (and will spend some of
>>>> tomorrow) improving the process for creating a resource. This involves
>>>> significant changes to the RebalancerConfig and its builders. The goal is
>>>> twofold: make the RebalancerConfig hierarchy much simpler and less
>>>> confusing for both users and developers.
>>>>
>>>> If this part of the code is done right, it should simplify the required
>>>> work related to CRUD for resources.
>>>>
>>>> Kanak
>>
>
RE: API Refactoring: Current Status
Posted by Kanak Biscuitwala <ka...@hotmail.com>.
I just pushed the new RebalancerConfig code. It's still in the controller.rebalancer.config package. For the most part, the usage is not dramatically different from what it was before. Here's how it's different:
1. The spaghetti code dealing with going between IdealState and RebalancerConfig is gone now, making things much more extensible.
2. No dependence on Jackson anymore. Our default config types will always be written as IdealState, and the user-defined rebalancers can define their own serializers.
3. Read-only with builders. The builders can take a previous RebalancerConfig as input to support updates.
4. Removed a lot of code duplication and if statements. When we get the HelixAdministrator wired up, creating a resource should look relatively clean.
5. Allow for a default base builder, as suggested by Sandeep.
Work left to do:
1. Get rid of AbstractRebalancerConfig. All of the methods there should be part of the API.
2. Validation on the builders. For instance, the resource ID should always be passed.
3. Integration with the rest of the API changes
Creating a full auto rebalancer config looks like this:
FullAutoRebalancerConfig rebalancerConfig =
new FullAutoRebalancerConfig.Builder().withResourceId(resourceId).withReplicaCount(1)
.withPartitionCount(8).withStateModelDefId(stateModelDef.getStateModelDefId()).build();
OR
RebalancerConfig rebalancerConfig =
new BasicRebalancerConfig.Builder().withResourceId(resourceId).withReplicaCount(1)
.withPartitionCount(8).withStateModelDefId(stateModelDef.getStateModelDefId())
.withMode(RebalanceMode.FULL_AUTO).build();
and both usages call the same methods underneath! When we decide which resource commands to support, we'll be able to wrap this code.
Kanak
----------------------------------------
> From: kanak.b@hotmail.com
> To: dev@helix.apache.org
> Subject: RE: API Refactoring: Current Status
> Date: Sun, 9 Mar 2014 11:21:31 -0700
>
>
> Hi Kishore,
>
> I will post it to the forked repo once I resolve all the compile errors.
>
> Kanak
>
> ----------------------------------------
>> Date: Sun, 9 Mar 2014 00:09:40 -0800
>> Subject: Re: API Refactoring: Current Status
>> From: g.kishore@gmail.com
>> To: dev@helix.apache.org
>>
>> Hi Kanak,
>>
>> Where can I look at the code.
>>
>> thanks,
>> Kishore G
>>
>>
>> On Sat, Mar 8, 2014 at 7:23 PM, Kanak Biscuitwala <ka...@hotmail.com>wrote:
>>
>>> Hi,
>>>
>>> Just as an update, I've spent some of today (and will spend some of
>>> tomorrow) improving the process for creating a resource. This involves
>>> significant changes to the RebalancerConfig and its builders. The goal is
>>> twofold: make the RebalancerConfig hierarchy much simpler and less
>>> confusing for both users and developers.
>>>
>>> If this part of the code is done right, it should simplify the required
>>> work related to CRUD for resources.
>>>
>>> Kanak
>
RE: API Refactoring: Current Status
Posted by Kanak Biscuitwala <ka...@hotmail.com>.
Hi Kishore,
I will post it to the forked repo once I resolve all the compile errors.
Kanak
----------------------------------------
> Date: Sun, 9 Mar 2014 00:09:40 -0800
> Subject: Re: API Refactoring: Current Status
> From: g.kishore@gmail.com
> To: dev@helix.apache.org
>
> Hi Kanak,
>
> Where can I look at the code.
>
> thanks,
> Kishore G
>
>
> On Sat, Mar 8, 2014 at 7:23 PM, Kanak Biscuitwala <ka...@hotmail.com>wrote:
>
>> Hi,
>>
>> Just as an update, I've spent some of today (and will spend some of
>> tomorrow) improving the process for creating a resource. This involves
>> significant changes to the RebalancerConfig and its builders. The goal is
>> twofold: make the RebalancerConfig hierarchy much simpler and less
>> confusing for both users and developers.
>>
>> If this part of the code is done right, it should simplify the required
>> work related to CRUD for resources.
>>
>> Kanak
Re: API Refactoring: Current Status
Posted by kishore g <g....@gmail.com>.
Hi Kanak,
Where can I look at the code.
thanks,
Kishore G
On Sat, Mar 8, 2014 at 7:23 PM, Kanak Biscuitwala <ka...@hotmail.com>wrote:
> Hi,
>
> Just as an update, I've spent some of today (and will spend some of
> tomorrow) improving the process for creating a resource. This involves
> significant changes to the RebalancerConfig and its builders. The goal is
> twofold: make the RebalancerConfig hierarchy much simpler and less
> confusing for both users and developers.
>
> If this part of the code is done right, it should simplify the required
> work related to CRUD for resources.
>
> Kanak