You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@helix.apache.org by Hahn Jiang <ha...@gmail.com> on 2015/07/28 11:26:20 UTC

Helix newer use Rebalance

I use Helix 0.7.1
I start three participants and I want to assign a resource to two of them.
    List<String> preferenceList = new ArrayList<>();
    preferenceList.add("localhost_6101");
    preferenceList.add("localhost_6102");

Then I add preferenceList to resource: partitionNum=6  replicationNum=2
    admin.addResource(clusterName, resourceKey,
partitionNum,StateModelDefId.MasterSlave.stringify(), "SEMI_AUTO");
    IdealState idealState = admin.getResourceIdealState(clusterName,
resourceKey);
    idealState.setRebalanceMode(IdealState.RebalanceMode.SEMI_AUTO);
    idealState.setReplicas(replicationNum+"");
    for (int i = 0; i < partitionNum; ++i) {
      idealState.setPreferenceList(resourceKey + "_" + i, preferenceList);
    }
    admin.setResourceIdealState(clusterName, resourceKey, idealState);

At Last, I will rebalence cluster:
    admin.rebalance(clusterName, resourceName, replcaNum);

But it doesn't work. I saw the helix source code.
ZNRecord newIdealState =
DefaultTwoStateStrategy.calculateIdealState(instanceNames, partitions,
replica, keyPrefix, masterStateValue, slaveStateValue);

// for now keep mapField in SEMI_AUTO mode and remove listField in
CUSTOMIZED mode
if (idealState.getRebalanceMode() == RebalanceMode.SEMI_AUTO) {
  idealState.getRecord().setListFields(newIdealState.getListFields());
  idealState.getRecord().setMapFields(newIdealState.getMapFields());
}
if (idealState.getRebalanceMode() == RebalanceMode.CUSTOMIZED) {
  idealState.getRecord().setMapFields(newIdealState.getMapFields());
}

If my idealState is SEMI_AUTO, it will setListFields, my option will
override.
Is my option right?

Thanks

RE: Helix newer use Rebalance

Posted by Kanak Biscuitwala <ka...@hotmail.com>.
Hi,


Can you describe your use case further? Would FULL_AUTO work for you? Otherwise, SEMI_AUTO should be fine. The important first step is that you add instances through HelixAdmin#addInstance() first. That way, when you add/rebalance a resource, it will see those two instances and properly populate the ideal state with them.


Kanak

________________________________
> From: kanak.b@hotmail.com 
> To: user@helix.apache.org 
> Subject: RE: Helix newer use Rebalance 
> Date: Sun, 2 Aug 2015 14:52:42 -0700 
> 
> Hi, Can you describe your use case further? Would FULL_AUTO work for 
> you? Otherwise, SEMI_AUTO should be fine. The important first step is 
> that you add instances through HelixAdmin#addInstance() first. That 
> way, when you add/rebalance a resource, it will see those two instances 
> and properly populate the ideal state with them. Kanak Date: Tue, 28 
> Jul 2015 17:26:20 +0800 Subject: Helix newer use Rebalance From: 
> hahn.jiang.pro@gmail.com To: user@helix.apache.org I use Helix 0.7.1 I 
> start three participants and I want to assign a resource to two of 
> them. List preferenceList = new ArrayList<>(); 
> preferenceList.add("localhost_6101"); 
> preferenceList.add("localhost_6102"); Then I add preferenceList to 
> resource: partitionNum=6 replicationNum=2 
> admin.addResource(clusterName, resourceKey, 
> partitionNum,StateModelDefId.MasterSlave.stringify(), "SEMI_AUTO"); 
> IdealState idealState = admin.getResourceIdealState(clusterName, 
> resourceKey); 
> idealState.setRebalanceMode(IdealState.RebalanceMode.SEMI_AUTO); 
> idealState.setReplicas(replicationNum+""); for (int i = 0; i < 
> partitionNum; ++i) { idealState.setPreferenceList(resourceKey + 
> "_" + i, preferenceList); } 
> admin.setResourceIdealState(clusterName, resourceKey, idealState); At 
> Last, I will rebalence cluster: admin.rebalance(clusterName, 
> resourceName, replcaNum); But it doesn't work. I saw the helix source 
> code. ZNRecord newIdealState = 
> DefaultTwoStateStrategy.calculateIdealState(instanceNames, partitions, 
> replica, keyPrefix, masterStateValue, slaveStateValue); // for now keep 
> mapField in SEMI_AUTO mode and remove listField in CUSTOMIZED mode if 
> (idealState.getRebalanceMode() == RebalanceMode.SEMI_AUTO) { 
> idealState.getRecord().setListFields(newIdealState.getListFields()); 
> idealState.getRecord().setMapFields(newIdealState.getMapFields()); } if 
> (idealState.getRebalanceMode() == RebalanceMode.CUSTOMIZED) { 
> idealState.getRecord().setMapFields(newIdealState.getMapFields()); } If 
> my idealState is SEMI_AUTO, it will setListFields, my option will 
> override. Is my option right? Thanks

RE: Helix newer use Rebalance

Posted by Kanak Biscuitwala <ka...@hotmail.com>.
Hi,

Can you describe your use case further? Would FULL_AUTO work for you? Otherwise, SEMI_AUTO should be fine.

The important first step is that you add instances through HelixAdmin#addInstance() first. That way, when you add/rebalance a resource, it will see those two instances and properly populate the ideal state with them.

Kanak

Date: Tue, 28 Jul 2015 17:26:20 +0800
Subject: Helix newer use Rebalance
From: hahn.jiang.pro@gmail.com
To: user@helix.apache.org

I use Helix 0.7.1
I start three participants and I want to assign a resource to two of them.
    List preferenceList = new ArrayList<>();
    preferenceList.add("localhost_6101");
    preferenceList.add("localhost_6102");
    
Then I add preferenceList to resource: partitionNum=6  replicationNum=2
    admin.addResource(clusterName, resourceKey, partitionNum,StateModelDefId.MasterSlave.stringify(), "SEMI_AUTO");
    IdealState idealState = admin.getResourceIdealState(clusterName, resourceKey);
    idealState.setRebalanceMode(IdealState.RebalanceMode.SEMI_AUTO);
    idealState.setReplicas(replicationNum+"");
    for (int i = 0; i < partitionNum; ++i) {
      idealState.setPreferenceList(resourceKey + "_" + i, preferenceList);
    }
    admin.setResourceIdealState(clusterName, resourceKey, idealState);

At Last, I will rebalence cluster:
    admin.rebalance(clusterName, resourceName, replcaNum);

But it doesn't work. I saw the helix source code. 
ZNRecord newIdealState = DefaultTwoStateStrategy.calculateIdealState(instanceNames, partitions, replica, keyPrefix, masterStateValue, slaveStateValue);

// for now keep mapField in SEMI_AUTO mode and remove listField in CUSTOMIZED mode
if (idealState.getRebalanceMode() == RebalanceMode.SEMI_AUTO) {
  idealState.getRecord().setListFields(newIdealState.getListFields());
  idealState.getRecord().setMapFields(newIdealState.getMapFields());
}
if (idealState.getRebalanceMode() == RebalanceMode.CUSTOMIZED) {
  idealState.getRecord().setMapFields(newIdealState.getMapFields());
}

If my idealState is SEMI_AUTO, it will setListFields, my option will override.
Is my option right?

Thanks