You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@helix.apache.org by Brent <br...@gmail.com> on 2020/09/21 18:08:33 UTC
Changing the rebalancing strategy to CrushED via CLI or REST
Hello,
I'm interested in using a topology-aware rebalancing strategy. It's my
understanding that the default AutoRebalanceStrategy is not able to be
topology-aware (based on "Partition Assignment Strategies" in
https://cwiki.apache.org/confluence/display/HELIX/Helix+Topology-aware+Rebalance+Strategy).
Based on the "Simple User Guide" at the bottom of
http://helix.apache.org/1.0.1-docs/design_crushed.html, I'm attempting to
set my resource to use the CrushED REBALANCE_STRATEGY via this command (I'd
like to avoid the Java API if I can):
./helix-admin.sh --zkSvr 1.2.3.4:2181 --setConfig RESOURCE MyCluster,MyDB
"REBALANCE_STRATEGY=org.apache.helix.controller.rebalancer.strategy.CrushEdRebalanceStrategy"
When I pull the data from the idealState or externalView, it doesn't seem
to reflect the change (only idealState shown below):
~/helix-binaries/helix-core-1.0.1/bin$ curl
http://localhost:8100/admin/v2/clusters/MyCluster/resources/MyDB/idealState
{
"id" : "MyDB",
"mapFields" : {
"MyDB_0" : {},
"MyDB_1" : {},
"MyDB_2" : {}
},
"simpleFields" : {
"IDEAL_STATE_MODE" : "AUTO_REBALANCE",
"NUM_PARTITIONS" : "3",
"REBALANCE_MODE" : "FULL_AUTO",
"REBALANCE_STRATEGY" : "DEFAULT",
"REPLICAS" : "1",
"STATE_MODEL_DEF_REF" : "MasterSlave",
"STATE_MODEL_FACTORY_NAME" : "DEFAULT"
},
"listFields" : {
"MyDB_0" : [ ],
"MyDB_1" : [ ],
"MyDB_2" : [ ]
}
}
However, when I pull the config for the resource itself, it seems to show
what I'd expect:
~/helix-binaries/helix-core-1.0.1/bin$ curl
http://localhost:8100/admin/v2/clusters/MyCluster/resources/MyDB/configs
{
"id" : "MyDB",
"mapFields" : {
},
"simpleFields" : {
"REBALANCE_STRATEGY" :
"org.apache.helix.controller.rebalancer.strategy.CrushEdRebalanceStrategy"
},
"listFields" : {
}
}
Am I doing this correctly? Has that setting properly taken effect? I'm a
bit confused by why I see it in the resource config, but not in the
IdealState or externalView and I want to make sure this is the proper way
to actually use an alternate rebalancing strategy.
Thank you!
Re: Changing the rebalancing strategy to CrushED via CLI or REST
Posted by Brent <br...@gmail.com>.
I think I figured out some of my confusion which is that "--setConfig" and
"--addResourceProperty" on the CLI are two different things.
If I do this, then the setting shows up in the Resource configuration:
./helix-admin.sh --zkSvr 1.2.3.4:2181 --setConfig RESOURCE MyCluster,MyDB
"REBALANCE_STRATEGY=org.apache.helix.controller.rebalancer.strategy.CrushEdRebalanceStrategy"
But if I do this, the setting shows up in the Resource IdealState (though
not the Resource ExternalView):
./helix-admin.sh --zkSvr 1.2.3.4:2181 --addResourceProperty MyCluster MyDB
REBALANCE_STRATEGY
"org.apache.helix.controller.rebalancer.strategy.CrushRebalanceStrategy"
How do I know when to set configuration vs. add a property? There are a
bunch of other settings I'd like to set, but I'm not sure if they're
configuration or properties.
Thank you!
~Brent
On Mon, Sep 21, 2020 at 11:08 AM Brent <br...@gmail.com> wrote:
> Hello,
>
> I'm interested in using a topology-aware rebalancing strategy. It's my
> understanding that the default AutoRebalanceStrategy is not able to be
> topology-aware (based on "Partition Assignment Strategies" in
> https://cwiki.apache.org/confluence/display/HELIX/Helix+Topology-aware+Rebalance+Strategy).
> Based on the "Simple User Guide" at the bottom of
> http://helix.apache.org/1.0.1-docs/design_crushed.html, I'm attempting to
> set my resource to use the CrushED REBALANCE_STRATEGY via this command (I'd
> like to avoid the Java API if I can):
>
> ./helix-admin.sh --zkSvr 1.2.3.4:2181 --setConfig RESOURCE MyCluster,MyDB
> "REBALANCE_STRATEGY=org.apache.helix.controller.rebalancer.strategy.CrushEdRebalanceStrategy"
>
> When I pull the data from the idealState or externalView, it doesn't seem
> to reflect the change (only idealState shown below):
>
> ~/helix-binaries/helix-core-1.0.1/bin$ curl
> http://localhost:8100/admin/v2/clusters/MyCluster/resources/MyDB/idealState
> {
> "id" : "MyDB",
> "mapFields" : {
> "MyDB_0" : {},
> "MyDB_1" : {},
> "MyDB_2" : {}
> },
> "simpleFields" : {
> "IDEAL_STATE_MODE" : "AUTO_REBALANCE",
> "NUM_PARTITIONS" : "3",
> "REBALANCE_MODE" : "FULL_AUTO",
> "REBALANCE_STRATEGY" : "DEFAULT",
> "REPLICAS" : "1",
> "STATE_MODEL_DEF_REF" : "MasterSlave",
> "STATE_MODEL_FACTORY_NAME" : "DEFAULT"
> },
> "listFields" : {
> "MyDB_0" : [ ],
> "MyDB_1" : [ ],
> "MyDB_2" : [ ]
> }
> }
>
> However, when I pull the config for the resource itself, it seems to show
> what I'd expect:
> ~/helix-binaries/helix-core-1.0.1/bin$ curl
> http://localhost:8100/admin/v2/clusters/MyCluster/resources/MyDB/configs
> {
> "id" : "MyDB",
> "mapFields" : {
> },
> "simpleFields" : {
> "REBALANCE_STRATEGY" :
> "org.apache.helix.controller.rebalancer.strategy.CrushEdRebalanceStrategy"
> },
> "listFields" : {
> }
> }
>
> Am I doing this correctly? Has that setting properly taken effect? I'm a
> bit confused by why I see it in the resource config, but not in the
> IdealState or externalView and I want to make sure this is the proper way
> to actually use an alternate rebalancing strategy.
>
> Thank you!
>