You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@helix.apache.org by Jianqiu Lu <lv...@yahoo-inc.com> on 2014/06/18 04:05:00 UTC
question on a state change scenario
Hi team,
First I want to thanks all for this great software.
I am using helix for our storage system. In our system we have 3 states read , weak read and offline. The state transactions are read-> <- weak read-> <-offline, very similar to the master/slave example. But the difference is that we need participate to control its status , which means a participate can decide what is itself's current ideal state. If a participate find he can’t follow write speed(we use a queue to do write , participate consumes message from this queue), then it can change its state from read to weak read. Is there a proper way to do this state change from participate side? I looked into Helix java doc and found in HelixAdmin there is a method “rebalance” seems can achieve my goal, but my scenario is only need to change one server’s state,“rebalance” seems too over. Or does this a Helix fit scenario?
The other solution I thought about is using zookeeper to manage read and weak read states, and helix only maintain the online offline state. Once I get an online server from helix , then I will look up into zookeeper to find out what is the current read state of this server(maybe push mode from zookeeper is more proper). In this solution we don’t need to change helix ideal state from participate side, but this seems like mix helix and zookeeper to maintain state machine together. Could you please give me some advise on my thoughts?
Thanks
Jianqiu
RE: question on a state change scenario
Posted by Kanak Biscuitwala <ka...@hotmail.com>.
The requested state feature is currently only supported in the task framework and otherwise ignored. Currently the only way for a participant to change its own state is to update the ideal state itself.
Kanak
________________________________
> Date: Wed, 18 Jun 2014 13:02:43 -0700
> Subject: Re: question on a state change scenario
> From: g.kishore@gmail.com
> To: lvjq@yahoo-inc.com
> CC: user@helix.apache.org; liyu@yahoo-inc.com
>
> Hi Jianqui,
>
> For updating the idealstate, yes race conditions are possible. There is
> a low level api that allows you to handle the race conditions.
>
> see HelixDataAccessor.update method: This method does a merge of
> existing and new values. For example, for a participant to go into
> read_only state for a partition. It can do
> acce
> is= new IdealState();
> is.set("participant_id", "partition_id", "read_only");
> helixdataaccessor.update(accessor.keyBuilder.idealState("resource_name"),is);
>
> This should handle the race conditions. Jason/Kanak, please correct me
> if I am wrong.
>
> Which version of Helix are you using. I think documentation for #2 is
> weak and its something used internally by Helix for Task Management.
> Kanak, do we have any example for this ?
>
> thanks,
> Kishore G
>
>
>
>
> On Wed, Jun 18, 2014 at 11:30 AM, Jianqiu Lu
> <lv...@yahoo-inc.com>> wrote:
> Hi Kishore,Kanak,
> Thanks for your reply. In my case it is a participant-level property,
> an instance can have different state for different partitions. Ex.
> Partition 1-3 are on instance 1, and partition 1 is read , partition 2
> is weak read ,partition 3 is read. It does not have a strong constrain
> in my case, but updates ideal state may have some race conditions. From
> my understand, I need to pass the whole ideal state(all partitions) as
> a map to the method, but what if two servers update their ideal state
> at the same time? By the way how does #2 work? Is there any example?
> For failure handling we have different replica consumes the same
> queue, and register both of the two servers in different replica to
> same partition, so unless both of them are down, we can always have
> some one reachable.
>
> Thanks
> Jianqiu
>
> From: kishore g <g....@gmail.com>>
> Date: Tuesday, June 17, 2014 at 10:22 PM
> To: "user@helix.apache.org<ma...@helix.apache.org>"
> <us...@helix.apache.org>>, Jianqiu Lv
> <lv...@yahoo-inc.com>>
> Cc: Gavin Li <li...@yahoo-inc.com>>
> Subject: Re: question on a state change scenario
>
> Hi Jianqiu,
>
> Thank you for the kind words. I agree with what Kanak said but it
> mostly applies to systems where state level constraint is important.
> For example, if the state model is master-slave, we don't want the
> slave to automatically say that its the new master.
>
> Looks like in your systems, there are no strong constraints on the
> state. In this case it is ok for the participant to control its state.
> You have two options
>
> #1. The participant can decide the state and update the idealstate to
> reflect that state. This will be reflected in the ExternalView and the
> spectators can use the state to route requests. You can do this if you
> don't intend to have any hard constraints on the state.
>
> #2. Another option is to use the feature of requested state, the
> participant can request the controller to go to a specific state. The
> controller however can decide if it can honor that request.
>
> For the application you have described #1 can work. Let us know if #1
> is acceptable.
>
> Actually, I can see this as a requirement as a common pattern. We can
> allow the participants to override the state in certain scenarios. We
> should be able to provide a much simpler api to accomplish this.
>
> Also what is your requirement for failure handling.
>
> thanks,
> Kishore G
>
>
>
> On Tue, Jun 17, 2014 at 8:46 PM, Kanak Biscuitwala
> <ka...@hotmail.com>> wrote:
> Hi Jianqiu,
>
> In Helix we typically discourage participants from changing the states
> of partitions they serve because then we have multiple brains in the
> system.
>
> It seems like the read property is a participant-level property rather
> than a partition-level state. Is this true? Can an instance have
> different speeds per partition on the same instance? If it is purely
> participant-level, then you can use HelixAdmin#setConfig to set this
> flag for the participant, and you can register an
> InstanceConfigChangeListener with HelixManager in order to listen for
> changes.
>
> It is certainly possible to use HelixDataAccessor#updateIdealState to
> only update the partitions that currently belong to an instance, but
> that feels more like a hack, and it requires you to fully customize the
> ideal state calculation logic, rather than using FULL_AUTO or SEMI_AUTO
> mode.
>
> Kanak
>
> ________________________________
>> From: lvjq@yahoo-inc.com<ma...@yahoo-inc.com>
>> To: user@helix.apache.org<ma...@helix.apache.org>
>> CC: liyu@yahoo-inc.com<ma...@yahoo-inc.com>
>> Subject: question on a state change scenario
>> Date: Wed, 18 Jun 2014 02:05:00 +0000
>>
>> Hi team,
>> First I want to thanks all for this great software.
>> I am using helix for our storage system. In our system we have 3
>> states read , weak read and offline. The state transactions are
>> read-> <- weak read-> <-offline, very similar to the master/slave
>> example. But the difference is that we need participate to control its
>> status , which means a participate can decide what is itself's current
>> ideal state. If a participate find he can’t follow write speed(we use a
>> queue to do write , participate consumes message from this queue), then
>> it can change its state from read to weak read. Is there a proper way
>> to do this state change from participate side? I looked into Helix java
>> doc and found in HelixAdmin there is a method “rebalance” seems can
>> achieve my goal, but my scenario is only need to change one server’s
>> state,“rebalance” seems too over. Or does this a Helix fit scenario?
>> The other solution I thought about is using zookeeper to manage read
>> and weak read states, and helix only maintain the online offline state.
>> Once I get an online server from helix , then I will look up into
>> zookeeper to find out what is the current read state of this
>> server(maybe push mode from zookeeper is more proper). In this
>> solution we don’t need to change helix ideal state from participate
>> side, but this seems like mix helix and zookeeper to maintain state
>> machine together. Could you please give me some advise on my thoughts?
>>
>> Thanks
>> Jianqiu
>
>
>
Re: question on a state change scenario
Posted by kishore g <g....@gmail.com>.
Hi Jianqui,
For updating the idealstate, yes race conditions are possible. There is a
low level api that allows you to handle the race conditions.
see HelixDataAccessor.update method: This method does a merge of existing
and new values. For example, for a participant to go into read_only state
for a partition. It can do
acce
is= new IdealState();
is.set("participant_id", "partition_id", "read_only");
helixdataaccessor.update(accessor.keyBuilder.idealState("resource_name"),is);
This should handle the race conditions. Jason/Kanak, please correct me if I
am wrong.
Which version of Helix are you using. I think documentation for #2 is weak
and its something used internally by Helix for Task Management. Kanak, do
we have any example for this ?
thanks,
Kishore G
On Wed, Jun 18, 2014 at 11:30 AM, Jianqiu Lu <lv...@yahoo-inc.com> wrote:
> Hi Kishore,Kanak,
> Thanks for your reply. In my case it is a participant-level property, an
> instance can have different state for different partitions. Ex. Partition
> 1-3 are on instance 1, and partition 1 is read , partition 2 is weak read
> ,partition 3 is read. It does not have a strong constrain in my case, but
> updates ideal state may have some race conditions. From my understand, I
> need to pass the whole ideal state(all partitions) as a map to the method,
> but what if two servers update their ideal state at the same time? By the
> way how does #2 work? Is there any example? For failure handling we have
> different replica consumes the same queue, and register both of the two
> servers in different replica to same partition, so unless both of them are
> down, we can always have some one reachable.
>
> Thanks
> Jianqiu
>
> From: kishore g <g....@gmail.com>
> Date: Tuesday, June 17, 2014 at 10:22 PM
> To: "user@helix.apache.org" <us...@helix.apache.org>, Jianqiu Lv <
> lvjq@yahoo-inc.com>
> Cc: Gavin Li <li...@yahoo-inc.com>
> Subject: Re: question on a state change scenario
>
> Hi Jianqiu,
>
> Thank you for the kind words. I agree with what Kanak said but it mostly
> applies to systems where state level constraint is important. For example,
> if the state model is master-slave, we don't want the slave to
> automatically say that its the new master.
>
> Looks like in your systems, there are no strong constraints on the state.
> In this case it is ok for the participant to control its state. You have
> two options
>
> #1. The participant can decide the state and update the idealstate to
> reflect that state. This will be reflected in the ExternalView and the
> spectators can use the state to route requests. You can do this if you
> don't intend to have any hard constraints on the state.
>
> #2. Another option is to use the feature of requested state, the
> participant can request the controller to go to a specific state. The
> controller however can decide if it can honor that request.
>
> For the application you have described #1 can work. Let us know if #1 is
> acceptable.
>
> Actually, I can see this as a requirement as a common pattern. We can
> allow the participants to override the state in certain scenarios. We
> should be able to provide a much simpler api to accomplish this.
>
> Also what is your requirement for failure handling.
>
> thanks,
> Kishore G
>
>
>
> On Tue, Jun 17, 2014 at 8:46 PM, Kanak Biscuitwala <ka...@hotmail.com>
> wrote:
>
>> Hi Jianqiu,
>>
>> In Helix we typically discourage participants from changing the states of
>> partitions they serve because then we have multiple brains in the system.
>>
>> It seems like the read property is a participant-level property rather
>> than a partition-level state. Is this true? Can an instance have different
>> speeds per partition on the same instance? If it is purely
>> participant-level, then you can use HelixAdmin#setConfig to set this flag
>> for the participant, and you can register an InstanceConfigChangeListener
>> with HelixManager in order to listen for changes.
>>
>> It is certainly possible to use HelixDataAccessor#updateIdealState to
>> only update the partitions that currently belong to an instance, but that
>> feels more like a hack, and it requires you to fully customize the ideal
>> state calculation logic, rather than using FULL_AUTO or SEMI_AUTO mode.
>>
>> Kanak
>>
>> ________________________________
>> > From: lvjq@yahoo-inc.com
>> > To: user@helix.apache.org
>> > CC: liyu@yahoo-inc.com
>> > Subject: question on a state change scenario
>> > Date: Wed, 18 Jun 2014 02:05:00 +0000
>> >
>> > Hi team,
>> > First I want to thanks all for this great software.
>> > I am using helix for our storage system. In our system we have 3
>> > states read , weak read and offline. The state transactions are
>> > read-> <- weak read-> <-offline, very similar to the master/slave
>> > example. But the difference is that we need participate to control its
>> > status , which means a participate can decide what is itself's current
>> > ideal state. If a participate find he can’t follow write speed(we use a
>> > queue to do write , participate consumes message from this queue), then
>> > it can change its state from read to weak read. Is there a proper way
>> > to do this state change from participate side? I looked into Helix java
>> > doc and found in HelixAdmin there is a method “rebalance” seems can
>> > achieve my goal, but my scenario is only need to change one server’s
>> > state,“rebalance” seems too over. Or does this a Helix fit scenario?
>> > The other solution I thought about is using zookeeper to manage read
>> > and weak read states, and helix only maintain the online offline state.
>> > Once I get an online server from helix , then I will look up into
>> > zookeeper to find out what is the current read state of this
>> > server(maybe push mode from zookeeper is more proper). In this
>> > solution we don’t need to change helix ideal state from participate
>> > side, but this seems like mix helix and zookeeper to maintain state
>> > machine together. Could you please give me some advise on my thoughts?
>> >
>> > Thanks
>> > Jianqiu
>>
>>
>
>
RE: question on a state change scenario
Posted by Kanak Biscuitwala <ka...@hotmail.com>.
Correction, line 4 should be:
delta.getRecord().getMapField(partitionName).put(participantName, newState);
----------------------------------------
> From: kanak.b@hotmail.com
> To: user@helix.apache.org; g.kishore@gmail.com
> CC: liyu@yahoo-inc.com
> Subject: RE: question on a state change scenario
> Date: Wed, 18 Jun 2014 13:04:08 -0700
>
>
> Hi Jianqiu,
>
> You need to use HelixDataAccessor#update to ensure atomicity, like:
>
> HelixManager manager = ...;
> HelixDataAccessor accessor = manager.getHelixDataAccessor();
> IdealState delta = new IdealState(resourceName);
> delta.getRecord().getMapFields().put(partitionName, newState);
> accessor.updateProperty(accessor.keyBuilder().idealStates(resourceName), delta);
>
> Kanak
> ________________________________
>> From: lvjq@yahoo-inc.com
>> To: g.kishore@gmail.com; user@helix.apache.org
>> CC: liyu@yahoo-inc.com
>> Subject: Re: question on a state change scenario
>> Date: Wed, 18 Jun 2014 18:30:17 +0000
>>
>> Hi Kishore,Kanak,
>> Thanks for your reply. In my case it is a participant-level property,
>> an instance can have different state for different partitions. Ex.
>> Partition 1-3 are on instance 1, and partition 1 is read , partition 2
>> is weak read ,partition 3 is read. It does not have a strong constrain
>> in my case, but updates ideal state may have some race conditions. From
>> my understand, I need to pass the whole ideal state(all partitions) as
>> a map to the method, but what if two servers update their ideal state
>> at the same time? By the way how does #2 work? Is there any example?
>> For failure handling we have different replica consumes the same
>> queue, and register both of the two servers in different replica to
>> same partition, so unless both of them are down, we can always have
>> some one reachable.
>>
>> Thanks
>> Jianqiu
>>
>> From: kishore g <g....@gmail.com>>
>> Date: Tuesday, June 17, 2014 at 10:22 PM
>> To: "user@helix.apache.org<ma...@helix.apache.org>"
>> <us...@helix.apache.org>>, Jianqiu Lv
>> <lv...@yahoo-inc.com>>
>> Cc: Gavin Li <li...@yahoo-inc.com>>
>> Subject: Re: question on a state change scenario
>>
>> Hi Jianqiu,
>>
>> Thank you for the kind words. I agree with what Kanak said but it
>> mostly applies to systems where state level constraint is important.
>> For example, if the state model is master-slave, we don't want the
>> slave to automatically say that its the new master.
>>
>> Looks like in your systems, there are no strong constraints on the
>> state. In this case it is ok for the participant to control its state.
>> You have two options
>>
>> #1. The participant can decide the state and update the idealstate to
>> reflect that state. This will be reflected in the ExternalView and the
>> spectators can use the state to route requests. You can do this if you
>> don't intend to have any hard constraints on the state.
>>
>> #2. Another option is to use the feature of requested state, the
>> participant can request the controller to go to a specific state. The
>> controller however can decide if it can honor that request.
>>
>> For the application you have described #1 can work. Let us know if #1
>> is acceptable.
>>
>> Actually, I can see this as a requirement as a common pattern. We can
>> allow the participants to override the state in certain scenarios. We
>> should be able to provide a much simpler api to accomplish this.
>>
>> Also what is your requirement for failure handling.
>>
>> thanks,
>> Kishore G
>>
>>
>>
>> On Tue, Jun 17, 2014 at 8:46 PM, Kanak Biscuitwala
>> <ka...@hotmail.com>> wrote:
>> Hi Jianqiu,
>>
>> In Helix we typically discourage participants from changing the states
>> of partitions they serve because then we have multiple brains in the
>> system.
>>
>> It seems like the read property is a participant-level property rather
>> than a partition-level state. Is this true? Can an instance have
>> different speeds per partition on the same instance? If it is purely
>> participant-level, then you can use HelixAdmin#setConfig to set this
>> flag for the participant, and you can register an
>> InstanceConfigChangeListener with HelixManager in order to listen for
>> changes.
>>
>> It is certainly possible to use HelixDataAccessor#updateIdealState to
>> only update the partitions that currently belong to an instance, but
>> that feels more like a hack, and it requires you to fully customize the
>> ideal state calculation logic, rather than using FULL_AUTO or SEMI_AUTO
>> mode.
>>
>> Kanak
>>
>> ________________________________
>>> From: lvjq@yahoo-inc.com<ma...@yahoo-inc.com>
>>> To: user@helix.apache.org<ma...@helix.apache.org>
>>> CC: liyu@yahoo-inc.com<ma...@yahoo-inc.com>
>>> Subject: question on a state change scenario
>>> Date: Wed, 18 Jun 2014 02:05:00 +0000
>>>
>>> Hi team,
>>> First I want to thanks all for this great software.
>>> I am using helix for our storage system. In our system we have 3
>>> states read , weak read and offline. The state transactions are
>>> read-> <- weak read-> <-offline, very similar to the master/slave
>>> example. But the difference is that we need participate to control its
>>> status , which means a participate can decide what is itself's current
>>> ideal state. If a participate find he can’t follow write speed(we use a
>>> queue to do write , participate consumes message from this queue), then
>>> it can change its state from read to weak read. Is there a proper way
>>> to do this state change from participate side? I looked into Helix java
>>> doc and found in HelixAdmin there is a method “rebalance” seems can
>>> achieve my goal, but my scenario is only need to change one server’s
>>> state,“rebalance” seems too over. Or does this a Helix fit scenario?
>>> The other solution I thought about is using zookeeper to manage read
>>> and weak read states, and helix only maintain the online offline state.
>>> Once I get an online server from helix , then I will look up into
>>> zookeeper to find out what is the current read state of this
>>> server(maybe push mode from zookeeper is more proper). In this
>>> solution we don’t need to change helix ideal state from participate
>>> side, but this seems like mix helix and zookeeper to maintain state
>>> machine together. Could you please give me some advise on my thoughts?
>>>
>>> Thanks
>>> Jianqiu
>>
>>
>
RE: question on a state change scenario
Posted by Kanak Biscuitwala <ka...@hotmail.com>.
Hi Jianqiu,
You need to use HelixDataAccessor#update to ensure atomicity, like:
HelixManager manager = ...;
HelixDataAccessor accessor = manager.getHelixDataAccessor();
IdealState delta = new IdealState(resourceName);
delta.getRecord().getMapFields().put(partitionName, newState);
accessor.updateProperty(accessor.keyBuilder().idealStates(resourceName), delta);
Kanak
________________________________
> From: lvjq@yahoo-inc.com
> To: g.kishore@gmail.com; user@helix.apache.org
> CC: liyu@yahoo-inc.com
> Subject: Re: question on a state change scenario
> Date: Wed, 18 Jun 2014 18:30:17 +0000
>
> Hi Kishore,Kanak,
> Thanks for your reply. In my case it is a participant-level property,
> an instance can have different state for different partitions. Ex.
> Partition 1-3 are on instance 1, and partition 1 is read , partition 2
> is weak read ,partition 3 is read. It does not have a strong constrain
> in my case, but updates ideal state may have some race conditions. From
> my understand, I need to pass the whole ideal state(all partitions) as
> a map to the method, but what if two servers update their ideal state
> at the same time? By the way how does #2 work? Is there any example?
> For failure handling we have different replica consumes the same
> queue, and register both of the two servers in different replica to
> same partition, so unless both of them are down, we can always have
> some one reachable.
>
> Thanks
> Jianqiu
>
> From: kishore g <g....@gmail.com>>
> Date: Tuesday, June 17, 2014 at 10:22 PM
> To: "user@helix.apache.org<ma...@helix.apache.org>"
> <us...@helix.apache.org>>, Jianqiu Lv
> <lv...@yahoo-inc.com>>
> Cc: Gavin Li <li...@yahoo-inc.com>>
> Subject: Re: question on a state change scenario
>
> Hi Jianqiu,
>
> Thank you for the kind words. I agree with what Kanak said but it
> mostly applies to systems where state level constraint is important.
> For example, if the state model is master-slave, we don't want the
> slave to automatically say that its the new master.
>
> Looks like in your systems, there are no strong constraints on the
> state. In this case it is ok for the participant to control its state.
> You have two options
>
> #1. The participant can decide the state and update the idealstate to
> reflect that state. This will be reflected in the ExternalView and the
> spectators can use the state to route requests. You can do this if you
> don't intend to have any hard constraints on the state.
>
> #2. Another option is to use the feature of requested state, the
> participant can request the controller to go to a specific state. The
> controller however can decide if it can honor that request.
>
> For the application you have described #1 can work. Let us know if #1
> is acceptable.
>
> Actually, I can see this as a requirement as a common pattern. We can
> allow the participants to override the state in certain scenarios. We
> should be able to provide a much simpler api to accomplish this.
>
> Also what is your requirement for failure handling.
>
> thanks,
> Kishore G
>
>
>
> On Tue, Jun 17, 2014 at 8:46 PM, Kanak Biscuitwala
> <ka...@hotmail.com>> wrote:
> Hi Jianqiu,
>
> In Helix we typically discourage participants from changing the states
> of partitions they serve because then we have multiple brains in the
> system.
>
> It seems like the read property is a participant-level property rather
> than a partition-level state. Is this true? Can an instance have
> different speeds per partition on the same instance? If it is purely
> participant-level, then you can use HelixAdmin#setConfig to set this
> flag for the participant, and you can register an
> InstanceConfigChangeListener with HelixManager in order to listen for
> changes.
>
> It is certainly possible to use HelixDataAccessor#updateIdealState to
> only update the partitions that currently belong to an instance, but
> that feels more like a hack, and it requires you to fully customize the
> ideal state calculation logic, rather than using FULL_AUTO or SEMI_AUTO
> mode.
>
> Kanak
>
> ________________________________
>> From: lvjq@yahoo-inc.com<ma...@yahoo-inc.com>
>> To: user@helix.apache.org<ma...@helix.apache.org>
>> CC: liyu@yahoo-inc.com<ma...@yahoo-inc.com>
>> Subject: question on a state change scenario
>> Date: Wed, 18 Jun 2014 02:05:00 +0000
>>
>> Hi team,
>> First I want to thanks all for this great software.
>> I am using helix for our storage system. In our system we have 3
>> states read , weak read and offline. The state transactions are
>> read-> <- weak read-> <-offline, very similar to the master/slave
>> example. But the difference is that we need participate to control its
>> status , which means a participate can decide what is itself's current
>> ideal state. If a participate find he can’t follow write speed(we use a
>> queue to do write , participate consumes message from this queue), then
>> it can change its state from read to weak read. Is there a proper way
>> to do this state change from participate side? I looked into Helix java
>> doc and found in HelixAdmin there is a method “rebalance” seems can
>> achieve my goal, but my scenario is only need to change one server’s
>> state,“rebalance” seems too over. Or does this a Helix fit scenario?
>> The other solution I thought about is using zookeeper to manage read
>> and weak read states, and helix only maintain the online offline state.
>> Once I get an online server from helix , then I will look up into
>> zookeeper to find out what is the current read state of this
>> server(maybe push mode from zookeeper is more proper). In this
>> solution we don’t need to change helix ideal state from participate
>> side, but this seems like mix helix and zookeeper to maintain state
>> machine together. Could you please give me some advise on my thoughts?
>>
>> Thanks
>> Jianqiu
>
>
Re: question on a state change scenario
Posted by Jianqiu Lu <lv...@yahoo-inc.com>.
Hi Kishore,Kanak,
Thanks for your reply. In my case it is a participant-level property, an instance can have different state for different partitions. Ex. Partition 1-3 are on instance 1, and partition 1 is read , partition 2 is weak read ,partition 3 is read. It does not have a strong constrain in my case, but updates ideal state may have some race conditions. From my understand, I need to pass the whole ideal state(all partitions) as a map to the method, but what if two servers update their ideal state at the same time? By the way how does #2 work? Is there any example? For failure handling we have different replica consumes the same queue, and register both of the two servers in different replica to same partition, so unless both of them are down, we can always have some one reachable.
Thanks
Jianqiu
From: kishore g <g....@gmail.com>>
Date: Tuesday, June 17, 2014 at 10:22 PM
To: "user@helix.apache.org<ma...@helix.apache.org>" <us...@helix.apache.org>>, Jianqiu Lv <lv...@yahoo-inc.com>>
Cc: Gavin Li <li...@yahoo-inc.com>>
Subject: Re: question on a state change scenario
Hi Jianqiu,
Thank you for the kind words. I agree with what Kanak said but it mostly applies to systems where state level constraint is important. For example, if the state model is master-slave, we don't want the slave to automatically say that its the new master.
Looks like in your systems, there are no strong constraints on the state. In this case it is ok for the participant to control its state. You have two options
#1. The participant can decide the state and update the idealstate to reflect that state. This will be reflected in the ExternalView and the spectators can use the state to route requests. You can do this if you don't intend to have any hard constraints on the state.
#2. Another option is to use the feature of requested state, the participant can request the controller to go to a specific state. The controller however can decide if it can honor that request.
For the application you have described #1 can work. Let us know if #1 is acceptable.
Actually, I can see this as a requirement as a common pattern. We can allow the participants to override the state in certain scenarios. We should be able to provide a much simpler api to accomplish this.
Also what is your requirement for failure handling.
thanks,
Kishore G
On Tue, Jun 17, 2014 at 8:46 PM, Kanak Biscuitwala <ka...@hotmail.com>> wrote:
Hi Jianqiu,
In Helix we typically discourage participants from changing the states of partitions they serve because then we have multiple brains in the system.
It seems like the read property is a participant-level property rather than a partition-level state. Is this true? Can an instance have different speeds per partition on the same instance? If it is purely participant-level, then you can use HelixAdmin#setConfig to set this flag for the participant, and you can register an InstanceConfigChangeListener with HelixManager in order to listen for changes.
It is certainly possible to use HelixDataAccessor#updateIdealState to only update the partitions that currently belong to an instance, but that feels more like a hack, and it requires you to fully customize the ideal state calculation logic, rather than using FULL_AUTO or SEMI_AUTO mode.
Kanak
________________________________
> From: lvjq@yahoo-inc.com<ma...@yahoo-inc.com>
> To: user@helix.apache.org<ma...@helix.apache.org>
> CC: liyu@yahoo-inc.com<ma...@yahoo-inc.com>
> Subject: question on a state change scenario
> Date: Wed, 18 Jun 2014 02:05:00 +0000
>
> Hi team,
> First I want to thanks all for this great software.
> I am using helix for our storage system. In our system we have 3
> states read , weak read and offline. The state transactions are
> read-> <- weak read-> <-offline, very similar to the master/slave
> example. But the difference is that we need participate to control its
> status , which means a participate can decide what is itself's current
> ideal state. If a participate find he can’t follow write speed(we use a
> queue to do write , participate consumes message from this queue), then
> it can change its state from read to weak read. Is there a proper way
> to do this state change from participate side? I looked into Helix java
> doc and found in HelixAdmin there is a method “rebalance” seems can
> achieve my goal, but my scenario is only need to change one server’s
> state,“rebalance” seems too over. Or does this a Helix fit scenario?
> The other solution I thought about is using zookeeper to manage read
> and weak read states, and helix only maintain the online offline state.
> Once I get an online server from helix , then I will look up into
> zookeeper to find out what is the current read state of this
> server(maybe push mode from zookeeper is more proper). In this
> solution we don’t need to change helix ideal state from participate
> side, but this seems like mix helix and zookeeper to maintain state
> machine together. Could you please give me some advise on my thoughts?
>
> Thanks
> Jianqiu
Re: question on a state change scenario
Posted by kishore g <g....@gmail.com>.
Hi Jianqiu,
Thank you for the kind words. I agree with what Kanak said but it mostly
applies to systems where state level constraint is important. For example,
if the state model is master-slave, we don't want the slave to
automatically say that its the new master.
Looks like in your systems, there are no strong constraints on the state.
In this case it is ok for the participant to control its state. You have
two options
#1. The participant can decide the state and update the idealstate to
reflect that state. This will be reflected in the ExternalView and the
spectators can use the state to route requests. You can do this if you
don't intend to have any hard constraints on the state.
#2. Another option is to use the feature of requested state, the
participant can request the controller to go to a specific state. The
controller however can decide if it can honor that request.
For the application you have described #1 can work. Let us know if #1 is
acceptable.
Actually, I can see this as a requirement as a common pattern. We can allow
the participants to override the state in certain scenarios. We should be
able to provide a much simpler api to accomplish this.
Also what is your requirement for failure handling.
thanks,
Kishore G
On Tue, Jun 17, 2014 at 8:46 PM, Kanak Biscuitwala <ka...@hotmail.com>
wrote:
> Hi Jianqiu,
>
> In Helix we typically discourage participants from changing the states of
> partitions they serve because then we have multiple brains in the system.
>
> It seems like the read property is a participant-level property rather
> than a partition-level state. Is this true? Can an instance have different
> speeds per partition on the same instance? If it is purely
> participant-level, then you can use HelixAdmin#setConfig to set this flag
> for the participant, and you can register an InstanceConfigChangeListener
> with HelixManager in order to listen for changes.
>
> It is certainly possible to use HelixDataAccessor#updateIdealState to only
> update the partitions that currently belong to an instance, but that feels
> more like a hack, and it requires you to fully customize the ideal state
> calculation logic, rather than using FULL_AUTO or SEMI_AUTO mode.
>
> Kanak
>
> ________________________________
> > From: lvjq@yahoo-inc.com
> > To: user@helix.apache.org
> > CC: liyu@yahoo-inc.com
> > Subject: question on a state change scenario
> > Date: Wed, 18 Jun 2014 02:05:00 +0000
> >
> > Hi team,
> > First I want to thanks all for this great software.
> > I am using helix for our storage system. In our system we have 3
> > states read , weak read and offline. The state transactions are
> > read-> <- weak read-> <-offline, very similar to the master/slave
> > example. But the difference is that we need participate to control its
> > status , which means a participate can decide what is itself's current
> > ideal state. If a participate find he can’t follow write speed(we use a
> > queue to do write , participate consumes message from this queue), then
> > it can change its state from read to weak read. Is there a proper way
> > to do this state change from participate side? I looked into Helix java
> > doc and found in HelixAdmin there is a method “rebalance” seems can
> > achieve my goal, but my scenario is only need to change one server’s
> > state,“rebalance” seems too over. Or does this a Helix fit scenario?
> > The other solution I thought about is using zookeeper to manage read
> > and weak read states, and helix only maintain the online offline state.
> > Once I get an online server from helix , then I will look up into
> > zookeeper to find out what is the current read state of this
> > server(maybe push mode from zookeeper is more proper). In this
> > solution we don’t need to change helix ideal state from participate
> > side, but this seems like mix helix and zookeeper to maintain state
> > machine together. Could you please give me some advise on my thoughts?
> >
> > Thanks
> > Jianqiu
>
>
RE: question on a state change scenario
Posted by Kanak Biscuitwala <ka...@hotmail.com>.
Hi Jianqiu,
In Helix we typically discourage participants from changing the states of partitions they serve because then we have multiple brains in the system.
It seems like the read property is a participant-level property rather than a partition-level state. Is this true? Can an instance have different speeds per partition on the same instance? If it is purely participant-level, then you can use HelixAdmin#setConfig to set this flag for the participant, and you can register an InstanceConfigChangeListener with HelixManager in order to listen for changes.
It is certainly possible to use HelixDataAccessor#updateIdealState to only update the partitions that currently belong to an instance, but that feels more like a hack, and it requires you to fully customize the ideal state calculation logic, rather than using FULL_AUTO or SEMI_AUTO mode.
Kanak
________________________________
> From: lvjq@yahoo-inc.com
> To: user@helix.apache.org
> CC: liyu@yahoo-inc.com
> Subject: question on a state change scenario
> Date: Wed, 18 Jun 2014 02:05:00 +0000
>
> Hi team,
> First I want to thanks all for this great software.
> I am using helix for our storage system. In our system we have 3
> states read , weak read and offline. The state transactions are
> read-> <- weak read-> <-offline, very similar to the master/slave
> example. But the difference is that we need participate to control its
> status , which means a participate can decide what is itself's current
> ideal state. If a participate find he can’t follow write speed(we use a
> queue to do write , participate consumes message from this queue), then
> it can change its state from read to weak read. Is there a proper way
> to do this state change from participate side? I looked into Helix java
> doc and found in HelixAdmin there is a method “rebalance” seems can
> achieve my goal, but my scenario is only need to change one server’s
> state,“rebalance” seems too over. Or does this a Helix fit scenario?
> The other solution I thought about is using zookeeper to manage read
> and weak read states, and helix only maintain the online offline state.
> Once I get an online server from helix , then I will look up into
> zookeeper to find out what is the current read state of this
> server(maybe push mode from zookeeper is more proper). In this
> solution we don’t need to change helix ideal state from participate
> side, but this seems like mix helix and zookeeper to maintain state
> machine together. Could you please give me some advise on my thoughts?
>
> Thanks
> Jianqiu