Communicating Master config to Slave

[Ming Fang <mi...@mac.com>]

I have a use case similar to rsync-replicated-file-system recipe.
One difference is that my Master needs to communicate the server host and port to the Slave.
Also the server port will be dynamic.

There seems to be three ways to communicate config changes to the Slave, InstanceConfig, ConfigScope, and ZkHelixPropertyStore.
I actually don't know what ConfigScope is for.

InstanceConfig at first appears to the right place since it is the Master instance config that is changing.
But there does not seem to be a way to update it once it is added the Admin.

That leaves me with ZkHelixPropertyStore.
Is ZkHelixPropertyStore the best way to do this?

Thanks

Re: Communicating Master config to Slave

[kishore g <g....@gmail.com>]

Got it, that should work.


On Sun, Mar 10, 2013 at 11:01 PM, Ming Fang <mi...@mac.com> wrote:

> Thanks Kishore.
>
> My use case requires starting a server socket when an instance becomes
> Master.
> Stop the server when that Master becomes Slave.
> And start a client socket when in Slave state.
> The Slave will have to lookup the Master's server socket info.
> I'm fairly far along with using HelixPropertyStore.
> One advantage I see is that using the PropertyStore I can set
> with AccessOption.EPHEMERAL option, which works for my use case.
>
> When I have time I'll look into the InstanceConfig option.
> Thanks again.
>
> On Mar 11, 2013, at 1:54 AM, kishore g <g....@gmail.com> wrote:
>
> Hi Ming,
>
> InstanceConfig is the right choice. You can update this during start up,
> there is a callback interface called preConnectCallback, this is invoked
> after you connect to cluster but before liveInstance znode is created. You
> can set the config here using
>
> manager.getConfigAccessor().set(ConfigScope, key, value)
>
> See TestConfigAccessor on how to create ConfigScope, it basically allows
> you to set config at a cluster, instance, resource, partition level.
>
> Hope this helps.
>
> Thanks,
> Kishore G
>
>
> On Sun, Mar 10, 2013 at 9:42 PM, Ming Fang <mi...@mac.com> wrote:
>
>> I have a use case similar to rsync-replicated-file-system recipe.
>> One difference is that my Master needs to communicate the server host and
>> port to the Slave.
>> Also the server port will be dynamic.
>>
>> There seems to be three ways to communicate config changes to the Slave,
>> InstanceConfig, ConfigScope, and ZkHelixPropertyStore.
>> I actually don't know what ConfigScope is for.
>>
>> InstanceConfig at first appears to the right place since it is the Master
>> instance config that is changing.
>> But there does not seem to be a way to update it once it is added the
>> Admin.
>>
>> That leaves me with ZkHelixPropertyStore.
>> Is ZkHelixPropertyStore the best way to do this?
>>
>> Thanks
>>
>>
>
>

Re: Communicating Master config to Slave

[Ming Fang <mi...@mac.com>]

Thanks Kishore.

My use case requires starting a server socket when an instance becomes Master.
Stop the server when that Master becomes Slave.
And start a client socket when in Slave state.
The Slave will have to lookup the Master's server socket info.
I'm fairly far along with using HelixPropertyStore.
One advantage I see is that using the PropertyStore I can set with AccessOption.EPHEMERAL option, which works for my use case.

When I have time I'll look into the InstanceConfig option.
Thanks again.

On Mar 11, 2013, at 1:54 AM, kishore g <g....@gmail.com> wrote:

> Hi Ming,
> 
> InstanceConfig is the right choice. You can update this during start up, there is a callback interface called preConnectCallback, this is invoked after you connect to cluster but before liveInstance znode is created. You can set the config here using
> 
> manager.getConfigAccessor().set(ConfigScope, key, value)
> 
> See TestConfigAccessor on how to create ConfigScope, it basically allows you to set config at a cluster, instance, resource, partition level.
> 
> Hope this helps. 
> 
> Thanks,
> Kishore G
> 
> 
> On Sun, Mar 10, 2013 at 9:42 PM, Ming Fang <mi...@mac.com> wrote:
> I have a use case similar to rsync-replicated-file-system recipe.
> One difference is that my Master needs to communicate the server host and port to the Slave.
> Also the server port will be dynamic.
> 
> There seems to be three ways to communicate config changes to the Slave, InstanceConfig, ConfigScope, and ZkHelixPropertyStore.
> I actually don't know what ConfigScope is for.
> 
> InstanceConfig at first appears to the right place since it is the Master instance config that is changing.
> But there does not seem to be a way to update it once it is added the Admin.
> 
> That leaves me with ZkHelixPropertyStore.
> Is ZkHelixPropertyStore the best way to do this?
> 
> Thanks
> 
> 

Re: Communicating Master config to Slave

[kishore g <g....@gmail.com>]

Hi Ming,

InstanceConfig is the right choice. You can update this during start up,
there is a callback interface called preConnectCallback, this is invoked
after you connect to cluster but before liveInstance znode is created. You
can set the config here using

manager.getConfigAccessor().set(ConfigScope, key, value)

See TestConfigAccessor on how to create ConfigScope, it basically allows
you to set config at a cluster, instance, resource, partition level.

Hope this helps.

Thanks,
Kishore G


On Sun, Mar 10, 2013 at 9:42 PM, Ming Fang <mi...@mac.com> wrote:

> I have a use case similar to rsync-replicated-file-system recipe.
> One difference is that my Master needs to communicate the server host and
> port to the Slave.
> Also the server port will be dynamic.
>
> There seems to be three ways to communicate config changes to the Slave,
> InstanceConfig, ConfigScope, and ZkHelixPropertyStore.
> I actually don't know what ConfigScope is for.
>
> InstanceConfig at first appears to the right place since it is the Master
> instance config that is changing.
> But there does not seem to be a way to update it once it is added the
> Admin.
>
> That leaves me with ZkHelixPropertyStore.
> Is ZkHelixPropertyStore the best way to do this?
>
> Thanks
>
>