You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@metron.apache.org by moshe jarusalem <tu...@gmail.com> on 2017/03/31 12:49:08 UTC
configuration update during runtime
Hi All,
I have been looking the codes for ConfiguredBolt and its derivatives. I
realized that updateConfig is actually not doing much?
Would you describe how you manage configuration changes might be needed
after bolts are initialized and running?
for convenience, I copied the code here
public void updateConfig(String path, byte[] data) throws IOException {
if (data.length != 0) {
String name = path.substring(path.lastIndexOf("/") + 1);
if (path.startsWith(ConfigurationType.ENRICHMENT.getZookeeperRoot())) {
getConfigurations().updateSensorEnrichmentConfig(name, data);
reloadCallback(name, ConfigurationType.ENRICHMENT);
} else if (ConfigurationType.GLOBAL.getZookeeperRoot().equals(path)) {
getConfigurations().updateGlobalConfig(data);
reloadCallback(name, ConfigurationType.GLOBAL);
}
}
Thanks,
Re: configuration update during runtime
Posted by Ryan Merriman <me...@gmail.com>.
The object containing the configuration is a ConcurrentHashMap.
On Fri, Mar 31, 2017 at 8:48 AM, moshe jarusalem <tu...@gmail.com> wrote:
> Thanks for the explanation.
>
> As zookeeper and storm are different frameworks I think the function gets
> called by a thread used by zookeeper and any access to configuration needs
> to be synchronized. For example, EnrichmentConfigurations is a class
> which doesn't have synchronized methods. Would this cause any problem?
>
>
>
>
>
>
> On Fri, Mar 31, 2017 at 4:36 PM, Ryan Merriman <me...@gmail.com>
> wrote:
>
>> That method does a lot actually. Anytime a zookeeper change happens
>> during runtime, the curator client created on line 84 in ConfiguredBolt
>> calls that method on line 91. Think of it as a callback for zookeeper
>> updates.
>>
>> Ryan
>>
>> On Fri, Mar 31, 2017 at 7:49 AM, moshe jarusalem <tu...@gmail.com>
>> wrote:
>>
>>> Hi All,
>>> I have been looking the codes for ConfiguredBolt and its derivatives. I
>>> realized that updateConfig is actually not doing much?
>>>
>>> Would you describe how you manage configuration changes might be needed
>>> after bolts are initialized and running?
>>>
>>>
>>> for convenience, I copied the code here
>>>
>>> public void updateConfig(String path, byte[] data) throws IOException {
>>> if (data.length != 0) {
>>> String name = path.substring(path.lastIndexOf("/") + 1);
>>> if (path.startsWith(ConfigurationType.ENRICHMENT.getZookeeperRoot())) {
>>> getConfigurations().updateSensorEnrichmentConfig(name, data);
>>> reloadCallback(name, ConfigurationType.ENRICHMENT);
>>> } else if (ConfigurationType.GLOBAL.getZookeeperRoot().equals(path)) {
>>> getConfigurations().updateGlobalConfig(data);
>>> reloadCallback(name, ConfigurationType.GLOBAL);
>>> }
>>> }
>>>
>>>
>>>
>>> Thanks,
>>>
>>>
>>
>
Re: configuration update during runtime
Posted by moshe jarusalem <tu...@gmail.com>.
Thanks for the explanation.
As zookeeper and storm are different frameworks I think the function gets
called by a thread used by zookeeper and any access to configuration needs
to be synchronized. For example, EnrichmentConfigurations is a class which
doesn't have synchronized methods. Would this cause any problem?
On Fri, Mar 31, 2017 at 4:36 PM, Ryan Merriman <me...@gmail.com> wrote:
> That method does a lot actually. Anytime a zookeeper change happens
> during runtime, the curator client created on line 84 in ConfiguredBolt
> calls that method on line 91. Think of it as a callback for zookeeper
> updates.
>
> Ryan
>
> On Fri, Mar 31, 2017 at 7:49 AM, moshe jarusalem <tu...@gmail.com> wrote:
>
>> Hi All,
>> I have been looking the codes for ConfiguredBolt and its derivatives. I
>> realized that updateConfig is actually not doing much?
>>
>> Would you describe how you manage configuration changes might be needed
>> after bolts are initialized and running?
>>
>>
>> for convenience, I copied the code here
>>
>> public void updateConfig(String path, byte[] data) throws IOException {
>> if (data.length != 0) {
>> String name = path.substring(path.lastIndexOf("/") + 1);
>> if (path.startsWith(ConfigurationType.ENRICHMENT.getZookeeperRoot())) {
>> getConfigurations().updateSensorEnrichmentConfig(name, data);
>> reloadCallback(name, ConfigurationType.ENRICHMENT);
>> } else if (ConfigurationType.GLOBAL.getZookeeperRoot().equals(path)) {
>> getConfigurations().updateGlobalConfig(data);
>> reloadCallback(name, ConfigurationType.GLOBAL);
>> }
>> }
>>
>>
>>
>> Thanks,
>>
>>
>
Re: configuration update during runtime
Posted by Ryan Merriman <me...@gmail.com>.
That method does a lot actually. Anytime a zookeeper change happens during
runtime, the curator client created on line 84 in ConfiguredBolt calls that
method on line 91. Think of it as a callback for zookeeper updates.
Ryan
On Fri, Mar 31, 2017 at 7:49 AM, moshe jarusalem <tu...@gmail.com> wrote:
> Hi All,
> I have been looking the codes for ConfiguredBolt and its derivatives. I
> realized that updateConfig is actually not doing much?
>
> Would you describe how you manage configuration changes might be needed
> after bolts are initialized and running?
>
>
> for convenience, I copied the code here
>
> public void updateConfig(String path, byte[] data) throws IOException {
> if (data.length != 0) {
> String name = path.substring(path.lastIndexOf("/") + 1);
> if (path.startsWith(ConfigurationType.ENRICHMENT.getZookeeperRoot())) {
> getConfigurations().updateSensorEnrichmentConfig(name, data);
> reloadCallback(name, ConfigurationType.ENRICHMENT);
> } else if (ConfigurationType.GLOBAL.getZookeeperRoot().equals(path)) {
> getConfigurations().updateGlobalConfig(data);
> reloadCallback(name, ConfigurationType.GLOBAL);
> }
> }
>
>
>
> Thanks,
>
>