[jira] [Commented] (STORM-2269) Dynamic reconfiguration for the nodes in Nimbus/Pacemaker clusters

["Robert Joseph Evans (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/STORM-2269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15798470#comment-15798470 ] 

Robert Joseph Evans commented on STORM-2269:
--------------------------------------------

[~sachingoyal],

I assigned the JIRA to you because in your e-mail you said you will start looking at the code.

https://github.com/apache/storm/blob/master/storm-core/src/jvm/org/apache/storm/pacemaker/PacemakerClientPool.java

is for the pacemaker client.

The nimbus code is similar at

https://github.com/apache/storm/blob/master/storm-core/src/jvm/org/apache/storm/utils/NimbusClient.java

Specifically getConfiguredClientAs.

On the NimbusCiient we do want to maintain backwards compatibility as much as possible.

I'm not sure the best way to deal with this.  It might be nice to create a new Config Map that reads in a file and will keep it cached in memory for a period of time, but we wrap the configs in other classes and transform them a lot so it is far from ideal.

It might be best to just add in some new APIs that will take the file location instead of the Config itself and have it deal with loading it each time, and or cacheing it.

> Dynamic reconfiguration for the nodes in Nimbus/Pacemaker clusters
> ------------------------------------------------------------------
>
>                 Key: STORM-2269
>                 URL: https://issues.apache.org/jira/browse/STORM-2269
>             Project: Apache Storm
>          Issue Type: Bug
>    Affects Versions: 2.0.0, 1.1.0
>            Reporter: Sachin Goyal
>            Assignee: Sachin Goyal
>              Labels: nimbus, pacemaker
>
> Reference: https://zookeeper.apache.org/doc/trunk/zookeeperReconfig.html
> It would be nice to have a similar functionality for Nimbus/Pacemaker clusters too.
> As that would eliminate the need for restarting servers in the Nimbus/Pacemaker clusters whenever a node exits or joins these clusters.
> ----------------------------------------------------------
> Reply from Bobby Evans on the dev group:
> ----------------------------------------------------------
> There is nothing for that right now on pacemaker.
> You can do it with nimbus so long as at least one of the original nodes is still up.
> But in either case it would not be too difficult to make it all fully functional.
> The two critical pieces would be in giving the workers and daemons a way to reload these specific configs dynamically.
> Then it would be documenting the order of operations to be sure nothing goes wrong.
> *Adding Pacemaker Node(s)*
> # bring up the new node(s).
> # update nimbus configs to start reading from the new nodes.
> # update all of the worker nodes to let workers start writing to the new node.
> *Removing Pacemaker Node(s)*
> # Shut down pacemaker nodes/update configs on workers (order should not matter so long as there are enough pacemaker nodes up to handle the load)
> # update the nimbus configs to not try and read from the old nodes
> *Adding new Nimbus Node(s)*
> # Bring up the new nimbus with the new config.
> # update all of the other nodes (including any machines that clients come from) with new config (order does not matter)
> *Removing Nimbus Node(s)*
> # Shut down the old nodes and update the configs on all the boxes in any order you want.
> # This should just work so long as you have at least one nimbus node still up.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)