You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@zookeeper.apache.org by Check Peck <co...@gmail.com> on 2018/02/02 20:12:48 UTC
replacing one zookeeper machine with brand new machine
I have a zookeeper ensemble of 5 servers and I am using exhibitor on top of
it. And I installed exhibitor and setup zookeeper by following this link:
https://github.com/soabase/exhibitor/wiki/Building-Exhibitor
Below is how all my zookeeper machines are setup in exhibitor
S:1:machineA,
S:2:machineB,
S:3:machineC,
S:4:machineD,
S:5:machineE
Now for some reasons, I need to replace "machineE" with brand new
"machineF". What is the best way by which I can safely remove one machine
and replace it with new machine?
Re: replacing one zookeeper machine with brand new machine
Posted by Check Peck <co...@gmail.com>.
I have a zookeeper ensemble of 5 servers and I am using exhibitor on top
of it. And I installed exhibitor and setup zookeeper by following this link:
https://github.com/soabase/exhibitor/wiki/Building-Exhibitor
Below is how all my zookeeper machines are setup in exhibitor
S:1:machineA,
S:2:machineB,
S:3:machineC,
S:4:machineD,
S:5:machineE
Now for some reasons, I need to replace "machineE" with brand new
"machineF". What is the best way by which I can safely remove one machine
and replace it with new machine?
Re: replacing one zookeeper machine with brand new machine
Posted by Check Peck <co...@gmail.com>.
I have a zookeeper ensemble of 5 servers and I am using exhibitor on top
of it. And I installed exhibitor and setup zookeeper by following this link:
https://github.com/soabase/exhibitor/wiki/Building-Exhibitor
Below is how all my zookeeper machines are setup in exhibitor
S:1:machineA,
S:2:machineB,
S:3:machineC,
S:4:machineD,
S:5:machineE
Now for some reasons, I need to replace "machineE" with brand new
"machineF". What is the best way by which I can safely remove one machine
and replace it with new machine?
Re: replacing one zookeeper machine with brand new machine
Posted by "Washko, Daniel" <dw...@gannett.com>.
There is a restart/stop button on the explorer page. If you turn off automatic restarts you can stop an instance. To remove the instance I believe there is a list of zookeeper nodes on the Config page. If you edit that and remove the zookeeper instances you are terminating and commit the change it should push it out. Once the restarts complete and all is green, you should be able to safely terminate the removed nodes.
--
Daniel S Washko
Solutions Architect
Phone: 757 667 1463
dwashko@gannett.comgannett.com <http://www.gannett.com/>
On 2/5/18, 3:27 PM, "Check Peck" <co...@gmail.com> wrote:
Is there an option to remove a zookeeper node from exhibitor? I am not sure
it is there I guess.
On Mon, Feb 5, 2018 at 10:21 AM, Washko, Daniel <dw...@gannett.com> wrote:
> The steps are the same whether Exhibitor is in the mix or not. Exhibitor
> will take care of management, though. I would recommend backing up the data
> in your Zookeeper ensemble just to be safe.
>
> 1) Spin up a new zookeeper and configure it to use exhibitor.
> 2) Let exhibitor bring it into the ensemble.
> 3) Use exhibitor to remove the old node.
> 4) Terminate the old node when exhibitor says it is no longer in the
> ensemble; or it is down.
>
> It has been a few years since I have worked with Exhibitor. It should
> automatically pull the new node into the ensemble. I believe there is an
> option to remove a node. You will be presented with a choice on how you
> want to initiate the changes - a rolling restart of restart all at once. I
> would recommend a rolling restart if you want to keep the ensemble live
> while you make the changes.
>
> If you have a problem with removing one of the nodes, you can edit the
> node list in exhibitor, remove that node, and save the configuration.
> Again, this will prompt for a rolling restart or parallel restart.
>
> Without exhibitor these are the steps I follow:
>
> 1) Backup the data
> 2) Spin up a new zookeeper
> 3) Identify the master
> 4) Alter the configuration on each zookeeper to add the new node and to
> add the other nodes to the new zookeeper. Be aware of the zookeeper ID, it
> has to be unique.
> 5) Perform a rolling restart of each node with the master last.
> 6) Verify the new master and that the data stored in zookeeper has
> migrated successfully to the new node.
> 7) Remove the old node from each config.
> 8) Stop zookeeper on the old node and do a rolling restart of the
> remaining zookeepr nodes with the master last.
> 9) Terminate the old node.
>
> --
> Daniel S Washko
> Solutions Architect
>
>
>
> Phone: 757 667 1463
> dwashko@gannett.comgannett.com <http://www.gannett.com/>
>
> On 2/2/18, 3:20 PM, "Check Peck" <co...@gmail.com> wrote:
>
> I have a zookeeper ensemble of 5 servers and I am using exhibitor on
> top of
> it. And I installed exhibitor and setup zookeeper by following this
> link:
>
> https://github.com/soabase/exhibitor/wiki/Building-Exhibitor
>
> Below is how all my zookeeper machines are setup in exhibitor
>
> S:1:machineA,
> S:2:machineB,
> S:3:machineC,
> S:4:machineD,
> S:5:machineE
>
> Now for some reasons, I need to replace "machineE" with brand new
> "machineF". What is the best way by which I can safely remove one
> machine
> and replace it with new machine?
>
>
>
Re: replacing one zookeeper machine with brand new machine
Posted by Check Peck <co...@gmail.com>.
Is there an option to remove a zookeeper node from exhibitor? I am not sure
it is there I guess.
On Mon, Feb 5, 2018 at 10:21 AM, Washko, Daniel <dw...@gannett.com> wrote:
> The steps are the same whether Exhibitor is in the mix or not. Exhibitor
> will take care of management, though. I would recommend backing up the data
> in your Zookeeper ensemble just to be safe.
>
> 1) Spin up a new zookeeper and configure it to use exhibitor.
> 2) Let exhibitor bring it into the ensemble.
> 3) Use exhibitor to remove the old node.
> 4) Terminate the old node when exhibitor says it is no longer in the
> ensemble; or it is down.
>
> It has been a few years since I have worked with Exhibitor. It should
> automatically pull the new node into the ensemble. I believe there is an
> option to remove a node. You will be presented with a choice on how you
> want to initiate the changes - a rolling restart of restart all at once. I
> would recommend a rolling restart if you want to keep the ensemble live
> while you make the changes.
>
> If you have a problem with removing one of the nodes, you can edit the
> node list in exhibitor, remove that node, and save the configuration.
> Again, this will prompt for a rolling restart or parallel restart.
>
> Without exhibitor these are the steps I follow:
>
> 1) Backup the data
> 2) Spin up a new zookeeper
> 3) Identify the master
> 4) Alter the configuration on each zookeeper to add the new node and to
> add the other nodes to the new zookeeper. Be aware of the zookeeper ID, it
> has to be unique.
> 5) Perform a rolling restart of each node with the master last.
> 6) Verify the new master and that the data stored in zookeeper has
> migrated successfully to the new node.
> 7) Remove the old node from each config.
> 8) Stop zookeeper on the old node and do a rolling restart of the
> remaining zookeepr nodes with the master last.
> 9) Terminate the old node.
>
> --
> Daniel S Washko
> Solutions Architect
>
>
>
> Phone: 757 667 1463
> dwashko@gannett.comgannett.com <http://www.gannett.com/>
>
> On 2/2/18, 3:20 PM, "Check Peck" <co...@gmail.com> wrote:
>
> I have a zookeeper ensemble of 5 servers and I am using exhibitor on
> top of
> it. And I installed exhibitor and setup zookeeper by following this
> link:
>
> https://github.com/soabase/exhibitor/wiki/Building-Exhibitor
>
> Below is how all my zookeeper machines are setup in exhibitor
>
> S:1:machineA,
> S:2:machineB,
> S:3:machineC,
> S:4:machineD,
> S:5:machineE
>
> Now for some reasons, I need to replace "machineE" with brand new
> "machineF". What is the best way by which I can safely remove one
> machine
> and replace it with new machine?
>
>
>
Re: replacing one zookeeper machine with brand new machine
Posted by "Washko, Daniel" <dw...@gannett.com>.
The steps are the same whether Exhibitor is in the mix or not. Exhibitor will take care of management, though. I would recommend backing up the data in your Zookeeper ensemble just to be safe.
1) Spin up a new zookeeper and configure it to use exhibitor.
2) Let exhibitor bring it into the ensemble.
3) Use exhibitor to remove the old node.
4) Terminate the old node when exhibitor says it is no longer in the ensemble; or it is down.
It has been a few years since I have worked with Exhibitor. It should automatically pull the new node into the ensemble. I believe there is an option to remove a node. You will be presented with a choice on how you want to initiate the changes - a rolling restart of restart all at once. I would recommend a rolling restart if you want to keep the ensemble live while you make the changes.
If you have a problem with removing one of the nodes, you can edit the node list in exhibitor, remove that node, and save the configuration. Again, this will prompt for a rolling restart or parallel restart.
Without exhibitor these are the steps I follow:
1) Backup the data
2) Spin up a new zookeeper
3) Identify the master
4) Alter the configuration on each zookeeper to add the new node and to add the other nodes to the new zookeeper. Be aware of the zookeeper ID, it has to be unique.
5) Perform a rolling restart of each node with the master last.
6) Verify the new master and that the data stored in zookeeper has migrated successfully to the new node.
7) Remove the old node from each config.
8) Stop zookeeper on the old node and do a rolling restart of the remaining zookeepr nodes with the master last.
9) Terminate the old node.
--
Daniel S Washko
Solutions Architect
Phone: 757 667 1463
dwashko@gannett.comgannett.com <http://www.gannett.com/>
On 2/2/18, 3:20 PM, "Check Peck" <co...@gmail.com> wrote:
I have a zookeeper ensemble of 5 servers and I am using exhibitor on top of
it. And I installed exhibitor and setup zookeeper by following this link:
https://github.com/soabase/exhibitor/wiki/Building-Exhibitor
Below is how all my zookeeper machines are setup in exhibitor
S:1:machineA,
S:2:machineB,
S:3:machineC,
S:4:machineD,
S:5:machineE
Now for some reasons, I need to replace "machineE" with brand new
"machineF". What is the best way by which I can safely remove one machine
and replace it with new machine?