You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by "Nehal J. Wani" <ne...@gmail.com> on 2013/05/27 07:25:06 UTC

Reg: Moving CloudStack Management Server from one machine to another

Suppose I have only one management server and I want to take it down for
maintenance. What is the procedure to deploy the management server on a
different machine?

I tried the following on the new machine:

(i) Completed all the steps from Chapter 2 in the
Apache_CloudStack-4.0.2-Quick_Install_Guide-en-US.pdf

(ii) Copied all files from /etc/cloud, /usr/lib64/cloud, /var/lib/cloud
from the old machine to the new one.

(iii) Took dumps of the databases cloud, cloudbridge, cloud_usage from old
machine and then put them in the new machine.

(iv) Changed the ip in /etc/cloud/agent/agent.properties and
/etc/cloud/management/db.properties from 10.3.3.57 to 10.3.3.56


But i still get the error

2013-05-27 10:51:16,581 WARN  [agent.manager.ClusteredAgentManagerImpl]
(Job-Executor-2:job-93) Unable to connect to peer management server:
90520731198133, ip: 10.3.3.57 due to Connection refused
java.net.ConnectException: Connection refused
    at sun.nio.ch.Net.connect(Native Method)
    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:525)
    at java.nio.channels.SocketChannel.open(SocketChannel.java:164)
    at
com.cloud.agent.manager.ClusteredAgentManagerImpl.connectToPeer(ClusteredAgentManagerImpl.java:442)
    at
com.cloud.agent.manager.ClusteredAgentAttache.send(ClusteredAgentAttache.java:172)
    at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:388)
    at
com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:505)
    at
com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:458)
    at
com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:1079)
    at
com.cloud.vm.UserVmManagerImpl.stopVirtualMachine(UserVmManagerImpl.java:2804)
    at
com.cloud.event.ActionEventCallback.intercept(ActionEventCallback.java:36)
    at com.cloud.api.commands.StopVMCmd.execute(StopVMCmd.java:119)
    at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
    at
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
    at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
    at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
    at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
    at java.lang.Thread.run(Thread.java:679)

How to rectify this? How can I achieve my goal?

Thanking You,
Nehal J. Wani
UG2, BTech CS+MS(CL)
IIIT-Hyderabad
http://commanlinewani.blogspot.com

Re: Reg: Moving CloudStack Management Server from one machine to another

Posted by guren san <sa...@gmail.com>.
HI

i am use haproxy to do that , this is my haproxy config file ,  you need
edit  "Global Settings"  in "host"  value change to your haproxy host IP


#---------------------------------------------------------------------
# Example configuration for a possible web application.  See the
# full configuration options online.
#
#   http://haproxy.1wt.eu/download/1.4/doc/configuration.txt
#
#---------------------------------------------------------------------

#---------------------------------------------------------------------
# Global settings
#---------------------------------------------------------------------
global
    # to have these messages end up in /var/log/haproxy.log you will
    # need to:
    #
    # 1) configure syslog to accept network log events.  This is done
    #    by adding the '-r' option to the SYSLOGD_OPTIONS in
    #    /etc/sysconfig/syslog
    #
    # 2) configure local2 events to go to the /var/log/haproxy.log
    #   file. A line like the following can be added to
    #   /etc/sysconfig/syslog
    #
    #    local2.*                       /var/log/haproxy.log
    #
    log         127.0.0.1 local2

    chroot      /var/lib/haproxy
    pidfile     /var/run/haproxy.pid
    maxconn     4000
    user        haproxy
    group       haproxy
    daemon

    # turn on stats unix socket
    stats socket /var/lib/haproxy/stats

defaults
        log global
        maxconn         2000
        contimeout      5000
        clitimeout      30000
        srvtimeout      30000
        retries         5   # when connect 3 times cant connect it it is bad

listen cloud-agent
mode            tcp
        bind :8250
        option          dontlognull
        balance         source
        option          abortonclose
        maxconn         400
        balance         roundrobin
        server  csm-01 10.40.0.10:8250  check port 8250
        server  csm-02 10.40.0.201:8250  check port 8250

listen  servers
bind *:80
option httpchk OPTIONS /client
log global
mode http
option httplog
option forwardfor
balance roundrobin
cookie JSESSIONID insert preserve
server msg01 10.40.0.10:8080 cookie JSESSIONID_SERVER_1 check inter 50000
server msg02 10.40.0.201:8080  cookie JSESSIONID_SERVER_2 check inter 50000


listen  admin_stats
        bind 0.0.0.0:1080
        mode http
        stats auth admin:123456
        log 127.0.0.1 local0 err
        stats refresh 30s
        stats uri /admin?stats





2013/5/27 Steven Liang <st...@yesup.com>

>  Hi Geoff,
>
> Can I use LVS to implement load balancer. If so, what ports should I use?
>
> Regards,
> Steven
>
>
> On 05/27/2013 03:24 AM, Geoff Higginbottom wrote:
>
> Nehal,
>
> I would approach this in a slightly different way.
>
> Set up your new management server but as a 2nd node rather than a replacement.  To do so you need to put a load balancer in front of both of them and update the global setting 'host' with the new load balanced IP.
>
> After updating the 'host' setting and restarting the 'cloud-management' service on each, you will need to restart all system VMs so they use this new setting.
>
> You should then add a 2nd MySQL server, initially as a slave and allow it to replicate, then 'fail over' making the slave the new master.  The docs detail how to deploy multiple MySQL servers and how to fail over.
>
> Whilst this may seem like more work, we always deploy with at least two Management Servers and two MySQL servers as this provides better resilience makes performing planned maintenance much easier.
>
> Regards
>
> Geoff Higginbottom
> CTO / Cloud Architect
>
>
> D: +44(0)20 3603 0542<tel:+442036030542> | S: +44(0)20 3603 0540<tel:+442036030540> | M: +44(0)7968161581<tel:+447968161581>
> geoff.higginbottom@shapeblue.com<ma...@shapeblue.com> <ge...@shapeblue.com> | www.shapeblue.com
>
> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS
>
>
>
> On 27 May 2013, at 06:25, "Nehal J. Wani" <ne...@gmail.com> <ne...@gmail.com>> wrote:
>
> Suppose I have only one management server and I want to take it down for
> maintenance. What is the procedure to deploy the management server on a
> different machine?
>
> I tried the following on the new machine:
>
> (i) Completed all the steps from Chapter 2 in the
> Apache_CloudStack-4.0.2-Quick_Install_Guide-en-US.pdf
>
> (ii) Copied all files from /etc/cloud, /usr/lib64/cloud, /var/lib/cloud
> from the old machine to the new one.
>
> (iii) Took dumps of the databases cloud, cloudbridge, cloud_usage from old
> machine and then put them in the new machine.
>
> (iv) Changed the ip in /etc/cloud/agent/agent.properties and
> /etc/cloud/management/db.properties from 10.3.3.57 to 10.3.3.56
>
>
> But i still get the error
>
> 2013-05-27 10:51:16,581 WARN  [agent.manager.ClusteredAgentManagerImpl]
> (Job-Executor-2:job-93) Unable to connect to peer management server:
> 90520731198133, ip: 10.3.3.57 due to Connection refused
> java.net.ConnectException: Connection refused
>    at sun.nio.ch.Net.connect(Native Method)
>    at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:525)
>    at java.nio.channels.SocketChannel.open(SocketChannel.java:164)
>    at
> com.cloud.agent.manager.ClusteredAgentManagerImpl.connectToPeer(ClusteredAgentManagerImpl.java:442)
>    at
> com.cloud.agent.manager.ClusteredAgentAttache.send(ClusteredAgentAttache.java:172)
>    at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:388)
>    at
> com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:505)
>    at
> com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:458)
>    at
> com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:1079)
>    at
> com.cloud.vm.UserVmManagerImpl.stopVirtualMachine(UserVmManagerImpl.java:2804)
>    at
> com.cloud.event.ActionEventCallback.intercept(ActionEventCallback.java:36)
>    at com.cloud.api.commands.StopVMCmd.execute(StopVMCmd.java:119)
>    at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
>    at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
>    at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>    at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>    at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>    at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>    at java.lang.Thread.run(Thread.java:679)
>
> How to rectify this? How can I achieve my goal?
>
> Thanking You,
> Nehal J. Wani
> UG2, BTech CS+MS(CL)
> IIIT-Hyderabadhttp://commanlinewani.blogspot.com
> This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>
>
>
>
> --
> *Steven Liang*
> *Linux System Admin*
> *Phone*: 1.416.499.8009 ext. 2865
> *Cell Phone*: 1.647.718.5292
> *Email*: stevenliang@yesup.com
> www.yesup.com | account.yesup.com
>
> [image: Yesup]
>

Re: Reg: Moving CloudStack Management Server from one machine to another

Posted by Steven Liang <st...@yesup.com>.
Hi Geoff,

Can I use LVS to implement load balancer. If so, what ports should I use?

Regards,
Steven


On 05/27/2013 03:24 AM, Geoff Higginbottom wrote:
> Nehal,
>
> I would approach this in a slightly different way.
>
> Set up your new management server but as a 2nd node rather than a replacement.  To do so you need to put a load balancer in front of both of them and update the global setting 'host' with the new load balanced IP.
>
> After updating the 'host' setting and restarting the 'cloud-management' service on each, you will need to restart all system VMs so they use this new setting.
>
> You should then add a 2nd MySQL server, initially as a slave and allow it to replicate, then 'fail over' making the slave the new master.  The docs detail how to deploy multiple MySQL servers and how to fail over.
>
> Whilst this may seem like more work, we always deploy with at least two Management Servers and two MySQL servers as this provides better resilience makes performing planned maintenance much easier.
>
> Regards
>
> Geoff Higginbottom
> CTO / Cloud Architect
>
>
> D: +44(0)20 3603 0542<tel:+442036030542> | S: +44(0)20 3603 0540<tel:+442036030540> | M: +44(0)7968161581<tel:+447968161581>
>
> geoff.higginbottom@shapeblue.com<ma...@shapeblue.com> | www.shapeblue.com
>
> ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS
>
>
>
> On 27 May 2013, at 06:25, "Nehal J. Wani" <ne...@gmail.com>> wrote:
>
> Suppose I have only one management server and I want to take it down for
> maintenance. What is the procedure to deploy the management server on a
> different machine?
>
> I tried the following on the new machine:
>
> (i) Completed all the steps from Chapter 2 in the
> Apache_CloudStack-4.0.2-Quick_Install_Guide-en-US.pdf
>
> (ii) Copied all files from /etc/cloud, /usr/lib64/cloud, /var/lib/cloud
> from the old machine to the new one.
>
> (iii) Took dumps of the databases cloud, cloudbridge, cloud_usage from old
> machine and then put them in the new machine.
>
> (iv) Changed the ip in /etc/cloud/agent/agent.properties and
> /etc/cloud/management/db.properties from 10.3.3.57 to 10.3.3.56
>
>
> But i still get the error
>
> 2013-05-27 10:51:16,581 WARN  [agent.manager.ClusteredAgentManagerImpl]
> (Job-Executor-2:job-93) Unable to connect to peer management server:
> 90520731198133, ip: 10.3.3.57 due to Connection refused
> java.net.ConnectException: Connection refused
>     at sun.nio.ch.Net.connect(Native Method)
>     at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:525)
>     at java.nio.channels.SocketChannel.open(SocketChannel.java:164)
>     at
> com.cloud.agent.manager.ClusteredAgentManagerImpl.connectToPeer(ClusteredAgentManagerImpl.java:442)
>     at
> com.cloud.agent.manager.ClusteredAgentAttache.send(ClusteredAgentAttache.java:172)
>     at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:388)
>     at
> com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:505)
>     at
> com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:458)
>     at
> com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:1079)
>     at
> com.cloud.vm.UserVmManagerImpl.stopVirtualMachine(UserVmManagerImpl.java:2804)
>     at
> com.cloud.event.ActionEventCallback.intercept(ActionEventCallback.java:36)
>     at com.cloud.api.commands.StopVMCmd.execute(StopVMCmd.java:119)
>     at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
>     at
> com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
>     at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>     at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>     at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>     at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>     at java.lang.Thread.run(Thread.java:679)
>
> How to rectify this? How can I achieve my goal?
>
> Thanking You,
> Nehal J. Wani
> UG2, BTech CS+MS(CL)
> IIIT-Hyderabad
> http://commanlinewani.blogspot.com
> This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>


-- 
*Steven Liang*
*Linux System Admin*
*Phone*: 1.416.499.8009 ext. 2865
*Cell Phone*: 1.647.718.5292
*Email*: stevenliang@yesup.com <ma...@yesup.com>
www.yesup.com <http://www.yesup.com> | account.yesup.com 
<http://account.yesup.com>

Yesup

Re: Reg: Moving CloudStack Management Server from one machine to another

Posted by Geoff Higginbottom <ge...@shapeblue.com>.
Nehal,

I would approach this in a slightly different way.

Set up your new management server but as a 2nd node rather than a replacement.  To do so you need to put a load balancer in front of both of them and update the global setting 'host' with the new load balanced IP.

After updating the 'host' setting and restarting the 'cloud-management' service on each, you will need to restart all system VMs so they use this new setting.

You should then add a 2nd MySQL server, initially as a slave and allow it to replicate, then 'fail over' making the slave the new master.  The docs detail how to deploy multiple MySQL servers and how to fail over.

Whilst this may seem like more work, we always deploy with at least two Management Servers and two MySQL servers as this provides better resilience makes performing planned maintenance much easier.

Regards

Geoff Higginbottom
CTO / Cloud Architect


D: +44(0)20 3603 0542<tel:+442036030542> | S: +44(0)20 3603 0540<tel:+442036030540> | M: +44(0)7968161581<tel:+447968161581>

geoff.higginbottom@shapeblue.com<ma...@shapeblue.com> | www.shapeblue.com

ShapeBlue Ltd, 53 Chandos Place, Covent Garden, London, WC2N 4HS



On 27 May 2013, at 06:25, "Nehal J. Wani" <ne...@gmail.com>> wrote:

Suppose I have only one management server and I want to take it down for
maintenance. What is the procedure to deploy the management server on a
different machine?

I tried the following on the new machine:

(i) Completed all the steps from Chapter 2 in the
Apache_CloudStack-4.0.2-Quick_Install_Guide-en-US.pdf

(ii) Copied all files from /etc/cloud, /usr/lib64/cloud, /var/lib/cloud
from the old machine to the new one.

(iii) Took dumps of the databases cloud, cloudbridge, cloud_usage from old
machine and then put them in the new machine.

(iv) Changed the ip in /etc/cloud/agent/agent.properties and
/etc/cloud/management/db.properties from 10.3.3.57 to 10.3.3.56


But i still get the error

2013-05-27 10:51:16,581 WARN  [agent.manager.ClusteredAgentManagerImpl]
(Job-Executor-2:job-93) Unable to connect to peer management server:
90520731198133, ip: 10.3.3.57 due to Connection refused
java.net.ConnectException: Connection refused
   at sun.nio.ch.Net.connect(Native Method)
   at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:525)
   at java.nio.channels.SocketChannel.open(SocketChannel.java:164)
   at
com.cloud.agent.manager.ClusteredAgentManagerImpl.connectToPeer(ClusteredAgentManagerImpl.java:442)
   at
com.cloud.agent.manager.ClusteredAgentAttache.send(ClusteredAgentAttache.java:172)
   at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:388)
   at
com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:505)
   at
com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:458)
   at
com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:1079)
   at
com.cloud.vm.UserVmManagerImpl.stopVirtualMachine(UserVmManagerImpl.java:2804)
   at
com.cloud.event.ActionEventCallback.intercept(ActionEventCallback.java:36)
   at com.cloud.api.commands.StopVMCmd.execute(StopVMCmd.java:119)
   at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
   at
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
   at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
   at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
   at java.util.concurrent.FutureTask.run(FutureTask.java:166)
   at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
   at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
   at java.lang.Thread.run(Thread.java:679)

How to rectify this? How can I achieve my goal?

Thanking You,
Nehal J. Wani
UG2, BTech CS+MS(CL)
IIIT-Hyderabad
http://commanlinewani.blogspot.com
This email and any attachments to it may be confidential and are intended solely for the use of the individual to whom it is addressed. Any views or opinions expressed are solely those of the author and do not necessarily represent those of Shape Blue Ltd or related companies. If you are not the intended recipient of this email, you must neither take any action based upon its contents, nor copy or show it to anyone. Please contact the sender if you believe you have received this email in error. Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue Services India LLP is operated under license from Shape Blue Ltd. ShapeBlue is a registered trademark.