You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by "Alex Mathiasen (JIRA)" <ji...@apache.org> on 2012/11/21 09:21:57 UTC
[jira] [Created] (CLOUDSTACK-525) Cloudstack Redundant Virtual
Router issues both MASTER.
Alex Mathiasen created CLOUDSTACK-525:
-----------------------------------------
Summary: Cloudstack Redundant Virtual Router issues both MASTER.
Key: CLOUDSTACK-525
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-525
Project: CloudStack
Issue Type: Bug
Components: Install and Setup, KVM, Management Server, Network Controller, Network Devices
Affects Versions: 4.0.0
Environment: Two servers with same hardware running Ubuntu 12.04 LTS.
Reporter: Alex Mathiasen
Priority: Critical
Steps:
- Two hosts added to cluster
- Add network offering with RVR set,
- Create instance using that network offering
- Look at the state of both virtual routers on each host being MASTER
Due to the state above, you will recieve errors with the following message:
More than one redundant virtual router is in MASTER state! Router r-15-VM and router r-14-VM - Virtual router (name: r-15-VM, id: 15 and router (name: r-14-VM, id: 15) are both in MASTER state! If the problem persist, restart both of routers.
No errors in the logfiles, I have tried rebooting the virtual routers. Migrating the second virtual router onto the same host will fix the state to MASTER and BACKUP, but it's not the point - As the virtual router is supposed to run on each host.
Looking at the console on both virtual routers, you will discover that keepalived, conntrackd and haproxy failed to start.
Trying to ping the virtual routers by their local-link address. from management-server, you will discover the following result:
This is the server running on same host as the instance.
root@cloudsrv1:/# ping 169.254.0.169
PING 169.254.0.169 (169.254.0.169) 56(84) bytes of data.
64 bytes from 169.254.0.169: icmp_req=1 ttl=64 time=0.193 ms
This is the server running on the second host, which should behave as BACKUP.
root@cloudsrv1:/mnt/mysql# ping 169.254.3.121
PING 169.254.3.121 (169.254.3.121) 56(84) bytes of data.
>From 169.254.0.1 icmp_seq=1 Destination Host Unreachable
View from management-server.log:
2012-11-21 09:19:42,066 DEBUG [agent.transport.Request] (RouterStatusMonitor-1:null) Seq 3-2029663984: Sending { Cmd , MgmtId: 279278805451002, via: 3, Ver: v1, Flags: 100011, [{"CheckRouterCommand":{"accessDetails":{"router.ip":"169.254.0.169","router.name":"r-12-VM"},"wait":60}}] }
2012-11-21 09:19:42,212 DEBUG [agent.transport.Request] (AgentManager-Handler-8:null) Seq 3-2029663984: Processing: { Ans: , MgmtId: 279278805451002, via: 3, Ver: v1, Flags: 10, [{"CheckRouterAnswer":{"state":"MASTER","isBumped":false,"result":true,"details":"Status: MASTER&Bumped: NO","wait":0}}] }
2012-11-21 09:19:42,212 DEBUG [agent.transport.Request] (RouterStatusMonitor-1:null) Seq 3-2029663984: Received: { Ans: , MgmtId: 279278805451002, via: 3, Ver: v1, Flags: 10, { CheckRouterAnswer } }
2012-11-21 09:19:42,213 DEBUG [agent.manager.AgentManagerImpl] (RouterStatusMonitor-1:null) Details from executing class com.cloud.agent.api.CheckRouterCommand: Status: MASTER&Bumped: NO
2012-11-21 09:19:42,267 DEBUG [agent.transport.Request] (RouterStatusMonitor-1:null) Seq 5-1237057653: Sending { Cmd , MgmtId: 279278805451002, via: 5, Ver: v1, Flags: 100011, [{"CheckRouterCommand":{"accessDetails":{"router.ip":"169.254.3.121","router.name":"r-13-VM"},"wait":60}}] }
2012-11-21 09:19:42,421 DEBUG [agent.transport.Request] (AgentManager-Handler-14:null) Seq 5-1237057653: Processing: { Ans: , MgmtId: 279278805451002, via: 5, Ver: v1, Flags: 10, [{"CheckRouterAnswer":{"state":"MASTER","isBumped":false,"result":true,"details":"Status: MASTER&Bumped: NO","wait":0}}] }
2012-11-21 09:19:42,421 DEBUG [agent.transport.Request] (RouterStatusMonitor-1:null) Seq 5-1237057653: Received: { Ans: , MgmtId: 279278805451002, via: 5, Ver: v1, Flags: 10, { CheckRouterAnswer } }
2012-11-21 09:19:42,421 DEBUG [agent.manager.AgentManagerImpl] (RouterStatusMonitor-1:null) Details from executing class com.cloud.agent.api.CheckRouterCommand: Status: MASTER&Bumped: NO
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CLOUDSTACK-525) Cloudstack Redundant Virtual
Router issues both MASTER.
Posted by "Alex Mathiasen (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13501765#comment-13501765 ]
Alex Mathiasen commented on CLOUDSTACK-525:
-------------------------------------------
The network offering is created with the following options enabled:
VPN, DHCP, DNS, Firewall, Load Balancer, User Data, Source NAT, Static NAT and Port forwarding.
Guest Type Isolated
Availability Optional - This doesnt seem to be configurable when setting up the network offering.
Conserve mode No
Traffic Type Guest
Supported Services Lb, Dns, Firewall, UserData, SourceNat, PortForwarding, Dhcp, StaticNat, Vpn
Service Capabilities SupportedLBIsolation: dedicated, ElasticLb: false, SupportedSourceNatTypes: peraccount, RedundantRouter: true, ElasticIp: false
> Cloudstack Redundant Virtual Router issues both MASTER.
> --------------------------------------------------------
>
> Key: CLOUDSTACK-525
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-525
> Project: CloudStack
> Issue Type: Bug
> Components: Install and Setup, KVM, Management Server, Network Controller, Network Devices
> Affects Versions: 4.0.0
> Environment: Two servers with same hardware running Ubuntu 12.04 LTS.
> Reporter: Alex Mathiasen
> Priority: Critical
> Labels: RVR
>
> Steps:
> - Two hosts added to cluster
> - Add network offering with RVR set,
> - Create instance using that network offering
> - Look at the state of both virtual routers on each host being MASTER
> Due to the state above, you will recieve errors with the following message:
> More than one redundant virtual router is in MASTER state! Router r-15-VM and router r-14-VM - Virtual router (name: r-15-VM, id: 15 and router (name: r-14-VM, id: 15) are both in MASTER state! If the problem persist, restart both of routers.
> No errors in the logfiles, I have tried rebooting the virtual routers. Migrating the second virtual router onto the same host will fix the state to MASTER and BACKUP, but it's not the point - As the virtual router is supposed to run on each host.
> Looking at the console on both virtual routers, you will discover that keepalived, conntrackd and haproxy failed to start.
> Trying to ping the virtual routers by their local-link address. from management-server, you will discover the following result:
> This is the server running on same host as the instance.
> root@cloudsrv1:/# ping 169.254.0.169
> PING 169.254.0.169 (169.254.0.169) 56(84) bytes of data.
> 64 bytes from 169.254.0.169: icmp_req=1 ttl=64 time=0.193 ms
> This is the server running on the second host, which should behave as BACKUP.
> root@cloudsrv1:/mnt/mysql# ping 169.254.3.121
> PING 169.254.3.121 (169.254.3.121) 56(84) bytes of data.
> From 169.254.0.1 icmp_seq=1 Destination Host Unreachable
> View from management-server.log:
> 2012-11-21 09:19:42,066 DEBUG [agent.transport.Request] (RouterStatusMonitor-1:null) Seq 3-2029663984: Sending { Cmd , MgmtId: 279278805451002, via: 3, Ver: v1, Flags: 100011, [{"CheckRouterCommand":{"accessDetails":{"router.ip":"169.254.0.169","router.name":"r-12-VM"},"wait":60}}] }
> 2012-11-21 09:19:42,212 DEBUG [agent.transport.Request] (AgentManager-Handler-8:null) Seq 3-2029663984: Processing: { Ans: , MgmtId: 279278805451002, via: 3, Ver: v1, Flags: 10, [{"CheckRouterAnswer":{"state":"MASTER","isBumped":false,"result":true,"details":"Status: MASTER&Bumped: NO","wait":0}}] }
> 2012-11-21 09:19:42,212 DEBUG [agent.transport.Request] (RouterStatusMonitor-1:null) Seq 3-2029663984: Received: { Ans: , MgmtId: 279278805451002, via: 3, Ver: v1, Flags: 10, { CheckRouterAnswer } }
> 2012-11-21 09:19:42,213 DEBUG [agent.manager.AgentManagerImpl] (RouterStatusMonitor-1:null) Details from executing class com.cloud.agent.api.CheckRouterCommand: Status: MASTER&Bumped: NO
> 2012-11-21 09:19:42,267 DEBUG [agent.transport.Request] (RouterStatusMonitor-1:null) Seq 5-1237057653: Sending { Cmd , MgmtId: 279278805451002, via: 5, Ver: v1, Flags: 100011, [{"CheckRouterCommand":{"accessDetails":{"router.ip":"169.254.3.121","router.name":"r-13-VM"},"wait":60}}] }
> 2012-11-21 09:19:42,421 DEBUG [agent.transport.Request] (AgentManager-Handler-14:null) Seq 5-1237057653: Processing: { Ans: , MgmtId: 279278805451002, via: 5, Ver: v1, Flags: 10, [{"CheckRouterAnswer":{"state":"MASTER","isBumped":false,"result":true,"details":"Status: MASTER&Bumped: NO","wait":0}}] }
> 2012-11-21 09:19:42,421 DEBUG [agent.transport.Request] (RouterStatusMonitor-1:null) Seq 5-1237057653: Received: { Ans: , MgmtId: 279278805451002, via: 5, Ver: v1, Flags: 10, { CheckRouterAnswer } }
> 2012-11-21 09:19:42,421 DEBUG [agent.manager.AgentManagerImpl] (RouterStatusMonitor-1:null) Details from executing class com.cloud.agent.api.CheckRouterCommand: Status: MASTER&Bumped: NO
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira