You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cloudstack.apache.org by "Abhinav Roy (JIRA)" <ji...@apache.org> on 2012/09/17 22:04:07 UTC
[jira] [Created] (CLOUDSTACK-127) Adding VMs to an existing LB rule
after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Abhinav Roy created CLOUDSTACK-127:
--------------------------------------
Summary: Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
Key: CLOUDSTACK-127
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
Project: CloudStack
Issue Type: Bug
Components: Network Controller
Affects Versions: pre-4.0.0
Environment: MS OS : Rhel 6.2
Hypervisor : Xen 6.0.2
BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
Reporter: Abhinav Roy
Fix For: pre-4.0.0
Steps :
=========================
1. Deploy an advanced zone setup with CS 3.0.2
2. Deploy couple of VMs.
3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
4. Upgrade the setup to ASF 4.0
5. Create a VM on the existing Network.
6. Add the VM created in step 5 to the LB rule created in step 3.
Expected Behaviour :
=========================
All the steps should be successful and no error should be seen.
Observed Behaviour :
=========================
Step 6 fails with the following exceptions
2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CLOUDSTACK-127) Adding VMs to an existing LB rule
after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Abhinav Roy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Abhinav Roy updated CLOUDSTACK-127:
-----------------------------------
Attachment: management-server.log
Attaching management server log
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Fix For: pre-4.0.0
>
> Attachments: management-server.log
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CLOUDSTACK-127) Adding VMs to an existing LB
rule after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Sheng Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458042#comment-13458042 ]
Sheng Yang commented on CLOUDSTACK-127:
---------------------------------------
I cannot find ipassoc fail in the SMlog you provided. Could you provide the log when this issue happened?
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log, messages, SMlog.10
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CLOUDSTACK-127) Adding VMs to an existing LB
rule after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "edison su (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457495#comment-13457495 ]
edison su commented on CLOUDSTACK-127:
--------------------------------------
On one of our demo machines, yas the same issue. The root cause is the file system of router vm is in read-only state. Destroy/recreate router solves the problem.
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (CLOUDSTACK-127) Adding VMs to an existing LB rule
after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Abhinav Roy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Abhinav Roy closed CLOUDSTACK-127.
----------------------------------
Thanks Sheng.
Closing the issue as per Sheng's comments.
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log, messages, SMlog.10, SMlog.11
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CLOUDSTACK-127) Adding VMs to an existing LB
rule after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Sudha Ponnaganti (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sudha Ponnaganti reassigned CLOUDSTACK-127:
-------------------------------------------
Assignee: Chip Childers
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Chip Childers
> Fix For: pre-4.0.0
>
> Attachments: management-server.log
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (CLOUDSTACK-127) Adding VMs to an existing LB
rule after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Sheng Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sheng Yang resolved CLOUDSTACK-127.
-----------------------------------
Resolution: Invalid
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log, messages, SMlog.10, SMlog.11
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CLOUDSTACK-127) Adding VMs to an existing LB rule
after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Abhinav Roy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Abhinav Roy updated CLOUDSTACK-127:
-----------------------------------
Attachment: SMlog.11
Sorry Sheng. Here is the log having that error.
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log, messages, SMlog.10, SMlog.11
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CLOUDSTACK-127) Adding VMs to an existing LB
rule after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Sheng Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458227#comment-13458227 ]
Sheng Yang commented on CLOUDSTACK-127:
---------------------------------------
Alex is correct.
[14655] 2012-09-18 00:32:21.052174 #### VMOPS enter routerProxy ####
[14655] 2012-09-18 00:32:21.052307 ['/bin/bash', '/opt/xensource/bin/router_proxy.sh', 'ipassoc.sh', '169.254.2.233', '-A', '', '-s', '-f', '-l', '10.102.125.69/24', '-c', 'eth2', '-g', '10.102.125.1']
[14655] 2012-09-18 00:32:21.202257 FAILED: (rc 127) stdout: '', stderr: 'bash: /opt/cloud/bin/ipassoc.sh: No such file or directory
'
[14655] 2012-09-18 00:32:21.202499 routerProxy command ipassoc.sh 169.254.2.233 -A -s -f -l 10.102.125.69/24 -c eth2 -g 10.102.125.1 failed
[14655] 2012-09-18 00:32:21.202608 #### VMOPS exit routerProxy ####
Please reboot your VR from CS to get the latest version of scripts.
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log, messages, SMlog.10, SMlog.11
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CLOUDSTACK-127) Adding VMs to an existing LB
rule after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Sheng Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13457488#comment-13457488 ]
Sheng Yang commented on CLOUDSTACK-127:
---------------------------------------
Yes, also please attach /var/log/SMlog in the XenServer, with /var/log/messages in the router for analysing.
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (CLOUDSTACK-127) Adding VMs to an existing LB rule
after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Abhinav Roy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Abhinav Roy updated CLOUDSTACK-127:
-----------------------------------
Attachment: messages
SMlog.10
Attaching SMlog of XenServer and
messages from Virtual Router
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log, messages, SMlog.10
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CLOUDSTACK-127) Adding VMs to an existing LB
rule after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Sheng Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458229#comment-13458229 ]
Sheng Yang commented on CLOUDSTACK-127:
---------------------------------------
Also it's possible you got the same issue with Edison, either way there is something wrong with your VR's scripts.
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log, messages, SMlog.10, SMlog.11
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CLOUDSTACK-127) Adding VMs to an existing LB
rule after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Sheng Yang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13459014#comment-13459014 ]
Sheng Yang commented on CLOUDSTACK-127:
---------------------------------------
It's by design, and documented in the upgrade instructions.
The reason here is we need to update the VR to the newer version. The old version VR wouldn't guarantee compatibility with new version mgmt server. But it's not always broke all the functions. That's why you can see that create VM successful, but assoicate new IP failed.
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log, messages, SMlog.10, SMlog.11
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (CLOUDSTACK-127) Adding VMs to an existing LB
rule after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Alex Huang (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alex Huang reassigned CLOUDSTACK-127:
-------------------------------------
Assignee: Sheng Yang (was: Chip Childers)
This looks like a version mismatch problem again with the virtual router is not upgraded after upgrading cloudstack. Please verify and if it is resolve it as so.
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CLOUDSTACK-127) Adding VMs to an existing LB
rule after upgrading from CS 3.0.2 to ASF 4.0 fails with
"ResourceUnavailableException: Resource [DataCenter:1] is unreachable:
Unable to apply ip association on router "
Posted by "Abhinav Roy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13458616#comment-13458616 ]
Abhinav Roy commented on CLOUDSTACK-127:
----------------------------------------
Yes , rebooting the router does work. But this happens everytime after I do a upgrade. I have tried this 3-4 times already. So does this mean everytime we need to reboot the router after upgrade?
Also why the creation of VMs with the same router succeeds? And why don't we see this error before upgrade ?
> Adding VMs to an existing LB rule after upgrading from CS 3.0.2 to ASF 4.0 fails with "ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router "
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-127
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-127
> Project: CloudStack
> Issue Type: Bug
> Components: Network Controller
> Affects Versions: pre-4.0.0
> Environment: MS OS : Rhel 6.2
> Hypervisor : Xen 6.0.2
> BUILDs : 3.0.2 and ASF 4.0 [ Git Revision: 54f9af1695bc359b02e9fc906b3b335cc0bfec41 Git URL: https://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git]
> Reporter: Abhinav Roy
> Assignee: Sheng Yang
> Fix For: pre-4.0.0
>
> Attachments: management-server.log, messages, SMlog.10, SMlog.11
>
>
> Steps :
> =========================
> 1. Deploy an advanced zone setup with CS 3.0.2
> 2. Deploy couple of VMs.
> 3. Acquire an IP Address for the Network created and create a LB rule for the VMs created in step2.
> 4. Upgrade the setup to ASF 4.0
> 5. Create a VM on the existing Network.
> 6. Add the VM created in step 5 to the LB rule created in step 3.
> Expected Behaviour :
> =========================
> All the steps should be successful and no error should be seen.
> Observed Behaviour :
> =========================
> Step 6 fails with the following exceptions
> 2012-09-18 00:32:07,036 DEBUG [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Adding VM[User|VM-3] to the load balancer pool
> 2012-09-18 00:32:07,130 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (Job-Executor-7:job-18) Applying ip association in network Ntwk[204|Guest|7]
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Sending { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,177 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Executing: { Cmd , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 100001, [{"routing.IpAssocCommand":{"ipAddresses":[{"accountId":2,"publicIp":"10.102.125.69","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:4d:b6:00:00:09","networkRate":200,"trafficType":"Public"},{"accountId":2,"publicIp":"10.102.125.66","sourceNat":false,"add":true,"oneToOneNat":false,"firstIP":false,"vlanId":"untagged","vlanGateway":"10.102.125.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:b2:8e:00:00:06","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.233","router.name":"r-4-VM"},"wait":0}}] }
> 2012-09-18 00:32:07,178 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Executing request
> 2012-09-18 00:32:07,718 ERROR [xen.resource.CitrixResourceBase] (DirectAgent-6:null) Ip Assoc failure on applying one ip due to exception:
> com.cloud.exception.InternalErrorException: Xen plugin "ipassoc" failed.
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.assignPublicIpAddress(CitrixResourceBase.java:1934)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:2030)
> at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:421)
> at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
> at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:191)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,719 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-6:null) Seq 1-2076573728: Response Received:
> 2012-09-18 00:32:07,719 DEBUG [agent.transport.Request] (DirectAgent-6:null) Seq 1-2076573728: Processing: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, [{"routing.IpAssocAnswer":{"results":["Failed",null],"result":false,"wait":0}}] }
> 2012-09-18 00:32:07,720 DEBUG [agent.transport.Request] (Job-Executor-7:job-18) Seq 1-2076573728: Received: { Ans: , MgmtId: 160940036302157, via: 1, Ver: v1, Flags: 0, { IpAssocAnswer } }
> 2012-09-18 00:32:07,723 WARN [network.lb.LoadBalancingRulesManagerImpl] (Job-Executor-7:job-18) Unable to apply the load balancer config because resource is unavaliable.
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to apply ip association on router
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3135)
> at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:2983)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:551)
> at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:435)
> at com.cloud.network.NetworkManagerImpl.applyIpAssociations(NetworkManagerImpl.java:966)
> at com.cloud.network.NetworkManagerImpl.applyRules(NetworkManagerImpl.java:3668)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerRules(LoadBalancingRulesManagerImpl.java:922)
> at com.cloud.utils.db.DatabaseCallback.intercept(DatabaseCallback.java:34)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.applyLoadBalancerConfig(LoadBalancingRulesManagerImpl.java:896)
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:467)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,731 ERROR [cloud.api.ApiDispatcher] (Job-Executor-7:job-18) Exception while executing AssignToLoadBalancerRuleCmd:
> com.cloud.utils.exception.CloudRuntimeException: Failed to add specified loadbalancerruleid for vms [6]
> at com.cloud.network.lb.LoadBalancingRulesManagerImpl.assignToLoadBalancer(LoadBalancingRulesManagerImpl.java:489)
> at com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
> at com.cloud.api.commands.AssignToLoadBalancerRuleCmd.execute(AssignToLoadBalancerRuleCmd.java:99)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2012-09-18 00:32:07,732 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-7:job-18) Complete async job-18, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: Failed to add specified loadbalancerruleid for vms [6]
> 2012-09-18 00:32:07,756 DEBUG [cloud.async.SyncQueueManagerImpl] (Job-Executor-7:job-18) Sync queue (1) is currently empty
> 2012-09-18 00:32:08,755 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-8:null) Ping from 1
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira