You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "manasaveloori (JIRA)" <ji...@apache.org> on 2013/12/19 08:35:07 UTC
[jira] [Reopened] (CLOUDSTACK-5325) Restart Network command should
be blocked on non upgraded router network
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
manasaveloori reopened CLOUDSTACK-5325:
---------------------------------------
Network restart is failing with KVM HV..But it is not failing using Xen HV.
The exceptions seen in log for KVM HV in MS logs:
2013-12-19 02:30:03,640 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Restarting network 215...
2013-12-19 02:30:03,640 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Skip the shutting down of network id=215
2013-12-19 02:30:03,640 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Implementing the network Ntwk[215|Guest|8] elements and resources as a part of network restart
2013-12-19 02:30:03,646 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Asking VirtualRouter to implemenet Ntwk[215|Guest|8]
2013-12-19 02:30:03,649 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Lock is acquired for network id 215 as a part of router startup in Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(3)-Pod(null)-Cluster(null)-Host(null)-Storage()]
2013-12-19 02:30:03,652 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Lock is released for network id 215 as a part of router startup in Dest[Zone(Id)-Pod(Id)-Cluster(Id)-Host(Id)-Storage(Volume(Id|Type-->Pool(Id))] : Dest[Zone(3)-Pod(null)-Cluster(null)-Host(null)-Storage()]
2013-12-19 02:30:03,652 DEBUG [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Reprogramming network Ntwk[215|Guest|8] as a part of network implement
2013-12-19 02:30:03,655 DEBUG [c.c.n.r.RulesManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) There are no static nat to apply for network id=215
2013-12-19 02:30:03,674 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Applying firewall rules in network Ntwk[215|Guest|8]
2013-12-19 02:30:03,691 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Seq 5-2014318288: Sending { Cmd , MgmtId: 233845177509765, via: 5(Rack1Pod1Host28), Ver: v1, Flags: 100001, [{"com.cloud.agent.api.routing.SetFirewallRulesCommand":{"rules":[{"id":11,"srcIp":"10.147.47.36","protocol":"udp","srcPortRange":[500,500],"revoked":false,"alreadyAdded":true,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Ingress","defaultEgressPolicy":false},{"id":13,"srcIp":"10.147.47.36","protocol":"udp","srcPortRange":[1701,1701],"revoked":false,"alreadyAdded":true,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Ingress","defaultEgressPolicy":false},{"id":15,"srcIp":"10.147.47.36","protocol":"udp","srcPortRange":[4500,4500],"revoked":false,"alreadyAdded":true,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Ingress","defaultEgressPolicy":false},{"id":16,"srcIp":"10.147.47.36","protocol":"tcp","srcPortRange":[1,65535],"revoked":false,"alreadyAdded":true,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Ingress","defaultEgressPolicy":false},{"id":26,"srcIp":"10.147.47.34","protocol":"tcp","srcPortRange":[1,65535],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Ingress","defaultEgressPolicy":false},{"id":29,"srcIp":"10.147.47.31","protocol":"tcp","srcPortRange":[1,65535],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Ingress","defaultEgressPolicy":false}],"accessDetails":{"router.guest.ip":"10.1.1.1","firewall.egress.default":"false","zone.network.type":"Advanced","router.ip":"169.254.2.217","router.name":"r-39-VM"},"wait":0}}] }
2013-12-19 02:30:03,946 DEBUG [c.c.a.t.Request] (StatsCollector-2:ctx-b2fc8c8c) Seq 5-2014318287: Received: { Ans: , MgmtId: 233845177509765, via: 5, Ver: v1, Flags: 10, { GetHostStatsAnswer } }
2013-12-19 02:30:03,982 DEBUG [c.c.a.t.Request] (AgentManager-Handler-4:null) Seq 5-2014318288: Processing: { Ans: , MgmtId: 233845177509765, via: 5, Ver: v1, Flags: 0, [{"com.cloud.agent.api.routing.SetFirewallRulesAnswer":{"result":true,"wait":0}}] }
2013-12-19 02:30:03,982 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Seq 5-2014318288: Received: { Ans: , MgmtId: 233845177509765, via: 5, Ver: v1, Flags: 0, { SetFirewallRulesAnswer } }
2013-12-19 02:30:04,001 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Applying ip association in network Ntwk[215|Guest|8]
2013-12-19 02:30:04,020 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Seq 5-2014318289: Sending { Cmd , MgmtId: 233845177509765, via: 5(Rack1Pod1Host28), Ver: v1, Flags: 100001, [{"com.cloud.agent.api.routing.IpAssocCommand":{"ipAddresses":[{"accountId":3,"publicIp":"10.147.47.36","sourceNat":true,"add":true,"oneToOneNat":false,"firstIP":true,"broadcastUri":"47","vlanGateway":"10.147.47.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:e3:00:00:00:17","networkRate":200,"trafficType":"Public"},{"accountId":3,"publicIp":"10.147.47.31","sourceNat":false,"add":false,"oneToOneNat":false,"firstIP":false,"broadcastUri":"47","vlanGateway":"10.147.47.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:5f:8d:00:00:17","networkRate":200,"trafficType":"Public"},{"accountId":3,"publicIp":"10.147.47.34","sourceNat":false,"add":false,"oneToOneNat":false,"firstIP":false,"broadcastUri":"47","vlanGateway":"10.147.47.1","vlanNetmask":"255.255.255.0","vifMacAddress":"06:5f:8d:00:00:17","networkRate":200,"trafficType":"Public"}],"accessDetails":{"router.guest.ip":"10.1.1.1","zone.network.type":"Advanced","router.ip":"169.254.2.217","router.name":"r-39-VM"},"wait":0}}] }
2013-12-19 02:30:04,839 DEBUG [c.c.a.t.Request] (AgentManager-Handler-14:null) Seq 5-2014318289: Processing: { Ans: , MgmtId: 233845177509765, via: 5, Ver: v1, Flags: 0, [{"com.cloud.agent.api.routing.IpAssocAnswer":{"results":["Failed","Failed","Failed"],"result":false,"wait":0}}] }
2013-12-19 02:30:04,839 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Seq 5-2014318289: Received: { Ans: , MgmtId: 233845177509765, via: 5, Ver: v1, Flags: 0, { IpAssocAnswer } }
2013-12-19 02:30:04,839 WARN [c.c.n.f.FirewallManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Failed to apply firewall rules due to
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:3] is unreachable: Unable to apply ip association on router
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3735)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.associatePublicIP(VirtualNetworkApplianceManagerImpl.java:3526)
at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.associatePublicIP(VpcVirtualNetworkApplianceManagerImpl.java:505)
at sun.reflect.GeneratedMethodAccessor321.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy240.associatePublicIP(Unknown Source)
at com.cloud.network.element.VirtualRouterElement.applyIps(VirtualRouterElement.java:476)
at com.cloud.network.IpAddressManagerImpl.applyIpAssociations(IpAddressManagerImpl.java:976)
at com.cloud.network.IpAddressManagerImpl.applyRules(IpAddressManagerImpl.java:514)
at com.cloud.network.firewall.FirewallManagerImpl.applyRules(FirewallManagerImpl.java:521)
at com.cloud.network.firewall.FirewallManagerImpl.applyFirewallRules(FirewallManagerImpl.java:641)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy175.applyFirewallRules(Unknown Source)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.reprogramNetworkRules(NetworkOrchestrator.java:1090)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1063)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.restartNetwork(NetworkOrchestrator.java:2370)
at com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1869)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy199.restartNetwork(Unknown Source)
at org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:521)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-12-19 02:30:04,841 WARN [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Failed to reapply Ingress firewall rule(s) as a part of network id=215 restart
2013-12-19 02:30:04,844 DEBUG [c.c.n.f.FirewallManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) applying default firewall egress rules
2013-12-19 02:30:04,860 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Applying firewall rules in network Ntwk[215|Guest|8]
2013-12-19 02:30:04,867 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Seq 5-2014318290: Sending { Cmd , MgmtId: 233845177509765, via: 5(Rack1Pod1Host28), Ver: v1, Flags: 100001, [{"com.cloud.agent.api.routing.SetFirewallRulesCommand":{"rules":[{"id":31,"srcIp":"","protocol":"tcp","srcPortRange":[1,65535],"revoked":true,"alreadyAdded":false,"sourceCidrList":[],"purpose":"Firewall","trafficType":"Egress","defaultEgressPolicy":false}],"accessDetails":{"router.guest.ip":"10.1.1.1","firewall.egress.default":"false","zone.network.type":"Advanced","router.ip":"169.254.2.217","router.name":"r-39-VM"},"wait":0}}] }
2013-12-19 02:30:05,003 DEBUG [c.c.a.t.Request] (AgentManager-Handler-13:null) Seq 5-2014318290: Processing: { Ans: , MgmtId: 233845177509765, via: 5, Ver: v1, Flags: 0, [{"com.cloud.agent.api.routing.SetFirewallRulesAnswer":{"results":["Failed"],"result":false,"wait":0}}] }
2013-12-19 02:30:05,004 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Seq 5-2014318290: Received: { Ans: , MgmtId: 233845177509765, via: 5, Ver: v1, Flags: 0, { SetFirewallRulesAnswer } }
2013-12-19 02:30:05,004 WARN [c.c.n.f.FirewallManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Failed to apply firewall rules due to
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:3] is unreachable: Unable to apply firewall rules on router
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyRules(VirtualNetworkApplianceManagerImpl.java:3735)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.applyFirewallRules(VirtualNetworkApplianceManagerImpl.java:3542)
at sun.reflect.GeneratedMethodAccessor353.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy240.applyFirewallRules(Unknown Source)
at com.cloud.network.element.VirtualRouterElement.applyFWRules(VirtualRouterElement.java:248)
at com.cloud.network.firewall.FirewallManagerImpl.applyRules(FirewallManagerImpl.java:567)
at com.cloud.network.IpAddressManagerImpl.applyRules(IpAddressManagerImpl.java:502)
at com.cloud.network.firewall.FirewallManagerImpl.applyRules(FirewallManagerImpl.java:521)
at com.cloud.network.firewall.FirewallManagerImpl.applyFirewallRules(FirewallManagerImpl.java:641)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy175.applyFirewallRules(Unknown Source)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.reprogramNetworkRules(NetworkOrchestrator.java:1106)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1063)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.restartNetwork(NetworkOrchestrator.java:2370)
at com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1869)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy199.restartNetwork(Unknown Source)
at org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:521)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-12-19 02:30:05,006 WARN [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Failed to reapply firewall Egress rule(s) as a part of network id=215 restart
2013-12-19 02:30:05,007 DEBUG [c.c.n.r.RulesManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) There are no port forwarding rules to apply for network id=215
2013-12-19 02:30:05,009 DEBUG [c.c.n.r.RulesManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) There are no static nat rules to apply for network id=215
2013-12-19 02:30:05,010 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Applying load balancer rules of scheme Public in network id=215
2013-12-19 02:30:05,010 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) There are no Load Balancing Rules to forward to the network elements
2013-12-19 02:30:05,011 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Applying load balancer rules of scheme Internal in network id=215
2013-12-19 02:30:05,011 DEBUG [c.c.n.l.LoadBalancingRulesManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) There are no Load Balancing Rules to forward to the network elements
2013-12-19 02:30:05,028 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Seq 5-2014318291: Sending { Cmd , MgmtId: 233845177509765, via: 5(Rack1Pod1Host28), Ver: v1, Flags: 100111, [{"com.cloud.agent.api.routing.VpnUsersCfgCommand":{"userpwds":[{"username":"root","add":true},{"username":"test","add":true}],"accessDetails":{"account.id":"3","zone.network.type":"Advanced","router.ip":"169.254.2.217","router.name":"r-39-VM"},"wait":0}},{"com.cloud.agent.api.routing.RemoteAccessVpnCfgCommand":{"create":true,"vpcEnabled":false,"vpnServerIp":"10.147.47.36","ipRange":"10.1.2.2-10.1.2.8","presharedKey":"hhtetEww7RvBOMHRXtx5k5jz","localIp":"10.1.2.1","localCidr":"10.1.1.0/24","publicInterface":"eth2","accessDetails":{"zone.network.type":"Advanced","router.ip":"169.254.2.217","router.name":"r-39-VM"},"wait":0}}] }
2013-12-19 02:30:05,336 DEBUG [c.c.a.t.Request] (AgentManager-Handler-1:null) Seq 5-2014318291: Processing: { Ans: , MgmtId: 233845177509765, via: 5, Ver: v1, Flags: 110, [{"com.cloud.agent.api.Answer":{"result":true,"wait":0}},{"com.cloud.agent.api.Answer":{"result":false,"details":"Configure VPN failed","wait":0}}] }
2013-12-19 02:30:05,337 DEBUG [c.c.a.m.AgentAttache] (AgentManager-Handler-1:null) Seq 5-2014318291: No more commands found
2013-12-19 02:30:05,337 DEBUG [c.c.a.t.Request] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Seq 5-2014318291: Received: { Ans: , MgmtId: 233845177509765, via: 5, Ver: v1, Flags: 110, { Answer, Answer } }
2013-12-19 02:30:05,337 ERROR [c.c.n.r.VirtualNetworkApplianceManagerImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Unable to start vpn in zone 3 for account 3 on domR: r-39-VM due to Configure VPN failed
2013-12-19 02:30:05,337 WARN [o.a.c.e.o.NetworkOrchestrator] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Failed to implement network Ntwk[215|Guest|8] elements and resources as a part of network restart due to
com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:3] is unreachable: Unable to start vpn in zone 3 for account 3 on domR: r-39-VM due to Configure VPN failed
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRemoteAccessVpn(VirtualNetworkApplianceManagerImpl.java:2624)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy240.startRemoteAccessVpn(Unknown Source)
at com.cloud.network.element.VirtualRouterElement.startVpn(VirtualRouterElement.java:430)
at com.cloud.network.vpn.RemoteAccessVpnManagerImpl.startRemoteAccessVpn(RemoteAccessVpnManagerImpl.java:442)
at com.cloud.network.vpn.RemoteAccessVpnManagerImpl.startRemoteAccessVpn(RemoteAccessVpnManagerImpl.java:92)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.reprogramNetworkRules(NetworkOrchestrator.java:1140)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.implementNetworkElementsAndResources(NetworkOrchestrator.java:1063)
at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.restartNetwork(NetworkOrchestrator.java:2370)
at com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1869)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:50)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at $Proxy199.restartNetwork(Unknown Source)
at org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:92)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:521)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-12-19 02:30:05,339 WARN [c.c.n.NetworkServiceImpl] (Job-Executor-55:ctx-4f7c042f ctx-3eeba0e9) Network id=215 failed to restart.
2013-12-19 02:30:05,412 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-55:ctx-4f7c042f) Complete async job-118, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed to restart network"}
The same should happen for Xen.
> Restart Network command should be blocked on non upgraded router network
> ------------------------------------------------------------------------
>
> Key: CLOUDSTACK-5325
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5325
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Upgrade
> Affects Versions: 4.3.0
> Reporter: shweta agarwal
> Assignee: Alena Prokharchyk
> Priority: Critical
> Fix For: 4.3.0
>
>
> Restart Network command should be blocked on non upgraded router network due to following reasons:
> 1. It fails when there are rules like PF /LB /firewall to be applied as a part of network restart . But don't give proper error message just says restart networks failed .
> 2. If user tries restart network with force cleanup =true ; his network will be restarted with new router . In some sense that account owner has bumped up his priority w.r.t time on router upgrade
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)