You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Animesh Chaturvedi (JIRA)" <ji...@apache.org> on 2014/09/23 23:31:34 UTC
[jira] [Updated] (CLOUDSTACK-7424) [Automation] Failed to restart
network
[ https://issues.apache.org/jira/browse/CLOUDSTACK-7424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Animesh Chaturvedi updated CLOUDSTACK-7424:
-------------------------------------------
Issue Type: Test (was: Bug)
> [Automation] Failed to restart network
> --------------------------------------
>
> Key: CLOUDSTACK-7424
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7424
> Project: CloudStack
> Issue Type: Test
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Automation, Test
> Affects Versions: 4.5.0
> Reporter: Chandan Purushothama
> Assignee: Chandan Purushothama
> Priority: Critical
> Fix For: 4.5.0
>
>
> Error Executing the test case *integration.component.test_multiple_ips_per_nic.TestVmNetworkOperations.test_network_restart_cleanup_false_3_VPC*
> *Error Message*
> Job failed: {jobprocstatus : 0, created : u'2014-08-24T07:31:03+0000', cmd : u'org.apache.cloudstack.api.command.user.network.RestartNetworkCmd', userid : u'77df13d2-2b4d-11e4-89bd-1e5d0e053e75', jobstatus : 2, jobid : u'01faacdc-294b-4209-a855-2dc8ea0aa2b4', jobresultcode : 530, jobresulttype : u'object', jobresult : {errorcode : 530, errortext : u'Failed to restart network'}, accountid : u'77df055e-2b4d-11e4-89bd-1e5d0e053e75'} Logs available at http://xenrt.hq.xensource.com/control/queue.cgi?action=testlogs&id=804076&phase=Parallel&test=multiple_ips_per_n
> *Stacktrace*
> File "/usr/lib/python2.7/unittest/case.py", line 332, in run
> testMethod()
> File "/usr/local/lib/python2.7/dist-packages/ddt.py", line 114, in wrapper
> return func(self, *args, **kwargs)
> File "/root/cloudstack/test/integration/component/test_multiple_ips_per_nic.py", line 1186, in test_network_restart_cleanup_false
> network.restart(self.apiclient, cleanup=False)
> File "/usr/local/lib/python2.7/dist-packages/marvin/lib/base.py", line 2626, in restart
> return(apiclient.restartNetwork(cmd))
> File "/usr/local/lib/python2.7/dist-packages/marvin/cloudstackAPI/cloudstackAPIClient.py", line 1899, in restartNetwork
> response = self.connection.marvinRequest(command, response_type=response, method=method)
> File "/usr/local/lib/python2.7/dist-packages/marvin/cloudstackConnection.py", line 379, in marvinRequest
> raise e
> 'Job failed: {jobprocstatus : 0, created : u\'2014-08-24T07:31:03+0000\', cmd : u\'org.apache.cloudstack.api.command.user.network.RestartNetworkCmd\', userid : u\'77df13d2-2b4d-11e4-89bd-1e5d0e053e75\', jobstatus : 2, jobid : u\'01faacdc-294b-4209-a855-2dc8ea0aa2b4\', jobresultcode : 530, jobresulttype : u\'object\', jobresult : {errorcode : 530, errortext : u\'Failed to restart network\'}, accountid : u\'77df055e-2b4d-11e4-89bd-1e5d0e053e75\'}\n
> Logs available at http://xenrt.hq.xensource.com/control/queue.cgi?action=testlogs&id=804076&phase=Parallel&test=multiple_ips_per_n
> ==============================
> *Error in the Management Server Log*:
> ==============================
> {code}
> 2014-08-24 07:31:03,602 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Router r-229-VM is in Stopping, so not sending apply static nat rules commands to the backend
> 2014-08-24 07:31:03,621 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Router r-229-VM is in Stopping, so not sending apply vpc ip association commands to the backend
> 2014-08-24 07:31:03,638 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Router r-229-VM is in Stopping, so not sending apply vpc ip association commands to the backend
> 2014-08-24 07:31:03,640 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Sending network shutdown to VpcVirtualRouter
> 2014-08-24 07:31:03,653 DEBUG [c.c.n.NetworkModelImpl] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Service SecurityGroup is not supported in the network id=314
> 2014-08-24 07:31:03,654 DEBUG [c.c.n.r.VpcVirtualNetworkApplianceManagerImpl] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Router r-229-VM is in Stopping, so not sending setup guest network command to the backend
> 2014-08-24 07:31:03,663 DEBUG [c.c.v.VirtualMachineManagerImpl] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Lock is acquired for nic id 476 as a part of remove vm VM[DomainRouter|r-229-VM] from network Ntwk[314|Guest|37]
> 2014-08-24 07:31:03,667 DEBUG [c.c.n.NetworkModelImpl] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Service SecurityGroup is not supported in the network id=314
> 2014-08-24 07:31:03,668 WARN [c.c.v.VirtualMachineManagerImpl] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Unable to remove vm VM[DomainRouter|r-229-VM] from network Ntwk[314|Guest|37]
> 2014-08-24 07:31:03,669 DEBUG [c.c.v.VirtualMachineManagerImpl] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Lock is released for nic id 476 as a part of remove vm VM[DomainRouter|r-229-VM] from network Ntwk[314|Guest|37]
> 2014-08-24 07:31:03,671 WARN [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Unable to complete shutdown of the network elements due to element: VpcVirtualRouter
> com.cloud.exception.ResourceUnavailableException: Resource [DataCenter:1] is unreachable: Unable to remove vm VM[DomainRouter|r-229-VM] from network, is not in the right state
> at com.cloud.vm.VirtualMachineManagerImpl.orchestrateRemoveVmFromNetwork(VirtualMachineManagerImpl.java:3079)
> at com.cloud.vm.VirtualMachineManagerImpl.removeVmFromNetwork(VirtualMachineManagerImpl.java:3009)
> at com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl.removeVpcRouterFromGuestNetwork(VpcVirtualNetworkApplianceManagerImpl.java:321)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy190.removeVpcRouterFromGuestNetwork(Unknown Source)
> at com.cloud.network.element.VpcVirtualRouterElement.shutdown(VpcVirtualRouterElement.java:261)
> at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.shutdownNetworkElementsAndResources(NetworkOrchestrator.java:2161)
> at org.apache.cloudstack.engine.orchestration.NetworkOrchestrator.restartNetwork(NetworkOrchestrator.java:2468)
> at com.cloud.network.NetworkServiceImpl.restartNetwork(NetworkServiceImpl.java:1874)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:106)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
> at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy157.restartNetwork(Unknown Source)
> at org.apache.cloudstack.api.command.user.network.RestartNetworkCmd.execute(RestartNetworkCmd.java:95)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
> at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 2014-08-24 07:31:03,673 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Failed to shutdown the network elements and resources as a part of network restart: Implemented
> 2014-08-24 07:31:03,673 DEBUG [o.a.c.e.o.NetworkOrchestrator] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Marking network Ntwk[314|Guest|37] with restartRequired=true
> 2014-08-24 07:31:03,675 WARN [c.c.n.NetworkServiceImpl] (API-Job-Executor-82:ctx-6738c61c job-1529 ctx-3eaba8e6) Network id=314 failed to restart.
> 2014-08-24 07:31:03,680 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-82:ctx-6738c61c job-1529) Complete async job-1529, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed to restart network"}
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)