You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Venkata Siva Vijayendra Bhamidipati (JIRA)" <ji...@apache.org> on 2013/06/10 20:08:20 UTC
[jira] [Closed] (CLOUDSTACK-2887) PVLAN - vmware dvswitch -
destroy VM fail with VM stuck in 'Stopping' state
[ https://issues.apache.org/jira/browse/CLOUDSTACK-2887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Venkata Siva Vijayendra Bhamidipati closed CLOUDSTACK-2887.
-----------------------------------------------------------
Resolution: Duplicate
Closing as dup of CLOUDSTACK-2881.
> PVLAN - vmware dvswitch - destroy VM fail with VM stuck in 'Stopping' state
> ----------------------------------------------------------------------------
>
> Key: CLOUDSTACK-2887
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2887
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Management Server
> Affects Versions: 4.2.0
> Environment: MS ASF 2.0 build CloudStack-non-OSS-MASTER-452-rhel6.3.tar.gz
> vcenter ESXi 5.0 hosts
> Reporter: angeline shen
> Assignee: Venkata Siva Vijayendra Bhamidipati
> Priority: Blocker
> Fix For: 4.2.0
>
> Attachments: management-server.log.gz, management-server.log.gz, Screenshot-CloudStack - Mozilla Firefox-7.png
>
>
> 1. zone with two PVLAN networks
> 2. deploy VMs in both PVLANs.
> Confirm VMs in same PVLAN cannot connect to each other.
> confirm VMs in 1 PVLAN can reach VMs in other PVLAN. VMs can reach outside world
> 3. Destroy VMs in both PVLANs.
> Result: VMS remain in Stopping state.
> 2013-06-06 15:46:57,467 DEBUG [cloud.api.ApiServlet] (catalina-exec-10:null) ===START=== 10.216.133.70 -- GET command=destroyVirtualMachine&id=c27541d9-f4d2-40fc-8fab-3edfadc4067e&response=json&sessionkey=RV1iRzQ3pg636vVZIwdb
> rJlAJXU%3D&_=1370559071830
> 2013-06-06 15:46:57,575 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-10:null) submit async job-34, details: AsyncJobVO {id:34, userId: 2, accountId: 2, sessionKey: null, instanceType: VirtualMachine, instanceId: 12, c
> md: org.apache.cloudstack.api.command.user.vm.DestroyVMCmd, cmdOriginator: null, cmdInfo: {"response":"json","id":"c27541d9-f4d2-40fc-8fab-3edfadc4067e","sessionkey":"RV1iRzQ3pg636vVZIwdbrJlAJXU\u003d","ctxUserId":"2","httpmeth
> od":"GET","_":"1370559071830","ctxAccountId":"2","ctxStartEventId":"132"}, cmdVersion: 0, callbackType: 0, callbackAddress: null, status: 0, processStatus: 0, resultCode: 0, result: null, initMsid: 206915885077197, completeMsid
> : null, lastUpdated: null, lastPolled: null, created: null}
> 2013-06-06 15:46:57,577 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-34:job-34) Executing org.apache.cloudstack.api.command.user.vm.DestroyVMCmd for job-34
> 2013-06-06 15:46:57,578 DEBUG [cloud.api.ApiServlet] (catalina-exec-10:null) ===END=== 10.216.133.70 -- GET command=destroyVirtualMachine&id=c27541d9-f4d2-40fc-8fab-3edfadc4067e&response=json&sessionkey=RV1iRzQ3pg636vVZIwdbrJlAJXU%3D&_=1370559071830
> 2013-06-06 15:46:57,630 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-34:job-34) Destroying vm VM[User|vm1612V61]
> 2013-06-06 15:46:57,667 DEBUG [cloud.capacity.CapacityManagerImpl] (Job-Executor-34:job-34) VM state transitted from :Running to Stopping with event: StopRequestedvm's original host id: 1 new host id: 2 host id before state transition: 2
> 2013-06-06 15:46:57,702 ERROR [cloud.async.AsyncJobManagerImpl] (Job-Executor-34:job-34) Unexpected exception while executing org.apache.cloudstack.api.command.user.vm.DestroyVMCmd
> java.lang.ClassCastException: com.cloud.vm.VMInstanceVO$$EnhancerByCGLIB$$a77eface cannot be cast to com.cloud.vm.UserVmVO
> at com.cloud.vm.UserVmManagerImpl.prepareStop(UserVmManagerImpl.java:4712)
> at com.cloud.vm.VirtualMachineManagerImpl.advanceStop(VirtualMachineManagerImpl.java:1168)
> at com.cloud.vm.VirtualMachineManagerImpl.destroy(VirtualMachineManagerImpl.java:1285)
> at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.destroyVirtualMachine(VMEntityManagerImpl.java:265)
> at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.destroy(VirtualMachineEntityImpl.java:225)
> at com.cloud.vm.UserVmManagerImpl.destroyVm(UserVmManagerImpl.java:3385)
> at com.cloud.vm.UserVmManagerImpl.destroyVm(UserVmManagerImpl.java:1898)
> at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at org.apache.cloudstack.api.command.user.vm.DestroyVMCmd.execute(DestroyVMCmd.java:100)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
> at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:437)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2013-06-06 15:46:57,703 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-34:job-34) Complete async job-34, jobStatus: 2, resultCode: 530, result: Error Code: 530 Error text: com.cloud.vm.VMInstanceVO$$EnhancerByCGLIB$$a77eface cannot be cast to com.cloud.vm.UserVmVO
> 2013-06-06 15:47:00,621 DEBUG [cloud.api.ApiServlet] (catalina-exec-20:null) ===START=== 10.216.133.70 -- GET command=queryAsyncJobResult&jobId=f8a46980-8409-4ead-a209-a799e202f959&response=json&sessionkey=RV1iRzQ3pg636vVZIwdbrJlAJXU%3D&_=1370559074987
> 2013-06-06 15:47:00,632 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-20:null) Async job-34 completed
> 2013-06-06 15:47:00,636 DEBUG [cloud.api.ApiServlet] (catalina-exec-20:null) ===END=== 10.216.133.70 -- GET command=queryAsyncJobResult&jobId=f8a46980-8409-4ead-a209-a799e202f959&response=json&sessionkey=RV1iRzQ3pg636vVZIwdbrJlAJXU%3D&_=1370559074987
> 2013-06-06 15:47:01,599 DEBUG [cloud.server.StatsCollec
> 4. In DB, patch VM entries to Stopped state, then try Destroy VM.
> VM remain in 'Expunging' state
> [agent.manager.DirectAgentAttache] (DirectAgent-452:null) Seq 2-224136613: Response Received:
> 2013-06-06 16:43:46,663 DEBUG [agent.transport.Request] (DirectAgent-452:null) Seq 2-224136613: Processing: { Ans: , MgmtId: 206915885077197, via: 2, Ver: v1, Flags: 10, [{"Answer":{"result":false,"details":"Expunge failed in vSphere. vm: i-2-3-VM","wait":0}}] }
> 2013-06-06 16:43:46,663 DEBUG [agent.transport.Request] (UserVm-Scavenger-1:null) Seq 2-224136613: Received: { Ans: , MgmtId: 206915885077197, via: 2, Ver: v1, Flags: 10, { Answer } }
> 2013-06-06 16:43:46,663 WARN [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Failed to expunge vm due to: Expunge failed in vSphere. vm: i-2-3-VM
> 2013-06-06 16:43:46,663 INFO [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Did not expunge VM[User|z1vmware1611V7]
> 2013-06-06 16:43:46,663 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Stopped called on VM[User|z1vmware1611V72] but the state is Expunging
> 2013-06-06 16:43:46,695 DEBUG [cloud.capacity.CapacityManagerImpl] (UserVm-Scavenger-1:null) VM state transitted from :Expunging to Expunging with event: ExpungeOperationvm's original host id: 2 new host id: 2 host id before state transition: 2
> 2013-06-06 16:43:46,696 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Destroying vm VM[User|z1vmware1611V72]
> 2013-06-06 16:43:46,696 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Cleaning up NICS
> 2013-06-06 16:43:46,696 DEBUG [cloud.network.NetworkManagerImpl] (UserVm-Scavenger-1:null) Cleaning network for vm: 6
> 2013-06-06 16:43:46,698 DEBUG [cloud.storage.VolumeManagerImpl] (UserVm-Scavenger-1:null) Cleaning storage for vm: 6
> 2013-06-06 16:43:46,700 DEBUG [agent.transport.Request] (UserVm-Scavenger-1:null) Seq 2-224136614: Sending { Cmd , MgmtId: 206915885077197, via: 2, Ver: v1, Flags: 100011, [{"UnregisterVMCommand":{"vmName":"i-2-6-VM","wait":0}}] }
> 2013-06-06 16:43:46,700 DEBUG [agent.transport.Request] (UserVm-Scavenger-1:null) Seq 2-224136614: Executing: { Cmd , MgmtId: 206915885077197, via: 2, Ver: v1, Flags: 100011, [{"UnregisterVMCommand":{"vmName":"i-2-6-VM","wait":0}}] }
> 2013-06-06 16:43:46,701 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-196:null) Seq 2-224136614: Executing request
> 2013-06-06 16:43:46,701 INFO [vmware.resource.VmwareResource] (DirectAgent-196:10.223.81.40) Executing resource UnregisterVMCommand: {"vmName":"i-2-6-VM","wait":0}
> 2013-06-06 16:43:46,701 DEBUG [vmware.mo.HostMO] (DirectAgent-196:10.223.81.40) find VM i-2-6-VM on host
> 2013-06-06 16:43:46,701 DEBUG [vmware.mo.HostMO] (DirectAgent-196:10.223.81.40) load VM cache on host
> 2013-06-06 16:43:46,735 WARN [vmware.resource.VmwareResource] (DirectAgent-196:10.223.81.40) We are not able to unregister VM Exception: com.vmware.vim25.InvalidPowerState
> message: []
> 2013-06-06 16:43:46,735 WARN [vmware.resource.VmwareResource] (DirectAgent-196:10.223.81.40) Expunge failed in vSphere. vm: i-2-6-VM
> 2013-06-06 16:43:46,735 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-196:null) Seq 2-224136614: Response Received:
> 2013-06-06 16:43:46,735 DEBUG [agent.transport.Request] (DirectAgent-196:null) Seq 2-224136614: Processing: { Ans: , MgmtId: 206915885077197, via: 2, Ver: v1, Flags: 10, [{"Answer":{"result":false,"details":"Expunge failed in vSphere. vm: i-2-6-VM","wait":0}}] }
> 2013-06-06 16:43:46,735 DEBUG [agent.transport.Request] (UserVm-Scavenger-1:null) Seq 2-224136614: Received: { Ans: , MgmtId: 206915885077197, via: 2, Ver: v1, Flags: 10, { Answer } }
> 2013-06-06 16:43:46,735 WARN [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Failed to expunge vm due to: Expunge failed in vSphere. vm: i-2-6-VM
> 2013-06-06 16:43:46,735 INFO [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Did not expunge VM[User|z1vmware1611V72]
> 2013-06-06 16:43:46,735 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Stopped called on VM[User|z1vmware1611V73] but the state is Expunging
> 2013-06-06 16:43:46,771 DEBUG [cloud.capacity.CapacityManagerImpl] (UserVm-Scavenger-1:null) VM state transitted from :Expunging to Expunging with event: ExpungeOperationvm's original host id: 1 new host id: 1 host id before stat:
> 2013-06-06 16:43:46,799 WARN [vmware.resource.VmwareResource] (DirectAgent-429:10.223.81.41) Expunge failed in vSphere. vm: i-2-7-VM
> 2013-06-06 16:43:46,799 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-429:null) Seq 1-1694436621: Response Received:
> 2013-06-06 16:43:46,799 DEBUG [agent.transport.Request] (DirectAgent-429:null) Seq 1-1694436621: Processing: { Ans: , MgmtId: 206915885077197, via: 1, Ver: v1, Flags: 10, [{"Answer":{"result":false,"details":"Expunge failed in vSphere. vm: i-2-7-VM","wait":0}}] }
> 2013-06-06 16:43:46,799 DEBUG [agent.transport.Request] (UserVm-Scavenger-1:null) Seq 1-1694436621: Received: { Ans: , MgmtId: 206915885077197, via: 1, Ver: v1, Flags: 10, { Answer } }
> 2013-06-06 16:43:46,799 WARN [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Failed to expunge vm due to: Expunge failed in vSphere. vm: i-2-7-VM
> 2013-06-06 16:43:46,800 INFO [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Did not expunge VM[User|z1vmware1611V73]
> 2013-06-06 16:43:46,800 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Stopped called on VM[User|vm1612V61] but the state is Expunging
> 2013-06-06 16:43:46,838 DEBUG [cloud.capacity.CapacityManagerImpl] (UserVm-Scavenger-1:null) VM state transitted from :Expunging to Expunging with event: ExpungeOperationvm's original host id: 1 new host id: 2 host id before state transition: 2
> 2013-06-06 16:43:46,838 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Destroying vm VM[User|vm1612V61]
> 2013-06-06 16:43:46,838 DEBUG [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Cleaning up NICS
> 2013-06-06 16:43:46,838 DEBUG [cloud.network.NetworkManagerImpl] (UserVm-Scavenger-1:null) Cleaning network for vm: 12
> 2013-06-06 16:43:46,839 DEBUG [cloud.storage.VolumeManagerImpl] (UserVm-Scavenger-1:null) Cleaning storage for vm: 12
> 2013-06-06 16:43:46,842 DEBUG [agent.transport.Request] (UserVm-Scavenger-1:null) Seq 2-224136615: Sending { Cmd , MgmtId: 206915885077197, via: 2, Ver: v1, Flags: 100011, [{"UnregisterVMCommand":{"vmName":"i-2-12-VM","wait":0}}] }
> 2013-06-06 16:43:46,842 DEBUG [agent.transport.Request] (UserVm-Scavenger-1:null) Seq 2-224136615: Executing: { Cmd , MgmtId: 206915885077197, via: 2, Ver: v1, Flags: 100011, [{"UnregisterVMCommand":{"vmName":"i-2-12-VM","wait":0}}] }
> 2013-06-06 16:43:46,842 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-130:null) Seq 2-224136615: Executing request
> 2013-06-06 16:43:46,843 INFO [vmware.resource.VmwareResource] (DirectAgent-130:10.223.81.40) Executing resource UnregisterVMCommand: {"vmName":"i-2-12-VM","wait":0}
> 2013-06-06 16:43:46,843 DEBUG [vmware.mo.HostMO] (DirectAgent-130:10.223.81.40) find VM i-2-12-VM on host
> 2013-06-06 16:43:46,843 DEBUG [vmware.mo.HostMO] (DirectAgent-130:10.223.81.40) load VM cache on host
> 2013-06-06 16:43:46,858 WARN [vmware.resource.VmwareResource] (DirectAgent-130:10.223.81.40) We are not able to unregister VM Exception: com.vmware.vim25.InvalidPowerState
> message: []
> 2013-06-06 16:43:46,858 WARN [vmware.resource.VmwareResource] (DirectAgent-130:10.223.81.40) Expunge failed in vSphere. vm: i-2-12-VM
> 2013-06-06 16:43:46,858 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-130:null) Seq 2-224136615: Response Received:
> 2013-06-06 16:43:46,859 DEBUG [agent.transport.Request] (DirectAgent-130:null) Seq 2-224136615: Processing: { Ans: , MgmtId: 206915885077197, via: 2, Ver: v1, Flags: 10, [{"Answer":{"result":false,"details":"Expunge failed in vSphere. vm: i-2-12-VM","wait":0}}] }
> 2013-06-06 16:43:46,859 DEBUG [agent.transport.Request] (UserVm-Scavenger-1:null) Seq 2-224136615: Received: { Ans: , MgmtId: 206915885077197, via: 2, Ver: v1, Flags: 10, { Answer } }
> 2013-06-06 16:43:46,859 WARN [cloud.vm.VirtualMachineManagerImpl] (UserVm-Scavenger-1:null) Failed to expunge vm due to: Expunge failed in vSphere. vm: i-2-12-VM
> 2013-06-06 16:43:46,859 INFO [cloud.vm.UserVmManagerImpl] (UserVm-Scavenger-1:null) Did not expunge VM[User|vm1612V61]
> 2013-06-06 16:43:47,371 DEBUG [cloud.api.ApiServlet] (catalina-exec-25:null) ===START=== 10.216.133.70 -- GET command=listDomains&id=ec9a7ec4-ce3c-11e2-add9-f923bf56723a&listAll=true&response=json&sessionkey=RV1iRzQ3pg636vVZIwdbrJlAJXU%3D&_=1370562481753
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira