You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Sailaja Mada (JIRA)" <ji...@apache.org> on 2013/09/01 11:02:51 UTC
[jira] [Created] (CLOUDSTACK-4587) [VMWARE]VM is failing to deploy
on a Legacy zone after adding zone wide primary storage and moving cluster
wide primary storage to maintenance mode
Sailaja Mada created CLOUDSTACK-4587:
----------------------------------------
Summary: [VMWARE]VM is failing to deploy on a Legacy zone after adding zone wide primary storage and moving cluster wide primary storage to maintenance mode
Key: CLOUDSTACK-4587
URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4587
Project: CloudStack
Issue Type: Bug
Security Level: Public (Anyone can view this level - this is the default.)
Components: Storage Controller, VMware
Affects Versions: 4.2.1
Reporter: Sailaja Mada
Priority: Critical
Steps:
1. Upgraded from 307 to 4.2 using VMWARE Cluster with Cluster level primary storage
2. Add Zone wide primary storage after upgrade
3. Move the cluster level scope storage to Maintenance mode
4. Try to deploy new VM.
Observation:
VM is failing to deploy on a Legacy zone after adding zone wide primary storage and moving cluster wide primary storage to maintenance mode
2013-09-01 14:21:10,323 DEBUG [cloud.network.NetworkManagerImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) Network id=207 is already implemented
2013-09-01 14:21:10,354 DEBUG [network.guru.PodBasedNetworkGuru] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) Allocated a nic NicProfile[119-35-4a8f547b-7bff-4b3f-ba10-b0146af4d1bb-10.102.195.111-null for VM[DomainRouter|r-35-VM]
2013-09-01 14:21:10,390 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) Checking if we need to prepare 1 volumes for VM[DomainRouter|r-35-VM]
2013-09-01 14:21:10,400 DEBUG [storage.image.TemplateDataFactoryImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) template 203 is already in store:2, type:Image
2013-09-01 14:21:10,409 DEBUG [storage.image.TemplateDataFactoryImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) template 203 is already in store:203, type:Primary
2013-09-01 14:21:10,410 DEBUG [storage.volume.VolumeServiceImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) Found template 203-2-3cfe22ca-3a33-39a0-bf5e-0dab154869fd in storage pool 203 with VMTemplateStoragePool id: 11
2013-09-01 14:21:10,421 DEBUG [storage.volume.VolumeServiceImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) Acquire lock on VMTemplateStoragePool 11 with timeout 3600 seconds
2013-09-01 14:21:10,423 INFO [storage.volume.VolumeServiceImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) lock is acquired for VMTemplateStoragePool 11
2013-09-01 14:21:10,431 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) copyAsync inspecting src type TEMPLATE copyAsync inspecting dest type TEMPLATE
2013-09-01 14:21:10,519 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) copy object failed:
java.lang.NullPointerException
at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyObject(AncientDataMotionStrategy.java:210)
at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:411)
at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:440)
at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:569)
at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2536)
at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2592)
at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888)
at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2740)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1872)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1972)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1950)
at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:188)
at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:2034)
at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1939)
at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2120)
at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:886)
at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578)
at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227)
at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3406)
at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2966)
at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2952)
at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-09-01 14:21:10,520 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) copy failed
com.cloud.utils.exception.CloudRuntimeException: java.lang.NullPointerException
at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyObject(AncientDataMotionStrategy.java:231)
at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:411)
at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:440)
at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:569)
at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2536)
at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2592)
at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888)
at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2740)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1872)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1972)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1950)
at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:188)
at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:2034)
at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1939)
at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2120)
at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:886)
at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578)
at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227)
at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3406)
at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2966)
at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2952)
at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-09-01 14:21:10,525 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-152:null) Seq 5-1348865999: Response Received:
2013-09-01 14:21:10,525 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 5-1348865999: Received: { Ans: , MgmtId: 227594284004867, via: 5, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
2013-09-01 14:21:10,530 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-172:null) Seq 6-1919947690: Executing request
2013-09-01 14:21:10,558 INFO [storage.volume.VolumeServiceImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) releasing lock for VMTemplateStoragePool 11
2013-09-01 14:21:10,558 WARN [utils.db.Merovingian2] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) Was unable to find lock for the key template_spool_ref11 and thread id 595423976
2013-09-01 14:21:10,558 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) Unable to create Vol[98|vm=35|ROOT]:com.cloud.utils.exception.CloudRuntimeException: java.lang.NullPointerException
2013-09-01 14:21:10,558 INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) Unable to contact resource.
2013-09-01 14:21:10,558 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) Unable to create Vol[98|vm=35|ROOT]:com.cloud.utils.exception.CloudRuntimeException: java.lang.NullPointerException
2013-09-01 14:21:10,558 INFO [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) Unable to contact resource.
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:203] is unreachable: Unable to create Vol[98|vm=35|ROOT]:com.cloud.utils.exception.CloudRuntimeException: java.lang.NullPointerException
at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2544)
at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2592)
at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:888)
at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2740)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1872)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1972)
at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1950)
at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:188)
at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:2034)
at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1939)
at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2120)
at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:886)
at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:578)
at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227)
at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3406)
at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2966)
at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2952)
at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:679)
2013-09-01 14:21:10,592 DEBUG [vmware.resource.VmwareResource] (DirectAgent-172:10.102.192.20) Datastore summary info, storageId: 55e6d095-7e22-3f4a-b533-874a9fede8df, localPath: /cpg_vol/sailaja/vmwareps1, poolType: NetworkFilesystem, capacity: 879609303040, free: 376848912384, used: 502760390656
2013-09-01 14:21:10,679 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-172:null) Seq 6-1919947690: Response Received:
2013-09-01 14:21:10,679 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 6-1919947690: Received: { Ans: , MgmtId: 227594284004867, via: 6, Ver: v1, Flags: 10, { GetStorageStatsAnswer } }
2013-09-01 14:21:10,685 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-271:null) Seq 6-1919947691: Executing request
2013-09-01 14:21:10,694 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) Cleaning up resources for the vm VM[DomainRouter|r-35-VM] in Starting state
2013-09-01 14:21:10,696 DEBUG [agent.transport.Request] (Job-Executor-119:job-199 = [ 4837d9ec-d365-4653-960c-83fa1dc4fa87 ]) Seq 6-1919947692: Sending { Cmd , MgmtId: 227594284004867, via: 6, Ver: v1, Flags: 100111, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":true,"vmName":"r-35-VM","wait":0}}] }
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira