You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Sailaja Mada (JIRA)" <ji...@apache.org> on 2013/07/15 07:42:49 UTC

[jira] [Updated] (CLOUDSTACK-3524) NPE while trying to start the router which is in stopped state

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-3524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sailaja Mada updated CLOUDSTACK-3524:
-------------------------------------

    Description: 
Steps:

1. Configure Advanced networking zone with VMWARE with Zone wide primary storage
2. Tried to start a Router which is in stopped state.

Observation:
NPE while trying to start the router which is in stopped state

2013-07-15 10:51:32,703 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Checking if we need to prepare 1 volumes for VM[DomainRouter|r-4-VM]
2013-07-15 10:51:32,705 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Volume Vol[14|vm=4|ROOT] will be recreated on storage pool Pool[1|NetworkFilesystem] assigned by deploymentPlanner
2013-07-15 10:51:32,717 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Created new volume Vol[15|vm=4|ROOT] for old volume Vol[14|vm=4|ROOT]
2013-07-15 10:51:32,799 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) copyAsync inspecting src type TEMPLATE copyAsync inspecting dest type VOLUME
2013-07-15 10:51:32,806 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Failed to send to storage pool
java.lang.NullPointerException
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cloneVolume(AncientDataMotionStrategy.java:264)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:343)
        at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromBaseImageAsync(VolumeServiceImpl.java:457)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:494)
        at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485)
        at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2727)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1867)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1967)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1945)
        at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:203)
        at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1964)
        at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1869)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2046)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:849)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:243)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3327)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2887)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2873)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
2013-07-15 10:51:32,819 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) copy failed
com.cloud.utils.exception.CloudRuntimeException: Failed to send to storage pool
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cloneVolume(AncientDataMotionStrategy.java:268)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:343)
        at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromBaseImageAsync(VolumeServiceImpl.java:457)
  at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromBaseImageAsync(VolumeServiceImpl.java:457)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:494)
        at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485)
        at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2727)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1867)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1967)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1945)
        at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:203)
        at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1964)
        at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1869)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2046)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:849)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:243)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3327)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2887)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2873)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.NullPointerException
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cloneVolume(AncientDataMotionStrategy.java:264)
        ... 42 more
2013-07-15 10:51:32,831 WARN  [storage.datastore.ObjectInDataStoreManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Unsupported data object (VOLUME, org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@4188dfd6)
2013-07-15 10:51:32,831 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Unable to create Vol[15|vm=4|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Failed to send to storage pool
2013-07-15 10:51:32,846 INFO  [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Unable to contact resource.
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create Vol[15|vm=4|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Failed to send to storage pool
        at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2493)
        at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)

 at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2727)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1867)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1967)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1945)
        at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:203)
        at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1964)
        at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1869)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2046)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:849)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:243)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3327)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2887)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2873)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
2013-07-15 10:51:32,853 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Cleaning up resources for the vm VM[DomainRouter|r-4-VM] in Starting state
2013-07-15 10:51:32,855 DEBUG [agent.transport.Request] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Seq 1-636099015: Sending  { Cmd , MgmtId: 187767034175903, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"r-4-VM","wait":0}}] }
2013-07-15 10:51:32,855 DEBUG [agent.transport.Request] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Seq 1-636099015: Executing:  { Cmd , MgmtId: 187767034175903, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"r-4-VM","wait":0}}] }
2013-07-15 10:51:32,856 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-123:null) Seq 1-636099015: Executing request
2013-07-15 10:51:32,856 INFO  [vmware.resource.VmwareResource] (DirectAgent-123:10.102.192.18) Executing resource StopCommand: {"isProxy":false,"executeInSequence":false,"vmName":"r-4-VM","wait":0}
2013-07-15 10:51:32,856 DEBUG [vmware.mo.HostMO] (DirectAgent-123:10.102.192.18) find VM r-4-VM on host
2013-07-15 10:51:32,856 DEBUG [vmware.mo.HostMO] (DirectAgent-123:10.102.192.18) load VM cache on host
2013-07-15 10:51:32,865 INFO  [vmware.resource.VmwareResource] (DirectAgent-123:10.102.192.18) VM r-4-VM is no longer in vSphere
2013-07-15 10:51:32,866 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-123:null) Seq 1-636099015: Response Received:
2013-07-15 10:51:32,867 DEBUG [agent.transport.Request] (DirectAgent-123:null) Seq 1-636099015: Processing:  { Ans: , MgmtId: 187767034175903, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"vncPort":0,"result":true,"details":"VM r-4-VM is no longer in vSphere","wait":0}}] }
2013-07-15 10:51:32,867 DEBUG [agent.transport.Request] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Seq 1-636099015: Received:  { Ans: , MgmtId: 187767034175903, via: 1, Ver: v1, Flags: 10, { StopAnswer } }



  was:
Steps:

1. Configure Advanced networking zone with VMWARE
2. Tried to start a Router which is in stopped state.

Observation:
NPE while trying to start the router which is in stopped state

2013-07-15 10:51:32,703 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Checking if we need to prepare 1 volumes for VM[DomainRouter|r-4-VM]
2013-07-15 10:51:32,705 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Volume Vol[14|vm=4|ROOT] will be recreated on storage pool Pool[1|NetworkFilesystem] assigned by deploymentPlanner
2013-07-15 10:51:32,717 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Created new volume Vol[15|vm=4|ROOT] for old volume Vol[14|vm=4|ROOT]
2013-07-15 10:51:32,799 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) copyAsync inspecting src type TEMPLATE copyAsync inspecting dest type VOLUME
2013-07-15 10:51:32,806 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Failed to send to storage pool
java.lang.NullPointerException
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cloneVolume(AncientDataMotionStrategy.java:264)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:343)
        at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromBaseImageAsync(VolumeServiceImpl.java:457)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:494)
        at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485)
        at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2727)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1867)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1967)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1945)
        at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:203)
        at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1964)
        at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1869)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2046)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:849)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:243)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3327)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2887)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2873)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
2013-07-15 10:51:32,819 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) copy failed
com.cloud.utils.exception.CloudRuntimeException: Failed to send to storage pool
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cloneVolume(AncientDataMotionStrategy.java:268)
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:343)
        at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromBaseImageAsync(VolumeServiceImpl.java:457)
  at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromBaseImageAsync(VolumeServiceImpl.java:457)
        at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:494)
        at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485)
        at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2727)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1867)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1967)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1945)
        at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:203)
        at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1964)
        at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1869)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2046)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:849)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:243)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3327)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2887)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2873)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
Caused by: java.lang.NullPointerException
        at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cloneVolume(AncientDataMotionStrategy.java:264)
        ... 42 more
2013-07-15 10:51:32,831 WARN  [storage.datastore.ObjectInDataStoreManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Unsupported data object (VOLUME, org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@4188dfd6)
2013-07-15 10:51:32,831 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Unable to create Vol[15|vm=4|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Failed to send to storage pool
2013-07-15 10:51:32,846 INFO  [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Unable to contact resource.
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create Vol[15|vm=4|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Failed to send to storage pool
        at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2493)
        at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)

 at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2727)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1867)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1967)
        at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1945)
        at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:203)
        at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1964)
        at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1869)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2046)
        at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:849)
        at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
        at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:243)
        at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3327)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2887)
        at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2873)
        at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
        at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
        at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
        at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:679)
2013-07-15 10:51:32,853 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Cleaning up resources for the vm VM[DomainRouter|r-4-VM] in Starting state
2013-07-15 10:51:32,855 DEBUG [agent.transport.Request] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Seq 1-636099015: Sending  { Cmd , MgmtId: 187767034175903, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"r-4-VM","wait":0}}] }
2013-07-15 10:51:32,855 DEBUG [agent.transport.Request] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Seq 1-636099015: Executing:  { Cmd , MgmtId: 187767034175903, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"r-4-VM","wait":0}}] }
2013-07-15 10:51:32,856 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-123:null) Seq 1-636099015: Executing request
2013-07-15 10:51:32,856 INFO  [vmware.resource.VmwareResource] (DirectAgent-123:10.102.192.18) Executing resource StopCommand: {"isProxy":false,"executeInSequence":false,"vmName":"r-4-VM","wait":0}
2013-07-15 10:51:32,856 DEBUG [vmware.mo.HostMO] (DirectAgent-123:10.102.192.18) find VM r-4-VM on host
2013-07-15 10:51:32,856 DEBUG [vmware.mo.HostMO] (DirectAgent-123:10.102.192.18) load VM cache on host
2013-07-15 10:51:32,865 INFO  [vmware.resource.VmwareResource] (DirectAgent-123:10.102.192.18) VM r-4-VM is no longer in vSphere
2013-07-15 10:51:32,866 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-123:null) Seq 1-636099015: Response Received:
2013-07-15 10:51:32,867 DEBUG [agent.transport.Request] (DirectAgent-123:null) Seq 1-636099015: Processing:  { Ans: , MgmtId: 187767034175903, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"vncPort":0,"result":true,"details":"VM r-4-VM is no longer in vSphere","wait":0}}] }
2013-07-15 10:51:32,867 DEBUG [agent.transport.Request] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Seq 1-636099015: Received:  { Ans: , MgmtId: 187767034175903, via: 1, Ver: v1, Flags: 10, { StopAnswer } }



    
> NPE while trying to start the router which is in stopped state
> --------------------------------------------------------------
>
>                 Key: CLOUDSTACK-3524
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3524
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Storage Controller
>    Affects Versions: 4.2.0
>            Reporter: Sailaja Mada
>            Priority: Critical
>
> Steps:
> 1. Configure Advanced networking zone with VMWARE with Zone wide primary storage
> 2. Tried to start a Router which is in stopped state.
> Observation:
> NPE while trying to start the router which is in stopped state
> 2013-07-15 10:51:32,703 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Checking if we need to prepare 1 volumes for VM[DomainRouter|r-4-VM]
> 2013-07-15 10:51:32,705 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Volume Vol[14|vm=4|ROOT] will be recreated on storage pool Pool[1|NetworkFilesystem] assigned by deploymentPlanner
> 2013-07-15 10:51:32,717 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Created new volume Vol[15|vm=4|ROOT] for old volume Vol[14|vm=4|ROOT]
> 2013-07-15 10:51:32,799 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) copyAsync inspecting src type TEMPLATE copyAsync inspecting dest type VOLUME
> 2013-07-15 10:51:32,806 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Failed to send to storage pool
> java.lang.NullPointerException
>         at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cloneVolume(AncientDataMotionStrategy.java:264)
>         at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:343)
>         at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromBaseImageAsync(VolumeServiceImpl.java:457)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:494)
>         at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485)
>         at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2727)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1867)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1967)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1945)
>         at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:203)
>         at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1964)
>         at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1869)
>         at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>         at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2046)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:849)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
>         at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:243)
>         at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
>         at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3327)
>         at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2887)
>         at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2873)
>         at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>         at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
>         at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-07-15 10:51:32,819 DEBUG [storage.motion.AncientDataMotionStrategy] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) copy failed
> com.cloud.utils.exception.CloudRuntimeException: Failed to send to storage pool
>         at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cloneVolume(AncientDataMotionStrategy.java:268)
>         at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:343)
>         at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromBaseImageAsync(VolumeServiceImpl.java:457)
>   at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromBaseImageAsync(VolumeServiceImpl.java:457)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:494)
>         at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485)
>         at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2727)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1867)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1967)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1945)
>         at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:203)
>         at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1964)
>         at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1869)
>         at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>         at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2046)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:849)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
>         at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:243)
>         at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
>         at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3327)
>         at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2887)
>         at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2873)
>         at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>         at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
>         at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> Caused by: java.lang.NullPointerException
>         at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.cloneVolume(AncientDataMotionStrategy.java:264)
>         ... 42 more
> 2013-07-15 10:51:32,831 WARN  [storage.datastore.ObjectInDataStoreManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Unsupported data object (VOLUME, org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@4188dfd6)
> 2013-07-15 10:51:32,831 DEBUG [cloud.storage.VolumeManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Unable to create Vol[15|vm=4|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Failed to send to storage pool
> 2013-07-15 10:51:32,846 INFO  [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Unable to contact resource.
> com.cloud.exception.StorageUnavailableException: Resource [StoragePool:1] is unreachable: Unable to create Vol[15|vm=4|ROOT]:com.cloud.utils.exception.CloudRuntimeException: Failed to send to storage pool
>         at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2493)
>         at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)
>  at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.start(VirtualNetworkApplianceManagerImpl.java:2727)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startVirtualRouter(VirtualNetworkApplianceManagerImpl.java:1867)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.startRouters(VirtualNetworkApplianceManagerImpl.java:1967)
>         at com.cloud.network.router.VirtualNetworkApplianceManagerImpl.deployVirtualRouterInGuestNetwork(VirtualNetworkApplianceManagerImpl.java:1945)
>         at com.cloud.network.element.VirtualRouterElement.implement(VirtualRouterElement.java:203)
>         at com.cloud.network.NetworkManagerImpl.implementNetworkElementsAndResources(NetworkManagerImpl.java:1964)
>         at com.cloud.network.NetworkManagerImpl.implementNetwork(NetworkManagerImpl.java:1869)
>         at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>         at com.cloud.network.NetworkManagerImpl.prepare(NetworkManagerImpl.java:2046)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:849)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
>         at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:243)
>         at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
>         at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3327)
>         at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2887)
>         at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:2873)
>         at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
>         at org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:420)
>         at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:155)
>         at com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:531)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-07-15 10:51:32,853 DEBUG [cloud.vm.VirtualMachineManagerImpl] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Cleaning up resources for the vm VM[DomainRouter|r-4-VM] in Starting state
> 2013-07-15 10:51:32,855 DEBUG [agent.transport.Request] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Seq 1-636099015: Sending  { Cmd , MgmtId: 187767034175903, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"r-4-VM","wait":0}}] }
> 2013-07-15 10:51:32,855 DEBUG [agent.transport.Request] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Seq 1-636099015: Executing:  { Cmd , MgmtId: 187767034175903, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"r-4-VM","wait":0}}] }
> 2013-07-15 10:51:32,856 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-123:null) Seq 1-636099015: Executing request
> 2013-07-15 10:51:32,856 INFO  [vmware.resource.VmwareResource] (DirectAgent-123:10.102.192.18) Executing resource StopCommand: {"isProxy":false,"executeInSequence":false,"vmName":"r-4-VM","wait":0}
> 2013-07-15 10:51:32,856 DEBUG [vmware.mo.HostMO] (DirectAgent-123:10.102.192.18) find VM r-4-VM on host
> 2013-07-15 10:51:32,856 DEBUG [vmware.mo.HostMO] (DirectAgent-123:10.102.192.18) load VM cache on host
> 2013-07-15 10:51:32,865 INFO  [vmware.resource.VmwareResource] (DirectAgent-123:10.102.192.18) VM r-4-VM is no longer in vSphere
> 2013-07-15 10:51:32,866 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-123:null) Seq 1-636099015: Response Received:
> 2013-07-15 10:51:32,867 DEBUG [agent.transport.Request] (DirectAgent-123:null) Seq 1-636099015: Processing:  { Ans: , MgmtId: 187767034175903, via: 1, Ver: v1, Flags: 10, [{"com.cloud.agent.api.StopAnswer":{"vncPort":0,"result":true,"details":"VM r-4-VM is no longer in vSphere","wait":0}}] }
> 2013-07-15 10:51:32,867 DEBUG [agent.transport.Request] (Job-Executor-24:job-24 = [ a8d8b93b-4a29-445a-844a-e75d34cc5346 ]) Seq 1-636099015: Received:  { Ans: , MgmtId: 187767034175903, via: 1, Ver: v1, Flags: 10, { StopAnswer } }

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira