You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Sanjay Tripathi (JIRA)" <ji...@apache.org> on 2013/12/19 13:23:07 UTC
[jira] [Commented] (CLOUDSTACK-4402) [deleteStoragePool] There is
no way to delete Primary storage if the last host with which it was
associated is already removed.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-4402?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13852849#comment-13852849 ]
Sanjay Tripathi commented on CLOUDSTACK-4402:
---------------------------------------------
Fix is available for review at: https://reviews.apache.org/r/16381/
> [deleteStoragePool] There is no way to delete Primary storage if the last host with which it was associated is already removed.
> -------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-4402
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-4402
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Management Server
> Affects Versions: 4.2.0
> Environment: commit id # 3c40e8bb3f6278f78c24c6317d513bd5ad599944
> Reporter: venkata swamybabu budumuru
> Assignee: Sanjay Tripathi
> Priority: Critical
> Fix For: 4.3.0
>
> Attachments: logs.tgz
>
>
> Steps to reproduce :
> 1. Have at least 1 advanced zone with XenServer Cluster
> 2. Doesn't actually matter whether VMs are deployed or not.
> In my case the exact environment is as mentioned below
> - 1 advanced zone
> - 1 KVM cluster with 1 host
> - 1 XenServer Cluster with 1 host
> - All systemVMs are running on KVM host
> 3. place the XenServer into maintenance mode
> 2013-08-19 19:05:31,905 DEBUG [cloud.api.ApiServlet] (catalina-exec-22:null) ===START=== 10.252.192.25 -- GET command=deleteHost&id=f498aa17-ed6c-4ca1-92e8-ef32c70c9290&forced=false&response=json&sessionkey=zJzyl0NdZA0QlMd8kSwWbynDxBI%3D&_=1376919346034
> 2013-08-19 19:05:31,919 DEBUG [cloud.resource.ResourceManagerImpl] (catalina-exec-22:null) Dispatching resource state event DELETE_HOST to NetworkUsageManagerImpl_EnhancerByCloudStack_aa8b2ad3
> 2013-08-19 19:05:31,919 DEBUG [cloud.resource.ResourceManagerImpl] (catalina-exec-22:null) Dispatching resource state event DELETE_HOST to KvmServerDiscoverer_EnhancerByCloudStack_1647b3bf
> 2013-08-19 19:05:31,920 DEBUG [cloud.resource.ResourceManagerImpl] (catalina-exec-22:null) Dispatching resource state event DELETE_HOST to OvmDiscoverer_EnhancerByCloudStack_829e4aea
> 2013-08-19 19:05:31,920 DEBUG [cloud.resource.ResourceManagerImpl] (catalina-exec-22:null) Dispatching resource state event DELETE_HOST to VmwareServerDiscoverer_EnhancerByCloudStack_6ca85485
> 2013-08-19 19:05:31,920 DEBUG [cloud.resource.ResourceManagerImpl] (catalina-exec-22:null) Dispatching resource state event DELETE_HOST to PremiumSecondaryStorageManagerImpl_EnhancerByCloudStack_4b3fda74
> 2013-08-19 19:05:31,920 DEBUG [cloud.resource.ResourceManagerImpl] (catalina-exec-22:null) Dispatching resource state event DELETE_HOST to XcpServerDiscoverer_EnhancerByCloudStack_1934d343
> 2013-08-19 19:05:31,920 DEBUG [cloud.resource.ResourceManagerImpl] (catalina-exec-22:null) Deleting Host: 9 Guid:d1f2d15f-17a5-4495-8f33-e8f5a5418413
> 2013-08-19 19:05:31,924 DEBUG [dc.dao.DataCenterIpAddressDaoImpl] (catalina-exec-22:null) Releasing ip address: 10.147.40.31 data center 1
> 2013-08-19 19:05:31,926 INFO [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Host 9 is disconnecting with event Remove
> 2013-08-19 19:05:31,928 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) The next status of agent 9is Removed, current status is Up
> 2013-08-19 19:05:31,928 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Deregistering link for 9 with state Removed
> 2013-08-19 19:05:31,928 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Remove Agent : 9
> 2013-08-19 19:05:31,928 DEBUG [agent.manager.DirectAgentAttache] (AgentTaskPool-9:null) Processing disconnect 9
> 2013-08-19 19:05:31,928 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer_EnhancerByCloudStack_1934d343
> 2013-08-19 19:05:31,928 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.deploy.DeploymentPlanningManagerImpl_EnhancerByCloudStack_dff95db9
> 2013-08-19 19:05:31,929 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.network.NetworkManagerImpl_EnhancerByCloudStack_f18a6f4c
> 2013-08-19 19:05:31,929 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.storage.secondary.SecondaryStorageListener
> 2013-08-19 19:05:31,929 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.hypervisor.vmware.manager.VmwareManagerImpl_EnhancerByCloudStack_88ff6e46
> 2013-08-19 19:05:31,929 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.network.security.SecurityGroupListener
> 2013-08-19 19:05:31,929 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.storage.listener.StoragePoolMonitor
> 2013-08-19 19:05:31,929 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.vm.ClusteredVirtualMachineManagerImpl_EnhancerByCloudStack_1e4b2050
> 2013-08-19 19:05:31,929 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.storage.LocalStoragePoolListener
> 2013-08-19 19:05:31,929 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.network.SshKeysDistriMonitor
> 2013-08-19 19:05:31,929 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.network.router.VirtualNetworkApplianceManagerImpl_EnhancerByCloudStack_b7bc8cf1
> 2013-08-19 19:05:31,929 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.network.SshKeysDistriMonitor
> 2013-08-19 19:05:31,930 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl_EnhancerByCloudStack_32a05514
> 2013-08-19 19:05:31,931 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.storage.upload.UploadListener
> 2013-08-19 19:05:31,931 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.storage.download.DownloadListener
> 2013-08-19 19:05:31,931 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.agent.manager.AgentMonitor
> 2013-08-19 19:05:31,932 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.capacity.StorageCapacityListener
> 2013-08-19 19:05:31,932 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.capacity.ComputeCapacityListener
> 2013-08-19 19:05:31,932 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener
> 2013-08-19 19:05:31,932 DEBUG [cloud.network.NetworkUsageManagerImpl] (AgentTaskPool-9:null) Disconnected called on 9 with status Removed
> 2013-08-19 19:05:31,932 DEBUG [agent.manager.AgentManagerImpl] (AgentTaskPool-9:null) Sending Disconnect to listener: com.cloud.consoleproxy.ConsoleProxyListener
> 2013-08-19 19:05:31,932 DEBUG [cloud.host.Status] (AgentTaskPool-9:null) Transition:[Resource state = Maintenance, Agent event = Remove, Host id = 9, name = Rack1Pod1Host31]
> 2013-08-19 19:05:31,950 DEBUG [cloud.resource.ResourceState] (catalina-exec-22:null) Resource state update: [id = 9; name = Rack1Pod1Host31; old state = Maintenance; event = DeleteHost; new state = Disabled]
> 2013-08-19 19:05:31,955 DEBUG [cloud.host.Status] (AgentTaskPool-9:null) Agent status update: [id = 9; name = Rack1Pod1Host31; old status = Up; event = Remove; new status = Removed; old update count = 2; new update count = 3]
> 2013-08-19 19:05:31,955 DEBUG [agent.manager.ClusteredAgentManagerImpl] (AgentTaskPool-9:null) Notifying other nodes of to disconnect
> 2013-08-19 19:05:31,961 DEBUG [cloud.api.ApiServlet] (catalina-exec-22:null) ===END=== 10.252.192.25 -- GET command=deleteHost&id=f498aa17-ed6c-4ca1-92e8-ef32c70c9290&forced=false&response=json&sessionkey=zJzyl0NdZA0QlMd8kSwWbynDxBI%3D&_=1376919346034
> 4. remove the host first
> 5. Go the primary storage that is associated with Xenserver and place that into maintenance mode
> 2013-08-19 19:06:44,458 DEBUG [cloud.api.ApiServlet] (catalina-exec-8:null) ===END=== 10.252.192.25 -- GET command=enableStorageMaintenance&id=a1a1a150-6709-3d81-a414-4cee93e62168&response=json&sessionkey=zJzyl0NdZA0QlMd8kSwWbynDxBI%3D&_=1376919418572
> 2013-08-19 19:06:44,461 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-63:job-62 = [ 0b3ebe91-0101-4ba2-bb36-2327b9e7566f ]) Executing org.apache.cloudstack.api.command.admin.storage.PreparePrimaryStorageForMaintenanceCmd for job-62 = [ 0b3ebe91-0101-4ba2-bb36-2327b9e7566f ]
> 2013-08-19 19:06:44,481 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-63:job-62 = [ 0b3ebe91-0101-4ba2-bb36-2327b9e7566f ]) Complete async job-62 = [ 0b3ebe91-0101-4ba2-bb36-2327b9e7566f ], jobStatus: 1, resultCode: 0, result: org.apache.cloudstack.api.response.StoragePoolResponse@5de2af25
> 2013-08-19 19:06:44,485 DEBUG [cloud.async.AsyncJobManagerImpl] (Job-Executor-63:job-62 = [ 0b3ebe91-0101-4ba2-bb36-2327b9e7566f ]) Done executing org.apache.cloudstack.api.command.admin.storage.PreparePrimaryStorageForMaintenanceCmd for job-62 = [ 0b3ebe91-0101-4ba2-bb36-2327b9e7566f ]
> 2013-08-19 19:06:47,504 DEBUG [cloud.api.ApiServlet] (catalina-exec-20:null) ===START=== 10.252.192.25 -- GET command=queryAsyncJobResult&jobId=0b3ebe91-0101-4ba2-bb36-2327b9e7566f&response=json&sessionkey=zJzyl0NdZA0QlMd8kSwWbynDxBI%3D&_=1376919421729
> 2013-08-19 19:06:47,512 DEBUG [cloud.async.AsyncJobManagerImpl] (catalina-exec-20:null) Async job-62 = [ 0b3ebe91-0101-4ba2-bb36-2327b9e7566f ] completed
> 6. Now try to remove the host with forced=false
> commanddeleteStoragePool&forced=false&id=a1a1a150-6709-3d81-a414-4cee93e62168responsejsonsessionkeyzJzyl0NdZA0QlMd8kSwWbynDxBI=
> Observations :-
> (i) It failed to delete with an error saying "Failed to delete storage pool on host"
> (ii) tried to delete it with forced=true and that as well failed with the same error.
> 2013-08-19 19:06:55,677 ERROR [cloud.api.ApiServer] (catalina-exec-22:null) unhandled exception executing api command: deleteStoragePool
> com.cloud.utils.exception.CloudRuntimeException: Failed to delete storage pool on host
> at org.apache.cloudstack.storage.datastore.lifecycle.CloudStackPrimaryDataStoreLifeCycleImpl.deleteDataStore(CloudStackPrimaryDataStoreLifeCycleImpl.java:478)
> at com.cloud.storage.StorageManagerImpl.deletePool(StorageManagerImpl.java:937)
> at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
> at org.apache.cloudstack.api.command.admin.storage.DeletePoolCmd.execute(DeletePoolCmd.java:78)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:158)
> at com.cloud.api.ApiServer.queueCommand(ApiServer.java:514)
> at com.cloud.api.ApiServer.handleRequest(ApiServer.java:372)
> at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:305)
> at com.cloud.api.ApiServlet.doGet(ApiServlet.java:66)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
> at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
> at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
> at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
> at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
> at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
> at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
> at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
> at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
> at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
> at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
> at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:721)
> at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2268)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:679)
> 2013-08-19 19:06:55,679 DEBUG [cloud.api.ApiServlet] (catalina-exec-22:null) ===END=== 10.252.192.25 -- GET command=deleteStoragePool&id=a1a1a150-6709-3d81-a414-4cee93e62168&forced=false&response=json&sessionkey=zJzyl0NdZA0QlMd8kSwWbynDxBI%3D&_=1376919429782
> Attaching all the required logs along with db dump to the bug.
> Expected Result:
> (i) Either we should not let admin remove the hypervisor host while it has primary storages associated and mounted on it
> OR
> (ii) we should let the deleteStoragePool with forced=true to delete the pool forcefully.
--
This message was sent by Atlassian JIRA
(v6.1.4#6159)