[jira] [Created] (CLOUDSTACK-2821) [XenServer] ModifyStoragePoolCommand fails with error "com.cloud.utils.exception.CloudRuntimeException: Can not see storage pool: a1a1a150-6709-3d81-a414-4cee93e62168 from on host:0afd552a-6c99-4c50-aa58-47b817786793"

["venkata swamybabu budumuru (JIRA)" <ji...@apache.org>]

venkata swamybabu budumuru created CLOUDSTACK-2821:
------------------------------------------------------

             Summary: [XenServer] ModifyStoragePoolCommand fails with error "com.cloud.utils.exception.CloudRuntimeException: Can not see storage pool: a1a1a150-6709-3d81-a414-4cee93e62168 from on host:0afd552a-6c99-4c50-aa58-47b817786793"
                 Key: CLOUDSTACK-2821
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-2821
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: XenServer
    Affects Versions: 4.2.0
         Environment: commit # b4969c4af5264879ccbec01b65b6c94886fc79d4
            Reporter: venkata swamybabu budumuru
            Priority: Critical
             Fix For: 4.2.0


Steps to reproduce:

1. Have the latest CloudStack setup with at least 1 adv zone (1 cluster with XenServer 6.1 with all latest patches) and only system VMs in running state.
2. Make sure that all the system vms are in "Stopped" state and no user vms are created and the zone is in disabled state.
3. Push the XenServer host into Maintenance mode
4. After the host is successfully placed in maintenance, remove the host.
5. Add a totally different freshly installed host to the same cluster.

Observations:-

(i) It failed with the following error.

2013-06-03 13:06:00,403 DEBUG [agent.transport.Request] (AgentTaskPool-2:null) Seq 8-690552836: Sending  { Cmd , MgmtId: 7280707764394, via: 8, Ver: v1, Flags: 100011, [{"ModifyStoragePoolCommand":{"add":true,"pool":{"id":2,"uuid":"a1a1a150-6709-3d81-a414-4cee93e62168","host":"10.147.28.7","path":"/export/home/swamy/primary.campo.xen.1","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//a1a1a150-6709-3d81-a414-4cee93e62168","wait":0}}] }
2013-06-03 13:06:00,407 DEBUG [agent.transport.Request] (AgentTaskPool-2:null) Seq 8-690552836: Executing:  { Cmd , MgmtId: 7280707764394, via: 8, Ver: v1, Flags: 100011, [{"ModifyStoragePoolCommand":{"add":true,"pool":{"id":2,"uuid":"a1a1a150-6709-3d81-a414-4cee93e62168","host":"10.147.28.7","path":"/export/home/swamy/primary.campo.xen.1","port":2049,"type":"NetworkFilesystem"},"localPath":"/mnt//a1a1a150-6709-3d81-a414-4cee93e62168","wait":0}}] }
2013-06-03 13:06:00,407 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-2:null) Seq 8-690552836: Executing request
2013-06-03 13:06:00,489 WARN  [xen.resource.CitrixResourceBase] (DirectAgent-2:null) ModifyStoragePoolCommand add XenAPIException:Can not see storage pool: a1a1a150-6709-3d81-a414-4cee93e62168 from on host:0afd552a-6c99-4c50-aa58-47b817786793 host:0afd552a-6c99-4c50-aa58-47b817786793 pool: 10.147.28.7/export/home/swamy/primary.campo.xen.1
com.cloud.utils.exception.CloudRuntimeException: Can not see storage pool: a1a1a150-6709-3d81-a414-4cee93e62168 from on host:0afd552a-6c99-4c50-aa58-47b817786793
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.getStorageRepository(CitrixResourceBase.java:7548)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:5568)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:510)
        at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
        at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:102)
        at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:679)


(ii) CS sent "modifyStoragePoolCmd" and it didn't fine any SR with the UUID of primary and it resulted in failure.

(iii) Due to the above mentioned error, Primary storage never gets mounted and no VMs can be started until the primary storage is completely removed from CS and added newly.

Attaching all the required logs along with db dump to the bug.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira