You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Min Chen (JIRA)" <ji...@apache.org> on 2013/07/20 01:04:49 UTC

[jira] [Resolved] (CLOUDSTACK-3472) [Object_Store_Refactor] System VMs are not coming up in initial attempt, but they are coming up after multiple attempts

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-3472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Min Chen resolved CLOUDSTACK-3472.
----------------------------------

    Resolution: Fixed
    
> [Object_Store_Refactor] System VMs are not coming up in initial attempt, but they are coming up after multiple attempts
> -----------------------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-3472
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-3472
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Install and Setup
>    Affects Versions: 4.2.0
>         Environment: Latest build from ACS4.2 branch:
> Cluster:XenServer6.1
>            Reporter: Sanjeev N
>            Assignee: Min Chen
>            Priority: Critical
>             Fix For: 4.2.0
>
>         Attachments: management-server.rar
>
>
> System VMs are not coming up in initial attempt, but they are coming up after multiple attempts.
> Failed to create VDI in SR initially but succeeding after several failed attempts.
> Copycommand is failing with Runtime exception:
> 2013-07-11 11:25:17,554 DEBUG [agent.transport.Request] (DirectAgent-73:null) Seq 1-557449293: Processing:  { Ans: , MgmtId: 6615759585382, via: 1, Ver: v1, Flags: 110, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"Catch Exception com.cloud.utils.exception.CloudRuntimeException for template +  due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr f3429124-745a-70d1-da86-ce97bfb2570d","wait":0}}] }
> sr-list on hypervisor:
> ================
> [root@Rack1Pod1Host13 ~]# xe sr-list type=nfs
> uuid ( RO)                : f3429124-745a-70d1-da86-ce97bfb2570d
>           name-label ( RW): c65a038a-750c-3b4f-bf26-7ce3b74e1c85
>     name-description ( RW): 1
>                 host ( RO): Rack1Pod1Host13
>                 type ( RO): nfs
>         content-type ( RO): user
> Log snippet from management server log:
> 2013-07-11 11:25:15,239 DEBUG [agent.transport.Request] (DirectAgent-15:null) Seq 1-557449293: Executing:  { Cmd , MgmtId: 6615759585382, via: 1, Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"template/tmpl/1/1/ddc39e82-13b4-4d30-9149-5ea6c1b801d5.vhd","origUrl":"http://10.147.28.7/templates/newsysvmtemp/systemvmtemplate-2013-06-23-master-xen.vhd.bz2","uuid":"39355f04-ea28-11e2-9610-06045a000066","id":1,"format":"VHD","accountId":1,"checksum":"7d76f106e5a5f6b68d114291c5938b41","hvm":false,"displayText":"SystemVM Template (XenServer)","imageDataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.147.28.7/export/home/sanjeev/sec_xen_os","_role":"ImageCache"}},"name":"routing-1"}},"destTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"origUrl":"http://10.147.28.7/templates/newsysvmtemp/systemvmtemplate-2013-06-23-master-xen.vhd.bz2","uuid":"39355f04-ea28-11e2-9610-06045a000066","id":1,"format":"VHD","accountId":1,"checksum":"7d76f106e5a5f6b68d114291c5938b41","hvm":false,"displayText":"SystemVM Template (XenServer)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"routing-1"}},"wait":10800}}] }
> 2013-07-11 11:25:15,241 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-73:null) Seq 1-557449293: Executing request
> 2013-07-11 11:25:16,175 DEBUG [agent.transport.Request] (secstorage-1:null) Seq 1-557449302: Waiting for Seq 557449293 Scheduling:  { Cmd , MgmtId: 6615759585382, via: 1, Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"5b6d5297-e270-4d01-ad4c-f5a75d96134c","origUrl":"http://10.147.28.7/templates/newsysvmtemp/systemvmtemplate-2013-06-23-master-xen.vhd.bz2","uuid":"39355f04-ea28-11e2-9610-06045a000066","id":1,"format":"VHD","accountId":1,"checksum":"7d76f106e5a5f6b68d114291c5938b41","hvm":false,"displayText":"SystemVM Template (XenServer)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"routing-1"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"a11bf731-9838-4f1a-9472-655086d2104d","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"ROOT-3","size":222905783,"volumeId":3,"vmName":"s-3-VM","accountId":1,"format":"VHD","id":3}},"wait":0}}] }
> 2013-07-11 11:25:17,551 WARN  [xen.resource.XenServerStorageProcessor] (DirectAgent-73:null) can not create vdi in sr f3429124-745a-70d1-da86-ce97bfb2570d
> 2013-07-11 11:25:17,552 WARN  [xen.resource.XenServerStorageProcessor] (DirectAgent-73:null) Catch Exception com.cloud.utils.exception.CloudRuntimeException for template +  due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr f3429124-745a-70d1-da86-ce97bfb2570d
> com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr f3429124-745a-70d1-da86-ce97bfb2570d
>         at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copy_vhd_from_secondarystorage(XenServerStorageProcessor.java:792)
>         at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.copyTemplateToPrimaryStorage(XenServerStorageProcessor.java:863)
>         at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:70)
>         at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHandlerBase.java:49)
>         at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:628)
>         at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:73)
>         at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:104)
>         at com.cloud.agent.manager.DirectAgentAttache$Task.run(DirectAgentAttache.java:186)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-07-11 11:25:17,553 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-73:null) Seq 1-557449293: Response Received:
> 2013-07-11 11:25:17,554 DEBUG [agent.transport.Request] (DirectAgent-73:null) Seq 1-557449293: Processing:  { Ans: , MgmtId: 6615759585382, via: 1, Ver: v1, Flags: 110, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"Catch Exception com.cloud.utils.exception.CloudRuntimeException for template +  due to com.cloud.utils.exception.CloudRuntimeException: can not create vdi in sr f3429124-745a-70d1-da86-ce97bfb2570d","wait":0}}] }
> 2013-07-11 11:25:17,555 DEBUG [agent.transport.Request] (consoleproxy-1:null) Seq 1-557449293: Received:  { Ans: , MgmtId: 6615759585382, via: 1, Ver: v1, Flags: 110, { CopyCmdAnswer } }
> 2013-07-11 11:25:17,558 DEBUG [storage.image.BaseImageStoreDriverImpl] (consoleproxy-1:null) Unable to destoy TEMPLATE: 1
> com.cloud.utils.exception.CloudRuntimeException: not implemented yet
>         at org.apache.cloudstack.storage.endpoint.DefaultEndPointSelector.select(DefaultEndPointSelector.java:230)
>         at org.apache.cloudstack.storage.endpoint.DefaultEndPointSelector.select(DefaultEndPointSelector.java:216)
>         at org.apache.cloudstack.storage.image.BaseImageStoreDriverImpl.deleteAsync(BaseImageStoreDriverImpl.java:189)
>         at org.apache.cloudstack.storage.image.store.ImageStoreImpl.delete(ImageStoreImpl.java:148)
>         at org.apache.cloudstack.storage.cache.manager.StorageCacheManagerImpl.deleteCacheObject(StorageCacheManagerImpl.java:258)
>         at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyObject(AncientDataMotionStrategy.java:193)
>         at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:351)
>         at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:390)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:490)
>         at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485)
>         at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:550)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157)
>         at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111)
>         at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
>         at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
>         at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-07-11 11:25:17,559 DEBUG [agent.manager.AgentAttache] (DirectAgent-73:null) Seq 1-557449302: Sending now.  is current sequence.
> 2013-07-11 11:25:17,563 DEBUG [agent.transport.Request] (DirectAgent-73:null) Seq 1-557449302: Executing:  { Cmd , MgmtId: 6615759585382, via: 1, Ver: v1, Flags: 100111, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.TemplateObjectTO":{"path":"5b6d5297-e270-4d01-ad4c-f5a75d96134c","origUrl":"http://10.147.28.7/templates/newsysvmtemp/systemvmtemplate-2013-06-23-master-xen.vhd.bz2","uuid":"39355f04-ea28-11e2-9610-06045a000066","id":1,"format":"VHD","accountId":1,"checksum":"7d76f106e5a5f6b68d114291c5938b41","hvm":false,"displayText":"SystemVM Template (XenServer)","imageDataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"routing-1"}},"destTO":{"org.apache.cloudstack.storage.to.VolumeObjectTO":{"uuid":"a11bf731-9838-4f1a-9472-655086d2104d","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"c65a038a-750c-3b4f-bf26-7ce3b74e1c85","id":1,"poolType":"NetworkFilesystem","host":"10.147.28.7","path":"/export/home/sanjeev/pri_xen_os","port":2049}},"name":"ROOT-3","size":222905783,"volumeId":3,"vmName":"s-3-VM","accountId":1,"format":"VHD","id":3}},"wait":0}}] }
> 2013-07-11 11:25:17,566 DEBUG [agent.manager.DirectAgentAttache] (DirectAgent-82:null) Seq 1-557449302: Executing request
> 2013-07-11 11:25:17,602 DEBUG [storage.volume.VolumeServiceImpl] (consoleproxy-1:null) failed to create template on storage
> java.lang.RuntimeException: InvocationTargetException when invoking RPC callback for command: copyBaseImageCallback
>         at org.apache.cloudstack.framework.async.AsyncCallbackDispatcher.dispatch(AsyncCallbackDispatcher.java:148)
>         at org.apache.cloudstack.framework.async.InplaceAsyncCallbackDriver.performCompletionCallback(InplaceAsyncCallbackDriver.java:26)
>         at org.apache.cloudstack.framework.async.AsyncCallbackDispatcher.complete(AsyncCallbackDispatcher.java:120)
>         at org.apache.cloudstack.storage.motion.AncientDataMotionStrategy.copyAsync(AncientDataMotionStrategy.java:363)
>         at org.apache.cloudstack.storage.motion.DataMotionServiceImpl.copyAsync(DataMotionServiceImpl.java:55)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:390)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:490)
>         at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485)
>         at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:550)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157)
>         at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111)
>         at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
>         at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
>         at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> Caused by: java.lang.reflect.InvocationTargetException
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at org.apache.cloudstack.framework.async.AsyncCallbackDispatcher.dispatch(AsyncCallbackDispatcher.java:142)
>         ... 28 more
> Caused by: com.cloud.utils.exception.CloudRuntimeException: Failed to update state
>         at org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:160)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.copyBaseImageCallback(VolumeServiceImpl.java:411)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         ... 32 more
> Caused by: com.cloud.utils.fsm.NoTransitionException: Unable to transition to a new state from Ready via OperationFailed
>         at com.cloud.utils.fsm.StateMachine2.getNextState(StateMachine2.java:83)
>         at com.cloud.utils.fsm.StateMachine2.transitTo(StateMachine2.java:100)
>         at org.apache.cloudstack.storage.datastore.ObjectInDataStoreManagerImpl.update(ObjectInDataStoreManagerImpl.java:247)
>         at org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:158)
>         ... 34 more
> 2013-07-11 11:25:17,622 WARN  [storage.datastore.ObjectInDataStoreManagerImpl] (consoleproxy-1:null) Template 1 is not found on storage pool 1, so no need to delete
> 2013-07-11 11:25:17,623 ERROR [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:null) Failed to start instance VM[ConsoleProxy|v-2-VM]
> com.cloud.utils.exception.CloudRuntimeException: can't find mapping in ObjectInDataStore table for: org.apache.cloudstack.storage.image.store.TemplateObject@111b863
>         at org.apache.cloudstack.storage.datastore.ObjectInDataStoreManagerImpl.update(ObjectInDataStoreManagerImpl.java:228)
>         at org.apache.cloudstack.storage.image.store.TemplateObject.processEvent(TemplateObject.java:158)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createBaseImageAsync(VolumeServiceImpl.java:393)
>         at org.apache.cloudstack.storage.volume.VolumeServiceImpl.createVolumeFromTemplateAsync(VolumeServiceImpl.java:490)
>         at com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2485)
>         at com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2542)
>         at com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:851)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:557)
>         at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:550)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.startProxy(ConsoleProxyManagerImpl.java:556)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.allocCapacity(ConsoleProxyManagerImpl.java:927)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:1664)
>         at com.cloud.consoleproxy.ConsoleProxyManagerImpl.expandPool(ConsoleProxyManagerImpl.java:157)
>         at com.cloud.vm.SystemVmLoadScanner.loadScan(SystemVmLoadScanner.java:111)
>         at com.cloud.vm.SystemVmLoadScanner.access$100(SystemVmLoadScanner.java:33)
>         at com.cloud.vm.SystemVmLoadScanner$1.reallyRun(SystemVmLoadScanner.java:81)
>         at com.cloud.vm.SystemVmLoadScanner$1.run(SystemVmLoadScanner.java:72)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>         at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
>         at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
>         at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>         at java.lang.Thread.run(Thread.java:679)
> 2013-07-11 11:25:17,632 DEBUG [cloud.vm.VirtualMachineManagerImpl] (consoleproxy-1:null) Cleaning up resources for the vm VM[ConsoleProxy|v-2-VM] in Starting state
> 2013-07-11 11:25:17,636 DEBUG [agent.transport.Request] (consoleproxy-1:null) Seq 1-557449303: Sending  { Cmd , MgmtId: 6615759585382, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"v-2-VM","wait":0}}] }
> 2013-07-11 11:25:17,637 DEBUG [agent.transport.Request] (consoleproxy-1:null) Seq 1-557449303: Executing:  { Cmd , MgmtId: 6615759585382, via: 1, Ver: v1, Flags: 100011, [{"com.cloud.agent.api.StopCommand":{"isProxy":false,"executeInSequence":false,"vmName":"v-2-VM","wait":0}}] }
> Image store info from cloud DB:
> mysql> select * from image_store;
> +----+--------------------------------------------------+---------------------+----------+--------------------------------------------------+----------------+--------+------------+--------------------------------------+--------+---------------------+---------------------+------------+------------+
> | id | name                                             | image_provider_name | protocol | url                                              | data_center_id | scope  | role       | uuid                                 | parent | created             | removed             | total_size | used_bytes |
> +----+--------------------------------------------------+---------------------+----------+--------------------------------------------------+----------------+--------+------------+--------------------------------------+--------+---------------------+---------------------+------------+------------+
> |  1 | nfs://10.147.28.7/export/home/sanjeev/sec_xen_os | NFS                 | nfs      | nfs://10.147.28.7/export/home/sanjeev/sec_xen_os |              1 | ZONE   | ImageCache | 8e43fa4b-cf0c-46c2-9e68-ead6bd481742 | NULL   | 2013-07-11 14:51:18 | NULL                |       NULL |       NULL |
> |  2 | obj_store                                        | S3                  | http     | NULL                                             |           NULL | REGION | Image      | 73729a9c-c221-4532-a5c2-745b834f6b3f | NULL   | 2013-07-11 14:51:18 | 2013-07-11 14:59:17 |       NULL |       NULL |
> |  3 | objstore                                         | S3                  | http     | NULL                                             |           NULL | REGION | Image      | 0c444304-7e5f-4137-8da1-a812c3b81cf7 | NULL   | 2013-07-11 15:07:56 | NULL                |       NULL |       NULL |
> +----+--------------------------------------------------+---------------------+----------+--------------------------------------------------+----------------+--------+------------+--------------------------------------+--------+---------------------+---------------------+------------+------------+
> 3 rows in set (0.03 sec)
> mysql> select * from storage_pool;
> +----+------------+--------------------------------------+-------------------+------+----------------+--------+------------+---------------+----------------+--------------+-----------+---------------------------------+---------------------+---------+-------------+--------+-----------------------+---------+------------+---------+---------------+
> | id | name       | uuid                                 | pool_type         | port | data_center_id | pod_id | cluster_id | used_bytes    | capacity_bytes | host_address | user_info | path                            | created             | removed | update_time | status | storage_provider_name | scope   | hypervisor | managed | capacity_iops |
> +----+------------+--------------------------------------+-------------------+------+----------------+--------+------------+---------------+----------------+--------------+-----------+---------------------------------+---------------------+---------+-------------+--------+-----------------------+---------+------------+---------+---------------+
> |  1 | pri_xen_os | c65a038a-750c-3b4f-bf26-7ce3b74e1c85 | NetworkFilesystem | 2049 |              1 |      1 |          1 | 2258215108608 |  5902284816384 | 10.147.28.7  | NULL      | /export/home/sanjeev/pri_xen_os | 2013-07-11 14:51:08 | NULL    | NULL        | Up     | DefaultPrimary        | CLUSTER | NULL       |       0 |          NULL |
> +----+------------+--------------------------------------+-------------------+------+----------------+--------+------------+---------------+----------------+--------------+-----------+---------------------------------+---------------------+---------+-------------+--------+-----------------------+---------+------------+---------+---------------+
> 1 row in set (0.00 sec)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira