[jira] [Created] (CLOUDSTACK-5357) Xenserver - Failed to create snapshot due to "unable to destroy task(com.xe nsource.xenapi.Task@67d312d6) on host(23af93a0-93ff-40cb-ba11-a11d1b884d37)" when secondary store was unavaiable for 1 and 1/2 hours and then brought up..

["Sangeetha Hariharan (JIRA)" <ji...@apache.org>]

Sangeetha Hariharan created CLOUDSTACK-5357:
-----------------------------------------------

             Summary: Xenserver - Failed to create snapshot due to "unable to destroy task(com.xe nsource.xenapi.Task@67d312d6) on host(23af93a0-93ff-40cb-ba11-a11d1b884d37)" when secondary store was unavaiable for 1 and 1/2 hours and then brought up..
                 Key: CLOUDSTACK-5357
                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5357
             Project: CloudStack
          Issue Type: Bug
      Security Level: Public (Anyone can view this level - this is the default.)
          Components: Management Server
    Affects Versions: 4.3.0
         Environment: Build from 4.3
            Reporter: Sangeetha Hariharan
            Priority: Critical
             Fix For: 4.3.0


Xenserver - Failed to create snapshot due to "unable to destroy task(com.xe nsource.xenapi.Task@67d312d6) on host(23af93a0-93ff-40cb-ba11-a11d1b884d37)" when secondary store was unavaiable for 1 and 1/2 hours and then brought up.

Set up:

Advanced Zone with 2 Xenserver 6.2 hosts:

Steps to reproduce the problem:

1. Deploy 5 Vms in each of the hosts with 10 GB ROOT volume size , so we start with 10 Vms.
2. Start concurrent hourly snapshots for ROOT volumes of all the Vms.
3. Shutdown the Secondary storage server when the snapshots are in the progress.
4. Bring the Secondary storage server up after 1and 1/2 hour.

After the secondary store was brought up , we see the snapshot process continue to be active ( I see the copy command that was halted when the secondary store was down , starts progressing when server is drought up online).

I also see that the snapshot process has been completed on the secondary store. But management server throws the following exception and leaves behind the snapshot in "CreatedOnPrimary" state in DB.

2013-12-03 19:24:36,296 WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-252:ctx-8ca50e48) unable to destroy task(com.xe
nsource.xenapi.Task@67d312d6) on host(23af93a0-93ff-40cb-ba11-a11d1b884d37) due to
You gave an invalid object reference.  The object may have recently been deleted.  The class parameter gives the type of
reference given, and the handle parameter echoes the bad value given.
        at com.xensource.xenapi.Types.checkResponse(Types.java:209)
        at com.xensource.xenapi.Connection.dispatch(Connection.java:368)
        at com.cloud.hypervisor.xen.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool
.java:909)
        at com.xensource.xenapi.Task.destroy(Task.java:616)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.cloudVDIcopy(CitrixResourceBase.java:3920)
        at com.cloud.hypervisor.xen.resource.XenServerStorageProcessor.backupSnapshot(XenServerStorageProcessor.java:1268
)
        at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.execute(StorageSubsystemCommandHandlerBase.java:
90)
        at com.cloud.storage.resource.StorageSubsystemCommandHandlerBase.handleStorageCommands(StorageSubsystemCommandHan
dlerBase.java:50)
        at com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:613)
        at com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59)
        at com.cloud.hypervisor.xen.resource.XenServer610Resource.executeRequest(XenServer610Resource.java:106)
        at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:10
3)
        at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
        at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.ja
va:178)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)
2013-12-03 19:24:36,297 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-252:ctx-8ca50e48) Host 10.223.59.67 OpaqueRef:50d81c75-f239-1746-ae06-c6228e788104: Removing SR
2013-12-03 19:24:36,305 DEBUG [c.c.h.x.r.CitrixResourceBase] (DirectAgent-252:ctx-8ca50e48) Host 10.223.59.67 OpaqueRef:e2bcc8f5-a04a-3f5a-c234-85beb190aa02: Unplugging pbd





[root@Rack3Host5 6]# ls -lth
total 1.3G
-rw-r--r--. 1 root root 6.7G Dec  3 23:24 1f4ec466-688f-4a53-bfcb-e5f2da3bccae.vhd
[root@Rack3Host5 6]#




--
This message was sent by Atlassian JIRA
(v6.1#6144)