You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "edison su (JIRA)" <ji...@apache.org> on 2013/12/30 19:55:50 UTC
[jira] [Resolved] (CLOUDSTACK-5395) When backup snapshot fails
becasue of backup.snapshot.wait time exceeding , the vhd entries form the
primary store is not getting cleared.
[ https://issues.apache.org/jira/browse/CLOUDSTACK-5395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
edison su resolved CLOUDSTACK-5395.
-----------------------------------
Resolution: Fixed
fixed by Min's 7cd125ca3905d7d9596bde3a7a4cf19a32a23c9b
> When backup snapshot fails becasue of backup.snapshot.wait time exceeding , the vhd entries form the primary store is not getting cleared.
> ------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CLOUDSTACK-5395
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5395
> Project: CloudStack
> Issue Type: Bug
> Security Level: Public(Anyone can view this level - this is the default.)
> Components: Management Server
> Affects Versions: 4.3.0
> Environment: Build from 4.3
> Reporter: Sangeetha Hariharan
> Assignee: edison su
> Priority: Critical
> Fix For: 4.3.0
>
> Attachments: 1205logs.rar
>
>
> Steps to reproduce the problem:
> Deploy 5 Vms in each of the hosts with 10 GB , so we start with 10 Vms.
> We will be constantly writing to the ROOT volume.
> Change the backup.snapshot.wait to 10 mts and restart management server.
> Start concurrent snapshots for ROOT volumes of all the Vms.
> After 10 mts , the snapshots fail. They are present in the database in "CreatedOnPrimary" state.
> Vhd entries from the primary store fails to be cleaned up once the backsnapshpot job fails.
> In this case ,API fails with - "Failed to create snapshot due to an internal error creating snapshot for volume 81" . We should be able to present a more meaningful error message in this case.
> Expected Behavior:
> We should be able to clean up the Vhd entries from the primary store when the backsnapshpot job fails.
> In such cases , instead of
> select * from snapshot_store_ref;
> 702 | 1 | 355 | 2013-12-06 01:25:43 | NULL | NULL | Primary | 0 | 0 | 0 | 2eedb23e-6c3f-4cae-832b-8ddb67c1fc60 | Ready | 2 | 0 | 2013-12-06 01:25:44 | 81 |
> | 703 | 1 | 356 | 2013-12-06 01:25:43 | NULL | NULL | Primary | 0 | 0 | 0 | 9d88bc01-9406-41ad-a134-e74dc1457954 | Ready | 2 | 0 | 2013-12-06 01:26:12 | 80 |
> | 704 | 1 | 357 | 2013-12-06 01:25:43 | NULL | NULL | Primary | 0 | 0 | 0 | 2667f2bc-6086-4ec3-a88d-20811eabde91 | Ready | 2 | 0 | 2013-12-06 01:26:08 | 79 |
> | 705 | 1 | 358 | 2013-12-06 01:25:44 | NULL | NULL | Primary | 0 | 0 | 0 | 522b2296-6960-46f2-af7d-10ddfbede1da | Ready | 2 | 0 | 2013-12-06 01:26:45 | 78 |
> | 706 | 1 | 359 | 2013-12-06 01:25:44 | NULL | NULL | Primary | 0 | 0 | 0 | 3b94fa9d-a5a5-4441-8f9f-275dcef90368 | Ready | 2 | 0 | 2013-12-06 01:26:04 | 77 |
> | 707 | 1 | 360 | 2013-12-06 01:25:44 | NULL | NULL | Primary | 0 | 0 | 0 | 1ec1d5ef-177f-4da4-8464-f0c6d71a4e84 | Ready | 2 | 0 | 2013-12-06 01:25:59 | 76 |
> | 708 | 1 | 361 | 2013-12-06 01:25:44 | NULL | NULL | Primary | 0 | 0 | 0 | 324e7552-b42a-4660-90d6-62015a7a478e | Ready | 2 | 0 | 2013-12-06 01:26:21 | 75 |
> | 709 | 1 | 362 | 2013-12-06 01:25:44 | NULL | NULL | Primary | 0 | 0 | 0 | 65bd522c-c2c8-471a-be37-095558d058f2 | Ready | 2 | 0 | 2013-12-06 01:26:16 | 74 |
> | 710 | 1 | 363 | 2013-12-06 01:25:44 | NULL | NULL | Primary | 0 | 0 | 0 | d45ca6c7-7284-4150-907c-9499e9737c47 | Ready | 2 | 0 | 2013-12-06 01:25:46 | 73 |
> | 711 | 1 | 364 | 2013-12-06 01:25:44 | NULL | NULL | Primary | 0 | 0 | 0 | 4422f362-0be5-4a10-b172-45678d56f807 | Ready | 2 | 0 | 2013-12-06 01:25:55 | 72 |
> | 712 | 1 | 365 | 2013-12-06 01:25:44 | NULL | NULL | Primary | 0 | 0 | 0 | 89ffd430-3c03-45d2-9c48-9384636b9cd8 | Ready | 2 | 0 | 2013-12-06 01:26:01 | 71 |
> | 714 | 1 | 366 | 2013-12-06 01:25:45 | NULL | NULL | Primary | 0 | 0 | 0 | fca5545c-9b83-4bc1-abd2-dd1bc82b23bd | Ready | 2 | 0 | 2013-12-06 01:25:53 | 70 |
> | 715 | 1 | 367 | 2013-12-06 01:25:45 | NULL | NULL | Primary | 0 | 0 | 0 | 033d8f55-8895-40b8-a120-11b28fa1f96e | Ready | 2 | 0 | 2013-12-06 01:25:50 | 69 |
> | 716 | 1 | 368 | 2013-12-06 01:25:45 | NULL | NULL | Primary | 0 | 0 | 0 | e4d02558-28c2-474e-a379-970b22f33f55 | Ready | 2 | 0 | 2013-12-06 01:26:23 | 68 |
> | 717 | 1 | 369 | 2013-12-06 01:25:45 | NULL | NULL | Primary | 0 | 0 | 0 | 6f7c1ca0-9877-44af-9f77-4db7b8efc934 | Ready | 2 | 0 | 2013-12-06 01:26:37 | 67 |
> | 718 | 1 | 370 | 2013-12-06 01:25:45 | NULL | NULL | Primary | 0 | 0 | 0 | e3860504-7e37-416a-8216-c4a021f5e8bf | Ready | 2 | 0 | 2013-12-06 01:25:47 | 66 |
> | 719 | 1 | 371 | 2013-12-06 01:25:45 | NULL | NULL | Primary | 0 | 0 | 0 | 48c2a9bd-e7ce-429f-be73-ef9c19964ad3 | Ready | 2 | 0 | 2013-12-06 01:26:17 | 65 |
> | 720 | 1 | 372 | 2013-12-06 01:25:45 | NULL | NULL | Primary | 0 | 0 | 0 | 3eabb081-d0d9-41d6-9c11-00e002031461 | Ready | 2 | 0 | 2013-12-06 01:25:51 | 64 |
> | 721 | 1 | 373 | 2013-12-06 01:25:46 | NULL | NULL | Primary | 0 | 0 | 0 | c9968fbb-0876-4846-94aa-e44ea7e3143d | Ready | 2 | 0 | 2013-12-06 01:26:06 | 63 |
> | 723 | 1 | 374 | 2013-12-06 01:25:46 | NULL | NULL | Primary | 0 | 0 | 0 | 72fdec4a-c2f1-48e1-80bf-806b6db46820 | Ready | 2 | 0 | 2013-12-06 01:26:02 | 62 |
> | 742 | 1 | 375 | 2013-12-06 01:40:43 | NULL | NULL | Primary | 0 | 0 | 0 | 8a692b5a-569e-473c-83c4-dc32c09c8491 | Ready | 2 | 0 | 2013-12-06 01:40:47 | 60 |
> +-----+----------+-------------+---------------------+--------------+--------+------------+------+---------------+--------------------+-----------------------------------------------------+-----------+--------------+---------+---------------------+-----------+
> 2013-12-05 20:51:41,212 DEBUG [c.c.s.s.SnapshotManagerImpl] (Job-Executor-42:ctx-c20be5fc ctx-5f8cfbf3) Failed to create snapshot
> com.cloud.utils.exception.CloudRuntimeException: BackupSnapshot Failed due to Async 600 seconds timeout for task com.xensource.xenapi.Task@8b2ca24f
> at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:275)
> at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:135)
> at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:294)
> at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:951)
> at sun.reflect.GeneratedMethodAccessor405.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy161.takeSnapshot(Unknown Source)
> at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1341)
> at com.cloud.storage.VolumeApiServiceImpl.takeSnapshot(VolumeApiServiceImpl.java:1408)
> at sun.reflect.GeneratedMethodAccessor404.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy233.takeSnapshot(Unknown Source)
> at org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:181)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
> at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
> at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:520)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 2013-12-05 20:51:41,217 DEBUG [o.a.c.s.v.VolumeServiceImpl] (Job-Executor-42:ctx-c20be5fc ctx-5f8cfbf3) Take snapshot: 60 failed
> com.cloud.utils.exception.CloudRuntimeException: Failed to create snapshot
> at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:972)
> at sun.reflect.GeneratedMethodAccessor405.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy161.takeSnapshot(Unknown Source)
> at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1341)
> at com.cloud.storage.VolumeApiServiceImpl.takeSnapshot(VolumeApiServiceImpl.java:1408)
> at sun.reflect.GeneratedMethodAccessor404.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy233.takeSnapshot(Unknown Source)
> at org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:181)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
> at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
> at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:520)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: com.cloud.utils.exception.CloudRuntimeException: BackupSnapshot Failed due to Async 600 seconds timeout for task com.xensource.xenapi.Task@8b2ca24f
> at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:275)
> at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:135)
> at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:294)
> at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:951)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy161.takeSnapshot(Unknown Source)
> at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1341)
> at com.cloud.storage.VolumeApiServiceImpl.takeSnapshot(VolumeApiServiceImpl.java:1408)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy233.takeSnapshot(Unknown Source)
> at org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:181)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
> at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
> at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:520)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> 2013-12-05 20:35:50,550 DEBUG [o.a.c.s.v.VolumeServiceImpl] (Job-Executor-22:ctx-ae0b13d4 ctx-1bc38e81) Take snapshot: 81 failed
> com.cloud.utils.exception.CloudRuntimeException: Failed to create snapshot
> at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:972)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy161.takeSnapshot(Unknown Source)
> at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1341)
> at com.cloud.storage.VolumeApiServiceImpl.takeSnapshot(VolumeApiServiceImpl.java:1408)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:601)
> at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at $Proxy233.takeSnapshot(Unknown Source)
> at org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd.execute(CreateSnapshotCmd.java:181)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:161)
> at com.cloud.api.ApiAsyncJobDispatcher.runJobInContext(ApiAsyncJobDispatcher.java:109)
> at com.cloud.api.ApiAsyncJobDispatcher$1.run(ApiAsyncJobDispatcher.java:66)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:63)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:520)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: com.cloud.utils.exception.CloudRuntimeException: BackupSnapshot Failed due to Async 600 seconds timeout for task com.xensource.xenapi.Task@db9844a9
> at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:275)
> at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:135)
> at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:294)
> at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:951)
> ... 44 more
> 2013-12-05 20:35:50,552 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-22:ctx-ae0b13d4) Complete async job-697, jobStatus: FAILED, resultCode: 530, result: org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Failed to create snapshot due to an internal error creating snapshot for volume 81"}
> 2013-12-05 20:35:50,557 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Job-Executor-22:ctx-ae0b13d4) Done executing org.apache.cloudstack.api.command.user.snapshot.CreateSnapshotCmd for job-697
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)