You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users-cn@cloudstack.apache.org by "aix.niuy" <ai...@gmail.com> on 2014/11/07 09:55:35 UTC
回复: Re: 回复:cloudstack4.4.1+KVM高级网络 给虚拟机root磁盘卷创建快照总是出错
宿主机是Centos吗?
virsh version 把版本信息贴一下
2014-11-07
aix.niuy
发件人:刘宇超 <ta...@hotmail.com>
发送时间:2014-11-07 16:50
主题:Re: 回复:cloudstack4.4.1+KVM高级网络 给虚拟机root磁盘卷创建快照总是出错
收件人:"赵涛"<zh...@cloudstack.apache.org>
抄送:
请问那位大神遇到过?
------------------ 原始邮件 ------------------
发件人: "赵涛";<zh...@qq.com>;
发送时间: 2014年11月5日(星期三) 中午11:05
收件人: "users-cn"<us...@cloudstack.apache.org>;
主题: cloudstack4.4.1+KVM高级网络 给虚拟机root磁盘卷创建快照总是出错
测试环境cloudstack4.4.1+KVM高级网络:管理节点、kvm host都安装CentOS6.3-64位在全局参数里设置了
kvm.snapshot.enabled 值为true,重启了管理节点。但是关闭虚拟机后给虚拟机root磁盘卷创建快照总是出错,cs
通知栏里提示创建成功,查看快照后发现快照显示列表里是Err。请问该如何解决?错误日志如下:
2014-11-05 09:24:21,505 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-bc8070de) Execute sync-queue item: SyncQueueItemVO {id:13, queueId: 11, contentType: AsyncJob, contentId: 73, lastProcessMsid: null, lastprocessNumber: null, lastProcessTime: null, created: Wed Nov 05 09:24:19 CST 2014}
2014-11-05 09:24:21,506 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-bc8070de) Schedule queued job-73
2014-11-05 09:24:21,510 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Add job-73 into job monitoring
2014-11-05 09:24:21,510 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Executing AsyncJobVO {id:73, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.storage.VmWorkTakeVolumeSnapshot, cmdInfo: rO0ABXNyACpjb20uY2xvdWQuc3RvcmFnZS5WbVdvcmtUYWtlVm9sdW1lU25hcHNob3QEvmAbguLVzwIABFoACXF1aWVzY2VWbUwACHBvbGljeUlkdAAQTGphdmEvbGFuZy9Mb25nO0wACnNuYXBzaG90SWRxAH4AAUwACHZvbHVtZUlkcQB-AAF4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1ldAASTGphdmEvbGFuZy9TdHJpbmc7eHAAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAd0ABRWb2x1bWVBcGlTZXJ2aWNlSW1wbABzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAABzcQB-AAYAAAAAAAAABnEAfgAJ, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 52238016467, completeMsid: null, lastUpdated: null, lastPolled: null, created: Wed Nov 05 09:24:19 CST 2014}
2014-11-05 09:24:21,511 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Run VM work job: com.cloud.storage.VmWorkTakeVolumeSnapshot for VM 7, job origin: 72
2014-11-05 09:24:21,512 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Execute VM work job: com.cloud.storage.VmWorkTakeVolumeSnapshot{"volumeId":6,"policyId":0,"snapshotId":6,"quiesceVm":false,"userId":2,"accountId":2,"vmId":7,"handlerName":"VolumeApiServiceImpl"}
2014-11-05 09:24:21,548 DEBUG [c.c.a.t.Request] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Seq 1-5995979953890398722: Sending { Cmd , MgmtId: 52238016467, via: 1(node1-kvm), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"e2671633-076c-4d3e-a139-ed3807159312","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e7a222aa-a542-300f-a7bb-7fb8b24c4bde","id":1,"poolType":"NetworkFilesystem","host":"10.6.26.21","path":"/export/primary","port":2049,"url":"NetworkFilesystem://10.6.26.21/export/primary/?ROLE=Primary&STOREUUID=e7a222aa-a542-300f-a7bb-7fb8b24c4bde"}},"name":"ROOT-7","size":53687091200,"path":"e2671633-076c-4d3e-a139-ed3807159312","volumeId":6,"vmName":"i-2-7-VM","accountId":2,"format":"QCOW2","id":6,"deviceId":0,"hypervisorType":"KVM"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e7a222aa-a542-300f-a7bb-7fb8b24c4bde","id":1,"poolType":"NetworkFilesystem","host":"10.6.26.21","path":"/export/primary","port":2049,"url":"NetworkFilesystem://10.6.26.21/export/primary/?ROLE=Primary&STOREUUID=e7a222aa-a542-300f-a7bb-7fb8b24c4bde"}},"vmName":"i-2-7-VM","name":"test03_ROOT-7_20141105012419","hypervisorType":"KVM","id":6,"quiescevm":false,"physicalSize":0}},"wait":0}}] }
2014-11-05 09:24:22,858 DEBUG [c.c.a.ApiServlet] (catalina-exec-23:ctx-2f001f7c) ===START=== 192.168.9.76 -- GET command=queryAsyncJobResult&jobId=b2479cee-d5b5-464a-bae4-6974027f0a67&response=json&sessionkey=tMEAJPWMmcthAjo86aaTwBSDozQ%3D&_=1415150657767
2014-11-05 09:24:22,872 DEBUG [c.c.a.ApiServlet] (catalina-exec-23:ctx-2f001f7c ctx-9fe6010e) ===END=== 192.168.9.76 -- GET command=queryAsyncJobResult&jobId=b2479cee-d5b5-464a-bae4-6974027f0a67&response=json&sessionkey=tMEAJPWMmcthAjo86aaTwBSDozQ%3D&_=1415150657767
2014-11-05 09:24:23,750 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-10:null) SeqA 2-19036: Processing Seq 2-19036: { Cmd , MgmtId: -1, via: 2, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n \"connections\": [\n {\n \"id\": 8,\n \"clientInfo\": \"\",\n \"host\": \"10.6.31.4\",\n \"port\": 5905,\n \"tag\": \"a53e9399-e0e5-4df0-b361-1da2380bceef\",\n \"createTime\": 1415150339521,\n \"lastUsedTime\": 1415150583469\n },\n {\n \"id\": 7,\n \"clientInfo\": \"\",\n \"host\": \"10.6.31.4\",\n \"port\": 5904,\n \"tag\": \"93e383ff-3143-4830-b3e5-19d6bd248eaa\",\n \"createTime\": 1415150282559,\n \"lastUsedTime\": 1415150580763\n }\n ]\n}","wait":0}}] }
2014-11-05 09:24:23,754 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-10:null) SeqA 2-19036: Sending Seq 2-19036: { Ans: , MgmtId: 52238016467, via: 2, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2014-11-05 09:24:25,851 DEBUG [c.c.a.ApiServlet] (catalina-exec-13:ctx-8d3d7287) ===START=== 192.168.9.76 -- GET command=queryAsyncJobResult&jobId=b2479cee-d5b5-464a-bae4-6974027f0a67&response=json&sessionkey=tMEAJPWMmcthAjo86aaTwBSDozQ%3D&_=1415150660759
2014-11-05 09:24:25,869 DEBUG [c.c.a.ApiServlet] (catalina-exec-13:ctx-8d3d7287 ctx-f46c1c3d) ===END=== 192.168.9.76 -- GET command=queryAsyncJobResult&jobId=b2479cee-d5b5-464a-bae4-6974027f0a67&response=json&sessionkey=tMEAJPWMmcthAjo86aaTwBSDozQ%3D&_=1415150660759
2014-11-05 09:24:27,606 DEBUG [c.c.a.t.Request] (AgentManager-Handler-7:null) Seq 1-5995979953890398722: Processing: { Ans: , MgmtId: 52238016467, via: 1, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CreateObjectAnswer":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"/mnt/e7a222aa-a542-300f-a7bb-7fb8b24c4bde/e2671633-076c-4d3e-a139-ed3807159312/00eb9935-f0a3-4af1-aa92-a39bcb8c060f","id":0,"quiescevm":false,"physicalSize":0}},"result":true,"wait":0}}] }
2014-11-05 09:24:27,607 DEBUG [c.c.a.t.Request] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Seq 1-5995979953890398722: Received: { Ans: , MgmtId: 52238016467, via: 1, Ver: v1, Flags: 10, { CreateObjectAnswer } }
2014-11-05 09:24:27,640 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) copyAsync inspecting src type SNAPSHOT copyAsync inspecting dest type SNAPSHOT
2014-11-05 09:24:27,675 DEBUG [c.c.a.t.Request] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Seq 1-5995979953890398723: Sending { Cmd , MgmtId: 52238016467, via: 1(node1-kvm), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"/mnt/e7a222aa-a542-300f-a7bb-7fb8b24c4bde/e2671633-076c-4d3e-a139-ed3807159312/00eb9935-f0a3-4af1-aa92-a39bcb8c060f","volume":{"uuid":"e2671633-076c-4d3e-a139-ed3807159312","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e7a222aa-a542-300f-a7bb-7fb8b24c4bde","id":1,"poolType":"NetworkFilesystem","host":"10.6.26.21","path":"/export/primary","port":2049,"url":"NetworkFilesystem://10.6.26.21/export/primary/?ROLE=Primary&STOREUUID=e7a222aa-a542-300f-a7bb-7fb8b24c4bde"}},"name":"ROOT-7","size":53687091200,"path":"e2671633-076c-4d3e-a139-ed3807159312","volumeId":6,"vmName":"i-2-7-VM","accountId":2,"format":"QCOW2","id":6,"deviceId":0,"hypervisorType":"KVM"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e7a222aa-a542-300f-a7bb-7fb8b24c4bde","id":1,"poolType":"NetworkFilesystem","host":"10.6.26.21","path":"/export/primary","port":2049,"url":"NetworkFilesystem://10.6.26.21/export/primary/?ROLE=Primary&STOREUUID=e7a222aa-a542-300f-a7bb-7fb8b24c4bde"}},"vmName":"i-2-7-VM","name":"test03_ROOT-7_20141105012419","hypervisorType":"KVM","id":6,"quiescevm":false,"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/6","volume":{"uuid":"e2671633-076c-4d3e-a139-ed3807159312","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e7a222aa-a542-300f-a7bb-7fb8b24c4bde","id":1,"poolType":"NetworkFilesystem","host":"10.6.26.21","path":"/export/primary","port":2049,"url":"NetworkFilesystem://10.6.26.21/export/primary/?ROLE=Primary&STOREUUID=e7a222aa-a542-300f-a7bb-7fb8b24c4bde"}},"name":"ROOT-7","size":53687091200,"path":"e2671633-076c-4d3e-a139-ed3807159312","volumeId":6,"vmName":"i-2-7-VM","accountId":2,"format":"QCOW2","id":6,"deviceId":0,"hypervisorType":"KVM"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.6.26.21/export/secondary","_role":"Image"}},"vmName":"i-2-7-VM","name":"test03_ROOT-7_20141105012419","hypervisorType":"KVM","id":6,"quiescevm":false,"physicalSize":0}},"executeInSequence":false,"options":{"fullSnapshot":"true"},"wait":21600}}] }
2014-11-05 09:24:28,034 DEBUG [c.c.a.t.Request] (AgentManager-Handler-9:null) Seq 1-5995979953890398723: Processing: { Ans: , MgmtId: 52238016467, via: 1, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"Failed to backup 00eb9935-f0a3-4af1-aa92-a39bcb8c060f for disk /mnt/e7a222aa-a542-300f-a7bb-7fb8b24c4bde/e2671633-076c-4d3e-a139-ed3807159312 to /mnt/cb13ca02-b0f4-371a-a676-768194da25f9/snapshots/2/6","wait":0}}] }
2014-11-05 09:24:28,034 DEBUG [c.c.a.t.Request] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Seq 1-5995979953890398723: Received: { Ans: , MgmtId: 52238016467, via: 1, Ver: v1, Flags: 10, { CopyCmdAnswer } }
2014-11-05 09:24:28,049 DEBUG [c.c.s.s.SnapshotManagerImpl] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Failed to create snapshot
com.cloud.utils.exception.CloudRuntimeException: Failed to backup 00eb9935-f0a3-4af1-aa92-a39bcb8c060f for disk /mnt/e7a222aa-a542-300f-a7bb-7fb8b24c4bde/e2671633-076c-4d3e-a139-ed3807159312 to /mnt/cb13ca02-b0f4-371a-a676-768194da25f9/snapshots/2/6
at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:282)
at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:137)
at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:300)
at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:966)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy179.takeSnapshot(Unknown Source)
at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1503)
at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:1766)
at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:2503)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at com.cloud.storage.VolumeApiServiceImpl.handleVmWorkJob(VolumeApiServiceImpl.java:2511)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy183.handleVmWorkJob(Unknown Source)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-11-05 09:24:28,058 DEBUG [o.a.c.s.v.VolumeServiceImpl] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Take snapshot: 6 failed
com.cloud.utils.exception.CloudRuntimeException: Failed to create snapshot
at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:983)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy179.takeSnapshot(Unknown Source)
at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1503)
at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:1766)
at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:2503)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at com.cloud.storage.VolumeApiServiceImpl.handleVmWorkJob(VolumeApiServiceImpl.java:2511)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy183.handleVmWorkJob(Unknown Source)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.cloud.utils.exception.CloudRuntimeException: Failed to backup 00eb9935-f0a3-4af1-aa92-a39bcb8c060f for disk /mnt/e7a222aa-a542-300f-a7bb-7fb8b24c4bde/e2671633-076c-4d3e-a139-ed3807159312 to /mnt/cb13ca02-b0f4-371a-a676-768194da25f9/snapshots/2/6
at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:282)
at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:137)
at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:300)
at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:966)
... 44 more
2014-11-05 09:24:28,058 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Done executing VM work job: com.cloud.storage.VmWorkTakeVolumeSnapshot{"volumeId":6,"policyId":0,"snapshotId":6,"quiesceVm":false,"userId":2,"accountId":2,"vmId":7,"handlerName":"VolumeApiServiceImpl"}
2014-11-05 09:24:28,058 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Complete async job-73, jobStatus: SUCCEEDED, resultCode: 0, result: rO0ABXNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cgAQamF2YS5sYW5nLk51bWJlcoaslR0LlOCLAgAAeHAAAAAAAAAABg
2014-11-05 09:24:28,063 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Done with run of VM work job: com.cloud.storage.VmWorkTakeVolumeSnapshot for VM 7, job origin: 72
2014-11-05 09:24:28,063 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Done executing com.cloud.storage.VmWorkTakeVolumeSnapshot for job-73
2014-11-05 09:24:28,070 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Sync queue (11) is currently empty
2014-11-05 09:24:28,071 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Remove job-73 from job monitoring
Re: 回复:cloudstack4.4.1+KVM高级网络 给虚拟机root磁盘卷创建快照总是出错
Posted by 刘宇超 <ta...@hotmail.com>.
Wei Zhou 说的对,升级编译新的qemu来解决。发件人: aix.niuy发送时间: 2014-11-07 16:55收件人: users-cn; 赵涛主题: 回复: Re: 回复:cloudstack4.4.1+KVM高级网络 给虚拟机root磁盘卷创建快照总是出错宿主机是Centos吗?
virsh version 把版本信息贴一下
2014-11-07
aix.niuy
发件人:刘宇超 <ta...@hotmail.com>
发送时间:2014-11-07 16:50
主题:Re: 回复:cloudstack4.4.1+KVM高级网络 给虚拟机root磁盘卷创建快照总是出错
收件人:"赵涛"<zh...@cloudstack.apache.org>
抄送:
请问那位大神遇到过?
------------------ 原始邮件 ------------------
发件人: "赵涛";<zh...@qq.com>;
发送时间: 2014年11月5日(星期三) 中午11:05
收件人: "users-cn"<us...@cloudstack.apache.org>;
主题: cloudstack4.4.1+KVM高级网络 给虚拟机root磁盘卷创建快照总是出错
测试环境cloudstack4.4.1+KVM高级网络:管理节点、kvm host都安装CentOS6.3-64位在全局参数里设置了
kvm.snapshot.enabled 值为true,重启了管理节点。但是关闭虚拟机后给虚拟机root磁盘卷创建快照总是出错,cs
通知栏里提示创建成功,查看快照后发现快照显示列表里是Err。请问该如何解决?错误日志如下:
2014-11-05 09:24:21,505 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-bc8070de) Execute sync-queue item: SyncQueueItemVO {id:13, queueId: 11, contentType: AsyncJob, contentId: 73, lastProcessMsid: null, lastprocessNumber: null, lastProcessTime: null, created: Wed Nov 05 09:24:19 CST 2014}
2014-11-05 09:24:21,506 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-bc8070de) Schedule queued job-73
2014-11-05 09:24:21,510 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Add job-73 into job monitoring
2014-11-05 09:24:21,510 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Executing AsyncJobVO {id:73, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.storage.VmWorkTakeVolumeSnapshot, cmdInfo: rO0ABXNyACpjb20uY2xvdWQuc3RvcmFnZS5WbVdvcmtUYWtlVm9sdW1lU25hcHNob3QEvmAbguLVzwIABFoACXF1aWVzY2VWbUwACHBvbGljeUlkdAAQTGphdmEvbGFuZy9Mb25nO0wACnNuYXBzaG90SWRxAH4AAUwACHZvbHVtZUlkcQB-AAF4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1ldAASTGphdmEvbGFuZy9TdHJpbmc7eHAAAAAAAAAAAgAAAAAAAAACAAAAAAAAAAd0ABRWb2x1bWVBcGlTZXJ2aWNlSW1wbABzcgAOamF2YS5sYW5nLkxvbmc7i-SQzI8j3wIAAUoABXZhbHVleHIAEGphdmEubGFuZy5OdW1iZXKGrJUdC5TgiwIAAHhwAAAAAAAAAABzcQB-AAYAAAAAAAAABnEAfgAJ, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 52238016467, completeMsid: null, lastUpdated: null, lastPolled: null, created: Wed Nov 05 09:24:19 CST 2014}
2014-11-05 09:24:21,511 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Run VM work job: com.cloud.storage.VmWorkTakeVolumeSnapshot for VM 7, job origin: 72
2014-11-05 09:24:21,512 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Execute VM work job: com.cloud.storage.VmWorkTakeVolumeSnapshot{"volumeId":6,"policyId":0,"snapshotId":6,"quiesceVm":false,"userId":2,"accountId":2,"vmId":7,"handlerName":"VolumeApiServiceImpl"}
2014-11-05 09:24:21,548 DEBUG [c.c.a.t.Request] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Seq 1-5995979953890398722: Sending { Cmd , MgmtId: 52238016467, via: 1(node1-kvm), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CreateObjectCommand":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"volume":{"uuid":"e2671633-076c-4d3e-a139-ed3807159312","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e7a222aa-a542-300f-a7bb-7fb8b24c4bde","id":1,"poolType":"NetworkFilesystem","host":"10.6.26.21","path":"/export/primary","port":2049,"url":"NetworkFilesystem://10.6.26.21/export/primary/?ROLE=Primary&STOREUUID=e7a222aa-a542-300f-a7bb-7fb8b24c4bde"}},"name":"ROOT-7","size":53687091200,"path":"e2671633-076c-4d3e-a139-ed3807159312","volumeId":6,"vmName":"i-2-7-VM","accountId":2,"format":"QCOW2","id":6,"deviceId":0,"hypervisorType":"KVM"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e7a222aa-a542-300f-a7bb-7fb8b24c4bde","id":1,"poolType":"NetworkFilesystem","host":"10.6.26.21","path":"/export/primary","port":2049,"url":"NetworkFilesystem://10.6.26.21/export/primary/?ROLE=Primary&STOREUUID=e7a222aa-a542-300f-a7bb-7fb8b24c4bde"}},"vmName":"i-2-7-VM","name":"test03_ROOT-7_20141105012419","hypervisorType":"KVM","id":6,"quiescevm":false,"physicalSize":0}},"wait":0}}] }
2014-11-05 09:24:22,858 DEBUG [c.c.a.ApiServlet] (catalina-exec-23:ctx-2f001f7c) ===START=== 192.168.9.76 -- GET command=queryAsyncJobResult&jobId=b2479cee-d5b5-464a-bae4-6974027f0a67&response=json&sessionkey=tMEAJPWMmcthAjo86aaTwBSDozQ%3D&_=1415150657767
2014-11-05 09:24:22,872 DEBUG [c.c.a.ApiServlet] (catalina-exec-23:ctx-2f001f7c ctx-9fe6010e) ===END=== 192.168.9.76 -- GET command=queryAsyncJobResult&jobId=b2479cee-d5b5-464a-bae4-6974027f0a67&response=json&sessionkey=tMEAJPWMmcthAjo86aaTwBSDozQ%3D&_=1415150657767
2014-11-05 09:24:23,750 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-10:null) SeqA 2-19036: Processing Seq 2-19036: { Cmd , MgmtId: -1, via: 2, Ver: v1, Flags: 11, [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":2,"_loadInfo":"{\n \"connections\": [\n {\n \"id\": 8,\n \"clientInfo\": \"\",\n \"host\": \"10.6.31.4\",\n \"port\": 5905,\n \"tag\": \"a53e9399-e0e5-4df0-b361-1da2380bceef\",\n \"createTime\": 1415150339521,\n \"lastUsedTime\": 1415150583469\n },\n {\n \"id\": 7,\n \"clientInfo\": \"\",\n \"host\": \"10.6.31.4\",\n \"port\": 5904,\n \"tag\": \"93e383ff-3143-4830-b3e5-19d6bd248eaa\",\n \"createTime\": 1415150282559,\n \"lastUsedTime\": 1415150580763\n }\n ]\n}","wait":0}}] }
2014-11-05 09:24:23,754 DEBUG [c.c.a.m.AgentManagerImpl] (AgentManager-Handler-10:null) SeqA 2-19036: Sending Seq 2-19036: { Ans: , MgmtId: 52238016467, via: 2, Ver: v1, Flags: 100010, [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2014-11-05 09:24:25,851 DEBUG [c.c.a.ApiServlet] (catalina-exec-13:ctx-8d3d7287) ===START=== 192.168.9.76 -- GET command=queryAsyncJobResult&jobId=b2479cee-d5b5-464a-bae4-6974027f0a67&response=json&sessionkey=tMEAJPWMmcthAjo86aaTwBSDozQ%3D&_=1415150660759
2014-11-05 09:24:25,869 DEBUG [c.c.a.ApiServlet] (catalina-exec-13:ctx-8d3d7287 ctx-f46c1c3d) ===END=== 192.168.9.76 -- GET command=queryAsyncJobResult&jobId=b2479cee-d5b5-464a-bae4-6974027f0a67&response=json&sessionkey=tMEAJPWMmcthAjo86aaTwBSDozQ%3D&_=1415150660759
2014-11-05 09:24:27,606 DEBUG [c.c.a.t.Request] (AgentManager-Handler-7:null) Seq 1-5995979953890398722: Processing: { Ans: , MgmtId: 52238016467, via: 1, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CreateObjectAnswer":{"data":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"/mnt/e7a222aa-a542-300f-a7bb-7fb8b24c4bde/e2671633-076c-4d3e-a139-ed3807159312/00eb9935-f0a3-4af1-aa92-a39bcb8c060f","id":0,"quiescevm":false,"physicalSize":0}},"result":true,"wait":0}}] }
2014-11-05 09:24:27,607 DEBUG [c.c.a.t.Request] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Seq 1-5995979953890398722: Received: { Ans: , MgmtId: 52238016467, via: 1, Ver: v1, Flags: 10, { CreateObjectAnswer } }
2014-11-05 09:24:27,640 DEBUG [o.a.c.s.m.AncientDataMotionStrategy] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) copyAsync inspecting src type SNAPSHOT copyAsync inspecting dest type SNAPSHOT
2014-11-05 09:24:27,675 DEBUG [c.c.a.t.Request] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Seq 1-5995979953890398723: Sending { Cmd , MgmtId: 52238016467, via: 1(node1-kvm), Ver: v1, Flags: 100011, [{"org.apache.cloudstack.storage.command.CopyCommand":{"srcTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"/mnt/e7a222aa-a542-300f-a7bb-7fb8b24c4bde/e2671633-076c-4d3e-a139-ed3807159312/00eb9935-f0a3-4af1-aa92-a39bcb8c060f","volume":{"uuid":"e2671633-076c-4d3e-a139-ed3807159312","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e7a222aa-a542-300f-a7bb-7fb8b24c4bde","id":1,"poolType":"NetworkFilesystem","host":"10.6.26.21","path":"/export/primary","port":2049,"url":"NetworkFilesystem://10.6.26.21/export/primary/?ROLE=Primary&STOREUUID=e7a222aa-a542-300f-a7bb-7fb8b24c4bde"}},"name":"ROOT-7","size":53687091200,"path":"e2671633-076c-4d3e-a139-ed3807159312","volumeId":6,"vmName":"i-2-7-VM","accountId":2,"format":"QCOW2","id":6,"deviceId":0,"hypervisorType":"KVM"},"dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e7a222aa-a542-300f-a7bb-7fb8b24c4bde","id":1,"poolType":"NetworkFilesystem","host":"10.6.26.21","path":"/export/primary","port":2049,"url":"NetworkFilesystem://10.6.26.21/export/primary/?ROLE=Primary&STOREUUID=e7a222aa-a542-300f-a7bb-7fb8b24c4bde"}},"vmName":"i-2-7-VM","name":"test03_ROOT-7_20141105012419","hypervisorType":"KVM","id":6,"quiescevm":false,"physicalSize":0}},"destTO":{"org.apache.cloudstack.storage.to.SnapshotObjectTO":{"path":"snapshots/2/6","volume":{"uuid":"e2671633-076c-4d3e-a139-ed3807159312","volumeType":"ROOT","dataStore":{"org.apache.cloudstack.storage.to.PrimaryDataStoreTO":{"uuid":"e7a222aa-a542-300f-a7bb-7fb8b24c4bde","id":1,"poolType":"NetworkFilesystem","host":"10.6.26.21","path":"/export/primary","port":2049,"url":"NetworkFilesystem://10.6.26.21/export/primary/?ROLE=Primary&STOREUUID=e7a222aa-a542-300f-a7bb-7fb8b24c4bde"}},"name":"ROOT-7","size":53687091200,"path":"e2671633-076c-4d3e-a139-ed3807159312","volumeId":6,"vmName":"i-2-7-VM","accountId":2,"format":"QCOW2","id":6,"deviceId":0,"hypervisorType":"KVM"},"dataStore":{"com.cloud.agent.api.to.NfsTO":{"_url":"nfs://10.6.26.21/export/secondary","_role":"Image"}},"vmName":"i-2-7-VM","name":"test03_ROOT-7_20141105012419","hypervisorType":"KVM","id":6,"quiescevm":false,"physicalSize":0}},"executeInSequence":false,"options":{"fullSnapshot":"true"},"wait":21600}}] }
2014-11-05 09:24:28,034 DEBUG [c.c.a.t.Request] (AgentManager-Handler-9:null) Seq 1-5995979953890398723: Processing: { Ans: , MgmtId: 52238016467, via: 1, Ver: v1, Flags: 10, [{"org.apache.cloudstack.storage.command.CopyCmdAnswer":{"result":false,"details":"Failed to backup 00eb9935-f0a3-4af1-aa92-a39bcb8c060f for disk /mnt/e7a222aa-a542-300f-a7bb-7fb8b24c4bde/e2671633-076c-4d3e-a139-ed3807159312 to /mnt/cb13ca02-b0f4-371a-a676-768194da25f9/snapshots/2/6","wait":0}}] }
2014-11-05 09:24:28,034 DEBUG [c.c.a.t.Request] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Seq 1-5995979953890398723: Received: { Ans: , MgmtId: 52238016467, via: 1, Ver: v1, Flags: 10, { CopyCmdAnswer } }
2014-11-05 09:24:28,049 DEBUG [c.c.s.s.SnapshotManagerImpl] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Failed to create snapshot
com.cloud.utils.exception.CloudRuntimeException: Failed to backup 00eb9935-f0a3-4af1-aa92-a39bcb8c060f for disk /mnt/e7a222aa-a542-300f-a7bb-7fb8b24c4bde/e2671633-076c-4d3e-a139-ed3807159312 to /mnt/cb13ca02-b0f4-371a-a676-768194da25f9/snapshots/2/6
at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:282)
at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:137)
at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:300)
at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:966)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy179.takeSnapshot(Unknown Source)
at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1503)
at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:1766)
at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:2503)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at com.cloud.storage.VolumeApiServiceImpl.handleVmWorkJob(VolumeApiServiceImpl.java:2511)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy183.handleVmWorkJob(Unknown Source)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-11-05 09:24:28,058 DEBUG [o.a.c.s.v.VolumeServiceImpl] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Take snapshot: 6 failed
com.cloud.utils.exception.CloudRuntimeException: Failed to create snapshot
at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:983)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy179.takeSnapshot(Unknown Source)
at org.apache.cloudstack.storage.volume.VolumeServiceImpl.takeSnapshot(VolumeServiceImpl.java:1503)
at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:1766)
at com.cloud.storage.VolumeApiServiceImpl.orchestrateTakeVolumeSnapshot(VolumeApiServiceImpl.java:2503)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at com.cloud.storage.VolumeApiServiceImpl.handleVmWorkJob(VolumeApiServiceImpl.java:2511)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy183.handleVmWorkJob(Unknown Source)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.cloud.utils.exception.CloudRuntimeException: Failed to backup 00eb9935-f0a3-4af1-aa92-a39bcb8c060f for disk /mnt/e7a222aa-a542-300f-a7bb-7fb8b24c4bde/e2671633-076c-4d3e-a139-ed3807159312 to /mnt/cb13ca02-b0f4-371a-a676-768194da25f9/snapshots/2/6
at org.apache.cloudstack.storage.snapshot.SnapshotServiceImpl.backupSnapshot(SnapshotServiceImpl.java:282)
at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.backupSnapshot(XenserverSnapshotStrategy.java:137)
at org.apache.cloudstack.storage.snapshot.XenserverSnapshotStrategy.takeSnapshot(XenserverSnapshotStrategy.java:300)
at com.cloud.storage.snapshot.SnapshotManagerImpl.takeSnapshot(SnapshotManagerImpl.java:966)
... 44 more
2014-11-05 09:24:28,058 DEBUG [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Done executing VM work job: com.cloud.storage.VmWorkTakeVolumeSnapshot{"volumeId":6,"policyId":0,"snapshotId":6,"quiesceVm":false,"userId":2,"accountId":2,"vmId":7,"handlerName":"VolumeApiServiceImpl"}
2014-11-05 09:24:28,058 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73 ctx-b877aaf9) Complete async job-73, jobStatus: SUCCEEDED, resultCode: 0, result: rO0ABXNyAA5qYXZhLmxhbmcuTG9uZzuL5JDMjyPfAgABSgAFdmFsdWV4cgAQamF2YS5sYW5nLk51bWJlcoaslR0LlOCLAgAAeHAAAAAAAAAABg
2014-11-05 09:24:28,063 DEBUG [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Done with run of VM work job: com.cloud.storage.VmWorkTakeVolumeSnapshot for VM 7, job origin: 72
2014-11-05 09:24:28,063 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Done executing com.cloud.storage.VmWorkTakeVolumeSnapshot for job-73
2014-11-05 09:24:28,070 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Sync queue (11) is currently empty
2014-11-05 09:24:28,071 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-8:ctx-8fc5eb5a job-72/job-73) Remove job-73 from job monitoring