You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by J on 2017/04/21 06:19:47 UTC
Unable to start a VM due to insufficient capacity
Hi!
Just like that it is impossible to start any vm. ACS4.8, Xenservers 6.5
Unable to start a VM due to insufficient capacity. I can see in
xencenter it is trying to start VM on all hosts but fails. Also there is
enough resources on hosts.
Maybe its an XenServer issue, because it is also unable to stop VMS.
In log when starting VM:
2017-04-21 08:50:35,806 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Task failed! Task
record: uuid: a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI,
OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2017-04-21 08:50:35,818 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to start
VM(i-95-292-VM) on host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI,
OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI,
OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:461)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:4807)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:126)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,826 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) unable to destroy
task(com.xensource.xenapi.Task@27d940e) on
host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to You gave an invalid
object reference. The object may have recently been deleted. The class
parameter gives the type of reference given, and the handle parameter
echoes the bad value given.
2017-04-21 08:50:35,826 WARN [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Catch Exception: class
com.cloud.utils.exception.CloudRuntimeException due to
com.cloud.utils.exception.CloudRuntimeException: Unable to start
VM(i-95-292-VM) on host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI,
OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
com.cloud.utils.exception.CloudRuntimeException: Unable to start
VM(i-95-292-VM) on host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI,
OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:4826)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:126)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,828 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to start
i-95-292-VM due to
com.cloud.utils.exception.CloudRuntimeException: Unable to start
VM(i-95-292-VM) on host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI,
OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:4826)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:126)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,889 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to clean up VBD due to
You gave an invalid object reference. The object may have recently been
deleted. The class parameter gives the type of reference given, and the
handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:693)
at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
at
com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3445)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:208)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,901 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to clean up VBD due to
You gave an invalid object reference. The object may have recently been
deleted. The class parameter gives the type of reference given, and the
handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:693)
at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
at
com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3445)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:208)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,913 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to cleanup VIF
You gave an invalid object reference. The object may have recently been
deleted. The class parameter gives the type of reference given, and the
handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:693)
at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
at
com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
at com.xensource.xenapi.VIF.unplug(VIF.java:921)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3453)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:208)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:36,091 DEBUG [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) The VM is in stopped
state, detected problem during startup : i-95-292-VM
--
J Andersons
Re: Unable to start a VM due to insufficient capacity
Posted by Yiping Zhang <yz...@marketo.com>.
When you say there is enough resources on hosts, exactly what’s memory and cpu allocation on each of the hosts? Unless you have updated following global settings, the default settings are to disable VM allocation if cluster’s overall CPU or memory allocation is over 75%, and you will get insufficient resources error when you creating new instances.
cluster.cpu.allocated.capacity.disablethreshold = 0.75
cluster.memory.allocated.capacity.disablethreshold = 0.75
There are other situations where you could also receive this error, such as if you specified wrong network or the tagged resources are not available (storage, host, etc, or wrong tag was specified) and so on. This error message is pretty much useless as it is given, not to cloud operators or administrators anyway.
Yiping
On 4/21/17, 4:21 PM, "Marty Godsey" <ma...@gonsource.com> wrote:
Without looking too deep into this it seems that there maybe a stuck task trying to snapshot the VHD.
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
Again, based on what you sent, I would look at any hung xapi tasks on the host in question.
Regards,
Marty Godsey
nSource Solutions
-----Original Message-----
From: J Andersons
Sent: Friday, April 21, 2017 2:20 AM
To: users@cloudstack.apache.org
Subject: Unable to start a VM due to insufficient capacity
Hi!
Just like that it is impossible to start any vm. ACS4.8, Xenservers 6.5 Unable to start a VM due to insufficient capacity. I can see in xencenter it is trying to start VM on all hosts but fails. Also there is enough resources on hosts.
Maybe its an XenServer issue, because it is also unable to stop VMS.
In log when starting VM:
2017-04-21 08:50:35,806 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Task failed! Task
record: uuid: a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2017-04-21 08:50:35,818 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to start
VM(i-95-292-VM) on host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:461)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:4807)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:126)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,826 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) unable to destroy
task(com.xensource.xenapi.Task@27d940e) on
host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
2017-04-21 08:50:35,826 WARN [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Catch Exception: class com.cloud.utils.exception.CloudRuntimeException due to
com.cloud.utils.exception.CloudRuntimeException: Unable to start
VM(i-95-292-VM) on host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
com.cloud.utils.exception.CloudRuntimeException: Unable to start
VM(i-95-292-VM) on host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:4826)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:126)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,828 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to start i-95-292-VM due to
com.cloud.utils.exception.CloudRuntimeException: Unable to start
VM(i-95-292-VM) on host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:4826)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:126)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,889 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to clean up VBD due to You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:693)
at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
at
com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3445)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:208)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,901 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to clean up VBD due to You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:693)
at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
at
com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3445)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:208)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,913 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to cleanup VIF You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:693)
at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
at
com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
at com.xensource.xenapi.VIF.unplug(VIF.java:921)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3453)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:208)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:36,091 DEBUG [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) The VM is in stopped state, detected problem during startup : i-95-292-VM
--
J Andersons
RE: Unable to start a VM due to insufficient capacity
Posted by Marty Godsey <ma...@gonsource.com>.
Without looking too deep into this it seems that there maybe a stuck task trying to snapshot the VHD.
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
Again, based on what you sent, I would look at any hung xapi tasks on the host in question.
Regards,
Marty Godsey
nSource Solutions
-----Original Message-----
From: J Andersons
Sent: Friday, April 21, 2017 2:20 AM
To: users@cloudstack.apache.org
Subject: Unable to start a VM due to insufficient capacity
Hi!
Just like that it is impossible to start any vm. ACS4.8, Xenservers 6.5 Unable to start a VM due to insufficient capacity. I can see in xencenter it is trying to start VM on all hosts but fails. Also there is enough resources on hosts.
Maybe its an XenServer issue, because it is also unable to stop VMS.
In log when starting VM:
2017-04-21 08:50:35,806 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Task failed! Task
record: uuid: a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
2017-04-21 08:50:35,818 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to start
VM(i-95-292-VM) on host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.checkForSuccess(CitrixResourceBase.java:461)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:4807)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:126)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,826 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) unable to destroy
task(com.xensource.xenapi.Task@27d940e) on
host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
2017-04-21 08:50:35,826 WARN [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Catch Exception: class com.cloud.utils.exception.CloudRuntimeException due to
com.cloud.utils.exception.CloudRuntimeException: Unable to start
VM(i-95-292-VM) on host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
com.cloud.utils.exception.CloudRuntimeException: Unable to start
VM(i-95-292-VM) on host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:4826)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:126)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,828 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to start i-95-292-VM due to
com.cloud.utils.exception.CloudRuntimeException: Unable to start
VM(i-95-292-VM) on host(a6601054-b730-460a-b654-a29f1a5b8a1c) due to
Task failed! Task record: uuid:
a9ac4c12-a6ba-6d9a-556f-b5d88f8cd2d3
nameLabel: Async.VM.start_on
nameDescription:
allowedOperations: []
currentOperations: {}
created: Fri Apr 21 08:48:34 EEST 2017
finished: Fri Apr 21 08:50:35 EEST 2017
status: failure
residentOn: com.xensource.xenapi.Host@c07df7eb
progress: 1.0
type: <none/>
result:
errorInfo: [OTHER_OPERATION_IN_PROGRESS, VDI, OpaqueRef:1d681250-d950-81ec-9f87-f7eb3d8b0cc1, snapshot]
otherConfig: {}
subtaskOf: com.xensource.xenapi.Task@aaf13f6f
subtasks: []
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.startVM(CitrixResourceBase.java:4826)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:126)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,889 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to clean up VBD due to You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:693)
at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
at
com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3445)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:208)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,901 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to clean up VBD due to You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:693)
at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
at
com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3445)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:208)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:35,913 WARN [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) Unable to cleanup VIF You gave an invalid object reference. The object may have recently been deleted. The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
at com.xensource.xenapi.Types.checkResponse(Types.java:693)
at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
at
com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
at com.xensource.xenapi.VIF.unplug(VIF.java:921)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3453)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:208)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
at
com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
at
com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1677)
at
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2017-04-21 08:50:36,091 DEBUG [c.c.h.x.r.w.x.CitrixStartCommandWrapper]
(DirectAgent-28:ctx-6bbd1566) (logid:91c36627) The VM is in stopped state, detected problem during startup : i-95-292-VM
--
J Andersons