You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@cloudstack.apache.org by "Rajani Karuturi (JIRA)" <ji...@apache.org> on 2017/03/08 10:47:38 UTC

[jira] [Assigned] (CLOUDSTACK-9821) Unable to deploy user VM in Basic Zone

     [ https://issues.apache.org/jira/browse/CLOUDSTACK-9821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rajani Karuturi reassigned CLOUDSTACK-9821:
-------------------------------------------

    Assignee: Jayapal Reddy

> Unable to deploy user VM in Basic Zone
> --------------------------------------
>
>                 Key: CLOUDSTACK-9821
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9821
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Network Controller
>    Affects Versions: 4.10.0.0
>         Environment: Ubuntu 12.04 for CS MS
>            Reporter: Mike Tutkowski
>            Assignee: Jayapal Reddy
>            Priority: Blocker
>             Fix For: 4.10.0.0
>
>
> From my e-mails on dev@ concerning this issue (messages listed oldest to newest):
> I see the following exception when trying to deploy a user VM in a Basic Zone with two XenServer 6.5 hosts in one cluster. My system VMs have all deployed properly. The user template gets downloaded fine. I can see the user VM begin to start on a XenServer host, then it goes away. We then automatically try on the other host. I can see the VM begin to start there for a moment, then it goes away.
> I am just deploying the user VM’s template and root disk to NFS (same place where the template and root disks of my system VMs are).
> I am using the built-in XenServer CentOS 5.6 (64 bit) template with 1 vCPU, 500 MHz, and 256 MB memory.
> WARN  [c.c.a.r.v.VirtualRoutingResource] (DirectAgent-7:ctx-35aded78) (logid:aab9c320) Expected 1 answers while executing VmDataCommand but received 2
> WARN  [c.c.v.VirtualMachinePowerStateSyncImpl] (DirectAgentCronJob-14:ctx-27fb1ac3) (logid:2c342f23) VM state was updated but update time is null?! vm id: 6
> INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce) (logid:a56a9a8c) Begin cleanup expired async-jobs
> INFO  [o.a.c.f.j.i.AsyncJobManagerImpl] (AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce) (logid:a56a9a8c) End cleanup expired async-jobs
> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c) (logid:541e9ba5) Found 0 removed accounts to cleanup
> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c) (logid:541e9ba5) Found 0 disabled accounts to cleanup
> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c) (logid:541e9ba5) Found 0 inactive domains to cleanup
> INFO  [c.c.u.AccountManagerImpl] (AccountChecker-1:ctx-383a632c) (logid:541e9ba5) Found 0 disabled projects to cleanup
> WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-16:ctx-7c901443) (logid:aab9c320) callHostPlugin failed for cmd: default_network_rules with args secIps: 0:, vmName: i-2-6-VM, vmID: 6, vmIP: 10.117.40.53, vmMAC: 06:b2:f4:00:00:22,  due to There was a failure communicating with the plugin.
> WARN  [c.c.h.x.r.w.x.CitrixStartCommandWrapper] (DirectAgent-16:ctx-7c901443) (logid:aab9c320) Catch Exception: class com.cloud.utils.exception.CloudRuntimeException due to com.cloud.utils.exception.CloudRuntimeException: callHostPlugin failed for cmd: default_network_rules with args secIps: 0:, vmName: i-2-6-VM, vmID: 6, vmIP: 10.117.40.53, vmMAC: 06:b2:f4:00:00:22,  due to There was a failure communicating with the plugin.
> com.cloud.utils.exception.CloudRuntimeException: callHostPlugin failed for cmd: default_network_rules with args secIps: 0:, vmName: i-2-6-VM, vmID: 6, vmIP: 10.117.40.53, vmMAC: 06:b2:f4:00:00:22,  due to There was a failure communicating with the plugin.
>     at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.callHostPlugin(CitrixResourceBase.java:338)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:188)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
>     at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1691)
>     at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at java.lang.Thread.run(Thread.java:745)
> WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-16:ctx-7c901443) (logid:aab9c320) Unable to start i-2-6-VM due to
> com.cloud.utils.exception.CloudRuntimeException: callHostPlugin failed for cmd: default_network_rules with args secIps: 0:, vmName: i-2-6-VM, vmID: 6, vmIP: 10.117.40.53, vmMAC: 06:b2:f4:00:00:22,  due to There was a failure communicating with the plugin.
>     at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.callHostPlugin(CitrixResourceBase.java:338)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:188)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
>     at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1691)
>     at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at java.lang.Thread.run(Thread.java:745)
> INFO  [o.a.c.e.o.NetworkOrchestrator] (Network-Scavenger-1:ctx-2058d5ac) (logid:bf8885a0) NetworkGarbageCollector uses '20' seconds for GC interval.
> WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-16:ctx-7c901443) (logid:aab9c320) Unable to clean up VBD due to
> You gave an invalid object reference.  The object may have recently been deleted.  The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
>     at com.xensource.xenapi.Types.checkResponse(Types.java:693)
>     at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
>     at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
>     at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
>     at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3576)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:210)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
>     at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1691)
>     at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at java.lang.Thread.run(Thread.java:745)
> WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-16:ctx-7c901443) (logid:aab9c320) Unable to clean up VBD due to
> You gave an invalid object reference.  The object may have recently been deleted.  The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
>     at com.xensource.xenapi.Types.checkResponse(Types.java:693)
>     at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
>     at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
>     at com.xensource.xenapi.VBD.unplug(VBD.java:1109)
>     at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3576)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:210)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
>     at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1691)
>     at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at java.lang.Thread.run(Thread.java:745)
> WARN  [c.c.h.x.r.CitrixResourceBase] (DirectAgent-16:ctx-7c901443) (logid:aab9c320) Unable to cleanup VIF
> You gave an invalid object reference.  The object may have recently been deleted.  The class parameter gives the type of reference given, and the handle parameter echoes the bad value given.
>     at com.xensource.xenapi.Types.checkResponse(Types.java:693)
>     at com.xensource.xenapi.Connection.dispatch(Connection.java:395)
>     at com.cloud.hypervisor.xenserver.resource.XenServerConnectionPool$XenServerConnection.dispatch(XenServerConnectionPool.java:457)
>     at com.xensource.xenapi.VIF.unplug(VIF.java:921)
>     at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.handleVmStartFailure(CitrixResourceBase.java:3584)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:210)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixStartCommandWrapper.execute(CitrixStartCommandWrapper.java:53)
>     at com.cloud.hypervisor.xenserver.resource.wrapper.xenbase.CitrixRequestWrapper.execute(CitrixRequestWrapper.java:122)
>     at com.cloud.hypervisor.xenserver.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:1691)
>     at com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:315)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
>     at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at java.lang.Thread.run(Thread.java:745)
> INFO  [c.c.v.VirtualMachineManagerImpl] (Work-Job-Executor-2:ctx-bc104380 job-25/job-27 ctx-8c946a5b) (logid:aab9c320) Unable to start VM on Host[-2-Routing] due to Unable to start i-2-6-VM due to
> ERROR [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-2:ctx-bc104380 job-25/job-27 ctx-8c946a5b) (logid:aab9c320) Invocation exception, caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-6-VM]Scope=interface com.cloud.dc.DataCenter; id=1
> INFO  [c.c.v.VmWorkJobHandlerProxy] (Work-Job-Executor-2:ctx-bc104380 job-25/job-27 ctx-8c946a5b) (logid:aab9c320) Rethrow exception com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-6-VM]Scope=interface com.cloud.dc.DataCenter; id=1
> ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-2:ctx-bc104380 job-25/job-27) (logid:aab9c320) Unable to complete AsyncJobVO {id:27, userId: 2, accountId: 2, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAACAAAAAAAAAAIAAAAAAAAABnQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwc3IAEWphdmEudXRpbC5IYXNoTWFwBQfawcMWYNEDAAJGAApsb2FkRmFjdG9ySQAJdGhyZXNob2xkeHA_QAAAAAAADHcIAAAAEAAAAAF0AApWbVBhc3N3b3JkdAAcck8wQUJYUUFEbk5oZG1Wa1gzQmhjM04zYjNKa3hw, cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: null, initMsid: 52237617797, completeMsid: null, lastUpdated: null, lastPolled: null, created: Wed Mar 01 12:51:32 MST 2017}, job origin:25
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-6-VM]Scope=interface com.cloud.dc.DataCenter; id=1
>     at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:961)
>     at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4661)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:498)
>     at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
>     at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:4822)
>     at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
>     at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>     at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at java.lang.Thread.run(Thread.java:745)
> INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-2:ctx-bc104380 job-25/job-27) (logid:aab9c320) Remove job-27 from job monitoring
> WARN  [o.a.c.alerts] (API-Job-Executor-1:ctx-f787201d job-25 ctx-56356c1a) (logid:aab9c320)  alertType:: 8 // dataCenterId:: 1 // podId:: 1 // clusterId:: null // message:: Failed to deploy Vm with Id: 6, on Host with Id: null
> ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-1:ctx-f787201d job-25) (logid:aab9c320) Unexpected exception while executing org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin
> com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM due to insufficient capacity
>     at com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:623)
>     at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:242)
>     at org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:212)
>     at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:4084)
>     at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3682)
>     at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3670)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.lang.reflect.Method.invoke(Method.java:498)
>     at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
>     at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
>     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
>     at org.apache.cloudstack.network.contrail.management.EventUtils$EventInterceptor.invoke(EventUtils.java:107)
>     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168)
>     at com.cloud.event.ActionEventInterceptor.invoke(ActionEventInterceptor.java:51)
>     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168)
>     at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
>     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
>     at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
>     at com.sun.proxy.$Proxy186.startVirtualMachine(Unknown Source)
>     at org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin.execute(DeployVMCmdByAdmin.java:50)
>     at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
>     at com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
>     at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:554)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>     at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>     at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
>     at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
>     at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>     at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at java.lang.Thread.run(Thread.java:745)
> Caused by: com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[User|i-2-6-VM]Scope=interface com.cloud.dc.DataCenter; id=1
>     at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:961)
>     at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:4661)
>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>     ... 18 more
> **********
> According to where the code fails, though, it appears to be a networking problem. If I set a breakpoint before the failure and change a variable to say that security groups are not being used, then the VM starts.
> I think this is a recently introduced problem because I have another branch based off of a slightly older version of master and it works fine here.
> **********
> I rolled back to my master branch at da66b06e7d562393da2e4b52206943f8bad49d10 and it works.
> It appears something that went into after that commit has broken this. It looks like this SHA is about two weeks old and that 43 commits have gone into master since it.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)