[jira] [Commented] (CLOUDSTACK-7864) CPVM continues to be in "STOPPED' state after failure to start because of a management server restart.

["ASF subversion and git services (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/CLOUDSTACK-7864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14203030#comment-14203030 ] 

ASF subversion and git services commented on CLOUDSTACK-7864:
-------------------------------------------------------------

Commit a275bd7ca4191f19defcfe106229a1bdc33c6d99 in cloudstack's branch refs/heads/master from [~minchen07]
[ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=a275bd7 ]

CLOUDSTACK-7864: CPVM continues to be in Stopped state after a failure to start because of a management server restart.


> CPVM continues to be in "STOPPED' state after failure to start because of a management server restart.
> ------------------------------------------------------------------------------------------------------
>
>                 Key: CLOUDSTACK-7864
>                 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7864
>             Project: CloudStack
>          Issue Type: Bug
>      Security Level: Public(Anyone can view this level - this is the default.) 
>          Components: Management Server
>    Affects Versions: 4.5.0
>            Reporter: Min Chen
>            Assignee: Min Chen
>            Priority: Critical
>             Fix For: 4.5.0
>
>
> CPVM continues to be in "STOPPED' state after failure to start because of a management server restart.
> Setup:
> Advanced zone set up with KVM hosts.
> Steps that led to the problem:
> As soon as the zone was set up and enabled , management server was restarted.
> I see that the CPVM failed to come up because there were no hosts at the time it was attempting to start.
> After this , even manually trying to start the CPVM does not succeed and the jobs are hanging.
> 2014-11-04 21:29:02,613 ERROR [c.c.v.VmWorkJobDispatcher] (Work-Job-Executor-1:ctx-5e9c6a9d job-3/job-16) Unable to complete AsyncJobVO
> {id:16, userId: 1, accountId: 1, instanceType: null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo: rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIACkoABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAAAAAAAAAABAAAAAAAAAAEAAAAAAAAAAXQAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAAAAAAAAAHBwcHBwcHBwcA, cmdVersion: 0, status: FAILED, processStatus: 0, resultCode: 530, result: job cancelled because of management server restart or shutdown, initMsid: 235397185403991, completeMsid: null, lastUpdated: null, lastPolled: null, created: Tue Nov 04 21:28:01 UTC 2014}
> , job origin:3
> com.cloud.exception.InsufficientServerCapacityException: Unable to create a deployment for VM[ConsoleProxy|v-1-VM]Scope=interface com.cloud.dc.DataCenter; id=1
> at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:932)
> at com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:5012)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
> at com.cloud.vm.VirtualMachineManagerImpl.handleVmWorkJob(VirtualMachineManagerImpl.java:5156)
> at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:101)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:540)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:50)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:47)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:497)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:679)
> 2014-11-04 21:29:02,620 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-1:ctx-5e9c6a9d job-3/job-16) Complete async job-16, jobStatus: FAILED, resultCode: 0, result: rO0ABXNyABpqYXZhLmxhbmcuUnVudGltZUV4Y2VwdGlvbp5fBkcKNIPlAgAAeHIAE2phdmEubGFuZy5FeGNlcHRpb27Q_R8-GjscxAIAAHhyABNqYXZhLmxhbmcuVGhyb3dhYmxl1cY1Jzl3uMsDAANMAAVjYXVzZXQAFUxqYXZhL2xhbmcvVGhyb3dhYmxlO0wADWRldGFpbE1lc3NhZ2V0ABJMamF2YS9sYW5nL1N0cmluZztbAApzdGFja1RyYWNldAAeW0xqYXZhL2xhbmcvU3RhY2tUcmFjZUVsZW1lbnQ7eHBxAH4ABnQAVUpvYiBmYWlsZWQgZHVlIHRvIGV4Y2VwdGlvbiBVbmFibGUgdG8gY3JlYXRlIGEgZGVwbG95bWVudCBmb3IgVk1bQ29uc29sZVByb3h5fHYtMS1WTV11cgAeW0xqYXZhLmxhbmcuU3RhY2tUcmFjZUVsZW1lbnQ7AkYqPDz9IjkCAAB4cAAAAA5zcgAbamF2YS5sYW5nLlN0YWNrVHJhY2VFbGVtZW50YQnFmiY23YUCAARJAApsaW5lTnVtYmVyTAAOZGVjbGFyaW5nQ2xhc3NxAH4ABEwACGZpbGVOYW1lcQB-AARMAAptZXRob2ROYW1lcQB-AAR4cAAAAHF0ACBjb20uY2xvdWQudm0uVm1Xb3JrSm9iRGlzcGF0Y2hlcnQAGFZtV29ya0pvYkRpc3BhdGNoZXIuamF2YXQABnJ1bkpvYnNxAH4ACgAAAhx0AD9vcmcuYXBhY2hlLmNsb3Vkc3RhY2suZnJhbWV3b3JrLmpvYnMuaW1wbC5Bc3luY0pvYk1hbmFnZXJJbXBsJDV0ABhBc3luY0pvYk1hbmFnZXJJbXBsLmphdmF0AAxydW5JbkNvbnRleHRzcQB-AAoAAAAydAA-b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlJDF0ABtNYW5hZ2VkQ29udGV4dFJ1bm5hYmxlLmphdmF0AANydW5zcQB-AAoAAAA4dABCb3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5pbXBsLkRlZmF1bHRNYW5hZ2VkQ29udGV4dCQxdAAaRGVmYXVsdE1hbmFnZWRDb250ZXh0LmphdmF0AARjYWxsc3EAfgAKAAAAZ3QAQG9yZy5hcGFjaGUuY2xvdWRzdGFjay5tYW5hZ2VkLmNvbnRleHQuaW1wbC5EZWZhdWx0TWFuYWdlZENvbnRleHRxAH4AGXQAD2NhbGxXaXRoQ29udGV4dHNxAH4ACgAAADVxAH4AHHEAfgAZdAAOcnVuV2l0aENvbnRleHRzcQB-AAoAAAAvdAA8b3JnLmFwYWNoZS5jbG91ZHN0YWNrLm1hbmFnZWQuY29udGV4dC5NYW5hZ2VkQ29udGV4dFJ1bm5hYmxlcQB-ABVxAH4AFnNxAH4ACgAAAfFxAH4AEHEAfgARcQB-ABZzcQB-AAoAAAHXdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuRXhlY3V0b3JzJFJ1bm5hYmxlQWRhcHRlcnQADkV4ZWN1dG9ycy5qYXZhcQB-ABpzcQB-AAoAAAFOdAAkamF2YS51dGlsLmNvbmN1cnJlbnQuRnV0dXJlVGFzayRTeW5jdAAPRnV0dXJlVGFzay5qYXZhdAAIaW5uZXJSdW5zcQB-AAoAAACmdAAfamF2YS51dGlsLmNvbmN1cnJlbnQuRnV0dXJlVGFza3EAfgAocQB-ABZzcQB-AAoAAARWdAAnamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9ydAAXVGhyZWFkUG9vbEV4ZWN1dG9yLmphdmF0AAlydW5Xb3JrZXJzcQB-AAoAAAJbdAAuamF2YS51dGlsLmNvbmN1cnJlbnQuVGhyZWFkUG9vbEV4ZWN1dG9yJFdvcmtlcnEAfgAucQB-ABZzcQB-AAoAAAKndAAQamF2YS5sYW5nLlRocmVhZHQAC1RocmVhZC5qYXZhcQB-ABZ4
> 2014-11-04 21:29:02,636 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-1:ctx-5e9c6a9d job-3/job-16) job-16 is already completed.
> 2014-11-04 21:29:02,636 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (Work-Job-Executor-1:ctx-5e9c6a9d job-3/job-16) Done executing com.cloud.vm.VmWorkStart for job-16
> 2014-11-04 21:29:02,650 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (Work-Job-Executor-1:ctx-5e9c6a9d job-3/job-16) Queue (queue id, sync type, sync id) - (1,VmWorkJobQueue, 1) is reaching concurrency limit 1
> 2014-11-04 21:29:02,651 DEBUG [o.a.c.f.j.i.SyncQueueManagerImpl] (Work-Job-Executor-1:ctx-5e9c6a9d job-3/job-16) There is a pending process in sync queue(id: 1)
> 2014-11-04 21:29:02,651 INFO [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-1:ctx-5e9c6a9d job-3/job-16) Remove job-16 from job monitoring



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)