You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@cloudstack.apache.org by Martin Emrich <ma...@empolis.com> on 2015/01/27 16:16:35 UTC
AW: ALL Hosts Stuck in Maintenance
Hi!
Any news on this issue? I just fell in to this pit again :(
Regards,
Martin
-----Ursprüngliche Nachricht-----
Von: Martin Emrich [mailto:martin.emrich@empolis.com]
Gesendet: Dienstag, 25. November 2014 13:39
An: users@cloudstack.apache.org
Betreff: Re: ALL Hosts Stuck in Maintenance
Hi!
Same problem here with CS 4.4.1 and 5 XenServers in one Cluster: Two hosts are in maintenance mode, and canceling maintenance mode results in "Internal Server Error". Here's my backtrace:
2014-11-25 13:08:45,448 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-54:ctx-a076520e job-765) Unexpected exception while executing org.apache.cloudstack.api.command.admin.host.CancelMaintenanceCmd
java.lang.NullPointerException
at
com.cloud.resource.ResourceManagerImpl.doCancelMaintenance(ResourceManagerImpl.java:2083)
at
com.cloud.resource.ResourceManagerImpl.cancelMaintenance(ResourceManagerImpl.java:2140)
at
com.cloud.resource.ResourceManagerImpl.cancelMaintenance(ResourceManagerImpl.java:1127)
at sun.reflect.GeneratedMethodAccessor400.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy148.cancelMaintenance(Unknown Source)
at
org.apache.cloudstack.api.command.admin.host.CancelMaintenanceCmd.execute(CancelMaintenanceCmd.java:102)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
at
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-11-25 13:08:45,449 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-54:ctx-a076520e job-765) Complete async job-765,
jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530}
Ciao
Martin
AW: ALL Hosts Stuck in Maintenance
Posted by Martin Emrich <ma...@empolis.com>.
Ok thanks... We found out that the Host can be safely removed and readded to CloudStack without further disturbance, so that's currently our Way To Go ;)
This even works for XenServer pool masters.
Ciao
Martin
-----Ursprüngliche Nachricht-----
Von: Daan Hoogland [mailto:daan.hoogland@gmail.com]
Gesendet: Mittwoch, 28. Januar 2015 11:34
An: users@cloudstack.apache.org
Betreff: Re: ALL Hosts Stuck in Maintenance
Martin, My diagnosis hasn't been altered studied or challanged ;)
for a quick fix vm != null && should be added in the expression line
2083 of ResourceManagerImpl. A more robust solution must be possible but i don't have time to look into that right now.
On Tue, Jan 27, 2015 at 4:16 PM, Martin Emrich <ma...@empolis.com> wrote:
> Hi!
>
> Any news on this issue? I just fell in to this pit again :(
>
> Regards,
>
> Martin
>
> -----Ursprüngliche Nachricht-----
> Von: Martin Emrich [mailto:martin.emrich@empolis.com]
> Gesendet: Dienstag, 25. November 2014 13:39
> An: users@cloudstack.apache.org
> Betreff: Re: ALL Hosts Stuck in Maintenance
>
> Hi!
>
> Same problem here with CS 4.4.1 and 5 XenServers in one Cluster: Two hosts are in maintenance mode, and canceling maintenance mode results in "Internal Server Error". Here's my backtrace:
>
> 2014-11-25 13:08:45,448 ERROR [c.c.a.ApiAsyncJobDispatcher]
> (API-Job-Executor-54:ctx-a076520e job-765) Unexpected exception while
> executing
> org.apache.cloudstack.api.command.admin.host.CancelMaintenanceCmd
> java.lang.NullPointerException
> at
> com.cloud.resource.ResourceManagerImpl.doCancelMaintenance(ResourceManagerImpl.java:2083)
> at
> com.cloud.resource.ResourceManagerImpl.cancelMaintenance(ResourceManagerImpl.java:2140)
> at
> com.cloud.resource.ResourceManagerImpl.cancelMaintenance(ResourceManagerImpl.java:1127)
> at sun.reflect.GeneratedMethodAccessor400.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at com.sun.proxy.$Proxy148.cancelMaintenance(Unknown Source)
> at
> org.apache.cloudstack.api.command.admin.host.CancelMaintenanceCmd.execute(CancelMaintenanceCmd.java:102)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
> at
> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
> at
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2014-11-25 13:08:45,449 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-54:ctx-a076520e job-765) Complete async job-765,
> jobStatus: FAILED, resultCode: 530, result:
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":
> [],"errorcode":530}
>
>
> Ciao
>
> Martin
--
Daan
Re: ALL Hosts Stuck in Maintenance
Posted by Daan Hoogland <da...@gmail.com>.
Martin, My diagnosis hasn't been altered studied or challanged ;)
for a quick fix vm != null && should be added in the expression line
2083 of ResourceManagerImpl. A more robust solution must be possible
but i don't have time to look into that right now.
On Tue, Jan 27, 2015 at 4:16 PM, Martin Emrich
<ma...@empolis.com> wrote:
> Hi!
>
> Any news on this issue? I just fell in to this pit again :(
>
> Regards,
>
> Martin
>
> -----Ursprüngliche Nachricht-----
> Von: Martin Emrich [mailto:martin.emrich@empolis.com]
> Gesendet: Dienstag, 25. November 2014 13:39
> An: users@cloudstack.apache.org
> Betreff: Re: ALL Hosts Stuck in Maintenance
>
> Hi!
>
> Same problem here with CS 4.4.1 and 5 XenServers in one Cluster: Two hosts are in maintenance mode, and canceling maintenance mode results in "Internal Server Error". Here's my backtrace:
>
> 2014-11-25 13:08:45,448 ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-54:ctx-a076520e job-765) Unexpected exception while executing org.apache.cloudstack.api.command.admin.host.CancelMaintenanceCmd
> java.lang.NullPointerException
> at
> com.cloud.resource.ResourceManagerImpl.doCancelMaintenance(ResourceManagerImpl.java:2083)
> at
> com.cloud.resource.ResourceManagerImpl.cancelMaintenance(ResourceManagerImpl.java:2140)
> at
> com.cloud.resource.ResourceManagerImpl.cancelMaintenance(ResourceManagerImpl.java:1127)
> at sun.reflect.GeneratedMethodAccessor400.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
> at
> org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
> at com.sun.proxy.$Proxy148.cancelMaintenance(Unknown Source)
> at
> org.apache.cloudstack.api.command.admin.host.CancelMaintenanceCmd.execute(CancelMaintenanceCmd.java:102)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:141)
> at
> com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:108)
> at
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:503)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
> at
> org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
> at
> org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
> at
> org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:460)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2014-11-25 13:08:45,449 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] (API-Job-Executor-54:ctx-a076520e job-765) Complete async job-765,
> jobStatus: FAILED, resultCode: 530, result:
> org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530}
>
>
> Ciao
>
> Martin
--
Daan