You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@aurora.apache.org by "David McLaughlin (JIRA)" <ji...@apache.org> on 2014/09/26 22:00:35 UTC
[jira] [Created] (AURORA-759) Scheduler updater stuck in
ROLLING_BACK
David McLaughlin created AURORA-759:
---------------------------------------
Summary: Scheduler updater stuck in ROLLING_BACK
Key: AURORA-759
URL: https://issues.apache.org/jira/browse/AURORA-759
Project: Aurora
Issue Type: Story
Components: Scheduler
Reporter: David McLaughlin
When using the new updater, I ran into a situation where my update was stuck in ROLLING_BACK state.
My sequence of events:
1) Create a job where your package is being fetched from a local directory.
2) Have that local directory wiped clean.
3) Update your job with a new config, with a bad path.
Stack traces from scheduler log:
{code}
I0926 18:50:56.888 THREAD278 org.apache.aurora.scheduler.updater.JobUpdateControllerImpl$4.execute: Forwarding task change for www-data/devel/hello_world/1
E0926 18:50:56.888 THREAD278 org.apache.aurora.scheduler.updater.JobUpdateEventSubscriber.taskChangedState: Failed to handle state change: java.lang.IllegalStateException: Attempted
to evaluate an inactive job updater.
java.lang.IllegalStateException: Attempted to evaluate an inactive job updater.
at com.google.common.base.Preconditions.checkState(Preconditions.java:176)
at org.apache.aurora.scheduler.updater.OneWayJobUpdater.evaluate(OneWayJobUpdater.java:145)
at org.apache.aurora.scheduler.updater.JobUpdateControllerImpl.evaluateUpdater(JobUpdateControllerImpl.java:451)
at org.apache.aurora.scheduler.updater.JobUpdateControllerImpl.access$700(JobUpdateControllerImpl.java:92)
at org.apache.aurora.scheduler.updater.JobUpdateControllerImpl$4.execute(JobUpdateControllerImpl.java:239)
at org.apache.aurora.scheduler.storage.Storage$MutateWork$NoResult.apply(Storage.java:132)
at org.apache.aurora.scheduler.storage.Storage$MutateWork$NoResult$Quiet.apply(Storage.java:149)
at org.apache.aurora.scheduler.storage.log.LogStorage.write(LogStorage.java:551)
at org.apache.aurora.scheduler.storage.CallOrderEnforcingStorage.write(CallOrderEnforcingStorage.java:130)
at org.apache.aurora.scheduler.updater.JobUpdateControllerImpl.instanceChanged(JobUpdateControllerImpl.java:231)
at org.apache.aurora.scheduler.updater.JobUpdateControllerImpl.instanceChangedState(JobUpdateControllerImpl.java:216)
at org.apache.aurora.scheduler.updater.JobUpdateEventSubscriber.taskChangedState(JobUpdateEventSubscriber.java:55)
at sun.reflect.GeneratedMethodAccessor39.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.google.common.eventbus.EventSubscriber.handleEvent(EventSubscriber.java:74)
at com.google.common.eventbus.SynchronizedEventSubscriber.handleEvent(SynchronizedEventSubscriber.java:47)
at com.google.common.eventbus.EventBus.dispatch(EventBus.java:322)
at com.google.common.eventbus.EventBus.dispatchQueuedEvents(EventBus.java:304)
at com.google.common.eventbus.EventBus.post(EventBus.java:275)
at org.apache.aurora.scheduler.events.PubsubEventModule$2.post(PubsubEventModule.java:68)
at org.apache.aurora.scheduler.state.StateManagerImpl.updateTaskAndExternalState(StateManagerImpl.java:443)
at org.apache.aurora.scheduler.state.StateManagerImpl.access$100(StateManagerImpl.java:80)
at org.apache.aurora.scheduler.state.StateManagerImpl$5.apply(StateManagerImpl.java:273)
at org.apache.aurora.scheduler.state.StateManagerImpl$5.apply(StateManagerImpl.java:259)
at org.apache.aurora.scheduler.storage.log.LogStorage$7.apply(LogStorage.java:559)
at org.apache.aurora.scheduler.storage.log.LogStorage$7.apply(LogStorage.java:556)
at org.apache.aurora.scheduler.storage.mem.MemStorage.doWork(MemStorage.java:175)
at org.apache.aurora.scheduler.storage.mem.MemStorage.access$400(MemStorage.java:59)
at org.apache.aurora.scheduler.storage.mem.MemStorage$4.apply(MemStorage.java:200)
at org.apache.aurora.scheduler.storage.mem.MemStorage$4.apply(MemStorage.java:197)
{code}
{code}
java.lang.IllegalArgumentException: A terminal state should not specify actions: EvaluationResult{status=FAILED, sideEffects={4=SideEffect{action=Optional.absent(), statusChanges=[W
ORKING, FAILED]}, 3=SideEffect{action=Optional.absent(), statusChanges=[WORKING, FAILED]}, 2=SideEffect{action=Optional.of(WATCH_TASK), statusChanges=[WORKING]}}}
at org.apache.aurora.scheduler.updater.JobUpdateControllerImpl.evaluateUpdater(JobUpdateControllerImpl.java:490)
at org.apache.aurora.scheduler.updater.JobUpdateControllerImpl.changeJobUpdateStatus(JobUpdateControllerImpl.java:402)
at org.apache.aurora.scheduler.updater.JobUpdateControllerImpl.recordAndChangeJobUpdateStatus(JobUpdateControllerImpl.java:334)
at org.apache.aurora.scheduler.updater.JobUpdateControllerImpl.changeUpdateStatus(JobUpdateControllerImpl.java:319)
at org.apache.aurora.scheduler.updater.JobUpdateControllerImpl.evaluateUpdater(JobUpdateControllerImpl.java:497)
at org.apache.aurora.scheduler.storage.db.DbStorage.write(DbStorage.java:150)
at org.apache.aurora.scheduler.storage.db.DbStorage$$EnhancerByGuice$$44df2497.CGLIB$write$4(<generated>)
at org.apache.aurora.scheduler.storage.db.DbStorage$$EnhancerByGuice$$44df2497$$FastClassByGuice$$d23df519.invoke(<generated>)
at com.google.inject.internal.cglib.proxy.$MethodProxy.invokeSuper(MethodProxy.java:228)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at org.mybatis.guice.transactional.TransactionalMethodInterceptor.invoke(TransactionalMethodInterceptor.java:101)
at com.google.inject.internal.InterceptorStackCallback$InterceptedMethodInvocation.proceed(InterceptorStackCallback.java:72)
at com.google.inject.internal.InterceptorStackCallback.intercept(InterceptorStackCallback.java:52)
at org.apache.aurora.scheduler.storage.db.DbStorage$$EnhancerByGuice$$44df2497.write(<generated>)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)