[GitHub] QiuMM commented on issue #6991: run pending tasks when assigned a task that is already pending

[GitBox <gi...@apache.org>]

QiuMM commented on issue #6991: run pending tasks when assigned a task that is already pending
URL: https://github.com/apache/incubator-druid/pull/6991#issuecomment-464017959
 
 
   I think I know why the pending tasks aren't going to run, two cases:
   
   - If one worker is added to the black list, it will backoff for 15 minutes. The `workerBlackListCleanupPeriod` is 5 minutes, so 15 minutes later the worker still need to wait for several minutes (max to 5 minutes) to be removed from the black list and start to run the pending tasks. 
   
   - Every time I update and restart the Overlord, some tasks may fail to be assigned and I get the debug log "Unsuccessful task-assign attempt for task...".  If there are no new tasks to be added to the task queue or any other condition to trigger the `runPendingTasks()`, the pending tasks would never go to run (I have observed this situation several times). 
   
   Have updated this PR since just modified the `RemoteTaskRunner` didn't fix this issue, we also need to modify the `TaskQueue`.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org