[GitHub] [druid] jihoonson opened a new issue #11445: Improve `TaskRunnerListener` to be more useful

[GitBox <gi...@apache.org>]

jihoonson opened a new issue #11445:
URL: https://github.com/apache/druid/issues/11445


   Druid ingestion service has a callback system for task status updates. In the current system, you can register a callback listener, i.e., `TaskRunnerListener`, on the `TaskRunner`. The `TaskRunnerListener` interface provides 2 methods to define the callback behavior upon task location or status changes. This currently has a couple of issues.
   
   1. `TaskRunnerListener` can be attached to the `TaskRunner` only when it is in the same JVM process. This limits the use case of the callback system. Suppose that the coordinator can attach a callback for auto compaction tasks in the overlord. It can safely mark the intervals compacted successfully to avoid further compaction. Or suppose that the parallel task can attach a callback for its subtasks. The overlord will send notifications for subtask status updates instead of parallel task poking the overlord constantly. I believe this limitation is the reason of why we have few use cases of `TaskRunnerListener` today.
   2. Multiple sources of task status change. Because `TaskRunner.run()` returns a future of `TaskStatus`, the caller should handle exceptions from the future. We usually handle these exceptions well, but sometimes [we create a new taskStatus to feed callback listeners that is different from what is propagated to the overlord](https://github.com/apache/druid/blob/master/indexing-service/src/main/java/org/apache/druid/indexing/overlord/SingleTaskBackgroundRunner.java#L211). This is quite confusing because those taskStatus fed to the listeners is not source of truth.
   3. The callback system is currently racy. If there are 2 or more threads calling `TaskRunnerUtils.notifyStatusChanged` at the same time, the listener can be notified the updates in random order.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org