[GitHub] [druid] a2l007 commented on issue #9940: Improve Performance With Multithreaded Kill Unused Segments Task

[GitBox <gi...@apache.org>]

a2l007 commented on issue #9940:
URL: https://github.com/apache/druid/issues/9940#issuecomment-642111692


   @jihoonson I believe the rationale behind having a multi-threaded Kill Task is with regard to clusters that heavily rely on just Hadoop based batch indexing. In this case, the cluster may not have several middlemanagers or the middlemanagers may not be tuned for native parallel indexing and thus might not have several task slots. If we end up making kill tasks spawn subtasks, it might use up more task slots which can cause subsequent indexing tasks to wait. 
   I agree that your idea of a task slot model for kill tasks is better in terms of fine grained resource management and is in line with the community's effort towards native parallel indexing tasks, but considering the usecase I mentioned above, it might be useful to support multi threading within a Kill Task itself as well.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org