[GitHub] [druid] maytasm commented on issue #12055: Auto Compaction Tasks Unexpectedly Running in Parallel (maxNumConcurrentSubTasks=1)

[GitBox <gi...@apache.org>]

maytasm commented on issue #12055:
URL: https://github.com/apache/druid/issues/12055#issuecomment-1005193925


   @pchang388 
   I suspect that the multiple compaction tasks for the same datasource are running for different intervals. The auto compaction create compaction tasks in order of most recent data regardless of datasource. For example, if you have 5 slots for auto compaction and datasource A with 5 segments of interval 2000-01-01/2000-01-02, 2000-01-02/2000-01-03, 2000-01-03/2000-01-04, 2000-01-04/2000-01-05, 2000-01-05/2000-01-06 and datasource B with 1 segment of interval 1999-01-01/1999-01-02. Auto compaction will create 5 compaction tasks for datasource A, one compaction task for each segment of datasource A. Auto compaction will not create any compaction task for datasource B as all 5 available slots are used for datasource A since all segments of datasource A is more recent than those of the datasource B. 
   
   Do you have segments that needs compaction for datasources that was not run that are more recent than the one being compacted in the tasks?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org