[GitHub] [druid] suneet-s commented on pull request #13842: Make CompactionSearchPolicy injectable

["suneet-s (via GitHub)" <gi...@apache.org>]

suneet-s commented on PR #13842:
URL: https://github.com/apache/druid/pull/13842#issuecomment-1444759552

   > Could you please elaborate this part? I assume the intention here is not to make `CompactSegments` have a different period than the coordinator `period.indexingPeriod` as that is already possible through custom duties.
   
   If CompactSegments runs faster than the segment metadata refresh interval (which I think is 1 min by default) it doesn't realize the segments that were selected for compaction were compacted already, and the task fails until the metadata is refreshed. Now that the policy is available in the coordinator, the CompactSegments duty can be split into 2 - one that refreshes the iterator which can take a long time and another that keeps polling for the next available interval to be compacted and schedules the compaction task if there is capacity on the cluster to do so. I'll try to write up something more detailed in the next PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org