[GitHub] [incubator-druid] jihoonson commented on issue #8489: Stateful auto compaction

[GitBox <gi...@apache.org>]

jihoonson commented on issue #8489: Stateful auto compaction
URL: https://github.com/apache/incubator-druid/issues/8489#issuecomment-535288014
 
 
   @himanshug thank you for taking a look!
   
   > 1. Does coordinator auto compaction always do "minor compaction" ( described in #7491 )?
   
   For now, auto compaction doesn't support minor compaction. You can do minor compaction with the compaction task by setting `forceGuaranteedRollup`.
   
   The future plan I'm thinking of is supporting both minor and major compaction in auto compaction. Like, it will do minor compaction for recent segments while doing major compaction for old ones. There are a couple reasons why we still need major compaction. One reason could be, we can do secondary partitioning with major compaction which could accelerate query performance by further pruning segments.
   
   > 2. Same `CompactionTask` is used for both "minor" as well as "major" compaction. It is major compaction when either of "interval" or "segmentGranularity" fields are explicitly provided ?
   
   Good question. It is the major compaction if 1) `forceGuaranteedRollup` is true or 2) `segmentGranularity` is provided explicitly. The second condition should actually be "if the provided segment granularity is different from that of input segments". But I was lazy to implement it in #7547..
   
   > > If it exists, the coordinator compares the compactionPartitionsSpec with the partitionsSpec in auto compaction configuration. The segment will be a compaction candidate if it has a different partitionsSpec.
   > 
   > or else auto/minor compaction would produce exactly same result because it doesn't change segmentGranularity... right ?
   
   You mean if the compaction task has the exactly same tuningConfig? Then I think yes.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@druid.apache.org
For additional commands, e-mail: commits-help@druid.apache.org