[GitHub] [pinot] klsince commented on issue #8258: MergeRollupTask and RealtimeToOfflineSegmentsTask don't clean up old segments

[GitBox <gi...@apache.org>]

klsince commented on issue #8258:
URL: https://github.com/apache/pinot/issues/8258#issuecomment-1054519053


   I think those two minion tasks don't remove the segments directly. For segments in RT table, they should be removed based on table retention. For segments in OFFLINE table, in addition to table retention, they also should be removed based on segment lineage. As you may have noticed in the design doc, segment lineage is used by MergeRollup task for atomic segment swap. 
   
   So, did you see any logs from [this manageSegmentLineageCleanupForTable method](https://github.com/apache/pinot/blob/master/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/retention/RetentionManager.java#L196) and [this manageRetentionForTable method](https://github.com/apache/pinot/blob/master/pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/retention/RetentionManager.java#L100) at the controller side?  to see if deletion kicked in or not.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org