[GitHub] [pinot] chenboat commented on a change in pull request #6778: upload missing LLC segment to segment store by controller periodic task

[GitBox <gi...@apache.org>]

chenboat commented on a change in pull request #6778:
URL: https://github.com/apache/pinot/pull/6778#discussion_r708471267



##########
File path: pinot-controller/src/main/java/org/apache/pinot/controller/ControllerConf.java
##########
@@ -173,6 +173,16 @@
     public static final String SEGMENT_RELOCATOR_INITIAL_DELAY_IN_SECONDS =
         "controller.segmentRelocator.initialDelayInSeconds";
 
+    // The flag to indicate if controller periodic job will fix the missing LLC segment deep store copy.
+    // Default value is false.
+    public static final String ENABLE_DEEP_STORE_RETRY_UPLOAD_LLC_SEGMENT =
+        "controller.realtime.segment.deepStoreUploadRetryEnabled";
+    // For realtime ingestion, we care more about recent data. This config indicates the fix range for missing LLC
+    // segment deep store copy, i.e. controller periodic job will only fix the segments created within this
+    // range. Note that smaller value puts less pressure on servers and zookeeper.
+    public static final String DEEP_STORE_RETRY_UPLOAD_LLC_SEGMENT_CREATED_IN_DAYS =

Review comment:
       This could be a good ramp-up control parameter we can tune -- we might start with 7 days and test of the impact. We want to protect our zk cluster as much as possible from large volume of reads.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org