[GitHub] [incubator-pinot] Jackie-Jiang commented on a change in pull request #7178: Merge/Rollup task scheduler which supports multi-level CONCAT tasks f…

[GitBox <gi...@apache.org>]

Jackie-Jiang commented on a change in pull request #7178:
URL: https://github.com/apache/incubator-pinot/pull/7178#discussion_r672670245



##########
File path: pinot-core/src/main/java/org/apache/pinot/core/common/MinionConstants.java
##########
@@ -67,16 +68,22 @@ private MinionConstants() {
     public static final String GRANULARITY_KEY = "granularity";
 
     // Rollup aggregate function related configs
-    public static final String AGGREGATE_KEY_PREFIX = "aggregate";
+    public static final String AGGREGATION_TYPE_KEY_SUFFIX = ".aggregationType";
 
     // Merge properties related configs
-    public static final String MERGE_KEY_PREFIX = "merge";
-    public static final String BUFFER_TIME = "bufferTime";
+    public static final String BUFFER_TIME_PERIOD = "bufferTimePeriod";
+    public static final String BUCKET_TIME_PERIOD = "bucketTimePeriod";
+    public static final String ROUND_BUCKET_TIME_PERIOD = "roundBucketTimePeriod";
     public static final String MAX_NUM_RECORDS_PER_SEGMENT = "maxNumRecordsPerSegment";
     public static final String MAX_NUM_RECORDS_PER_TASK = "maxNumRecordsPerTask";
 
     // Segment name generator related configs
-    public static final String MERGED_SEGMENT_NAME_KEY = "mergedSegmentNameKey";
+    public static final String SEGMENT_NAME_PREFIX_KEY = "segmentNamePrefix";
+    public static final String SEGMENT_NAME_PREFIX = "merged";
+
+    // Window start and window end set by task generator
+    public static final String WINDOW_START_MS_KEY = "windowStartMs";

Review comment:
       For merge/rollup task, we should not use the window start/end, but use the partition bucket instead. Different from the realtime-to-offline task, we need to keep all the records (cannot filter the records) for merge/rollup task.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@pinot.apache.org
For additional commands, e-mail: commits-help@pinot.apache.org